Rikku
15th August 2009, 16:02
After working for a few months now building different clusters based on Linux and server hardware I've decided to try something a little different. Running 10-20 servers 24/7 crunching numbers is very costly and as I don't have anywhere much more practical than the end of my bed to run them, it can be quite noisy also! Size is also a problem, so far I've been working with 1U and 3U servers, and as I've discovered, a stack of 25 3U servers is rather large. So now I'm a bit more experienced with general Linux cluster building, now for something new.
After looking at the main problems with my previous clusters (the big secretly beige one that never really had a name and Chobits the HP one) I've decided to take things to an extreme, the main problems being power, noise and size, so what better to use than mini-ITX hardware! At first I looked at using nano-ITX but this was going to be too costly and underpowered for the task. So what was I looking for, something mini-itx that packs a decent punch and wasn't too costly. I also aimed to get something that had onboard gigabit ethernet and ideally multi-core. I shopped around for a few days and eventually came across a new Intel board, the D945GCLF2. It had everything I needed, the new dual core Atom-330, gigabit ethernet with PXE support, low power and best of all it was only £70! So I added it to my basket along with 1Gb 667Mhz DDR2 RAM, a 4Gb IDE flash card and a really, really small DC-DC power supply, the PicoPSU-90.
All the kit went together nicely and I couldn't wait to get the thing up and running crunching numbers to see what it was capable of! I was really impressed with how quiet the board ran, it only had one small fan on it that was pretty quiet, but then I suppose anything is after listening to the original Chobits cluster! After running it for a while under load I was impressed with how cool it kept, however, there was one chip (possibly the southbridge) that had no heatsink and got quite warm under load, time for the boards first mod!
A relatively cheap chipset cooling kit from Akasa had a heatsink in it that was the perfect size for the chip and came with an adhesive pad for mounting it. While I was there I also removed the original CPU heatsink and replaced it with the Akasa one that came with a fan. While I was at it I cut some of the cables down, mainly the HDD power cable, which was huge, and the fan cables to tidy the board up and also bought some bright white 3mm LED's from Maplin and set at making a power and HDD activity light also.
Now for something a little different in the software department, until now all my clusters have been Linux based, I've used everything from Fedora Core 3 to Slackware based Zenwalk. I even once had a dabble at Windows Server 2008, but that's a different story altogether. Anyway, this time I decided to learn something new and go with FreeBSD, it seemed to be a good choice for embedded systems and has out of the box diskless client support, great stuff.
First impressions, easy enough to install, took some getting used to the naming conventions of things i.e. re0 instead of eth0 and /dev/ad0sl1a rather than /dev/sda1. After I'd installed it and played with it a bit I managed to destroy it somehow, I fat-fingered a command and somehow removed all my bins, never a good thing. So I started fresh and installed it from the tty rather than the installer and managed to get the system running with everything I needed at around 300Mb.
I decided to have a go at using MPICH for my message passing interface rather than OpenMPI which I had used up until now, I got it installed and ran in to all kinds of random problems and I soon lost my patience and uninstalled it and opted for OpenMPI instead.
OpenMPI installation went smoothly and within minutes I had it up and running with John the Ripper MPI. I ran a few tests running John on it's own and got around 2200K c/s, not too bad. I then ran John with mpirun and after some playing around with the number of processes I managed to peak at around 8300k c/s, not bad at all, that's almost 4 times the speed of running single threaded, almost perfect considering it's a dual core CPU with Hyper-Threading so SMP picks it up as 4 cores. It will never be quite perfect though as there will always be some power used on background tasks, especially on the master node. Now all I need is more hardware...
For pics and more info go here http://nullconnection.com/rikkuness/
After looking at the main problems with my previous clusters (the big secretly beige one that never really had a name and Chobits the HP one) I've decided to take things to an extreme, the main problems being power, noise and size, so what better to use than mini-ITX hardware! At first I looked at using nano-ITX but this was going to be too costly and underpowered for the task. So what was I looking for, something mini-itx that packs a decent punch and wasn't too costly. I also aimed to get something that had onboard gigabit ethernet and ideally multi-core. I shopped around for a few days and eventually came across a new Intel board, the D945GCLF2. It had everything I needed, the new dual core Atom-330, gigabit ethernet with PXE support, low power and best of all it was only £70! So I added it to my basket along with 1Gb 667Mhz DDR2 RAM, a 4Gb IDE flash card and a really, really small DC-DC power supply, the PicoPSU-90.
All the kit went together nicely and I couldn't wait to get the thing up and running crunching numbers to see what it was capable of! I was really impressed with how quiet the board ran, it only had one small fan on it that was pretty quiet, but then I suppose anything is after listening to the original Chobits cluster! After running it for a while under load I was impressed with how cool it kept, however, there was one chip (possibly the southbridge) that had no heatsink and got quite warm under load, time for the boards first mod!
A relatively cheap chipset cooling kit from Akasa had a heatsink in it that was the perfect size for the chip and came with an adhesive pad for mounting it. While I was there I also removed the original CPU heatsink and replaced it with the Akasa one that came with a fan. While I was at it I cut some of the cables down, mainly the HDD power cable, which was huge, and the fan cables to tidy the board up and also bought some bright white 3mm LED's from Maplin and set at making a power and HDD activity light also.
Now for something a little different in the software department, until now all my clusters have been Linux based, I've used everything from Fedora Core 3 to Slackware based Zenwalk. I even once had a dabble at Windows Server 2008, but that's a different story altogether. Anyway, this time I decided to learn something new and go with FreeBSD, it seemed to be a good choice for embedded systems and has out of the box diskless client support, great stuff.
First impressions, easy enough to install, took some getting used to the naming conventions of things i.e. re0 instead of eth0 and /dev/ad0sl1a rather than /dev/sda1. After I'd installed it and played with it a bit I managed to destroy it somehow, I fat-fingered a command and somehow removed all my bins, never a good thing. So I started fresh and installed it from the tty rather than the installer and managed to get the system running with everything I needed at around 300Mb.
I decided to have a go at using MPICH for my message passing interface rather than OpenMPI which I had used up until now, I got it installed and ran in to all kinds of random problems and I soon lost my patience and uninstalled it and opted for OpenMPI instead.
OpenMPI installation went smoothly and within minutes I had it up and running with John the Ripper MPI. I ran a few tests running John on it's own and got around 2200K c/s, not too bad. I then ran John with mpirun and after some playing around with the number of processes I managed to peak at around 8300k c/s, not bad at all, that's almost 4 times the speed of running single threaded, almost perfect considering it's a dual core CPU with Hyper-Threading so SMP picks it up as 4 cores. It will never be quite perfect though as there will always be some power used on background tasks, especially on the master node. Now all I need is more hardware...
For pics and more info go here http://nullconnection.com/rikkuness/