Proof of Concept: Mixing Clustering with Virtualization #openmosix #linux #vserver #cluster
For over 7 years I’ve been thing of the possibilities of mixing clustered computing with virtualization. Distributed computing is in essence combining multiple physical computers to create one big virtual computer, virtualization is in essence about creating multiple smaller computers in the context of one physical computer.
I originally thought of creating a Beowulf cluster, for which I had ordered a CD in 1998, although my experience with Beowulf had been practically zero. So I decided to start a new homegrown project and investigated the existing tools which cold be used to implement this. After due diligence I took two platforms which I liked for their potential and FOSS nature OpenMosix and Linux VServer and integrated those kernel patches to make it possible to run both together. I was primarily going to using Gentoo in both the hosts as the guests servers, with the exception of one RedHat, one Mandrake (Mandriva) and one Debian guest for the compiler packaging farm. My plan was to have a hetrogenous cluster with an underlying clustered filesystem.
At the time I was doing this distributed filesystems where not my forte, and I wanted a working proof of concept which I could use without spending too much time on getting the filesystem working. So I chose for NFS, with locking disabled. This did mean that I needed to have a master server which could do the primary updating of the NFS. This master server would double as my Gentoo package server and my syslog server, this would avoid me needing to make any changes in the VServers themselves. A drawback of a hetrogenous cluster is the need to compile for the lowest common denominator CPU, and although the gain of sharing binaries did make up for that the guarantee that the whole cluster would be able to run the distributed threads and the investment cost, which was close to zero as I’d saved most of the computers from the garbage.
In the end I was running 6 hosts, each with between 2 and 5 guests.
Image source: Martinez Zea