General Musing

blaze your trail

Archive for the ‘hardware’ Category

Proof of Concept: Use Everywhere #bash

leave a comment »

For a while I’ve been using custom tools to perform certain task that I wasn’t able to do with the shell tools provided. Or which worked differently on each of the Unix platforms I worked on. I mostly solved this by adding any custom scripts to the bundle that I deployed. This naturally didn’t work for compiled programs, due to different processors and kernel stacks.

When I came to a platform for which I had not yet compiled my custom tool I would compile it and add it to my custom share, USB-stick or CVS. I would name the tool <name>-<machine>-<system> which would allow me to ship the bundle with a version which was compiled specifically for that platform, or for multiple platforms. I would then call my wrapper script which would call the specific binary for the current platform.

I […] am rarely happier than when spending an entire day programming my computer to perform automatically a task that it would otherwise take me a good ten seconds to do by hand. Ten seconds, I tell myself, is ten seconds. Time is valuable and ten seconds’ worth of it is well worth the investment of a day’s happy activity working out a way of saving it.
Douglas Adams

Naturally I borrowed the idea from somebody, although I can no longer remember who.

Image source: pastebin


Written by Daniël W. Crompton (webhat)

March 10, 2012 at 12:48 pm

Posted in hardware, OS, programming

Tagged with , ,

On the Border of the Internet #risk

with 3 comments

The age old lie told by ISP support desks: ” The Internet is down,” was briefly reality again yesterday.

The past couple of days I’d been seeing and hearing comments that there was a disturbance in the force of the Internet. Initially a NANOG message was posted about a general malaise or instability in the Internet, some humorous quips were posted in response and the matter was soon forgotten.

A network operator looking with hindsight said that they had been able to see more than normal numbers of updates coming on BGP which is normally an indicator of network instability being solved by rerouting round the problem. That is all part of the normal operation of the Internet. And sometime yesterday morning as the east coast of the US was getting to work the looming disaster struck.

Juniper network devices started core dumping and restarting due to a bug in the code which handled the BGP UPDATE messages as another large updated was arriving. The self healing properties of the Internet broke and the Internet went with it. The Great Juniper Outage of 2011 was born.


Almost certainly. The reliance on the hardware of one specific vendor on the part of large ISPs – backbone carriers – creates a single point of failure which is bad – mkay. A fail over situation should always be in place, not just at the ISPs. Companies who rely on the Internet for business should take this into account too. A recent outages at some of companies I consulted said that by placing their faith in one specific vendor they had created a single point of failure which had caused some high profile repercussions.

Do you have a single point of failure?

Written by Daniël W. Crompton (webhat)

November 8, 2011 at 7:13 am

Posted in hardware, risk, technology

Tagged with , , , ,

Proof of Concept: Mixing Clustering with Virtualization #openmosix #linux #vserver #cluster

leave a comment »

Cluster armado con hardware viejo y donado - Cluster armed with old and donated hardware

For over 7 years I’ve been thing of the possibilities of mixing clustered computing with virtualization. Distributed computing is in essence combining multiple physical computers to create one big virtual computer, virtualization is in essence about creating multiple smaller computers in the context of one physical computer.

I originally thought of creating a Beowulf cluster, for which I had ordered a CD in 1998, although my experience with Beowulf had been practically zero. So I decided to start a new homegrown project and investigated the existing tools which cold be used to implement this. After due diligence I took two platforms which I liked for their potential and FOSS nature OpenMosix and Linux VServer and integrated those kernel patches to make it possible to run both together. I was primarily going to using Gentoo in both the hosts as the guests servers, with the exception of one RedHat, one Mandrake (Mandriva) and one Debian guest for the compiler packaging farm. My plan was to have a hetrogenous cluster with an underlying clustered filesystem.

At the time I was doing this distributed filesystems where not my forte, and I wanted a working proof of concept which I could use without spending too much time on getting the filesystem working. So I chose for NFS, with locking disabled. This did mean that I needed to have a master server which could do the primary updating of the NFS. This master server would double as my Gentoo package server and my syslog server, this would avoid me needing to make any changes in the VServers themselves. A drawback of a hetrogenous cluster is the need to compile for the lowest common denominator CPU, and although the gain of sharing binaries did make up for that the guarantee that the whole cluster would be able to run the distributed threads and the investment cost, which was close to zero as I’d saved most of the computers from the garbage.

In the end I was running 6 hosts, each with between 2 and 5 guests.

Image source: Martinez Zea

Written by Daniël W. Crompton (webhat)

February 16, 2011 at 12:17 pm

I want a GPS Video Camera and a little Laptop

leave a comment »

Written by Daniël W. Crompton (webhat)

July 6, 2008 at 3:27 pm

%d bloggers like this: