Mark Shuttleworth | here be dragons

As we move from “tens” to “hundreds” to “thousands” of nodes in a typical data centre we need new tools and practices. This hyperscale story – of hyper-dense racks with wimpy nodes – is the big shift in the physical world which matches the equally big shift to cloud computing in the virtualised world. Ubuntu’s popularity in the cloud comes in part from being leaner, faster, more agile. And MAAS – Metal as a Service – is bringing that agility back to the physical world for hyperscale deployments.

Servers used to aspire to being expensive. Powerful. Big. We gave them names like “Hercules” or “Atlas”. The bigger your business, or the bigger your data problem, the bigger the servers you bought. It was all about being beefy – with brands designed to impress, like POWER and Itanium.

Things are changing.

Today, server capacity can be bought as a commodity, based on the total cost of compute: the cost per teraflop, factoring in space, time, electricity. We can get more power by adding more nodes to our clusters, rather than buying beefier nodes. We can increase reliability by doubling up, so services keep running when individual nodes fail. Much as RAID changed the storage game, this scale-out philosophy, pioneered by Google, is changing the server landscape.

In this hyperscale era, each individual node is cheap, wimpy and, by historical standards for critical computing, unreliable. But together, they’re unstoppable. The horsepower now resides in the cluster, not the node. Likewise, the reliability of the infrastructure now depends on redundancy, rather than heroic performances from specific machines. There is, as they say, safety in numbers.

We don’t even give hyperscale nodes proper names any more – ask “node-0025904ce794”. Of course, you can still go big with the cluster name. I’m considering “Mark’s Magnificent Mountain of Metal” – significantly more impressive than “Mark’s Noisy Collection of Fans in the Garage”, which is what Claire will probably call it. And that’s not the knicker-throwing kind of fan, either.

The catch to this massive multiplication in node density, however, is in the cost of provisioning. Hyperscale won’t work economically if every server has to be provisioned, configured  and managed as if it were a Hercules or an Atlas. To reap the benefits, we need leaner provisioning processes. We need deployment tools to match the scale of the new physical reality.

That’s where Metal as a Service (MAAS) comes in. MAAS makes it easy to set up the hardware on which to deploy any service that needs to scale up and down dynamically – a cloud being just one example. It lets you provision your servers dynamically, just like cloud instances – only in this case, they’re whole physical nodes. “Add another node to the Hadoop cluster, and make sure it has at least 16GB RAM” is as easy as asking for it.

With a simple web interface, you can  add, commission, update and recycle your servers at will.  As your needs change, you can respond rapidly, by adding new nodes and dynamically re-deploying them between services. When the time comes, nodes can be retired for use outside the MAAS.

As we enter an era in which ATOM is as important in the data centre as XEON, an operating system like Ubuntu makes even more sense. Its freedom from licensing restrictions, together with the labour saving power of tools like MAAS, make it cost-effective, finally, to deploy and manage hundreds of nodes at a time

Here’s another way to look at it: Ubuntu is bringing cloud semantics to the bare metal world. What a great foundation for your IAAS.

Reports of beta testing for 8.04 LTS seem very positive all round, to the great credit of the desktop and server teams who have been working so hard to make Hardy Heron rock. I have been running Hardy on my laptop through most of the cycle, but took the plunge on my home firewall and desktop (Kubuntu) machine this weekend.

The coolest part of the firewall upgrade is the fact that Michael has made the release upgrade tool independent of the GUI, so you can use it for server upgrades too.

So, now would be a great time to test the upgrade! File bugs if you run into any issues with your particular configuration. Apparently, this is upported on both Ubuntu 6.06 LTS (Dapper) and Ubuntu 7.10 (Gutsy), so feedback on either upgrade path would be most welcome.

sudo aptitude install update-manager-core

That should make sure you have the release upgrade tool installed. Now you can trigger the upgrade process to the current beta:

sudo do-release-upgrade --devel-release

This should fetch the latest version of the update tool, which knows about various transitions in library versions etc so that it can attempt to update your machine smoothly without leaving large amounts of dangling packages. You can say “no” if you don’t like the proposed package install and removal plan (in which case, your feedback would be very valuable!).

For fresh installs, 8.04 LTS should be good to go on any high-volume server platform available in the market today – let the server team know if you run into any problems at all. They are hoping to meet the desktop team’s “Just Works” standard, so the bar is set pretty high. From my perspective, the upgrade was smooth – full marks and my thanks to everyone involved.