Archive for April, 2012

Quality has a new name

Monday, April 23rd, 2012

This quirky scheme of adjectives and animals presents a pretty puzzle every six months. What mix of characteristics do we want to celebrate in the next release? Here we are, busily finalizing the precise pangolin (which was a rather perfect product placement for a scaly anteater, all things considered) and before one realises it’s time to talk turkey, so to speak, about Q! Our code names may raise a quizzical eyebrow here and there, but they capture the zeitgeist of a cycle and shape our discussions in surprising ways. The quest for a name has no quick answer unless, of course, you jump to the last paragraph ;)

12.04 being an LTS we’ve been minding our P’s and Q’s, but many of our quality-oriented practices from 12.04 LTS will continue into Q territory. We’ll keep the platform usable throughout the cycle, because that helped hugely to encourage daily use of the release, which in turn gives us much better feedback on questions of quality. And we’ll ratchet up the continuous integration, smoke testing and automated benchmarking of the release, since we can do it all in the cloud. We have, so to speak, stacks and stacks of cloud to use. So quality is quotidian rather than quarterly. And it is both qualitative and quantitative, with user research and testing continuing to shape our design decisions. The effort we put into polishing Unity and the rest of the platform in 12.04 seem to have paid off handsomely, with many quondam quarrelsome suddenly quiescent in the face of a surge in support for the work.

But the finest quality is that without a name, so support for “quality” as a codename would at best be qualified. Every release has quality first these days – they all get used, on the server, on devices, and while the term of maintenance might vary, our commitment to interim releases is just as important as that to an LTS.

Our focus on quality permeates from the platform up to the code we write upstream, and our choices of upstream components too. We require tests and gated trunks for all Canonical codebases, and prefer upstreams that share the same values. Quality starts at the source, it’s not something that can be patched in after the fact. And I’m delighted that we have many upstreams using our tools to improve their quality too! We have awesome tools for daily builds from branches, continuous integration support in Launchpad, the ability to provide a gated trunk with tests run in the cloud for projects that really care about quality. Rumours and allegations of a move from Upstart to systemd are unfounded: Upstart has a huge battery of tests, the competition has virtually none. Upstart knows everything it wants to be, the competition wants to be everything. Quality comes from focus and clarity of purpose, it comes from careful design and rigorous practices. After a review by the Ubuntu Foundations team our course is clear: we’re committed to Upstart, it’s the better choice for a modern init, innit. For our future on cloud and client, Upstart is crisp, clean and correct. It will be a pleasure to share all the Upstart-enablement patches we carry with other family friends as soon as their release is ready and they can take a breath, so to speak.

From a styling point of view, we think in terms of quadruples: this next release starts a cycle of four, which will culminate in 14.04 LTS. So there’s an opportunity to refresh the look. That will kick off with a project on typography to make sure we are expressing ourselves with crystal clarity – making the most of Ubuntu’s Light and Medium font weights for a start. And a project on iconography, with the University of Reading, to refine the look of apps and interfaces throughout the platform. It’s amazing how quaint the early releases of Ubuntu look compared to the current style. And we’re only just getting started! In our artistic explorations we want to embrace tessellation as an expression of the part-digital, part-organic nature of Ubuntu. We love the way tessellated art expresses both the precision and reliability of our foundations, and the freedom and collaboration of a project driven by people making stuff for people. There’s nothing quixotic in our desire to make Ubuntu the easiest, steadiest, and most beautiful way to live digitally.

On the fauna front, the quotable campaign for the Queer Quokka is quorate but, it must sometimes be said, this is not a democracy. One man’s favourite furball is another’s mangy marsupial. No, the quintessential stories of Q will be all about style on the client, with a refresh of our theme and typography, a start on new iconography and perhaps even a new form factor taking flight. So brown is out and something colourful and light is called for. On the cloud front, the new virtualized network madness called Quantum will make its appearance. Being a first cut, it’s more likely to be Folsom than wholesome, but it’s going to be worth calling out, and the name is reminiscent of our package-oriented practices, where goodness is delivered one piece at a time. And so the stage is set for a decision:  I give you the Quantal Quetzal, soon to be dressed in tessellated technicolour, now open for toolchains, kernels and other pressing preparatory packages.

It may not be riding rockets, but bee-keeping has a certain edge to it when you’re allergic to their stings.

Last autumn we set up three hives on a corner of the garden. Andrew and the botanical team have since given the hives proper plinths, we’ve registered them with the local authorities, and it’s become a rite of passage for willing guests to join a hive inspection or feeding.

Claire agreed to join, if I could find a pink bee suit. Turns out, white ones dye really well:

Claire in her new bee suit

The bees just call her "Blossom" with a capital B

For those of you who, like me, know virtually nothing about bees or beekeeping, here’s the short short version.

Beehives are cleverly designed as stacks of rectangular boxes without a floor or a ceiling. You combine different types of boxes to get different things done. For example, the main body of the hive lives in the base box, called a brood box. That’s where the queen hangs out, the comb there has both honey cells and hatching cells (think honey-eating bee-maggots, more charmingly called brood). The worker bees move freely up and down the hive, from box to box. You put a filter above the brood box to stop the queen, who is too big to fit through the grille, from laying eggs upstairs.

You get boxes for feeding the bees, and boxes for collecting honey. The honey-collecting boxes are called “supers”, and the others, which are more like spacing boxes with room inside for gadgets and gizmos are called ekes (possibly Norse, for “augment”?). Since our colonies are young, they haven’t yet filled out their brood boxes, so I’ve focused on feeding them. I accidentally drowned half the bees in each of the colonies in the first attempt by failing to install the feeders properly so, all in all, I’ve come to appreciate how fragile the colonies can be.

Throughout the course of the year, there are different things to watch out for, or get done. Early in spring you give the hives a dose of syrup to get them started. We started feeding them in March.

This Easter weekend, with my brother Bradley & guest visiting, and spring springing, seemed like a perfect occasion to see how the colonies were doing. We made some feeding syrup last night, woke up at sparrowfart this morning and suited up. Our goal was to check out the hives, make sure they seemed healthy & happy, and replenish the food to help the colonies grow quickly.

Things got a little adventurous while we were inspecting the brood frames of the first and most vigorous of the hives. Two of us ended up with bees inside our veils, so we closed up quickly and beat a hasty retreat for a brush-off and tea. I got stung on the neck, but I think the sting failed to set properly so the epi-pens are all intact. Some disappointment in the household that nobody got to jab me in the heart with one.

Restored, we went back to finish off the other hives, and Bradley spotted a queen – first time I’ve seen one of them. Once we all relaxed it was a pleasure to work through the frames together one by one. As Bradley said, there are millions of years of evolution telling you that you urgently have to be somewhere else every time the hive buzzes. We’ve got to figure out the best way to clear each frame without ending up surrounded by a cloud of upset bees. But it all went smoothly.

So, all things being well, next year we’ll have honey that is genuinely local. And we’ll get to test that old story about local honey being good for hayfever. All in the name of science, of course.

As we move from “tens” to “hundreds” to “thousands” of nodes in a typical data centre we need new tools and practices. This hyperscale story – of hyper-dense racks with wimpy nodes – is the big shift in the physical world which matches the equally big shift to cloud computing in the virtualised world. Ubuntu’s popularity in the cloud comes in part from being leaner, faster, more agile. And MAAS – Metal as a Service – is bringing that agility back to the physical world for hyperscale deployments.

Servers used to aspire to being expensive. Powerful. Big. We gave them names like “Hercules” or “Atlas”. The bigger your business, or the bigger your data problem, the bigger the servers you bought. It was all about being beefy – with brands designed to impress, like POWER and Itanium.

Things are changing.

Today, server capacity can be bought as a commodity, based on the total cost of compute: the cost per teraflop, factoring in space, time, electricity. We can get more power by adding more nodes to our clusters, rather than buying beefier nodes. We can increase reliability by doubling up, so services keep running when individual nodes fail. Much as RAID changed the storage game, this scale-out philosophy, pioneered by Google, is changing the server landscape.

In this hyperscale era, each individual node is cheap, wimpy and, by historical standards for critical computing, unreliable. But together, they’re unstoppable. The horsepower now resides in the cluster, not the node. Likewise, the reliability of the infrastructure now depends on redundancy, rather than heroic performances from specific machines. There is, as they say, safety in numbers.

We don’t even give hyperscale nodes proper names any more – ask “node-0025904ce794”. Of course, you can still go big with the cluster name. I’m considering “Mark’s Magnificent Mountain of Metal” – significantly more impressive than “Mark’s Noisy Collection of Fans in the Garage”, which is what Claire will probably call it. And that’s not the knicker-throwing kind of fan, either.

The catch to this massive multiplication in node density, however, is in the cost of provisioning. Hyperscale won’t work economically if every server has to be provisioned, configured  and managed as if it were a Hercules or an Atlas. To reap the benefits, we need leaner provisioning processes. We need deployment tools to match the scale of the new physical reality.

That’s where Metal as a Service (MAAS) comes in. MAAS makes it easy to set up the hardware on which to deploy any service that needs to scale up and down dynamically – a cloud being just one example. It lets you provision your servers dynamically, just like cloud instances – only in this case, they’re whole physical nodes. “Add another node to the Hadoop cluster, and make sure it has at least 16GB RAM” is as easy as asking for it.

With a simple web interface, you can  add, commission, update and recycle your servers at will.  As your needs change, you can respond rapidly, by adding new nodes and dynamically re-deploying them between services. When the time comes, nodes can be retired for use outside the MAAS.

As we enter an era in which ATOM is as important in the data centre as XEON, an operating system like Ubuntu makes even more sense. Its freedom from licensing restrictions, together with the labour saving power of tools like MAAS, make it cost-effective, finally, to deploy and manage hundreds of nodes at a time

Here’s another way to look at it: Ubuntu is bringing cloud semantics to the bare metal world. What a great foundation for your IAAS.