What if your cloud instances could be updated with the same certainty and precision as your mobile phone – with carrier grade assurance that an update applies perfectly or is not applied at all? What if your apps could be isolated from one another completely, so there’s no possibility that installing one app could break another, and stronger assurance that a compromise of one app won’t compromise the data from another? When we set out to build the Ubuntu Phone we took on the challenge of raising the bar for reliability and security in the mobile market. And today that same technology is coming to the cloud, in the form of a new “snappy” image called Ubuntu Core, which is in beta today on Azure and as a KVM image you can run on any Linux machine.

This is in a sense the biggest break with tradition in 10 years of Ubuntu, because snappy Ubuntu Core doesn’t use debs or apt-get. We call it “snappy” because that’s the new bullet-proof mechanism for app delivery and system updates; it’s completely different to the traditional package-based Ubuntu server and desktop. The snappy system keeps each part of Ubuntu in a separate, read-only file, and does the same for each application. That way, developers can deliver everything they need to be confident their app will work exactly as they intend, and we can take steps to keep the various apps isolated from one another, and ensure that updates are always perfect. Of course, that means that apt-get won’t work, but that’s OK since developers can reuse debs to make their snappy apps, and the core system is exactly the same as any other Ubuntu system – server or desktop.

Whenever we make a fix to packages in Ubuntu, we’ll publish the same fix to Ubuntu Core, and systems can get that fix transactionally. In fact, updates to Ubuntu Core are even smaller than package updates because we only need to send the precise difference between the old and new versions, not the whole package. Of course, Ubuntu Core is in addition to all the current members of the Ubuntu family – desktop, server, and cloud images that use apt-get and debs, and all the many *buntu remixes which bring their particular shine to our community. You still get all the Ubuntu you like, and there’s a new snappy Core image on all the clouds for the sort of deployment where precision, specialism and security are the top priority.

This is the biggest new thing in Ubuntu since we committed to deliver a mobile phone platform, and it’s very delicious that it’s borne of exactly the same amazing technology that we’ve been perfecting for these last three years. I love it when two completely different efforts find underlying commonalities, and it’s wonderful to me that the work we’ve done for the phone, where carriers and consumers are the audience, might turn out to be so useful in the cloud, which is all about back-end infrastructure.

Why is this so interesting?

Transactional updates have lots of useful properties: if they are done well, you can know EXACTLY what’s running on a particular system, and you can coordinate updates with very high precision across thousands of instances in the cloud. You can run systems as canaries, getting updates ahead of other identical systems to see if they cause unexpected problems. You can roll updates back, because each version is a complete, independent image. That’s very nice indeed.

There have been interesting developments in the transaction systems field over the past few years. ChromeOS is updated transactionally, when you turn it on, it makes sure it’s running the latest version of the OS. CoreOS brought aspects of Chrome OS and Gentoo to the cloud, Red Hat has a beta of Atomic as a transactional version of RHEL, and of course Docker is a way of delivering apps transactionally too (it combines app and system files very neatly). Ubuntu Core raises the bar for certainty, extensibility and security in the transactional systems game. What I love about Ubuntu Core is the way it embraces transactional updates not just for the base system but for applications on top of the system as well. The system is just one layer that can be updated transactionally, and so are each of the apps on the system. You get an extensible platform that retains the lovely properties of transactionality but lets you choose exactly the capabilities you want for yourself, rather than having someone else force you to use a particular tool.

For example, in CoreOS, things like Fleet are built-in, you can’t opt out. In Ubuntu Core, we aim for a much smaller Core, and then enable you to install Docker or any other container system as a framework, with snappy. We’re working with all the different container vendors, and app systems, and container coordination systems, to help them make snappy versions of their tools. That way, you get the transactional semantics you want with the freedom to use whichever tools suit you. And the whole thing is smaller and more secure because we baked fewer assumptions into the core.

The snappy system is also designed to provide security guarantees across diverse environments. Because there is a single repository of frameworks and packages, and each of them has a digital fingerprint that cannot be faked, two people on opposite ends of the world can compare their systems and know that they are running exactly the same versions of the system and apps. Atomic might allow you to roll back, but it’s virtually impossible to customise the system for your own preferences rather than Red Hat’s, and still know you are running the same secure bits as anybody else.

Developers of snappy apps get much more freedom to bundle the exact versions of libraries that they want to use with their apps. It’s much easier to make a snappy package than a traditional Ubuntu package – just bundle up everything you want in one place, and ship it. We use strong application isolation to keep data confidential between apps. If you install a bad app, it only has access to the data you create with that app, not to data from other applications. This is a key piece of security that comes from our efforts to bring Ubuntu to the mobile market, where malware is a real problem today. And as a result, we can enable developers to go much faster – they can publish their app on whatever schedule suits them, regardless of the Ubuntu release cadence. Want the very latest app? Snappy makes that easiest.

This is also why I think snappy will result in much simpler systems management. Instead of having literally thousands of packages on your Ubuntu server, with tons of dependencies, a snappy system just has a single package for each actual app or framework that’s installed. I bet the average system on the cloud ends up with about three packages installed, total! Try this sort of output:

$ snappy info
release: ubuntu-core/devel
frameworks: docker, panamax
apps: owncloud

That’s much easier to manage and reason about at scale. We recently saw how complicated things can get in the old packaging system, when Owncloud upstream wanted to remove the original packages of Owncloud from an old Ubuntu release. With snappy Ubuntu, Owncloud can publish exactly what they want you to use as a snappy package, and can update that for you directly, in a safe transactional manner with full support for rolling back. I think upstream developers are going to love being in complete control of their app on snappy Ubuntu Core.

$ sudo snappy install hello-world

Welcome to a snappy new world!

Things here are really nice and simple:

$ snappy info
$ snappy build .
$ snappy install foo
$ snappy update foo
$ snappy rollback foo
$ snappy remove foo
$ snappy update-versions
$ snappy versions

Just for fun, download the image and have a play. I’m delighted that Ubuntu Core is today’s Qemu Advent Calendar image too! Or launch it on Azure, coming soon to all the clouds.

It’s important for Ubuntu to continue to find new ways to bring free software to a wider audience. The way people think about software is changing, and I think Ubuntu Core becomes a very useful tool for people doing stuff at huge scale in the cloud. If you want crisp, purposeful, tightly locked down systems that are secure by design, Ubuntu Core and snappy packages are the right tool for the job. Running docker farms? Running transcode farms? I think you’ll like this very much!

We have the world’s biggest free software community because we find ways to recognise all kinds of contributions and to support people helping one another to bring their ideas to fruition. One of the goals of snappy was to reduce the overhead and bureaucracy of packaging software to make it incredibly easy for anybody to publish code they care about to other Ubuntu users. We have built a great community of developers using this toolchain for the phone, I think it’s going to be even better on the cloud where Ubuntu is already so popular. There is a lot to do in making the most of existing debs in the snappy environment, and I’m excited that there is a load of amazing software on github that can now flow more easily to Ubuntu users on any cloud.

Welcome to the family, Ubuntu Core!

V is for Vivid

Monday, October 20th, 2014

Release week! Already! I wouldn’t call Trusty ‘vintage’ just yet, but Utopic is poised to leap into the torrent stream. We’ve all managed to land our final touches to *buntu and are excited to bring the next wave of newness to users around the world. Glad to see the unicorn theme went down well, judging from the various desktops I see on G+.

And so it’s time to open the vatic floodgates and invite your thoughts and contributions to our soon-to-be-opened iteration next. Our ventrous quest to put GNU as you love it on phones is bearing fruit, with final touches to the first image in a new era of convergence in computing. From tiny devices to personal computers of all shapes and sizes to the ventose vistas of cloud computing, our goal is to make a platform that is useful, versal and widely used.

Who would have thought – a phone! Each year in Ubuntu brings something new. It is a privilege to celebrate our tenth anniversary milestone with such vernal efforts. New ecosystems are born all the time, and it’s vital that we refresh and renew our thinking and our product in vibrant ways. That we have the chance to do so is testament to the role Linux at large is playing in modern computing, and the breadth of vision in our virtual team.

To our fledgling phone developer community, for all your votive contributions and vocal participation, thank you! Let’s not be vaunty: we have a lot to do yet, but my oh my what we’ve made together feels fantastic. You are the vigorous vanguard, the verecund visionaries and our venerable mates in this adventure. Thank you again.

This verbose tract is a venial vanity, a chance to vector verbal vibes, a map of verdant hills to be climbed in months ahead. Amongst those peaks I expect we’ll find new ways to bring secure, free and fabulous opportunities for both developers and users. This is a time when every electronic thing can be an Internet thing, and that’s a chance for us to bring our platform, with its security and its long term support, to a vast and important field. In a world where almost any device can be smart, and also subverted, our shared efforts to make trusted and trustworthy systems might find fertile ground. So our goal this next cycle is to show the way past a simple Internet of things, to a world of Internet things-you-can-trust.

In my favourite places, the smartest thing around is a particular kind of monkey. Vexatious at times, volant and vogie at others, a vervet gets in anywhere and delights in teasing cats and dogs alike. As the upstart monkey in this business I can think of no better mascot. And so let’s launch our vicenary cycle, our verist varlet, the Vivid Vervet!

This is a series of posts on reasons to choose Ubuntu for your public or private cloud work & play. When you see Ubuntu on a cloud it means that Canonical has a working relationship with that cloud vendor, and the Ubuntu images there come with a set of guarantees:

  1. Those images are up to date and secure.
  2. They have also been optimised on that cloud, both for performance and cost.
  3. The images provide a standard experience for app compatibility.

That turns out to be a lot of work for us to achieve, but it makes your life really easy.

Fresh, secure and tasty images

We update the cloud images across all clouds on a regular basis. Updating the image means that you have more of the latest updates pre-installed so launching a new machine is much faster – fewer updates to install on boot for a fully secured and patched machine.

  1. At least every two weeks, typically, if there are just a few small updates across the board to roll into the freshest image.
  2. Immediately if there is a significant security issue, so starting a fresh image guarantees you to have no known security gotchas.
  3. Sooner than usual if there are a lot of updates which would make launching and updating a machine slow.

Updates might include fixes to the kernel, or any of the packages we install by default in the “core” cloud images. We also make sure that these updated images are used by default in any “quick launch” UI that the cloud provides, so you don’t have to go hunt for the right image identity. And there are automated tools that will tell you the ID for the current image of Ubuntu on your cloud of choice. So you can script “give me a fresh Ubuntu machine” for any cloud, trivially. It’s all very nice.

Optimised for your pocket and your workload

Every cloud behaves differently – both in terms of their architecture, and their economics. When we engage with the cloud operator we figure out how to ensure that Ubuntu is “optimal” on that cloud. Usually that means we figure out things like storage mechanisms (the classic example is S3 but we have to look at each cloud to see what they provide and how to take advantage of it) and ensure that data-heavy operations like system updates draw on those resources in the most cost-efficient manner. This way we try to ensure that using Ubuntu is a guarantee of the most cost-effective base OS experience on any given cloud. In the case of more sophisticated clouds, we are digging in to kernel parameters and drivers to ensure that performance is first class. On Azure there is a LOT of deep engineering between Canonical and Microsoft to ensure that Ubuntu gets the best possible performance out of the Hyper-V substrate, and we are similarly engaged with other cloud operators and solution providers that use highly-specialised hypervisors, such as Joyent and VMware. Even the network can be tweaked for efficiency in a particular cloud environment once we know exactly how that cloud works under the covers. And we do that tweaking in the standard images so EVERYBODY benefits and you can take it for granted – if you’re using Ubuntu, it’s optimal. The results of this work can be pretty astonishing. In the case of one cloud we reduced the Ubuntu startup time by 23x from what their team had done internally; not that they were ineffective, it’s just that we see things through the eyes of a large-scale cloud user and care about things that a single developer might not care about as much. When you’re doing something at scale, even small efficiencies add up to big numbers.

Standard, yummy

Before we had this program in place, every cloud vendor hacked their own Ubuntu images, and they were all slightly different in unpredictable ways. We all have our own favourite way of doing things, so if every cloud has a lead engineer who rigged the default Ubuntu the way they like it, end users have to figure out the differences the hard way, stubbing their toes on them. In some cases they had default user accounts with different behaviour, in others they had different default packages installed. EMACS, Vi, nginx, the usual tweaks. In a couple of cases there were problems with updates or security, and we realised that Ubuntu users would be much better off if we took responsibility for this and ensured that the name is an assurance of standard behaviour and quality across all clouds. So now we have that, and if you see Ubuntu on a public cloud you can be sure it’s done to that standard, and we’re responsible. If it isn’t, please let us know and we’ll fix it for you. That means that you can try out a new cloud really easily – your stuff should work exactly the same way with those images, and differences between the clouds will have been considered and abstracted in the base OS. We’ll have tweaked the network, kernel, storage, update mechanisms and a host of other details so that you don’t have to, we’ll have installed appropriate tools for that specific cloud, and we’ll have lined things up so that to the best of our ability none of those changes will break your apps, or updates. If you haven’t recently tried a new cloud, go ahead and kick the tires on the base Ubuntu images in two or three of them. They should all Just Work TM.   It’s frankly a lot of fun for us to work with the cloud operators – this is the frontline of large-scale systems engineering, and the guys driving architecture at public cloud providers are innovating like crazy but doing so in a highly competitive and operationally demanding environment. Our job in this case is to make sure that end-users don’t have to worry about how the base OS is tuned – it’s already tuned for them. We’re taking that to the next level in many cases by optimising workloads as well, in the form of Juju charms, so you can get whole clusters or scaled-out services that are tuned for each cloud as well. The goal is that you can create a cloud account and have complex scale-out infrastructure up and running in a few minutes. Devops, distilled.

Kudos to all the speakers, panellists, designers and engineers who made ODS Atlanta such a great event last week. And thanks in particular to the team at Canonical that helped pull together our keynote, I had a very large number of compliments that really belong to all of you!

For those that didn’t make it, here are a few highlights.

First, Ubuntu is the leading OpenStack distribution, with 55% of all production are using Ubuntu, nearly 5x the number for RHEL. There is a big squabble at the moment between vendors in the RHEL camp; for the record, Canonical is happy to work with vendors of alternative OpenStack distributions on Ubuntu as long as we have a commercial agreement that enables us to support users. Nonetheless, the standard way to do OpenStack starts with Ubuntu followed by the addition of Canonical’s cloud archive, installing OpenStack using those packages.

Second, vendors are focused on interoperability through Canonical’s OpenStack Interop Lab (OIL). We build OpenStack thousands of ways every month with permutations and combinations of code from many vendors. Bring us a Juju charm of your work, sign up to the OIL program and we’ll tell you which other vendors you need to do more work with if you want to be interoperable with their OpenStack offerings.

Third, Juju and MAAS are growing support for Windows and CentOS, with other operating systems on the horizon too (patches welcome!). Thanks to contributions from CloudBase Solutions, you’ll get amazing orchestration of Windows and Linux apps on any cloud or bare metal. If you have a Windows app that you want charmed up, they are the guys to talk to! We did a live on-stage install of OpenStack with Ubuntu KVM and Windows Hyper-V with the beta code, and expect it to land in production Juju / MAAS in the coming weeks.

 

I’m particularly excited about a new product we’ve announced, which is a flat-fee fully managed on-premise OpenStack solution. Using our architecture and tools, and your hardware, we can give you a best-of-breed OpenStack deployment with SLA for a fixed fee of $15 per server per day. Pretty amazing, and if you are considering OpenStack, definitely an option to evaluate.  Give us a call!

U talking to me?

Wednesday, April 23rd, 2014

This upstirring undertaking Ubuntu is, as my colleague MPT explains, performance art. Not only must it be art, it must also perform, and that on a deadline. So many thanks and much credit to the teams and individuals who made our most recent release, the Trusty Tahr, into the gem of 14.04 LTS. And after the uproarious ululation and post-release respite, it’s time to open the floodgates to umpteen pent-up changes and begin shaping our next show.

The discipline of an LTS constrains our creativity – our users appreciate the results of a focused effort on performance and stability and maintainability, and we appreciate the spring cleaning that comes with a focus on technical debt. But the point of spring cleaning is to make room for fresh ideas and new art, and our next release has to raise the roof in that regard. And what a spectacular time to be unleashing creativity in Ubuntu. We have the foundations of convergence so beautifully demonstrated by our core apps teams – with examples that shine on phone and tablet and PC. And we have equally interesting innovation landed in the foundational LXC 1.0, the fastest, lightest virtual machines on the planet, born and raised on Ubuntu. With an LTS hot off the press, now is the time to refresh the foundations of the next generation of Linux: faster, smaller, better scaled and better maintained. We’re in a unique position to bring useful change to the ubiquitary Ubuntu developer, that hardy and precise pioneer of frontiers new and potent.

That future Ubuntu developer wants to deliver app updates instantly to users everywhere; we can make that possible. They want to deploy distributed brilliance instantly on all the clouds and all the hardware. We’ll make that possible. They want PAAS and SAAS and an Internet of Things that Don’t Bite, let’s make that possible. If free software is to fulfil its true promise it needs to be useful for people putting precious parts into production, and we’ll stand by our commitment that Ubuntu be the most useful platform for free software developers who carry the responsibilities of Dev and Ops.

It’s a good time to shine a light on umbrageous if understandably imminent undulations in the landscape we love – time to bring systemd to the centre of Ubuntu, time to untwist ourselves from Python 2.x and time to walk a little uphill and, thereby, upstream. Time to purge the ugsome and prune the unusable. We’ve all got our ucky code, and now’s a good time to stand united in favour of the useful over the uncolike and the utile over the uncous. It’s not a time to become unhinged or ultrafidian, just a time for careful review and consideration of business as usual.

So bring your upstanding best to the table – or the forum – or the mailing list – and let’s make something amazing. Something unified and upright, something about which we can be universally proud. And since we’re getting that once-every-two-years chance to make fresh starts and dream unconstrained dreams about what the future should look like, we may as well go all out and give it a dreamlike name. Let’s get going on the utopic unicorn. Give it stick. See you at vUDS.

The very best edge of all

Saturday, March 8th, 2014

Check out “loving the bottom edge” for the most important bit of design guidance for your Ubuntu mobile app.

This work has been a LOT of fun. It started when we were trying to find the zen of each edge of the screen, a long time back. We quickly figured out that the bottom edge is by far the most fun, by far the most accessible. You can always get to it easily, it feels great. I suspect that’s why Apple has used the bottom edge for their quick control access on IOS.

progresion

We started in the same place as Apple, thinking that the bottom edge was so nice we wanted it for ourselves, in the system. But as we discussed it, we started to think that the app developer was the one who deserved to do something really distinctive in their app with it instead. It’s always tempting to grab the tastiest bit for oneself, but the mark of civility is restraint in the use of power and this felt like an appropriate time to exercise that restraint.

Importantly you can use it equally well if we split the screen into left and right stages. That made it a really important edge for us because it meant it could be used equally well on the Ubuntu phone, with a single app visible on the screen, and on the Ubuntu tablet, where we have the side stage as a uniquely cool way to put phone apps on tablet screens alongside a bigger, tablet app.

The net result is that you, the developer, and you, the user, have complete creative freedom with that bottom edge. There are of course ways to judge how well you’ve exercised that freedom, and the design guidance tries to leave you all the freedom in the world while still providing a framework for evaluating how good the result will feel to your users. If you want, there are some archetypes and patterns to choose from, but what I’d really like to see is NEW patterns and archetypes coming from diverse designs in the app developer community.

Here’s the key thing – that bottom edge is the one thing you are guaranteed to want to do more innovatively on Ubuntu than on any other mobile platform. So if you are creating a portable app, targeting a few different environments, that’s the thing to take extra time over for your Ubuntu version. That’s the place to brainstorm, try out ideas on your friends, make a few mockups. It’s the place you really express the single most important aspects of your application, because it’s the fastest, grooviest gesture in the book, and it’s all yours on Ubuntu.

Have fun!

OpenStack has emerged as the consensus forum for open source private cloud software. That of course makes it a big and complex community, with complex governance and arguably even more complex politics, but it has survived several rounds of competition and is now settling down as THE place to get diverse vendors to work together on a IAAS that anybody can deploy for themselves. It is a big enough forum with sufficient independent leadership that no one vendor will ever control it (despite some fantastically impressive efforts to do so!). In short, OpenStack is what you want if you are trying to figure out how to build yourself a cloud.

And by quite a large majority, most of the people who have actually chosen to deploy OpenStack in production, have done so on Ubuntu.

At the latest OpenStack summit, an official survey of production OpenStack deployments found 55% of them on Ubuntu, a stark contrast with the 10% of OpenStack deployments on RHEL.

Canonical and Ubuntu play an interesting role in OpenStack. We do not seek to control any particular part of the project, although some of our competitors clearly think that would be useful for them to achieve, we think OpenStack would be greatly diminished in importance if it was perceived to be controlled by a single vendor, and we think there are enough contributors and experts around the table to ensure that the end result cannot actually be controlled by a single party. To a certain extent, the battle for notional control of key aspects of OpenStack just holds the project back; it’s a distraction from the real task at hand, which is to deliver a high quality, high performance open cloud story. So our focus is on supporting the development of OpenStack, supporting the broadest range of vendors who want to offer OpenStack solutions, components and services, and enabling a large ecosystem to accelerate the adoption of OpenStack in their markets.

It’s a point of pride for us that you can get an OpenStack cloud built on Ubuntu from just about every participant in the OpenStack ecosystem – Dell, HP, Mirantis, and many more – we think the healthiest approach is for us to ensure that people have great choices when it comes to their cloud solution.

We were founding members and are platinum sponsors of the OpenStack Foundation. But what’s more important to us, is that most OpenStack development happens on Ubuntu. We take the needs of OpenStack developers very seriously – for 14.04 LTS, our upcoming bi-annual enterprise release, a significant part of our product requirements were driven by the goal of supporting large-scale enterprise deployments of OpenStack with high availability as a baseline. Our partners like HP, who run one of the largest OpenStack public cloud offerings, invest heavily in OpenStack’s CI and test capabilities, ensuring that OpenStack on Ubuntu is of high quality for anybody who chooses the same base platform.

We publish stable, maintained archives of each OpenStack release for the LTS releases of Ubuntu. That means you can ALWAYS deploy the latest version of OpenStack on the current LTS of Ubuntu, and there is a clear upgrade path as new versions of both OpenStack and Ubuntu are released. And the fact that the OpenStack release cadence and the Ubuntu release cadence are perfectly aligned is no accident – it ensures that the OpenStack developers can always deliver their latest code straight to a very large audience of developers and operators. That’s important because of the extraordinary pace of innovation inside OpenStack; there are significant and valuable improvements in each six-month release, so customers, even enterprise customers, find themselves wanting a more aggressive upgrade schedule for OpenStack than is normal for them in platform environments. We support that and have committed to continue doing so, though we do expect the urgency of those upgrades to diminish as OpenStack matures over the next three years.

For commercial support of OpenStack, we are happy for industry to engage either with our partners who can provide local talent combined with an escalation path to Canonical for L3 support of the whole solution, or directly with Canonical if the circumstances warrant it. That means building on Ubuntu opens up a wide range of solution providers who can make the same high commitment to SLAs and upgrades.

For Canonical itself, our focus is on scale and quality. Our direct customers run the very largest production deployments of OpenStack, both private and public, and we enjoy collaborating with their architects to push the limits of the stack as it stands today. That gives us a lot of insight into the approaches being taken by a wide range of architects in telco, finance and media. We ourselves invest very heavily in testing, continuous integration, and interoperability, with the largest OpenStack interop program (OIL) that gives us the ability to speak with confidence about what combinations of vendor offerings will actually work, and in many cases, how they will perform together for different applications.

The fact that the traditional enterprise Linux vendors have now joined OpenStack is a tremendous validation of the role that OpenStack has assumed in industry: THE open cloud forum. But for all the reasons outlined above, most of the actual production deployments of OpenStack are not on traditional, legacy enterprise Linux. This mirrors the public cloud, where even the largest and most mission-critical deployments tend not to be on proprietary Linux offerings; the economics of HA single-node solutions just don’t apply in a scale-out environment. So just as Ubuntu is by far the most widely used platform for public cloud guests, it is also on track to be the enterprise choice for scale-out infrastructure like IAAS, storage, and big data. Even if you have always done Linux a particular way, the transition to scale-out thinking is an opportunity to reset expectations about your base OS; and for the fastest-moving players in telco, media and finance, Ubuntu turns out to be a great way to get more done, more efficiently.

In a series of 12 posts, I’ll make the case for Ubuntu as the platform of choice for public clouds, enterprise clouds and related scale-out initiatives.

Cloud computing is largely being defined on public clouds today. There are a range of initiatives for private cloud computing – some proprietary, some open – but for sheer scale and traction, the game today is all about public cloud services. Azure, AWS, a range of offerings from telco’s and service providers together with innovative takes on the concept from hardware OEMs have been the leading edge of the cloud market for the past five years. We do expect private clouds to flourish around OpenStack, but we expect the gene pool of innovation to stay on the public clouds for some time.

And what do people run on public clouds? By substantial majority, most of that innovation, most of that practical experience and most of the insights being generated are on Ubuntu.

Digital Ocean, the fastest growing new challenger in the US public cloud market, published definitive statistics on the share of operating systems that customers choose on their cloud:

Ubuntu has 67% share of the Digital Ocean public cloud

Ubuntu is the most popular OS on public clouds, by far.

AWS hasn’t spoken publicly on the topic but there are a number of measurements by third parties that provide some insight. For example,  SCALR offer a management service that is used by enterprises looking for more institutional management control of the way their teams use Amazon. One might think that an enterprise management perspective would be skewed away from Ubuntu towards traditional, legacy enterprise Linux, but in fact they find that Ubuntu is more than 70% of all the images they see, three times as popular as CentOS.

There is no true safety in numbers, but there is certainly reassurance. Using a platform that is being used by most other people means that the majority of the content you find about how to get things done efficiently is immediately relevant to you. Version skew – subtle differences in the versions of components that are available by default on your platform of choice – is much less of an issue if the guidebook you are reading assumes you’re on the same platform they used.

There is also the question of talent – finding people to get amazing things done on the cloud is a lot easier if you let them use the platforms they have already grown comfortable with. They can be more productive, and there are many more of them around to hire. Talking to companies about cloud computing today it’s clear their biggest constraint is knowledge acquisition; the time it takes to grow own internal skills or to hire in the necessary skills to get the job done. Building on Ubuntu gives you a much broader talent and knowledge base to work with. Training your own team to use Ubuntu if they are familiar with another Linux is a relatively minor switch compared to the fundamental challenge of adopting a IAAS-based architecture. Switching to Ubuntu is the fastest way to tame that dragon, and the economics are great, too.

That’s why we see many companies that have been doing Linux one way for a decade switching to Ubuntu when they switch to the cloud. Even if what they are doing on the cloud is essentially the same as something they already do on another platform, it’s “easier with Ubuntu on the cloud”, so they switch.

All Star ‘Buntu

Wednesday, February 12th, 2014

As prep for the upcoming 14.04 LTS release of Ubuntu I spent some quality time with each of the main flavours that I track – Kubuntu, Ubuntu GNOME, Xubuntu, and Ubuntu with the default DE, Unity.

They are all in really great shape! Thanks and congratulations to the teams that are racing to deliver Trusty versions of their favourite DE’s. I get the impression that all the major environments are settling down from periods of rapid change and stress, and the timing for an LTS release in 14.04 is perfect. Lucky us :)

The experience reminded me of something people say about Ubuntu all the time – that it’s a place where great people bring diverse but equally important interests together, and a place where people create options for others of which they are proud. You want options? This is the place to get them. You want to collaborate with amazing people? This is the place to find them. I’m very grateful to the people who create those options – for all of them it’s as much a labour of love as a professional concern, and their attention to detail is what makes the whole thing sing.

Of course, my testing was relatively lightweight. I saw tons of major improvements in shared apps like LibreOffice and Firefox and Chromium, and each of the desktop environments feels true to its values, diverse as those are. What I bet those teams would appreciate is all of you taking 14.04 for a spin yourselves. It’s stable enough for any of us who use Linux heavily as an engineering environment, and of course you can use a live boot image off USB if you just want to test drive the future. Cloud images are also available for server testing on all the major clouds.

Having the whole team, and broader community, focus on processes that support faster development at higher quality has really paid off. I’ve upgraded all my systems to Trusty and those I support from afar, too, without any issues. While that’s mere anecdata, the team has far more real data to support a rigorous assessment of 14.04′s quality than any other open platform on the planet, and it’s that rigour that we can all celebrate as the release date approached. There’s still time for tweaks and polish; if you are going to be counting on Trusty, give it a spin and let’s make sure it’s perfect.

Rigor and its results

Friday, December 20th, 2013

Perhaps the biggest change in Ubuntu since 12.04 LTS has been our shift, under Rick’s leadership, towards rigorous, highly automated, test-based QA across all of Ubuntu – server, desktop and mobile.

And what I love about the process is:

  • it’s completely transparent – check out http://ci.ubuntu.com/ and drill down to see the individual test runs
  • it spans every product – server, desktop and mobile, across a wide and growing range of hardware
  • the team takes it absolutely seriously and “stops the line” to fix issues on a regular basis

This is the result of two years hard work by an amazing team – at the package testing and system test level – to help us raise the bar for free software platforms.

This is what the 64-bit x86 server test run looks like today:

Automated testing of the Ubuntu Server platform gives us quick feedback on breaking changes.

Automated testing of the Ubuntu Server platform gives us quick feedback on breaking changes.

 

Over time, thanks to contributions of tests by community, partners and Canonical folks, the number of tests has grown substantially. In the mobile environment we run over 400 smoke tests on every build of every image:

 

Automated test results on a Nexus device.

Automated test results on a Nexus device.

 

In addition to the system image testing you see here, there is a growing portfolio of package-level tests, and processes that test changes both for problems inside the modified package AND for packages that depend on it. So increasingly, we are able to pick up on a problem before it spreads to any developer desktops that are tracking the tip of development.

Testing makes us smarter

It’s significantly more challenging to create test harnesses than code itself. Building this capability exercised our best contributors for the better part of two years; every team has had to figure out how to “get meta” on the parts of Ubuntu they care about. And in the process, we come to a deeper understanding of what it is that users care about, how our platform fits together, and the magic that lives inside the kernel that enables much of this work to happen at scale and in an automated fashion. Grappling with hard problems is like training; the more you do it, the better you understand what’s possible.

Testing helps us go faster

It’s a curious phenomenon that taking time to work on the stuff around the code helps to get the code done faster. But in something as large and complex as a free software platform, change is both your friend and your enemy. Every week thousands of changes flow into Ubuntu from a huge range of sources, and it’s impossible for any on person to anticipate the consequences of every change in advance. Having a rigorous, automated test framework tells us immediately when a change causes a problem somewhere else. Greater confidence that problems will be caught lets us move faster with changes, knowing we can either revert them quickly or stop the line to concentrate everyone’s attention on the issue when it touches a broad swath of the platform.

Testing brings more eyeballs

I run the tip of development – Trusty today – because I can trust the team to spot a problem before it affects me almost every time. That gives me a better view on how Ubuntu is evolving day by day and the ability to ask questions when they are relevant, not right before release. For developers at upstreams or in companies where Trusty is going to be a key platform, the ability to exercise it personally is a huge advantage; you can directly influence the trajectory best if you know where things stand at any given moment. So more rigour translates into more eyeballs which translates into a better result for everybody down the line.

 

We expect to ship our first Ubuntu mobile devices in 2014, and this initiative gives me confidence that we can bring new features and capabilities and improvements to those users fast. And that’s one of the things that makes Ubuntu great.