This work grows out of observations of new and established / sophisticated users making extensive use of the broader set of capabilities in their applications. We noticed that both groups of users spent a lot of time, relatively speaking, navigating the menus of their applications, either to learn about the capabilities of the app, or to take a specific action. We were also conscious of the broader theme in Unity design of leading from user intent. And that set us on a course which lead to today’s first public milestone on what we expect will  be a long, fruitful and exciting journey.

The menu has been a central part of the GUI since Xerox PARC invented ‘em in the 70′s. It’s the M in WIMP and has been there, essentially unchanged, for 30 years.

The original Macintosh desktop, circa 1984, courtesy of Wikipedia

We can do much better!

Say hello to the Head-Up Display, or HUD, which will ultimately replace menus in Unity applications. Here’s what we hope you’ll see in 12.04 when you invoke the HUD from any standard Ubuntu app that supports the global menu:

Snapshot of the HUD in Ubuntu 12.04

The intenterface – it maps your intent to the interface

This is the HUD. It’s a way for you to express your intent and have the application respond appropriately. We think of it as “beyond interface”, it’s the “intenterface”.  This concept of “intent-driven interface” has been a primary theme of our work in the Unity shell, with dash search as a first class experience pioneered in Unity. Now we are bringing the same vision to the application, in a way which is completely compatible with existing applications and menus.

The HUD concept has been the driver for all the work we’ve done in unifying menu systems across Gtk, Qt and other toolkit apps in the past two years. So far, that’s shown up as the global menu. In 12.04, it also gives us the first cut of the HUD.

Menus serve two purposes. They act as a standard way to invoke commands which are too infrequently used to warrant a dedicated piece of UI real-estate, like a toolbar button, and they serve as a map of the app’s functionality, almost like a table of contents that one can scan to get a feel for ‘what the app does’. It’s command invocation that we think can be improved upon, and that’s where we are focusing our design exploration.

As a means of invoking commands, menus have some advantages. They are always in the same place (top of the window or screen). They are organised in a way that’s quite easy to describe over the phone, or in a text book (“click the Edit->Preferences menu”), they are pretty fast to read since they are generally arranged in tight vertical columns. They also have some disadvantages: when they get nested, navigating the tree can become fragile. They require you to read a lot when you probably already know what you want. They are more difficult to use from the keyboard than they should be, since they generally require you to remember something special (hotkeys) or use a very limited subset of the keyboard (arrow navigation). They force developers to make often arbitrary choices about the menu tree (“should Preferences be in Edit or in Tools or in Options?”), and then they force users to make equally arbitrary effort to memorise and navigate that tree.

The HUD solves many of these issues, by connecting users directly to what they want. Check out the video, based on a current prototype. It’s a “vocabulary UI”, or VUI, and closer to the way users think. “I told the application to…” is common user paraphrasing for “I clicked the menu to…”. The tree is no longer important, what’s important is the efficiency of the match between what the user says, and the commands we offer up for invocation.

In 12.04 LTS, the HUD is a smart look-ahead search through the app and system (indicator) menus. The image is showing Inkscape, but of course it works everywhere the global menu works. No app modifications are needed to get this level of experience. And you don’t have to adopt the HUD immediately, it’s there if you want it, supplementing the existing menu mechanism.

It’s smart, because it can do things like fuzzy matching, and it can learn what you usually do so it can prioritise the things you use often. It covers the focused app (because that’s where you probably want to act) as well as system functionality; you can change IM state, or go offline in Skype, all through the HUD, without changing focus, because those apps all talk to the indicator system. When you’ve been using it for a little while it seems like it’s reading your mind, in a good way.

We’ll resurrect the  (boring) old ways of displaying the menu in 12.04, in the app and in the panel. In the past few releases of Ubuntu, we’ve actively diminished the visual presence of menus in anticipation of this landing. That proved controversial. In our defence, in user testing, every user finds the menu in the panel, every time, and it’s obviously a cleaner presentation of the interface. But hiding the menu before we had the replacement was overly aggressive. If the HUD lands in 12.04 LTS, we hope you’ll find yourself using the menu less and less, and be glad to have it hidden when you are not using it. You’ll definitely have that option, alongside more traditional menu styles.

Voice is the natural next step

Searching is fast and familiar, especially once we integrate voice recognition, gesture and touch. We want to make it easy to talk to any application, and for any application to respond to your voice. The full integration of voice into applications will take some time. We can start by mapping voice onto the existing menu structures of your apps. And it will only get better from there.

But even without voice input, the HUD is faster than mousing through a menu, and easier to use than hotkeys since you just have to know what you want, not remember a specific key combination. We can search through everything we know about the menu, including descriptive help text, so pretty soon you will be able to find a menu entry using only vaguely related text (imagine finding an entry called Preferences when you search for “settings”).

There is lots to discover, refine and implement. I have a feeling this will be a lot of fun in the next two years

Even better for the power user

The results so far are rather interesting: power users say things like “every GUI app now feels as powerful as VIM”. EMACS users just grunt and… nevermind . Another comment was “it works so well that the rare occasions when it can’t read my mind are annoying!”. We’re doing a lot of user testing on heavy multitaskers, developers and all-day-at-the-workstation personas for Unity in 12.04, polishing off loose ends in the experience that frustrated some in this audience in 11.04-10. If that describes you, the results should be delightful. And the HUD should be particularly empowering.

Even casual users find typing faster than mousing. So while there are modes of interaction where it’s nice to sit back and drive around with the mouse, we observe people staying more engaged and more focused on their task when they can keep their hands on the keyboard all the time. Hotkeys are a sort of mental gymnastics, the HUD is a continuation of mental flow.

Ahead of the competition

There are other teams interested in a similar problem space. Perhaps the best-known new alternative to the traditional menu is Microsoft’s Ribbon. Introduced first as part of a series of changes called Fluent UX in Office, the ribbon is now making its way to a wider set of Windows components and applications. It looks like this:

You can read about the ribbon from a supporter (like any UX change, it has its supporters and detractors ) and if you’ve used it yourself, you will have your own opinion about it. The ribbon is highly visual, making options and commands very visible. It is however also a hog of space (I’m told it can be minimised). Our goal in much of the Unity design has been to return screen real estate to the content with which the user is working; the HUD meets that goal by appearing only when invoked.

Instead of cluttering up the interface ALL the time, let’s clear out the chrome, and show users just what they want, when they want it.

Time will tell whether users prefer the ribbon, or the HUD, but we think it’s exciting enough to pursue and invest in, both in R&D and in supporting developers who want to take advantage of it.

Other relevant efforts include Enso and Ubiquity from the original Humanized team (hi Aza &co), then at Mozilla.

Our thinking is inspired by many works of science, art and entertainment; from Minority Report to Modern Warfare and Jef Raskin’s Humane Interface. We hope others will join us and accelerate the shift from pointy-clicky interfaces to natural and efficient ones.

Roadmap for the HUD

There’s still a lot of design and code still to do. For a start, we haven’t addressed the secondary aspect of the menu, as a visible map of the functionality in an app. That discoverability is of course entirely absent from the HUD; the old menu is still there for now, but we’d like to replace it altogether not just supplement it. And all the other patterns of interaction we expect in the HUD remain to be explored. Regardless, there is a great team working on this, including folk who understand Gtk and Qt such as Ted Gould, Ryan Lortie, Gord Allott and Aurelien Gateau, as well as designers Xi Zhu, Otto Greenslade, Oren Horev and John Lea. Thanks to all of them for getting this initial work to the point where we are confident it’s worthwhile for others to invest time in.

We’ll make sure it’s easy for developers working in any toolkit to take advantage of this and give their users a better experience. And we’ll promote the apps which do it best – it makes apps easier to use, it saves time and screen real-estate for users, and it creates a better impression of the free software platform when it’s done well.

From a code quality and testing perspective, even though we consider this first cut a prototype-grown-up, folk will be glad to see this:

Overall coverage rate:
   lines......: 87.1% (948 of 1089 lines)
   functions..: 97.7% (84 of 86 functions)
   branches...: 63.0% (407 of 646 branches)

Landing in 12.04  LTS is gated on more widespread testing.  You can of course try this out from a PPA or branch the code in Launchpad (you will need these two branches). Or dig deeper with blogs on the topic from Ted Gould, Olli Ries and Gord Allott. Welcome to 2012 everybody!

The three legs of our engineering practice are cadence, quality and design. For those teams which maintain their own codebases (unity, juju, bzr, lp and many more) the quality position is a easier to define, because we can make test coverage and continuous tested integration standard practices. It’s more challenging for the platform team and Ubuntu community, who integrate thousands of packages from all sorts of places into one product: Ubuntu. We’ve traditionally focused on items like security, where participation in a global security process helps us ensure Ubuntu gets world-class security support and has established a world-leading track record of security patches and proactive security.

Nevertheless, the last year has seen some amazing leaps forward in our ability to manage quality across the entire platform. In large part, that’s thanks to the leadership of Rick Spencer and Pete Graner, who made smoke-testing and benchmarking a rigorous part of the process for every change to the platform, and lead the work to make that commitment sane in practice across all the hundreds of people, inside and outside Canonical, who needed to be on board with it. And it’s thanks to tools like Jenkins and LAVA which automate the testing and reporting across a vast array of problem spaces, architectures and packages.

So we have a daily weather report for Precise, which gives you a feeling for where things stand right now, as well as tighter integration of the test suites being run by Canonical upstreams on code destined for Precise with the test harness used by the platform team integrating that work into the distribution.  I’ll take the liberty of repeating some of Rick’s core points here:

For upstreams, it boils down to “treat your trunk as sacred”. Practically, it requires:

• There is a trunk of code bound for Ubuntu.
• This trunk always builds automatically.
• This trunk has tests that are always passing automatically.
• All branches are properly reviewed for having both good tests and good implementation before merged into trunk.
• Any branch that breaks trunk by causing automated tests to fail or causes trunk to stop building, are immediately reverted.

For Ubuntu Engineering, the responsibilities include:

• Every maintainer in Ubuntu must have a test plan for upstream trunks that are run before uploading to the development release.
• Tests in the test plan that are automated can be run with the help of the QA team.
• Tests in the test plan that are manual can be run with the help of Nicholas, the new community QA Lead
• Refrain from uploading a trunk into Ubuntu if there are serious bugs found in testing that will slow down people using the development release for testing and development.
• Revert uploads that break Ubuntu, as there is no point in having the latest of a trunk in Ubuntu if it’s broken and just slowing everyone down.
• Add tests to upstream projects for the Ubuntu test plan if serious bugs do get through that cause a revert.

This isn’t limited to Canonical team members; didrocks and the French Musketeers have built a Unity SRU testing process which should let us crowdsource perspectives on the quality improvements or regressions of changes in Unity. Ara’s ongoing work around component and system testing is giving us a very useful database of known issues at the hardware level. Work on Checkbox and related tools continues to ensure that people can contribute data and help prioritise the issues which will have the widest benefit for millions of community adopters.

Upstream quality

Where upstreams have test suites, we’re integrating those into the automated QA framework. In an ideal world, whenever a package is changed, we’d have an upstream test suite to run for that package AND for every package which depends on it. That way, we’d catch breakage in the package itself, but more importantly, we’d catch consequential damage elsewhere, which is much harder for upstreams to catch themselves.

We’re already running that program, and as upstreams start to take testing more seriously, coverage across the whole platform will improve significantly. It’s been Canonical practice to have test suites for several years, and it’s very encouraging to see other upstreams adopting TDD and at the least rigorous unit and functional testing, one at a time. Open source projects love to talk about quality – but it’s important to back that with measurable practices and data. As an example in a complex case, we run the LTP against every kernel SRU, in addition to our own kernel and hardware cert tests.

In future, it should be possible to link this to the existing daily builds of tip (we have over 500 upstreams running daily builds on Launchpad, which is fantastic). THAT would give upstreams the ability to know when commits to their tip break tests in dependent packages. It would suck a large amount of compute, but it would provide a fantastic early warning system of collisions between independent changes in diverse but related projects.

There’s a lot more we will do, by integrating Apport for crash data collection, and routing those reports through a big data sieve we should be able to identify the issues which are having the biggest impact on the most users. But that’s a blog for another day. For now, well done, team Ubuntu!

Even the sheep are running scared.

It’s the last day of Movember.

Inspired by your comments, and frankly, more inspired by your donations to Movember in the name of this metrosexual disaster, I’ll take on the challenge of wearing this mating handicap for an extra week for every additional USD 1,000 donated today. About USD 2,100 raised so far, and I’ll match the total when the event wraps up.

Go on, make me do it, at  mobro.co/smonaut.

Update

Your generosity was incredible – thank you everybody. All that hard currency meant I had to wear the mo for an extra week (Claire was a little worried that you’d have her looking around for somebody else to kiss at New Years eve!) and the folks at ZA.Movember seem delighted. I matched all donations made by the end of the month, and will match one other challenge that came in shortly after. Would never have thought to top the charts; kudos is all yours.

Earlier this week the guys in #ubuntu-tv (on Freenode) generated an Etherpad with their thoughts and then arranged a meeting to discuss priorities.  Alan Bell produced some designs:  http://people.ubuntu.com/~alanbell/unitytelly/

The mailing list has seen some decent traffic as well, with people talking mostly about what the future of the Connected TV might be and features they’d like to see.

Essential
- 10′ interface- Watching Media (DVR, Live, Network(TV Guide is part of DVR/other services))- Control via remote controlHigh priority- Plugin support- Cloud and/or server storage (for home grown media)

- Playback of physical media (USB cd/dvd/bluray drive)

- Installable image

- Easy configuration of new devices (eg. installing same plugins, mounting same network shares)

- Ubuntu One Accounts

- Push media to/from other Ubuntu devices / Media syncing capabilities (Pause on one device, resume from same spot on another device)

- Collaborate with other Ubuntu devices (context: https://lists.launchpad.net/ubuntu-phone/msg00006.html )

- Control from portable devices (phones/tablets/web interface/PC) (collaboration with Ubuntu Phone/Tablet?)

Medium priority

- Sharing media with friends (social network connectivity)

- Purchasing media through online stores (Ubuntu one/Amazon/Netflix)

 

A smooth-running community is not a miracle, it’s the result of dedication, energy, organisation and empathy. Daniel has all of those gifts and qualities in abundance, and the impact he’s had on the governance of Ubuntu is profound.

We pride ourselves on being a meritocracy and a do-ocracy. Those who have the capacity and the will and the commitment and the values and the energy to lead, and who are recognised as leaders by their peers in the project, get the opportunity to take on responsibilities in one of the many parts of the project, or overall in the CC or TB. But it takes insight and effort to recognise those with that potential, and it takes leadership to encourage them to step forward to shine, and it takes organisation to coordinate diverse efforts so as to ensure a harmonious result. For all of the work and play he brings to that, I thank him.

In a galaxy of many stars, it’s perhaps impolite to single out one in particular. So please counterbalance that impertinence by picking your own star to thank; in our community, the depth and strength don’t come from the headline acts so much as the diversity of contributions from an enormous number of people. Thank you all.

"It will only take me a minute to fetch the lube."

You can contribute to the cause (male health, grin and bare it etc) via the excellent Movember and if you do it as a result of my embarrassment then I’ll match it up to a somewhat unreasonable figure. I think they take plastic, and for the globally challenged $1 is about R8, so your donation will look much more impressive than it is. Just like my tache. Think big

If however you would like to substitute wit for cash, and this is a rare occasion when that is a possibility, add your caption for this 70′s throwback to the comments below.

In the past year we’ve had to refine our thinking about a number of issues. The question about whether contributions outside the project, with a specific emphasis on Ubuntu, should be considered on a par with contributions directly to the project was resolved inclusively. So we are delighted to welcome members who do work in Debian to ensure that Ubuntu and Debian stay on rails together, and we are delighted to welcome members who contribute to projects elsewhere with the aim of improving the experience for Ubuntu users.

It remains true that there is no aspect of Ubuntu that a community participant cannot influence. At UDS this week it was impossible to tell, across hundreds of sessions, which voices were from Canonical, or Dell, or ARM, or Linaro, or from folk who have no corporate affiliation but have a passion for getting things done, and getting them in front of millions of users, and getting them right. From the artwork we ship, to the way we evaluate contributions, and the versions of software we include by default, to the toolchain and kernel and infrastructure that makes it happen, the degree of diverse participation is something we can be proud of. So thank you to everyone, whether participating for personal or corporate interests, for your engagement with Ubuntu.

It was a pleasure to meet the (mostly) new Community Council, and to have a session in person. And it was wonderful to see the vibrancy of the Community Leadership Track at UDS, and the participation in those discussions by leaders of other communities like GNOME and Debian. We have a lot to learn, and a lot to teach.

As a community, we will flourish if two things remain true:

• We continue to attract and empower motivated and energetic participants
• We defend our core values and the tone of our discussions

Unity, the desktop interface in today’s Ubuntu 11.10, was designed with this specific vision in mind. While the interface for each form factor is shaped appropriately, Unity’s core elements are arranged in exactly the way we need to create coherence across all of those devices. This was the origin of the name Unity – a single core interface framework, that scales across all screens, and supports all toolkits.

Canonical and the Ubuntu community have established Ubuntu’s place in desktop, server and cloud deployments. We have also invested in the design and engineering of Unity, motivated by the belief that desktop interfaces would merge with mobile, touch interfaces into a seamless personal computing platform in the future. Today we are inviting the whole Ubuntu community – both commercial and personal – to shape that possibility and design that future; a world where Ubuntu runs on mobile phones, tablets, televisions and traditional PC’s, creating a world where content is instantly available on all devices, in a form that is delightful to use.

A constantly changing world

The way we access the Internet, connect to our friends, listen to music, watch films and go about our daily lives is rapidly evolving. We now use a diverse set of devices with an array of operating systems, which have a range of connectivity. Few people are exclusively loyal to a single technology provider.

Consider this quote from Paul Maritz of VMWare:

“Three years ago over 95 percent of the devices connected to the Internet were personal computers. Three years from now that number will probably be less than 20 percent. More than 80 percent of the devices connected to the Internet will not be Windows-based personal computers.” Paul Maritz, 29 August 2011 VM World Keynote.

Make no mistake – just as the world is changing for manufacturers so is it changing for Linux distributions. Today, 70% of people in Egypt access the Internet solely via the phone. Even in the US that figure is a startling 25%.

Ubuntu is well positioned

Ubuntu will thrive in this new reality.

Our established collaboration with the silicon vendors that are driving this converging market are critical. Intel, ARM and AMD will make the chip-sets that will power this future and Ubuntu works with all of them on all technologies.

Our engagement with the PC market will help bring the results of this work to a huge audience – partnerships with the likes of Dell, HP, Asus, Lenovo, Acer, IBM, Vodafone and more are a gateway to users who want continuous, connected, cross-device computing.

We are determined to bring more free software to more people around the world, and building that future hand in hand with device manufacturers is the best way to do it. There is no winner in place yet. This opportunity remains wide open, but only to products that deliver excellent experiences for users, across a full range of device categories.

The investment we have already made in the interface accommodates the touch scenarios required in some form factors and, with a little love and attention, will work equally well in mouse, keyboard or stylus-driven environments. Ubuntu will not be restricted to small screen or large screen environments but encompasses both and all the form factors in between. We will see our work on the Ubuntu platform land in a variety of formats current and yet to be invented. It is without doubt the most exciting phase in the history of Ubuntu.

Ubuntu One and the software centre

Ubuntu’s personal cloud and app centre services are appropriate for all these environments. They deliver the required storage, syncing and sharing capabilities that are not just a convenience but a requirement as we move to a universe where content is increasingly shared but the devices that access them become more diverse. Ubuntu One’s support for other OSes show the ability of Ubuntu to play nice with others, recognising that the divergence is strength.  It allows users to choose the devices they prefer but still delivering the benefits of Ubuntu-centred strategy.

The next steps

We are describing this at UDS to energize the entire Ubuntu ecosystem around this challenge. Canonical will provide the heavy lifting needed to put us in the ball park, but there are opportunities for participation, contribution and engagement by all elements of the broader Ubuntu community, both corporate and individual.

Our developers, our partners’ developers and the broader open source development community share this opportunity. There is a great deal to discuss, and an array of strands we need to pull together at UDS. But the direction is clear and the prize is great – to bring more free software to more people in more delightful ways than ever before.

Orchestra is one of the most exciting new capabilities in 11.10. It provides automated installation of Ubuntu across sets of machines. Typically, it’s used by people bringing up a cluster or farm of servers, but the way it’s designed makes it very easy to bring up rich services, where there may be a variety of different kinds of nodes that all need to be installed together.

There’s a long history of tools that have been popular at one time or another for automated installation. FAI is the one I knew best before Orchestra came along and I was interested in the rationale for a new tool, and the ways in which it would enhance the experience of people building clusters, clouds and other services at scale. Dustin provided some of that in his introduction to Orchestra, but the short answer is that Orchestra is savvy to the service orchestration model of Juju, which means that the intelligence distilled in Juju charms can easily be harnessed in any deployment that uses Orchestra on bare metal.

What’s particularly cool about THAT is that it unifies the new world of private cloud with the old approach of Linux deployment in a cluster. So, for example, Orchestra can be used to deploy Hadoop across 3,000 servers on bare metal, and that same Juju charm can also deploy Hadoop on AWS or an OpenStack cloud. And soon it should be possible to deploy Hadoop across n physical machines with automatic bursting to your private or favourite public cloud, all automatically built in. Brilliant. Kudos to the conductor

Private cloud is very exciting – and with Ubuntu 11.10 it’s really easy to set up a small cloud to kick the tires, then scale that up as needed for production. But there are still lots of reasons why you might want to deploy a service onto bare metal, and Orchestra is a neat way to do that while at the same time preparing for a cloud-oriented future, because the work done to codify policies or practices in the physical environment should be useful immediately in the cloud, too.

For 12.04 LTS, where supporting larger-scale deployments will be a key goal, Orchestra becomes a tool that every Ubuntu administrator will find useful. I bet it will be the focus of a lot of discussion at UDS next week, and a lot of work in this cycle.

Advocates of big iron talk about reliability. But with public clouds, we’re learning how to build services that achieve very high levels of reliability despite having low individual node reliability. It doesn’t matter if a single node in the cloud fails – cloud-style architectures route around that damage and keep the overall service available. Just as we dial storage reliability up or down by designing RAID arrays for the right balance of performance and resilience to failure, you can dial service reliability up or down in the cloud by allowing for redundancy. That comes at a price, of course, but the price of an extra 9 is substantially lower when you tackle it cloud-style than when you try and achieve it on a single piece of hardware.

The other big strength of big iron was always throughput. Customers will pay for it, so mainframe vendors were always happy to oblige them. But again, it’s hard to beat the throughput of a Hadoop cluster, and even harder to scale the throughput of a mainframe as cost-effectively as one can scale a private cloud infrastructure underneath Hadoop.

I’m not suggesting insurance companies will throw away their mainframes. They’re working, they’re paid for, so they’ll stick around. But the rapid adoption of cloud-based architectures is going to make it very difficult to consolidate future IT onto mainframes (something that happened in every prior generation) and is also going to reduce the incentive for doing so in the first place. After 20 years of imminent irrelevance, there’s finally a real reason to think their time is up.