Mark Shuttleworth | here be dragons

It makes perfect sense to integrate Amazon search results in the Dash, because the Home Lens of the Dash should let you find *anything* anywhere. Over time, we’ll make the Dash smarter and smarter, so you can just ask for whatever you want, and it will Just Work.

The Home Lens of the Dash is a “give me X” experience. You hit the Super key, and say what you want, and we do our best to figure out what you mean, and give you that. Of course, you can narrow the scope of that search if you want. For example, you can hit Super-A and just search applications. But if you throw your query out to the Dash, we need to be a smart as possible about where we go looking for answers for you.

In 12.10 we’ll take the first step of looking both online and locally for possible results. The Home lens will show you local things like apps and music, as it always has, as well as results from Amazon. Note – these are not ads, they are results to your search. We don’t promote any product or service speculatively, these are not banners or spyware. These are results from underlying scopes, surfaced to the Home lens, because you didn’t narrow the scope to a specific, well scope.

What we have in 12.10 isn’t the full experience, so those who leap to judgement are at maximum risk of having to eat their words later. Chill out. If the first cut doesn’t work for you, remove it, or just search the specific scope you want (there are hotkeys for all the local scopes).

Here’s a quick Q&A on the main FUD-points.

Why are you putting ads in Ubuntu?

We’re not putting ads in Ubuntu. We’re integrating online scope results into the home lens of the dash. This is to enable you to hit “Super” and then ask for anything you like, and over time, with all of the fantastic search scopes that people are creating, we should be able to give you the right answer.

These are not ads because they are not paid placement, they are straightforward Amazon search results for your search. So the Dash becomes a super-search of any number of different kinds of data. Right now, it’s not dynamically choosing what to search, it’s just searching local scopes and Amazon, but it will get smarter over time.

I don’t wan’t to search Amazon for the Terminal

Use Super-A. You can tell Unity exactly what you want to search. And in future you’ll be able to do that from the home lens, too, more easily than the current Lens Bar at the bottom of the Dash.

I want to control what is searched on the Home Lens

So do I! Designs and patches welcome in the usual places. I’m pretty sure by 14.04 LTS we’ll have the kinks unkinked. Till then, come along for the ride, or stick with 12.04 LTS. We can’t wait till it’s perfect before landing everything, because the only way to learn what’s not perfect is to have other people – real people – use it.

I can’t believe you just changed Ubuntu. I liked it the way it was.

Looks like those six months are nearly up again 😉

This is just a moneymaking scheme.

We picked Amazon as a first place to start because most of our users are also regular users of Amazon, and it pays us to make your Amazon journey  get off to a faster start. Typing Super “queen marking cage”  Just Worked for me this morning. I am now looking forward to my game of Ultimate Where’s Waldo hunting down the queens in my bee colonies, Ubuntu will benefit from the fact that I chose to search Amazon that way, Amazon benefits from being more accessible to a very discerning, time-conscious and hotkey-friendly audience.

But there are many more kinds of things you can search through with Unity scopes. Most of them won’t pay Ubuntu a cent, but we’ll still integrate them into the coolest just-ask-and-you’ll-receive experience. I want us to do this because I think we can make the desktop better.

Why are you telling Amazon what I am searching for?

We are not telling Amazon what you are searching for. Your anonymity is preserved because we handle the query on your behalf. Don’t trust us? Erm, we have root. You do trust us with your data already. You trust us not to screw up on your machine with every update. You trust Debian, and you trust a large swathe of the open source community. And most importantly, you trust us to address it when, being human, we err.

The query should be encrypted!

It will be in the release version.

I want to control this in the settings!

Yes, we agree, and designs and work are under way to make that possible. That should land in 12.10 too, or as an update, or in 13.04.

In summary – please don’t feed the trolls. We’re interested in feedback in what sorts of things would be useful to search straight from the home lens, and how to improve the search results, as well as provide better control of the process to you.

Here is the key question, as I see it:

Can Canonical and the Ubuntu community handle the responsibility associated with this sort of service?

Well, if we want to run a modern platform, that updates automatically and provides users with the full benefit of living in a connected world, then we have to be able to do that. If we can’t we won’t be relevant. So we should talk about the appropriate kinds of privacy policy, appropriate encryption, appropriate settings and preferences, to make this all world class.

In the open source community, we celebrate having pieces that “do one thing well”, with lots of orthogonal tools compounding to give great flexibility. But that same philosophy leads to shortcomings on the GUI / UX front, where we want all the pieces to be aware of each other in a deeper way.

For example, we consciously place the notifications in the top right of the screen, avoiding space that is particularly precious (like new tab titles, and search boxes). But the indicators are also in the top right, and they make menus, which drop down into the same space a notification might occupy.

Since we know that notifications are queued, no notification is guaranteed to be displayed instantly, so a smarter notification experience would stay out of the way while you were using indicator menus, or get out of the way when you invoke them. The design story of focusayatana, where we balance the need for focus with the need for awareness, would suggest that we should suppress awareness-oriented things in favour of focus things. So when you’re interacting with an indicator menu, we shouldn’t pop up the notification. Since the notification system, and the indicator menu system, are separate parts, the UNIX philosophy sells us short in designing a smart, smooth experience because it says they should each do their thing individually.

Going further, it’s silly that the sound menu next/previous track buttons pop up a notification, because the same menu shows the new track immediately anyway. So the notification, which is purely for background awareness, is distracting from your focus, which is conveying exactly the same information!

But it’s not just the system menus. Apps can play in that space too, and we could be better about shaping the relationship between them. For example, if I’m moving the mouse around in the area of a notification, we should be willing to defer it a few seconds to stay out of the focus. When I stop moving the mouse, or typing in a window in that region, then it’s OK to pop up the notification.

It’s only by looking at the whole, that we can design great experiences. And only by building a community of both system and application developers that care about the whole, that we can make those designs real. So, thank you to all of you who approach things this way, we’ve made huge progress, and hopefully there are some ideas here for low-hanging improvements too 🙂

The desktop remains central to our everyday work and play, despite all the excitement around tablets, TV’s and phones. So it’s exciting for us to innovate in the desktop too, especially when we find ways to enhance the experience of both heavy “power” users and casual users at the same time. The desktop will be with us for a long time, and for those of us who spend hours every day using a wide diversity of applications, here is some very good news: 12.04 LTS will include the first step in a major new approach to application interfaces.

This work grows out of observations of new and established / sophisticated users making extensive use of the broader set of capabilities in their applications. We noticed that both groups of users spent a lot of time, relatively speaking, navigating the menus of their applications, either to learn about the capabilities of the app, or to take a specific action. We were also conscious of the broader theme in Unity design of leading from user intent. And that set us on a course which led to today’s first public milestone on what we expect will  be a long, fruitful and exciting journey.

The menu has been a central part of the GUI since Xerox PARC invented ’em in the 70’s. It’s the M in WIMP and has been there, essentially unchanged, for 30 years.

The original Macintosh desktop, circa 1984, courtesy of Wikipedia

We can do much better!

Say hello to the Head-Up Display, or HUD, which will ultimately replace menus in Unity applications. Here’s what we hope you’ll see in 12.04 when you invoke the HUD from any standard Ubuntu app that supports the global menu:

Snapshot of the HUD in Ubuntu 12.04

The intenterface – it maps your intent to the interface

This is the HUD. It’s a way for you to express your intent and have the application respond appropriately. We think of it as “beyond interface”, it’s the “intenterface”.  This concept of “intent-driven interface” has been a primary theme of our work in the Unity shell, with dash search as a first class experience pioneered in Unity. Now we are bringing the same vision to the application, in a way which is completely compatible with existing applications and menus.

The HUD concept has been the driver for all the work we’ve done in unifying menu systems across Gtk, Qt and other toolkit apps in the past two years. So far, that’s shown up as the global menu. In 12.04, it also gives us the first cut of the HUD.

Menus serve two purposes. They act as a standard way to invoke commands which are too infrequently used to warrant a dedicated piece of UI real-estate, like a toolbar button, and they serve as a map of the app’s functionality, almost like a table of contents that one can scan to get a feel for ‘what the app does’. It’s command invocation that we think can be improved upon, and that’s where we are focusing our design exploration.

As a means of invoking commands, menus have some advantages. They are always in the same place (top of the window or screen). They are organised in a way that’s quite easy to describe over the phone, or in a text book (“click the Edit->Preferences menu”), they are pretty fast to read since they are generally arranged in tight vertical columns. They also have some disadvantages: when they get nested, navigating the tree can become fragile. They require you to read a lot when you probably already know what you want. They are more difficult to use from the keyboard than they should be, since they generally require you to remember something special (hotkeys) or use a very limited subset of the keyboard (arrow navigation). They force developers to make often arbitrary choices about the menu tree (“should Preferences be in Edit or in Tools or in Options?”), and then they force users to make equally arbitrary effort to memorise and navigate that tree.

The HUD solves many of these issues, by connecting users directly to what they want. Check out the video, based on a current prototype. It’s a “vocabulary UI”, or VUI, and closer to the way users think. “I told the application to…” is common user paraphrasing for “I clicked the menu to…”. The tree is no longer important, what’s important is the efficiency of the match between what the user says, and the commands we offer up for invocation.

In 12.04 LTS, the HUD is a smart look-ahead search through the app and system (indicator) menus. The image is showing Inkscape, but of course it works everywhere the global menu works. No app modifications are needed to get this level of experience. And you don’t have to adopt the HUD immediately, it’s there if you want it, supplementing the existing menu mechanism.

It’s smart, because it can do things like fuzzy matching, and it can learn what you usually do so it can prioritise the things you use often. It covers the focused app (because that’s where you probably want to act) as well as system functionality; you can change IM state, or go offline in Skype, all through the HUD, without changing focus, because those apps all talk to the indicator system. When you’ve been using it for a little while it seems like it’s reading your mind, in a good way.

We’ll resurrect the  (boring) old ways of displaying the menu in 12.04, in the app and in the panel. In the past few releases of Ubuntu, we’ve actively diminished the visual presence of menus in anticipation of this landing. That proved controversial. In our defence, in user testing, every user finds the menu in the panel, every time, and it’s obviously a cleaner presentation of the interface. But hiding the menu before we had the replacement was overly aggressive. If the HUD lands in 12.04 LTS, we hope you’ll find yourself using the menu less and less, and be glad to have it hidden when you are not using it. You’ll definitely have that option, alongside more traditional menu styles.

Voice is the natural next step

Searching is fast and familiar, especially once we integrate voice recognition, gesture and touch. We want to make it easy to talk to any application, and for any application to respond to your voice. The full integration of voice into applications will take some time. We can start by mapping voice onto the existing menu structures of your apps. And it will only get better from there.

But even without voice input, the HUD is faster than mousing through a menu, and easier to use than hotkeys since you just have to know what you want, not remember a specific key combination. We can search through everything we know about the menu, including descriptive help text, so pretty soon you will be able to find a menu entry using only vaguely related text (imagine finding an entry called Preferences when you search for “settings”).

There is lots to discover, refine and implement. I have a feeling this will be a lot of fun in the next two years 🙂

Even better for the power user

The results so far are rather interesting: power users say things like “every GUI app now feels as powerful as VIM”. EMACS users just grunt and… nevermind ;-). Another comment was “it works so well that the rare occasions when it can’t read my mind are annoying!”. We’re doing a lot of user testing on heavy multitaskers, developers and all-day-at-the-workstation personas for Unity in 12.04, polishing off loose ends in the experience that frustrated some in this audience in 11.04-10. If that describes you, the results should be delightful. And the HUD should be particularly empowering.

Even casual users find typing faster than mousing. So while there are modes of interaction where it’s nice to sit back and drive around with the mouse, we observe people staying more engaged and more focused on their task when they can keep their hands on the keyboard all the time. Hotkeys are a sort of mental gymnastics, the HUD is a continuation of mental flow.

Ahead of the competition

There are other teams interested in a similar problem space. Perhaps the best-known new alternative to the traditional menu is Microsoft’s Ribbon. Introduced first as part of a series of changes called Fluent UX in Office, the ribbon is now making its way to a wider set of Windows components and applications. It looks like this:

You can read about the ribbon from a supporter (like any UX change, it has its supporters and detractors ;-)) and if you’ve used it yourself, you will have your own opinion about it. The ribbon is highly visual, making options and commands very visible. It is however also a hog of space (I’m told it can be minimised). Our goal in much of the Unity design has been to return screen real estate to the content with which the user is working; the HUD meets that goal by appearing only when invoked.

Instead of cluttering up the interface ALL the time, let’s clear out the chrome, and show users just what they want, when they want it.

Time will tell whether users prefer the ribbon, or the HUD, but we think it’s exciting enough to pursue and invest in, both in R&D and in supporting developers who want to take advantage of it.

Other relevant efforts include Enso and Ubiquity from the original Humanized team (hi Aza &co), then at Mozilla.

Our thinking is inspired by many works of science, art and entertainment; from Minority Report to Modern Warfare and Jef Raskin’s Humane Interface. We hope others will join us and accelerate the shift from pointy-clicky interfaces to natural and efficient ones.

Roadmap for the HUD

There’s still a lot of design and code still to do. For a start, we haven’t addressed the secondary aspect of the menu, as a visible map of the functionality in an app. That discoverability is of course entirely absent from the HUD; the old menu is still there for now, but we’d like to replace it altogether not just supplement it. And all the other patterns of interaction we expect in the HUD remain to be explored. Regardless, there is a great team working on this, including folk who understand Gtk and Qt such as Ted Gould, Ryan Lortie, Gord Allott and Aurelien Gateau, as well as designers Xi Zhu, Otto Greenslade, Oren Horev and John Lea. Thanks to all of them for getting this initial work to the point where we are confident it’s worthwhile for others to invest time in.

We’ll make sure it’s easy for developers working in any toolkit to take advantage of this and give their users a better experience. And we’ll promote the apps which do it best – it makes apps easier to use, it saves time and screen real-estate for users, and it creates a better impression of the free software platform when it’s done well.

From a code quality and testing perspective, even though we consider this first cut a prototype-grown-up, folk will be glad to see this:

Overall coverage rate:
   lines......: 87.1% (948 of 1089 lines)
   functions..: 97.7% (84 of 86 functions)
   branches...: 63.0% (407 of 646 branches)

Landing in 12.04  LTS is gated on more widespread testing.  You can of course try this out from a PPA or branch the code in Launchpad (you will need these two branches). Or dig deeper with blogs on the topic from Ted Gould, Olli Ries and Gord Allott. Welcome to 2012 everybody!

Earlier this week the guys in #ubuntu-tv (on Freenode) generated an Etherpad with their thoughts and then arranged a meeting to discuss priorities.  Alan Bell produced some designs:  http://people.ubuntu.com/~alanbell/unitytelly/

The mailing list has seen some decent traffic as well, with people talking mostly about what the future of the Connected TV might be and features they’d like to see.

Essential
– 10′ interface- Watching Media (DVR, Live, Network(TV Guide is part of DVR/other services))- Control via remote controlHigh priority– Plugin support- Cloud and/or server storage (for home grown media)

– Playback of physical media (USB cd/dvd/bluray drive)

– Installable image

– Easy configuration of new devices (eg. installing same plugins, mounting same network shares)

– Ubuntu One Accounts

– Push media to/from other Ubuntu devices / Media syncing capabilities (Pause on one device, resume from same spot on another device)

– Collaborate with other Ubuntu devices (context: https://lists.launchpad.net/ubuntu-phone/msg00006.html )

– Control from portable devices (phones/tablets/web interface/PC) (collaboration with Ubuntu Phone/Tablet?)

Medium priority

– Sharing media with friends (social network connectivity)

– Purchasing media through online stores (Ubuntu one/Amazon/Netflix)

 

By 14.04 LTS Ubuntu will power tablets, phones, TVs and smart screens from the car to the office kitchen, and it will connect those devices cleanly and seamlessly to the desktop, the server and the cloud.

Unity, the desktop interface in today’s Ubuntu 11.10, was designed with this specific vision in mind. While the interface for each form factor is shaped appropriately, Unity’s core elements are arranged in exactly the way we need to create coherence across all of those devices. This was the origin of the name Unity – a single core interface framework, that scales across all screens, and supports all toolkits.

Canonical and the Ubuntu community have established Ubuntu’s place in desktop, server and cloud deployments. We have also invested in the design and engineering of Unity, motivated by the belief that desktop interfaces would merge with mobile, touch interfaces into a seamless personal computing platform in the future. Today we are inviting the whole Ubuntu community – both commercial and personal – to shape that possibility and design that future; a world where Ubuntu runs on mobile phones, tablets, televisions and traditional PC’s, creating a world where content is instantly available on all devices, in a form that is delightful to use.

A constantly changing world

The way we access the Internet, connect to our friends, listen to music, watch films and go about our daily lives is rapidly evolving. We now use a diverse set of devices with an array of operating systems, which have a range of connectivity. Few people are exclusively loyal to a single technology provider.

Consider this quote from Paul Maritz of VMWare:

“Three years ago over 95 percent of the devices connected to the Internet were personal computers. Three years from now that number will probably be less than 20 percent. More than 80 percent of the devices connected to the Internet will not be Windows-based personal computers.” Paul Maritz, 29 August 2011 VM World Keynote.

Make no mistake – just as the world is changing for manufacturers so is it changing for Linux distributions. Today, 70% of people in Egypt access the Internet solely via the phone. Even in the US that figure is a startling 25%.

Ubuntu is well positioned

Ubuntu will thrive in this new reality.

Our established collaboration with the silicon vendors that are driving this converging market are critical. Intel, ARM and AMD will make the chip-sets that will power this future and Ubuntu works with all of them on all technologies.

Our engagement with the PC market will help bring the results of this work to a huge audience – partnerships with the likes of Dell, HP, Asus, Lenovo, Acer, IBM, Vodafone and more are a gateway to users who want continuous, connected, cross-device computing.

We are determined to bring more free software to more people around the world, and building that future hand in hand with device manufacturers is the best way to do it. There is no winner in place yet. This opportunity remains wide open, but only to products that deliver excellent experiences for users, across a full range of device categories.

The investment we have already made in the interface accommodates the touch scenarios required in some form factors and, with a little love and attention, will work equally well in mouse, keyboard or stylus-driven environments. Ubuntu will not be restricted to small screen or large screen environments but encompasses both and all the form factors in between. We will see our work on the Ubuntu platform land in a variety of formats current and yet to be invented. It is without doubt the most exciting phase in the history of Ubuntu.

Ubuntu One and the software centre

Ubuntu’s personal cloud and app centre services are appropriate for all these environments. They deliver the required storage, syncing and sharing capabilities that are not just a convenience but a requirement as we move to a universe where content is increasingly shared but the devices that access them become more diverse. Ubuntu One’s support for other OSes show the ability of Ubuntu to play nice with others, recognising that the divergence is strength.  It allows users to choose the devices they prefer but still delivering the benefits of Ubuntu-centred strategy.

The next steps

We are describing this at UDS to energize the entire Ubuntu ecosystem around this challenge. Canonical will provide the heavy lifting needed to put us in the ball park, but there are opportunities for participation, contribution and engagement by all elements of the broader Ubuntu community, both corporate and individual.

Our developers, our partners’ developers and the broader open source development community share this opportunity. There is a great deal to discuss, and an array of strands we need to pull together at UDS. But the direction is clear and the prize is great – to bring more free software to more people in more delightful ways than ever before.

Our goal with Unity is unprecedented ease of use, visual style and performance on the Linux desktop. With feature freeze behind us, we have a refined target render of the Dash for Oneiric, and here it is:

… click for the full size render.

Scopes and Lenses

We’ve moved from the idea of “Places” to a richer set of “Scopes and Lenses”. Scopes are data sources, and can tap into any online or offline data set as long as they can generate categorised results for a search, describe a set of filters and support some standard interfaces. Lenses are various ways to present the data that come from Scopes.

The Scopes have a range of filtering options they can use, such as ratings (“show me all the 5 star apps in the Software Center please”) and categories (“… that are games or media related”). Over time, the sophistication of this search system will grow but the goal is to keep it visual and immediate – something anyone can drive at first attempt.

This delivers on the original goal of creating a device-like experience that was search driven. Collaboration with the always-excellent Zeitgeist crew (quite a few of whom are now full time on the Unity team!) has improved the search experience substantially, kudos to them for the awesome work they’ve put in over the past six months. Since we introduced the Dash as a full screen device-like search experience, the same idea has made its way into several other shells, most notably Mac OS X Lion. While we’re definitely the outsider in this contest, I think we can stay one step ahead in the game given the support of our community.

The existing Places are all in the process of being updated to the Scopes and Lenses model, it’s a bit of a construction site at the moment so hard-hats are advised but dive in if you have good ideas for some more interesting scopes. I’ve heard all sorts of rumours about cool scopes in the pipeline 😉 and I bet this will be fertile ground for innovation. It’s pretty straightforward to make a scope, I’m sure others will blog and document the precise mechanisms but for those who want a head start, just use the source, Luke.

Panel evolution

In the panel, you’ll see that the top left corner is now consistently used to close whatever has the focus. Maximising a window keeps the window controls in the same position relative to the window – the top left corner. We have time to refine the behaviour of this based on user testing of the betas; for example, in one view, one should be able to close successive windows from that top left corner even if they are not maximised.

It’s taken several releases of careful iteration to get to this point. Even though we had a good idea where we were headed, each step needed to be taken one release at a time. Perhaps this might make a little clearer the reasons for the move of window controls to the left – it was the only place where we could ultimately keep them consistent all the way up to a maximised window with the title bar integrated into the panel. I’m confident this part will all be settled by 12.04.

As part of this two-step shuffle, the Dash invocation is now integrated in the Launcher. While this is slightly less of a Fitts-fantastic location, we consider it appropriate for a number of reasons. First, it preserves the top left corner for closing windows. Second, the Dash is best invoked with the Super key (sometimes erroneously and anachronistically referred to as the “Windows” key, for some reason ;-)). And finally, observations during user testing showed people as more inclined to try clicking on items in the Launcher than on the top left icon in the panel, unless that icon was something explicit like a close button for the window. Evidence based design rules.

Visual refinements

Rather than a flat darkening, we’re introducing a wash based on the desktop colour. The dash thus adjusts to your preferred palette based on your wallpaper. The same principle will drive some of the login experience – choosing a user will shift the login screen towards that users wallpaper and palette.

We’ve also integrated the panel and the dash, so indicators are rendered in a more holographic fashion inside the dash. Together with efforts to mute the contrast of Launcher icons the result is a more striking dash altogether: content is presented more dramatically.

Since we have raw access to the GL pipeline, we’re taking advantage of that with some real-time blur effects to help the readability and presentation of overlay content in the Dash, too. Both Nux in the case of Unity-3D and Qt in the case of Unity-2D have rich GL capabilities, and we’d like to make the most of whatever graphics stack you have on your hardware, while still running smoothly on the low end.

Growing community and ecosystem

A project like this needs diverse perspectives, talents and interests to make it feel rounded and complete. While Canonical is carrying the core load, and we’re happy to do so in order to bring this level of quality to the Ubuntu desktop user experience, what makes me particularly optimistic is the energy of the contributors both to Unity directly and to the integration of many other components and applications with the platform.

On the contribution front, a key goal for the Unity community is to maintain velocity in contributor patch flows. You should expect a rapid review and, all being well, landing, for contributions to Unity that are in line with the design goals. In a few cases we’ve also accepted patches that make it possible to use Unity in ways that are different to the design goals, especially where the evidence doesn’t lean very heavily one way or the other. Such contributions add some complexity but also give us the opportunity to test alternatives in a very rich way; the winning alternative would certainly survive, the other might not.

Contrary to common prognostication, this community is shaping up to be happy and productive. For those who do so for love and personal interest, participating in such a community should be fun and stimulating, an opportunity to exercise skills or pursue interests, give something back that will be appreciated and enjoyed by many, and help raise the bar for Linux experiences. I’d credit Jorge and others for their stewardship of this so far, and my heartfelt thanks to all of those who have helped make Unity better just for the fun of it.

Outside of the core, the growing number of apps that integrate sweetly with the launcher (quicklists), dash (scopes), indicators (both app-specific and category indicators) is helping to ensure that API’s are useful, refined and well implemented, as well as improving the experience of Ubuntu users everywhere. Now that we’re moving to Unity by default for both 2D and 3D, that’s even more valuable.

Under the hood

In this round, Unity-3D and Unity-2D have grown together and become twin faces on the same underlying model. They now share a good deal of common code and common services and – sigh – common bugs :-). But we’re now at the point where we can be confident that the Unity experience is available on the full range of hardware, from lightweight thin client systems made of ARM or Atom CPU’s to CADstations with oodles of GPU horsepower.

There’s something of a competition under way between proponents of the QML based Unity-2D, who believe that the GL support here is good enough to compete both at the high end and on the low end, and the GL-heads in Unity-3D, who think that the enhanced experiences possible with raw GL access justify the additional complexity of working in C++ and GL on the metal. Time will tell! Since a lot of the design inspiration for Unity came from game interfaces, I lean to the “let’s harness all the GL we can for the full 3D experience” side of the spectrum, but I’m intrigued with what the QML team are doing.

Users first, on free software. That has always been our mission: we set out to bring the joys and freedoms and innovation and performance and security that have always been part of the Linux platform, to a consumer audience. And yesterday marked the biggest leap forward in that mission that Ubuntu has ever taken, because in addition to the work we always do to make sure that the world’s best free software is polished and integrated, we brought something new to the very core of the user experience of the free platform: Unity.

We put user’s first because we committed to test and iterate Unity’s design with real users, and evolve it based on those findings. We’ve documented the process we’re following in that regard, so that other free software projects can decide for themselves if they also want to bring professional design into their process. I very much hope that this will become standard practice across all of free software, because in my view the future of free software is no longer just about inner beauty (architecture, performance, efficiency) it’s also about usability and style.

In the design of Unity we chose to be both humble and bold. Humble, because we have borrowed consciously from the work of other successful platforms, like Windows and MacOS. We borrowed what worked best, but then we took advantage of the fact that we are unconstrained by legacy and can innovate faster than they can, and took some bold leaps forward. In category indicators, the dash, overlay scrollbars and other innovations we are pioneering desktop experiences that I am sure will be emulated elsewhere, in both the free and proprietary platforms. This is the public “1.0”, there are rough points which will affect some users more than others, but we will iterate and polish them up one by one. Our goal should be to continue to set the pace and push free software to the forefront of usability and experience, growing the awesome Ubuntu and Unity community that shares those values and is excited by those ideas.

Ubuntu’s killer feature remains that community. The spirit of Ubuntu is about understanding that the measure of our own lives is in the way we improve the lives of others. Ubuntu has both economic and human dimensions: it is unique in bringing those together in a way which enables them to support one another. The fact that so many people recognise that their time, energy and expertise can have the biggest possible impact when expressed through Ubuntu is what makes their individual contributions so much more valuable. By recognising that it’s not just about bits, or licenses, or artwork, or documentation, or advocacy, or support, or assurance, or services, but that it’s about the whole of those in synthesis, we make something different to what the world has ever seen before. So to everyone who has helped bring Ubuntu 11.04 to fruition: thank you, and well done.

Of course, Ubuntu is far bigger than Unity. And the needs of the Ubuntu community, and users of Ubuntu, are far more diverse than simply Unity could address. So I’m proud of the fact that the Ubuntu community publishes the whole expression of software freedom across its archives. Kubuntu continues to improve and set a very high standard for the KDE experience. Lubuntu, the LXDE based expression of Ubuntu, is moving towards being 100% integrated. There is unique work being done in Ubuntu for users of the cloud and other server-oriented configurations. While we can be proud of what’s been achieved in Unity, we are equally proud of the efforts that go into ensuring that the full range of experiences is accommodated, to the extent possible with the effort put in by our huge community, under the Ubuntu umbrella.

We’re committed to keeping that the case. By welcoming all participants, and finding ways to accommodate and celebrate their differences rather than using them as grounds for divisiveness, we make something that is bigger than all our individual dreams.

The next major transition for Unity will be to deliver it on Wayland, the OpenGL-based display management system. We’d like to embrace Wayland early, as much of the work we’re doing on uTouch and other input systems will be relevant for Wayland and it’s an area we can make a useful contribution to the project.

We’re confident we’ll be able to retain the ability to run X applications in a compatibility mode, so this is not a transition that needs to reset the world of desktop free software. Nor is it a transition everyone needs to make at the same time: for the same reason we’ll keep investing in the 2D experience on Ubuntu despite also believing that Unity, with all its GL dependencies, is the best interface for the desktop. We’ll help GNOME and KDE with the transition, there’s no reason for them not to be there on day one either.

Timeframes are difficult. I’m sure we could deliver *something* in six months, but I think a year is more realistic for the first images that will be widely useful in our community. I’d love to be proven conservative on that 🙂 but I suspect it’s more likely to err the other way. It might take four or more years to really move the ecosystem. Progress on Wayland itself is sufficient for me to be confident that no other initiative could outrun it, especially if we deliver things like Unity and uTouch with it. And also if we make an early public statement in support of the project. Which this is!

In coming to this view, several scenarios were considered.

One is the continued improvement of X, which is a more vibrant project these days than it once was. X will be around a long time, hence the importance of our confidence levels on the idea of a compatibility environment. But we don’t believe X is setup to deliver the user experience we want, with super-smooth graphics and effects. I understand that it’s *possible* to get amazing results with X, but it’s extremely hard, and isn’t going to get easier. Some of the core goals of X make it harder to achieve these user experiences on X than on native GL, we’re choosing to prioritize the quality of experience over those original values, like network transparency.

We considered the Android compositing environment. It’s great for Android, but we felt it would be more difficult to bring the whole free software stack along with us if we pursued that direction.

We considered and spoke with several proprietary options, on the basis that they might be persuaded to open source their work for a new push, and we evaluated the cost of building a new display manager, informed by the lessons learned in Wayland. We came to the conclusion that any such effort would only create a hard split in the world which wasn’t worth the cost of having done it. There are issues with Wayland, but they seem to be solvable, we’d rather be part of solving them than chasing a better alternative. So Wayland it is.

In general, this will all be fine – actually *great* – for folks who have good open source drivers for their graphics hardware. Wayland depends on things they are all moving to support: kernel modesetting, gem buffers and so on. The requirement of EGL is new but consistent with industry standards from Khronos – both GLES and GL will be supported. We’d like to hear from vendors for whom this would be problematic, but hope it provides yet another (and perhaps definitive) motive to move to open source drivers for all Linux work.

In Maverick we’re adding the new Ayatana indicator for sound, Conor Curran’s very classy implementation of MPT’s very classy spec. It’s a Category Indicator, like the messaging menu, so it allows apps to embed themselves into it in a standard and appropriate way. You can have multiple players represented there, and control them directly from the menu, without needing a custom AppIndicator or windows open for the player(s). The integration with Rhythmbox and, via the MPRIS dbus API, several other players is coming along steadily.

One issue I’ve noticed is that the layout of the track and album art means we are almost always ellipsizing some of the track / album /artist data. I wondered whether it wouldn’t be reasonable to lay the metadata over the album art, if one used a drop shadow to ensure a more readable text:

Here’s the current layout:

Note the tight space for the track data, and hence the ellipsis

And here’s a GIMPfication, showing:

– the metadata right aligned,
– allowed to flow over the album art
– with a drop shadow to preserve contrast with the artwork

Metadata stretched over the artwork, with a drop shadow

And finally, I was a bit worried about the drop shadow over the non-art portion of the menu. It’s too different to anything else in the menu, so I cropped the shadows to limit them just to the area over the art:

Drop shadow is only used on the artwork

Clearly, this is only appropriate in the case where one has artwork. The metadata should stay left-aligned (and use the full width of the menu, something it doesn’t currently do) when there is no artwork.

Thoughts? I’m off to bed. Jetlagged, back from Debconf (lovely to see everyone again, if briefly).

A few months ago we took on the challenge of building a version of Ubuntu for the dual-boot, instant-on market. We wanted to be surfing the web in under 10 seconds, and give people a fantastic web experience. We also wanted it to be possible to upgrade from that limited usage model to a full desktop.

The fruit of that R&D is both a new desktop experience codebase, called Unity, and a range of Light versions of Ubuntu, both netbook and desktop, that are optimised for dual-boot scenarios.

The dual-boot, web-focused use case is sufficiently different from general-purpose desktop usage to warrant a fresh look at the way the desktop is configured. We spent quite a bit of time analyzing screenshots of a couple of hundred different desktop configurations from the current Ubuntu and Kubuntu user base, to see what people used most. We also identified the things that are NOT needed in lightweight dual-boot instant-on offerings. That provided us both with a list of things to focus on and make rich, and a list of things we could leave out.

Instant-on products are generally used in a stateless fashion. These are “get me to the web asap” environments, with no need of heavy local file management. If there is content there, it would be best to think of it as “cloud like” and synchronize it with the local Windows environment, with cloud services and other devices. They are also not environments where people would naturally expect to use a wide range of applications: the web is the key, and there may be a few complementary capabilities like media playback, messaging, games, and the ability to connect to local devices like printers and cameras and pluggable media.

We also learned something interesting from users. It’s not about how fast you appear to boot. It’s about how fast you actually deliver a working web browser and Internet connection. It’s about how fast you have a running system that is responsive to the needs of the user.

Unity: a lightweight netbook interface

There are several driving forces behind the result.

The desktop screenshots we studied showed that people typically have between 3 and 10 launchers on their panels, for rapid access to key applications. We want to preserve that sense of having a few favorite applications that are instantly accessible. Rather than making it equally easy to access any installed application, we assume that almost everybody will run one of a few apps, and they need to switch between those apps and any others which might be running, very easily.

We focused on maximising screen real estate for content. In particular, we focused on maximising the available vertical pixels for web browsing. Netbooks have screens which are wide, but shallow. Notebooks in general are moving to wide screen formats. So vertical space is more precious than horizontal space.

We also want to embrace touch as a first class input. We want people to be able to launch and switch between applications using touch, so the launcher must be finger friendly.

Those constraints and values lead us to a new shape for the desktop, which we will adopt in Ubuntu’s Netbook Edition for 10.10 and beyond.

First, we want to move the bottom panel to the left of the screen, and devote that to launching and switching between applications. That frees up vertical space for web content, at the cost of horizontal space, which is cheaper in a widescreen world. In Ubuntu today the bottom panel also presents the Trash and Show Desktop options, neither of which is relevant in a stateless instant-on environment.

Second, we’ll expand that left-hand launcher panel so that it is touch-friendly. With relatively few applications required for instant-on environments, we can afford to be more generous with the icon size there. The Unity launcher will show what’s running, and support fast switching and drag-and-drop between applications.

Third, we will make the top panel smarter. We’ve already talked about adopting a single global menu, which would be rendered by the panel in this case. If we can also manage to fit the window title and controls into that panel, we will have achieved very significant space saving for the case where someone is focused on a single application at a time, and especially for a web browser.

We end up with a configuration like this:

Mockup of Unity Launcher and Panel with maximised application

The launcher and panel that we developed in response to this challenge are components of Unity. They are now in a state where they can be tested widely, and where we can use that testing to shape their evolution going forward. A development milestone of Unity is available today in a PPA, with development branches on Launchpad, and I’d very much like to get feedback from people trying it out on a netbook, or even a laptop with a wide screen. Unity is aimed at full screen applications and, as I described above, doesn’t really support traditional file management. But it’s worth a spin, and it’s very easy to try out if you have Ubuntu 10.04 LTS installed already.

Ubuntu Light

Instant-on, dual boot installations are a new frontier for us. Over the past two years we have made great leaps forward as a first class option for PC OEM’s, who today ship millions of PC’s around the world with Ubuntu pre-installed. But traditionally, it’s been an “either/or” proposition – either Windows in markets that prefer it, or Ubuntu in markets that don’t. The dual-boot opportunity gives us the chance to put a free software foot forward even in markets where people use Windows as a matter of course.

And it looks beautiful:

Ubuntu Light, showing the Unity launcher and panel

In those cases, Ubuntu Netbook Light, or Ubuntu Desktop Light, will give OEM’s the ability to differentiate themselves with fast-booting Linux offerings that are familiar to Ubuntu users and easy to use for new users, safe for web browsing in unprotected environments like airports and hotels, focused on doing that job very well, but upgradeable with a huge list of applications, on demand. The Light versions will also benefit from the huge amount of work done on every Ubuntu release to keep it maintained – instant-on environments need just as much protection as everyday desktops, and Ubuntu has a deep commitment to getting that right.

The Ubuntu Light range is available to OEM’s today. Each image will be hand-crafted to boot fastest on that specific hardware, the application load reduced to the minimum, and it comes with tools for Windows which assist in the management of the dual-boot experience. Initially, the focus is on the Netbook Light version based on Unity, but in future we expect to do a Light version of the desktop, too.

Given the requirement to customise the Light versions for specific hardware, there won’t be a general-purpose downloadable image of Ubuntu Light on ubuntu.com.

Evolving Unity for Ubuntu Netbook Edition 10.10

Unity exists today, and is great for the minimalist, stateless configurations that suit a dual-boot environment. But in order embrace it for our Netbook UI, we’ll need to design some new capabilities, and implement them during this cycle.

Those design conversations are taking place this week at UDS, just outside Brussels in Belgium. If you can’t be there in person, and are interested in the design challenges Unity presents for the netbook form factor, check out the conference schedule and participate in the discussion virtually.

The two primary pieces we need to put in place are:

• Support for many more applications, and adding / removing applications. Instant-on environments are locked down, while netbook environments should support anybody’s applications, not just those favored in the Launcher.
• Support for file management, necessary for an environment that will be the primary working space for the user rather than an occasional web-focused stopover.

We have an initial starting point for the design, called the Dash, which presents files and applications as an overlay. The inspiration for the Dash comes from consoles and devices, which use full-screen, media-rich presentation. We want the Dash to feel device-like, and use the capabilities of modern hardware.

The Unity Dash, showing the Applications Place

The instant-on requirements and constraints proved very useful in shaping our thinking, but the canvas is still blank for the more general, netbook use case. Unity gives us the chance to do something profoundly new and more useful, taking advantage of ideas that have emerged in computing from the console to the handheld.

Relationship to Gnome Shell

Unity and Gnome Shell are complementary for the Gnome Project. While Gnome Shell presents an expansive view of how people work in complex environments with multiple simultaneous activities, Unity is designed to address the other end of the spectrum, where people are focused on doing one thing at any given time.

Unity does embrace the key technologies of Gnome 3: Mutter, for window management, and Zeitgeist will be an anchor component of our file management approach. The interface itself is built in Clutter.

The design seed of Unity was in place before Gnome Shell, and we decided to build on that for the instant-on work rather than adopt Gnome Shell because most of the devices we expect to ship Ubuntu Light on are netbooks. In any event, Unity represents the next step for the Ubuntu Netbook UI, optimised for small screens.

The Ubuntu Netbook interface is popular with Gnome users and we’re fortunate to be working inside an open ecosystem that encourages that level of diversity. As a result, Gnome has offerings for mobile, netbook and desktop form factors. Gnome is in the lucky position of having multiple vendors participating and solving different challenges independently. That makes Gnome stronger.

Relationship to FreeDesktop and KDE

Unity complies with freedesktop.org standards, and is helping to shape them, too. We would like KDE applications to feel welcome on a Unity-based netbook. We’re using the Ayatana indicators in the panel, so KDE applications which use AppIndicators will Just Work. And to the extent that those applications take advantage of the Messaging Menu, Sound Indicator and Me Menu, they will be fully integrated into the Unity environment. We often get asked by OEM’s how they can integrate KDE applications into their custom builds of Ubuntu, and the common frameworks of freedesktop.org greatly facilitate doing so in a smooth fashion.

Looking forward to the Maverick Meerkat

It will be an intense cycle, if we want to get all of these pieces in line. But we think it’s achievable: the new launcher, the new panel, the new implementation of the global menu and an array of indicators. Things have accelerated greatly during Lucid so if we continue at this pace, it should all come together. Here’s to a great summer of code.