When once you have tasted flight, you will forever walk the earth with your eyes turned skyward, for there you have been and there you will always long to return.
— Leonardo da Vinci
In the open source community, we celebrate having pieces that “do one thing well”, with lots of orthogonal tools compounding to give great flexibility. But that same philosophy leads to shortcomings on the GUI / UX front, where we want all the pieces to be aware of each other in a deeper way.
For example, we consciously place the notifications in the top right of the screen, avoiding space that is particularly precious (like new tab titles, and search boxes). But the indicators are also in the top right, and they make menus, which drop down into the same space a notification might occupy.
Since we know that notifications are queued, no notification is guaranteed to be displayed instantly, so a smarter notification experience would stay out of the way while you were using indicator menus, or get out of the way when you invoke them. The design story of focusayatana, where we balance the need for focus with the need for awareness, would suggest that we should suppress awareness-oriented things in favour of focus things. So when you’re interacting with an indicator menu, we shouldn’t pop up the notification. Since the notification system, and the indicator menu system, are separate parts, the UNIX philosophy sells us short in designing a smart, smooth experience because it says they should each do their thing individually.
Going further, it’s silly that the sound menu next/previous track buttons pop up a notification, because the same menu shows the new track immediately anyway. So the notification, which is purely for background awareness, is distracting from your focus, which is conveying exactly the same information!
But it’s not just the system menus. Apps can play in that space too, and we could be better about shaping the relationship between them. For example, if I’m moving the mouse around in the area of a notification, we should be willing to defer it a few seconds to stay out of the focus. When I stop moving the mouse, or typing in a window in that region, then it’s OK to pop up the notification.
It’s only by looking at the whole, that we can design great experiences. And only by building a community of both system and application developers that care about the whole, that we can make those designs real. So, thank you to all of you who approach things this way, we’ve made huge progress, and hopefully there are some ideas here for low-hanging improvements too
The desktop remains central to our everyday work and play, despite all the excitement around tablets, TV’s and phones. So it’s exciting for us to innovate in the desktop too, especially when we find ways to enhance the experience of both heavy “power” users and casual users at the same time. The desktop will be with us for a long time, and for those of us who spend hours every day using a wide diversity of applications, here is some very good news: 12.04 LTS will include the first step in a major new approach to application interfaces.
This work grows out of observations of new and established / sophisticated users making extensive use of the broader set of capabilities in their applications. We noticed that both groups of users spent a lot of time, relatively speaking, navigating the menus of their applications, either to learn about the capabilities of the app, or to take a specific action. We were also conscious of the broader theme in Unity design of leading from user intent. And that set us on a course which led to today’s first public milestone on what we expect will be a long, fruitful and exciting journey.
The menu has been a central part of the GUI since Xerox PARC invented ‘em in the 70′s. It’s the M in WIMP and has been there, essentially unchanged, for 30 years.
The original Macintosh desktop, circa 1984, courtesy of Wikipedia
We can do much better!
Say hello to the Head-Up Display, or HUD, which will ultimately replace menus in Unity applications. Here’s what we hope you’ll see in 12.04 when you invoke the HUD from any standard Ubuntu app that supports the global menu:
Snapshot of the HUD in Ubuntu 12.04
The intenterface – it maps your intent to the interface
This is the HUD. It’s a way for you to express your intent and have the application respond appropriately. We think of it as “beyond interface”, it’s the “intenterface”. This concept of “intent-driven interface” has been a primary theme of our work in the Unity shell, with dash search as a first class experience pioneered in Unity. Now we are bringing the same vision to the application, in a way which is completely compatible with existing applications and menus.
The HUD concept has been the driver for all the work we’ve done in unifying menu systems across Gtk, Qt and other toolkit apps in the past two years. So far, that’s shown up as the global menu. In 12.04, it also gives us the first cut of the HUD.
Menus serve two purposes. They act as a standard way to invoke commands which are too infrequently used to warrant a dedicated piece of UI real-estate, like a toolbar button, and they serve as a map of the app’s functionality, almost like a table of contents that one can scan to get a feel for ‘what the app does’. It’s command invocation that we think can be improved upon, and that’s where we are focusing our design exploration.
As a means of invoking commands, menus have some advantages. They are always in the same place (top of the window or screen). They are organised in a way that’s quite easy to describe over the phone, or in a text book (“click the Edit->Preferences menu”), they are pretty fast to read since they are generally arranged in tight vertical columns. They also have some disadvantages: when they get nested, navigating the tree can become fragile. They require you to read a lot when you probably already know what you want. They are more difficult to use from the keyboard than they should be, since they generally require you to remember something special (hotkeys) or use a very limited subset of the keyboard (arrow navigation). They force developers to make often arbitrary choices about the menu tree (“should Preferences be in Edit or in Tools or in Options?”), and then they force users to make equally arbitrary effort to memorise and navigate that tree.
The HUD solves many of these issues, by connecting users directly to what they want. Check out the video, based on a current prototype. It’s a “vocabulary UI”, or VUI, and closer to the way users think. “I told the application to…” is common user paraphrasing for “I clicked the menu to…”. The tree is no longer important, what’s important is the efficiency of the match between what the user says, and the commands we offer up for invocation.
In 12.04 LTS, the HUD is a smart look-ahead search through the app and system (indicator) menus. The image is showing Inkscape, but of course it works everywhere the global menu works. No app modifications are needed to get this level of experience. And you don’t have to adopt the HUD immediately, it’s there if you want it, supplementing the existing menu mechanism.
It’s smart, because it can do things like fuzzy matching, and it can learn what you usually do so it can prioritise the things you use often. It covers the focused app (because that’s where you probably want to act) as well as system functionality; you can change IM state, or go offline in Skype, all through the HUD, without changing focus, because those apps all talk to the indicator system. When you’ve been using it for a little while it seems like it’s reading your mind, in a good way.
We’ll resurrect the (boring) old ways of displaying the menu in 12.04, in the app and in the panel. In the past few releases of Ubuntu, we’ve actively diminished the visual presence of menus in anticipation of this landing. That proved controversial. In our defence, in user testing, every user finds the menu in the panel, every time, and it’s obviously a cleaner presentation of the interface. But hiding the menu before we had the replacement was overly aggressive. If the HUD lands in 12.04 LTS, we hope you’ll find yourself using the menu less and less, and be glad to have it hidden when you are not using it. You’ll definitely have that option, alongside more traditional menu styles.
Voice is the natural next step
Searching is fast and familiar, especially once we integrate voice recognition, gesture and touch. We want to make it easy to talk to any application, and for any application to respond to your voice. The full integration of voice into applications will take some time. We can start by mapping voice onto the existing menu structures of your apps. And it will only get better from there.
But even without voice input, the HUD is faster than mousing through a menu, and easier to use than hotkeys since you just have to know what you want, not remember a specific key combination. We can search through everything we know about the menu, including descriptive help text, so pretty soon you will be able to find a menu entry using only vaguely related text (imagine finding an entry called Preferences when you search for “settings”).
There is lots to discover, refine and implement. I have a feeling this will be a lot of fun in the next two years
Even better for the power user
The results so far are rather interesting: power users say things like “every GUI app now feels as powerful as VIM”. EMACS users just grunt and… nevermind . Another comment was “it works so well that the rare occasions when it can’t read my mind are annoying!”. We’re doing a lot of user testing on heavy multitaskers, developers and all-day-at-the-workstation personas for Unity in 12.04, polishing off loose ends in the experience that frustrated some in this audience in 11.04-10. If that describes you, the results should be delightful. And the HUD should be particularly empowering.
Even casual users find typing faster than mousing. So while there are modes of interaction where it’s nice to sit back and drive around with the mouse, we observe people staying more engaged and more focused on their task when they can keep their hands on the keyboard all the time. Hotkeys are a sort of mental gymnastics, the HUD is a continuation of mental flow.
Ahead of the competition
There are other teams interested in a similar problem space. Perhaps the best-known new alternative to the traditional menu is Microsoft’s Ribbon. Introduced first as part of a series of changes called Fluent UX in Office, the ribbon is now making its way to a wider set of Windows components and applications. It looks like this:
You can read about the ribbon from a supporter (like any UX change, it has its supporters and detractors ) and if you’ve used it yourself, you will have your own opinion about it. The ribbon is highly visual, making options and commands very visible. It is however also a hog of space (I’m told it can be minimised). Our goal in much of the Unity design has been to return screen real estate to the content with which the user is working; the HUD meets that goal by appearing only when invoked.
Instead of cluttering up the interface ALL the time, let’s clear out the chrome, and show users just what they want, when they want it.
Time will tell whether users prefer the ribbon, or the HUD, but we think it’s exciting enough to pursue and invest in, both in R&D and in supporting developers who want to take advantage of it.
Other relevant efforts include Enso and Ubiquity from the original Humanized team (hi Aza &co), then at Mozilla.
Our thinking is inspired by many works of science, art and entertainment; from Minority Report to Modern Warfare and Jef Raskin’s Humane Interface. We hope others will join us and accelerate the shift from pointy-clicky interfaces to natural and efficient ones.
Roadmap for the HUD
There’s still a lot of design and code still to do. For a start, we haven’t addressed the secondary aspect of the menu, as a visible map of the functionality in an app. That discoverability is of course entirely absent from the HUD; the old menu is still there for now, but we’d like to replace it altogether not just supplement it. And all the other patterns of interaction we expect in the HUD remain to be explored. Regardless, there is a great team working on this, including folk who understand Gtk and Qt such as Ted Gould, Ryan Lortie, Gord Allott and Aurelien Gateau, as well as designers Xi Zhu, Otto Greenslade, Oren Horev and John Lea. Thanks to all of them for getting this initial work to the point where we are confident it’s worthwhile for others to invest time in.
We’ll make sure it’s easy for developers working in any toolkit to take advantage of this and give their users a better experience. And we’ll promote the apps which do it best – it makes apps easier to use, it saves time and screen real-estate for users, and it creates a better impression of the free software platform when it’s done well.
From a code quality and testing perspective, even though we consider this first cut a prototype-grown-up, folk will be glad to see this:
Overall coverage rate:
lines......: 87.1% (948 of 1089 lines)
functions..: 97.7% (84 of 86 functions)
branches...: 63.0% (407 of 646 branches)
Landing in 12.04 LTS is gated on more widespread testing. You can of course try this out from a PPA or branch the code in Launchpad (you will need thesetwo branches). Or dig deeper with blogs on the topic from Ted Gould, Olli Ries and Gord Allott. Welcome to 2012 everybody!
Our goal with Unity is unprecedented ease of use, visual style and performance on the Linux desktop. With feature freeze behind us, we have a refined target render of the Dash for Oneiric, and here it is:
We’ve moved from the idea of “Places” to a richer set of “Scopes and Lenses”. Scopes are data sources, and can tap into any online or offline data set as long as they can generate categorised results for a search, describe a set of filters and support some standard interfaces. Lenses are various ways to present the data that come from Scopes.
The Scopes have a range of filtering options they can use, such as ratings (“show me all the 5 star apps in the Software Center please”) and categories (“… that are games or media related”). Over time, the sophistication of this search system will grow but the goal is to keep it visual and immediate – something anyone can drive at first attempt.
This delivers on the original goal of creating a device-like experience that was search driven. Collaboration with the always-excellent Zeitgeist crew (quite a few of whom are now full time on the Unity team!) has improved the search experience substantially, kudos to them for the awesome work they’ve put in over the past six months. Since we introduced the Dash as a full screen device-like search experience, the same idea has made its way into several other shells, most notably Mac OS X Lion. While we’re definitely the outsider in this contest, I think we can stay one step ahead in the game given the support of our community.
The existing Places are all in the process of being updated to the Scopes and Lenses model, it’s a bit of a construction site at the moment so hard-hats are advised but dive in if you have good ideas for some more interesting scopes. I’ve heard all sorts of rumours about cool scopes in the pipeline and I bet this will be fertile ground for innovation. It’s pretty straightforward to make a scope, I’m sure others will blog and document the precise mechanisms but for those who want a head start, just use the source, Luke.
Panel evolution
In the panel, you’ll see that the top left corner is now consistently used to close whatever has the focus. Maximising a window keeps the window controls in the same position relative to the window – the top left corner. We have time to refine the behaviour of this based on user testing of the betas; for example, in one view, one should be able to close successive windows from that top left corner even if they are not maximised.
It’s taken several releases of careful iteration to get to this point. Even though we had a good idea where we were headed, each step needed to be taken one release at a time. Perhaps this might make a little clearer the reasons for the move of window controls to the left – it was the only place where we could ultimately keep them consistent all the way up to a maximised window with the title bar integrated into the panel. I’m confident this part will all be settled by 12.04.
As part of this two-step shuffle, the Dash invocation is now integrated in the Launcher. While this is slightly less of a Fitts-fantastic location, we consider it appropriate for a number of reasons. First, it preserves the top left corner for closing windows. Second, the Dash is best invoked with the Super key (sometimes erroneously and anachronistically referred to as the “Windows” key, for some reason ). And finally, observations during user testing showed people as more inclined to try clicking on items in the Launcher than on the top left icon in the panel, unless that icon was something explicit like a close button for the window. Evidence based design rules.
Visual refinements
Rather than a flat darkening, we’re introducing a wash based on the desktop colour. The dash thus adjusts to your preferred palette based on your wallpaper. The same principle will drive some of the login experience – choosing a user will shift the login screen towards that users wallpaper and palette.
We’ve also integrated the panel and the dash, so indicators are rendered in a more holographic fashion inside the dash. Together with efforts to mute the contrast of Launcher icons the result is a more striking dash altogether: content is presented more dramatically.
Since we have raw access to the GL pipeline, we’re taking advantage of that with some real-time blur effects to help the readability and presentation of overlay content in the Dash, too. Both Nux in the case of Unity-3D and Qt in the case of Unity-2D have rich GL capabilities, and we’d like to make the most of whatever graphics stack you have on your hardware, while still running smoothly on the low end.
Growing community and ecosystem
A project like this needs diverse perspectives, talents and interests to make it feel rounded and complete. While Canonical is carrying the core load, and we’re happy to do so in order to bring this level of quality to the Ubuntu desktop user experience, what makes me particularly optimistic is the energy of the contributors both to Unity directly and to the integration of many other components and applications with the platform.
On the contribution front, a key goal for the Unity community is to maintain velocity in contributor patch flows. You should expect a rapid review and, all being well, landing, for contributions to Unity that are in line with the design goals. In a few cases we’ve also accepted patches that make it possible to use Unity in ways that are different to the design goals, especially where the evidence doesn’t lean very heavily one way or the other. Such contributions add some complexity but also give us the opportunity to test alternatives in a very rich way; the winning alternative would certainly survive, the other might not.
Contrary to common prognostication, this community is shaping up to be happy and productive. For those who do so for love and personal interest, participating in such a community should be fun and stimulating, an opportunity to exercise skills or pursue interests, give something back that will be appreciated and enjoyed by many, and help raise the bar for Linux experiences. I’d credit Jorge and others for their stewardship of this so far, and my heartfelt thanks to all of those who have helped make Unity better just for the fun of it.
Outside of the core, the growing number of apps that integrate sweetly with the launcher (quicklists), dash (scopes), indicators (both app-specific and category indicators) is helping to ensure that API’s are useful, refined and well implemented, as well as improving the experience of Ubuntu users everywhere. Now that we’re moving to Unity by default for both 2D and 3D, that’s even more valuable.
Under the hood
In this round, Unity-3D and Unity-2D have grown together and become twin faces on the same underlying model. They now share a good deal of common code and common services and – sigh – common bugs . But we’re now at the point where we can be confident that the Unity experience is available on the full range of hardware, from lightweight thin client systems made of ARM or Atom CPU’s to CADstations with oodles of GPU horsepower.
There’s something of a competition under way between proponents of the QML based Unity-2D, who believe that the GL support here is good enough to compete both at the high end and on the low end, and the GL-heads in Unity-3D, who think that the enhanced experiences possible with raw GL access justify the additional complexity of working in C++ and GL on the metal. Time will tell! Since a lot of the design inspiration for Unity came from game interfaces, I lean to the “let’s harness all the GL we can for the full 3D experience” side of the spectrum, but I’m intrigued with what the QML team are doing.
A wit said of Google Wave “if your project depends on reinventing scrollbars, you are doing something wrong.” But occasionally, just occasionally, one gets to do exactly that.
Under the Ayatana banner, we’ve been on a mission to make the desktop have less chrome and more content. The goal is to help people immerse themselves in their stuff, without being burdened with large amounts of widgetry which isn’t relevant to the thing they are trying to concentrate on. And when it comes to widgetry, there are few things left which take up more space than scrollbars.
For example, I spend plenty of time in a full screen terminal, and it’s lovely to see how clean that experience is on Natty today:
…but that scrollbar on the right seems heavy and outdated. We took inspiration from mobile devices, and started exploring the idea of making scrollbars be more symbolic, and less physical. Something like this:
Of course, since the desktop isn’t often a touch device, we need to think through pointer interactions. We wanted to preserve the idea of keeping content exposed as much as possible, while still providing for pointer interaction when needed. We also decided to drop the “one line scroll” capability, while preserving the ability to page up and down. Take a look at the result:
The design work behind this has been done by Christian Giordano, who worked through the corner cases (literally) and provided a mockup for testing purposes. And the heavy lifting for Natty is being done by the indefatigable Andrea Cimitan, who is currently polishing up a gtk implementation of the concept for the release. Christian put together a blog post on the subject, and a great video which talks through the design process and a few of the challenges and solutions found:
Multitouch is just as useful on a desktop as it is on a phone or tablet, so I’m delighted that the first cut of Canonical’s UTouch framework has landed in Maverick and will be there for its release on 10.10.10.
You’ll need 4-finger touch or better to get the most out of it, and we’re currently targeting the Dell XT2 as a development environment so the lucky folks with that machine will get the best results today. By release, we expect you’ll be able to use it with a range of devices from major manufacturers, and with addons like Apple’s Magic Trackpad.
The design team has lead the way, developing a “touch language” which goes beyond the work that we’ve seen elsewhere. Rather than single, magic gestures, we’re making it possible for basic gestures to be chained, or composed, into more sophisticated “sentences”. The basic gestures, or primitives, are like individual verbs, and stringing them together allows for richer interactions. It’s not quite the difference between banging rocks together and conducting a symphony orchestra, but it feels like a good step in the right direction
The new underlying code is published on Launchpad under the GPLv3 and LGPLv3, and of course there are quite a lot of modules for things like X and Gtk which may be under licenses preferred by those projects. There’s a PPA if you’re interested in tracking the cutting edge, or just branch / push/ merge on LP if you want to make it better. Details in the official developer announcement. The bits depend on Peter Hutterer’s recently published update to the X input protocols related to multi-touch, and add gesture processing and gesture event delivery. I’d like to thank Duncan McGreggor for his leadership of the team which implemented this design, and of course all the folks who have worked on it so far: Henrik Rydberg, Rafi Rubin, Chase Douglas, Stephen Webb at the heart of it, and many others who have expanded on their efforts.
In Maverick, quite a few Gtk applications will support gesture-based scrolling. We’ll enhance Evince to show some of the richer interactions that developers might want to add to their apps. Window management will be gesture-enabled in Unity, so 10.10 Netbook Edition users with touch screens or multi-touch pads will have sophisticated window management at their fingertips. Install Unity on your desktop for a taste of it, just apt-get install ubuntu-netbook and choose the appropriate session at login.
The roadmap beyond 10.10 will flesh out the app developer API and provide system services related to gesture processing and touch. It would be awesome to have touch-aware versions of all the major apps – browser, email, file management, chat, photo management and media playback – for 11.04, but that depends on you! So if you are interested in this, let’s work up some branches Here’s the official Canonical blog post, too.
The Ayatana Indicators work has given us a crisp, clean basis for indicators in the panel. We’ve said they will all look a particular way, and behave a particular way. And we’ve said they will be placed on the right of the panel.
But why limit indicators to the panel? Let’s make it possible for applications to use indicators themselves, for all the things that indicators are good at:
Conveying a particular state, such as whether or not the application is connected,
Providing a handle for the indicator menu, to modify that state
We’ll start with “window indicators”, or “windicators” for fun. Windicators are indicators displayed in the window title bar that behave just like the indicators in the panel: they have an icon which shows state, and clicking on the icon brings up a menu. Applications can create, update and remove window indicators using an API more or less like the AppIndicator framework first put to use in 10.04 LTS.
Window indicators, or "windicators", shown in a sample application window.
We’ve carefully placed all the panel indicators on the right, and we’ve carefully put the window controls and window title on the left. So now we have all this space on the right. As a pattern, it would fit to put the window indicators there.
Cody Russell is leading some work in Canonical around the technology which actually draws the window title bar and borders. It’s called “client side window decorations”. We are moving the rendering of the window decorations into the app itself, so that you don’t have the window manager and application drawing those pieces separately. That simplifies certain things (of course it also makes some things harder).
One of the most interesting consequences of the client-side decorations work is that it means that the application could more easily draw into the titlebar (because the application is drawing the title bar). And that makes it even more natural for the application to control the right side of the window title bar as well.
Update: Several commenters correctly pointed out that window indicators could just as easily be rendered by window managers in cases where the theme is not CSD-based. CSD provided the inspiration for giving that space to the application, it’s not essential to the implementation. It would be fantastic for window indicators to be available on
Less chrome, more content: banish the status bar
I’m on a “less is more” kick with our design efforts, and one of the things I want to banish is wasted vertical space. For netbooks, that’s particularly important. And a lot of applications have status bars at the bottom, for no good reason other than it was that way in Windows 3.1.
Typically the application status bar has:
Some status icons (“online”)
Some tools (“Yslow”)
A transient status message (“Saving draft…”)
We can replace these with a combination of windicators and temporary, overlay status bars. I really liked the Chrome browser’s use of overlay status messages, so kudos and thanks to them for the inspiration. The net result of those two steps, in apps where we can, is to save about 5% of the vertical space for your stuff – real content.
Prioritising examples for implementation
If you’re interested in this idea, please join the Ayatana mailing list and participate in the design discussions there. We’d like to develop some patterns that are generic, so that we can use a common icon and possibly also common indicator menu entries for addressing the same issue in diverse applications. Of course, applications will be free to use the mechanism for things that are unique to them.
Candidates for 10.10
It would be fantastic to implement a few of these window indicators for 10.10. Please help us choose the most useful cases! Currently on the list are:
Online / offline status indicator and toggle options for the mail client, chat program or Gwibber, the broadcast messages application.
An “unsaved” indicator, that tells people that the contents of the file they are working on have changed and potentially lets them save it or set autosave properties.
Progress indicators, which show that an action is in progress, and possibly also indicate the extent of the progress. The associated menu would enable one to pause or cancel the operation, and perhaps define the behaviour on completion of the action.
A “basket” indicator, which shows if any items have been selected for purchase,
Sharing indicators, which would show if a document is shared with multiple people, and enable one to setup such a share.
Volume indicators, which would show the loudness of application audio streams, and enable one to set the volume for that specific application.
The key thing is that these indicators are entirely application-specific, and ideally only relevant to the window that you are actually looking at.
Just like Panel Indicators…
From a visual design point of view, again the goal would be to ensure that indicators are symbolic. They would follow the same styling as Ayatana indicators:
Monochrome by default, with shape indicating the function of the indicator
Semantically colored: with red for critical problems, orange for alerts, green for positive status changes and blue for informative states that are not the default or usual state.
Integrated with the Netbook Edition Smart Panel
Last week I blogged about our decision to adopt a single, global menu for all applications, in the panel. And I also said we would explore putting the window title *and* menu into the panel, when the window is maximised. Of course, that means that we need to accommodate the window indicators in the panel as well.
So: when the window is maximised, and we are using a smart which can include both indicators and window titles, the window indicators will be inserted into the panel as well. They will appear on the right of the panel, and be the leftmost indicators. For example, here is the application, maximised (note the dodgy Ubuntu logo in the top left – that’s the panel, not the window title bar you’re looking at):
In this configuration, the system achieves “singular purpose”: the entire screen is devoted to a single application, yet the Ayatana elements continue to serve their purpose, either systemic (the battery indicator) or application specific.
When we set up Project Ayatana to improve the usability of the whole desktop, we called it Ayatana because we were focused on the “sphere of consciousness”, one’s awareness of what’s going on outside of the current application. There are two key aspects to the work:
Notifications are “awareness distilled” in the sense that you cannot interact with them at all. We designed them as ephemeral “click-transparent” messages, implemented in Notify-OSD. Their sole purpose is to notify you of transient events.
Indicator Menus combine persistent awareness of a state with a set of options for modifying that state.
In this blog I’ll outline the arc of our work on indicator menus to date, and the trajectory we expect it to follow. We’re about a year into the effort, all told, and I think it will take another 18 months before we can consider it baked. It should be done by 12.04 LTS. This is an iterative process, and things are in flux right now. I hope, when we are happy that we can commit to ABI stability, that Gnome and KDE will adopt the work too. For the moment, the rapid pace of evolution has meant that we’re depending on fantastic upstreams to keep up with us as things move.
Goals of the Ayatana Indicators
The indicators are designed to create a persistent awareness of state, or an awareness of a persistent state. They complement notifications: they are persistent, when notifications are ephemeral. You might miss a notification, but you should always be able to check your indicators. You can interact with indicators, using their menus, in contrast with the un-clickable notifications.
We value:
Support for both GNOME and KDE. Both desktop environments are important in Ubuntu. We encourage the teams to reflect a pure vision of each, but it’s also the case that users will want to run a GNOME application on Kubuntu occasionally, or vice versa. So we have to make sure the work is considered from the perspectives of developers on either side, and we have to provide APIs and libraries that work in both environments.
Accessibility. Indicators are critical elements of awareness. Whether you are connected, what the time is, whether you are online, whether your battery will last long enough for you to finish your work, whether you have messages… these are all vital to a complete computing experience. We have to make sure that visual and other disabilities can be addressed.
Familiarity and Innovation. As always, these are in tension with one another. Innovation helps us put free software at the front of the curve, but it creates the risk of breaking people’s habits and expectations.
Consistency and Usability. We want the end result to be more usable in the whole, and we are willing to lose individual nuggets if that helps make the whole more valuable.
Streamlining. There are too many indicators, that aren’t clear enough about their intent. There are also many indicators from different applications which do roughly the same thing, but in slightly different ways. The value of all the indicators is enhanced if there are fewer of them, and they are more obvious to discover and use.
Some firm decisions
Those values lead us to some anchor decisions:
D-Bus for communications. A messaging approach makes it straightforward to adopt consistent patterns across different desktop environments. We will provide wrapper libraries for both Gtk and Qt applications to access the indicator capabilities. A Qt application running on Ubuntu should “feel native” when it’s using indicators correctly. And vice-versa. The messaging approach also lets us handle accessibility in a better way: we don’t have to accommodate every possible disability visually, because we can have agents that interpret the indicator messages and handle it in ways that are appropriate for a particular disability.
Opinionated placement. We will place all indicators at the top right of the screen on GNOME. We’ll place them in a particular order, too, with the “most fundamental” indicator, which controls the overall session, in the top right. The order will not be random, but predictable between sessions and screen sizes. There will be no GUI support for users to reorder the indicators.
Constrained behaviour. All the indicators will take the form of an indicator (icon or text), and a menu. Clicking on an indicator will open its menu. Keyboard navigation will always work, and left and right arrows will translate either into submenu navigation or flipping from indicator to indicator. The whole set of indicators on the panel will be navigable as a single menu, in essence. We won’t support “right click” on indicators differently from “left click”, and there’ll be no ability for arbitrary applications to define arbitrary behaviours to arbitrary events on indicators.
Symbolic visuals. We want to pare back the visual representation of status presented by the icons. We don’t believe that visual accessibility for the disabled need drive the design of the standard icon set, as there will be both alternative icons, renderings, and entirely different options such as speech or custom devices to handle those. Colors on the indicators should have semantic purpose and be used mainly for alerts and awareness, while the shape of the icon should define its purpose.
The first part of our work was pure housekeeping. The panel in Ubuntu is very generic, it lets you put all sorts of different gadgets in all sorts of different places, and those gadgets can behave in all sorts of different ways. The result has been to stimulate innovation, but it has also made the panel very inconsistent and ultimately less useful.
We reviewed the way Ubuntu-specific applications were using the panel, and set out to clean them up. Update-manager lost its persistent notification in favour of the more direct popup window. Others will follow.
We decided to introduce a new gadget on the panel which would be a container for all the indicators which follow our new Ubuntu Ayatana pattern. And we started work on a set of indicators that would fit inside that container. Thus far, we’ve done the session, “me”, and sound indicators.
We also created a framework for applications which want to display their own indicator. That’s the AppIndicators work, which has been fantastically lead in 10.04 LTS by Jorge Castro, coordinating with many upstreams to ensure that their applications feel tightly integrated into the panel.
The icon visual design turned into a conversation about “-symbolic” icons at UDS in Dallas, and is now being realised in the ubuntu-mono icon theme in 10.04 LTS. There is work under way to make symbolic icons a more formal and rigorous construct that can be themed, and we’ll participate in that effort or offer an alternative implementation.
9 parts perspiration, 1 part innovation
The detailed design of a large set of systemic indicators, together with the work to make them all look, feel and behave in a consistent fashion, has been substantial effort involving MPT, Ted Gould, Cody Russell and many others. There’s still a lot of work to do. Conor Curran and Kalle Valo joined the team in this latest cycle. There is a great deal that remains to be done.
We also aspire to introduce some new and innovative concepts.
Category Indicators
In order to reduce the number of indicators and improve the persistence and usefulness of the indicators that remain, we’ve introduced the idea of “category indicators”. These are indicators into which multiple, similar applications can embed themselves. Instead of having a different indicator each application, we have one indicator for the whole category.
The messaging indicator, which aggregates awareness about many different types of messages from real people, is an example. Instead of having three different icons for email, IM and Identi.ca or Twitter, Ubuntu has just one messaging indicator, which can make you aware of important messages in any of those applications.
The three default applications for those lines of communication all share the same indicator. They are part of the same category. There are custom API’s for messaging applications which let them:
Insert entries in the messaging menu which are displayed even when the application is not running. Useful for helping people go straight to the activity. Instead of having to check if the email client is running, then switching to it or launching it, then going to the message composition window, I should *always* be able to compose a new message with just two clicks, regardless of whether or not the mail client is running initially.
Add custom menu entries to the messaging menu that are relevant to them. Each applications gets a “section” in the category indicator menu, and they can add custom menu entries to their section.
Register themselves as applications that should be shown in the messaging menu, or remove themselves from that menu. The default applications will show up there unless they are uninstalled or expressly configured not to use the messaging menu. Other applications will put themselves there by default when they are run by that user, who can also configure them not to display there.
Show whether they are running, a state which is indicated with a small “play” style triangle next to the application icon in the menu.
There are also some behaviours which are collective across all the applications in the category. For example, any of the applications can set the messaging indicator to an alert state, signalling that it’s worth clicking on.
Each category indicator supports a unique API that’s relevant for that category. There are some common features, for example the ability of applications to register and de-register for the indicators and the ability to add menu entries, but the details might vary substantially from one category to another.
The underlying goal is to make it clearer to users “what all of those icons are about”. There are fewer of them, and the ones that are there are more persistent – they are always there, and they always do the same sort of thing. “You’ve got a message” is useful no matter which channel the message came through. The net result is that the whole set of indicators feels tighter and better defined.
The session indicator, which displays the shutdown / restart menu, has a similar capability that replaced the “restart required” panel icon in 10.04 LTS. Since the session menu already contains the “restart” menu option, the session menu will now be set into an alert state when you need to restart. The “Restart…” menu option is changed to “Restart Required…” (though I would now prefer something like “Restart, completing updates…”).
The battery indicator shows the status of all of your batteries, from laptop to UPS to mouse and wireless keyboard. Other applications and devices which have battery information should be able to insert themselves there appropriately.
Similarly, all the calendar and alarm applications might fit into the Clock Indicator. And perhaps all the applications which have downloads might use a single category for that – there’s some discussion along those lines on the Ayatana list at the moment.
Timelines and iterations
The basic “add an indicator with a menu” capability is there now, and was used for Application Indicators in 10.04 LTS.
What complicates the picture from a delivery perspective is our evolving understanding of how best to organise the category indicators. For example, at the moment we are aggregating received messages in the messaging indicator, but the send or broadcast elements of those same communications channels are accessed through the Me menu, where we track presence. That has been controversial – sensible folks think we should perhaps restructure that to bring the elements together.
Each arrangement of category indicators involves shaping the API’s in new ways, because the categories are fundamentally different from one another, and we want to design custom indicators for each category. Not only are the individual category indicator designs changing, but the arrangement of categories themselves is subject to active debate and experimentation, which is important to getting a crisp final result.
We can’t be certain that the current configuration is the best one, and want the flexibility to continue to evolve and reshape the APIs accordingly. We expect it will take at least three iterations of Ubuntu to be certain, and that we can commit to ABI stability for 12.04 LTS onwards.
One of the driving mantras for us is “less is more”. I want us to “clean up, simplify, streamline, focus” the user experience work that we lead. The idea is to recognize the cost of every bit of chrome, every gradient or animation or line or detail or option or gconf setting. It turns out that all of those extras add some value, but they also add clutter. There’s a real cost to them – in attention, in space, in code, in QA. So we’re looking for things to strip out, as much (or more) as things to put in.
I’m not sure we’ll go as far as Microsoft has with their new Windows Phone 7 UI (links to .PPTX), which uses a design language called Metro. It’s radically pared back, and very cool work. It will be interesting to see if they’ve gone too far, or if users take to the more abstract feel of it.
It’s not hard to get people enthusiastic about the idea that less is more. However, it’s quite hard to get people to agree on which bits can be less. It turns out that one person’s clutter is another person’s most useful and valued feature.
Less, it turns out, is still less.
So, for example, consider tooltips on the panel. In bug #527458, there’s some discussion about a decision I made to deprecate tooltips on panel indicators. For quite a lot of people, that’s a little less too far.
On that particular decision, we’ll have to let time tell. For the moment, the decision stands. I’m the first to admit fallibility but I also know that it would be impossible to get consensus around a change like that. If those tooltips are, on balance, really just clutter, then unless someone is willing to take a decision that will be unpopular, they will be clutter forever. And it’s easier for me to make a decision like that in Ubuntu than for virtually anybody else. I apologise in advance for the mistakes that I will certainly make, and which others on the design team may make too, but I think it’s important to defend our willingness to pare things back and let the core, essential goodness shine through. We have to balance innovation and change with clarification and focus. We can’t *stop* innovating and changing, and we have to be willing to remove things that someone will miss.
The bug is a good place to continue the discussion about that particular issue. But I thought it would be useful to issue a call to arms, and invite suggestions from people on the Ayatana list as to what elements of the existing Ubuntu desktop can be trimmed back, on balance making the whole better.
There’s a growing awareness and excitement about the importance of design in free software. A few years ago, folks laughed out loud when it was suggested that design is a good thing for the free software community to build expertise in. And it’s been slow going, admittedly. It’s hard to bring clarity in a crowd. Or mob. We’ve been doing our part to lead that at Canonical and in the Ubuntu community, both through internal work and through public forums. If you’re interested in design and Free Software, then Ubuntu and Ayatana and related forums are great places to participate. And your participation is welcome!
Jono Bacon, Alan Pope, and many others have written, yesterday we published a new visual story and style for Ubuntu. The core design work was lead by Marcus Haslam, Otto Greenslade and Dominic Edmunds, who are the three visual artists leading our efforts in the Canonical Design team. Once we had the base ideas in place we invited some anchor members of the Ubuntu Art community to a design sprint, to test that the concept had the legs to work with the full range of forums, websites, derivatives and other pieces of this huge and wonderful project. And apparently, it does!
Here are some additional thoughts.
Embracing both Ubuntu and Canonical
One of the real challenges for us has been to find a branding and design strategy which spans the spectrum of audiences, forums and dialogues that we cover. With Ubuntu, it’s my specific dream to find a constructive blend of commercial and community interests, not only for Canonical but for other companies. That has made our design and branding work difficult – the distinctive look of Ubuntu lent itself well to pure community messaging, but it was hard to do a brochure for Canonical data center services for Ubuntu on servers. We have not only Ubuntu, but also Kubuntu and an important range of derivatives that all have a role in our ecosystem.
So we spent a lot of time trying to distill the requirements down into a set of three dimensions:
We found a set of ideas which each represent those spectrums, and which work together.
For example, we identified a palette which includes both a fresh, lively Orange, and a rich, mature Aubergine, which work together. The use of Aubergine indicates Commercial involvement of one form or another, while Orange is a signal of community engagement. The Forums will use the Orange elements more strongly, and a formal product brochure, with descriptions of supporting services, would use more of the Aubergine.
On the consumer/enterprise spectrum, we took inspiration from the aerospace industry, and identified a texture of closely spaced dots. When you see more of that, it means we’re signalling that the story is more about the enterprise, less of that, and it’s more about the consumer. Of course, there are cross-overs, for example when we are talking about the corporate desktop, where we’ll use that closely space dot texture as a boundary area, or separator. We also identified shades of Aubergine that are more consumer, or more enterprise – the darker shades mapping to a stronger emphasis on enterprise work.
And on the end-user / engineer spectrum, we took inspiration from graph paper and engineering blue prints. When you see widely spaced patterns of dots, or outline images and figures, that’s signalling that the content is more engineering-oriented than end-user oriented.
And finally, we found a number of themes which enhanced and echoed those ideas. We use a warm gray supporting colour to give shape to pages and documents, and we built on the dots and circles to create a whole style for figures, illustrations and pictograms.
The beauty of this is that we can now publish content that spans the full range, and we generally know when we start the design process what sorts of visual cues we want to be signalling. Instead of having these different mental domains fight with one another, we can now convey quite subtle collaboration between community and corporate, or work which is aimed at engineers and developers from enterprises as opposed to developers working with consumers. Time will tell how it shapes up, but for now I’m celebrating the milestone and the efforts of the team that pulled it together. There’s something there for everyone who wants to participate in the great hubbub of Ubuntuness that is our shared experience of free software.
So, for example, here’s a conference banner. The strong use of Aubergine suggests that it’s more corporate messaging (Canonical is heavily involved). Orange is used here more as a highlight. The Aubergine is darker, and there’s quite a lot of the fine dot pattern. Below the image is a set of scales showing where on those spectra this work is pitched.
As another example, here’s a brochure with an emphasis on end-users who are thinking about adopting Ubuntu’s cloud infrastructure. Again, the fine dot patterns suggests a more enterprise focus, as does the use of the dark aubergine. You can see the circle metaphor used in the quote callout.
And here’s a similar brochure, but with a more developer or engineering oriented focus: note the use of the graph-paper theme with wide spaced dots, and outline shapes.
Finally, here’s an example of a brochure and CD cover for Ubuntu:
As you can see the idea is to signal a mix of both community and Canonical involvement in the message, addressing consumer audiences with a mix of developers and end-users.
A new Ubuntu font
We have commissioned a new font to be developed both for the logo’s of Ubuntu and Canonical, and for use in the interface. The font will be called Ubuntu, and will be a modern humanist font that is optimised for screen legibility. It will be published under an open font license, and considered part of the trade dress of Ubuntu, which will limit its relevance for software interfaces outside of Ubuntu but leave it free for use across the web and in printed documents.
It will take a few months for the font to be finalised, initial elements will be final in the next week which will be sufficient for the logo and other bits and pieces, but I expect to see that font widely used in 10.10. The work has been commissioned from world-renowned fontographers Dalton Maag, who have expressed excitement at the opportunity to publish an open font and also a font that they know will be used daily by millions of people.
Initial coverage will be Western, Arabic, Hebrew and Cyrillic character sets, but over time we may be able to extend that to being a full Unicode font, with great kerning and hinting for print and screen usage globally. We are considering an internship program, to support aspiring fontographers from all corners of the world to visit London and work with Dalton Maag to extend the font to their own regional glyph set.
The critical test of the font is screen efficiency and legibility, and its character and personality are secondary to its fitness for that purpose. Nevertheless, our hope is that the font has a look that is elegant and expresses the full set of values for both Canonical and Ubuntu: adroitness, accountability, precision, reliability, freedom and collaboration. We’ll publish more as soon as we have it.
A good start
It’s been an exciting process, but I have the sense that we are just getting started. The language will get richer, we will find new things that we want to communicate, and new treatments and visual themes that resonate well with these starting points. We’ll find new ways to integrate this on the web, and on the desktop (look out for the two new themes, Radiance and Ambiance). I hope we’ll see the language being used to good effect across everything we do, both commercial and community oriented. There’s a range of expression here that should be useful to artists across the spectrum. Let me know how it works for you.
Thanks to the concerted efforts of Martin Pitt, Sebastien Bacher and several others, notify-osd and several related components landed in Jaunty last week. Thanks very much to all involved! And thanks to David Barth, Mirco Muller and Ted Gould who lead the development of notify-osd and the related messaging indicator.
Notify-OSD handles both application notifications and keyboard special keys like brightness and volume
MPT has posted an overview of the conceptual framework for “attention management” at https://wiki.ubuntu.com/NotificationDesignGuidelines, which puts ephemeral notification into context as just one of several distinct tools that applications can use when they don’t have the focus but need to make users aware of something. That’s a draft, and when it’s at 1.0 we’ll move it to a new site which will host design patterns on Canonical.com.
There is also a detailed specification for our implementation of the notification display agent, notify-osd, which can be found at https://wiki.ubuntu.com/NotifyOSD and which defines not only the expected behaviour of notify-osd but also all of the consequential updates we need to make across the packages in main an universe to ensure that those applications use notification and other techniques consistently.
There are at least 35 apps that need tweaking, and there may well be others! If you find an app that isn’t using notifications elegantly, please add it to the notification design guidelines page, and if you file a bug on the package, please tag it “notifications” so we can track these issues in a single consistent way.
Together with notify-osd, we’ve uploaded a new panel indicator which is used to provide a way to respond to messaging events, such as email and IRC pings. If someone IM’s you, then you should see an ephemeral notification, and the messaging indicator will give you a way to respond immediately. Same for email. Pidgin and Evolution are the primary focuses of the work, over time we’ll broaden that to the full complement of IM and email apps in the archive – patches welcome
There will be rough patches. Apps which don’t comply with the FreeDesktop.org spec and send actions on notifications even when the display agent says it does not support them, will have their notifications translated into alerts. That’s the primary focus of the effort now, the find and fix those apps. Also, we know there are several cases where a persistent response framework is required. The messaging indicator gets most of them, we will have additional persistent tools in place for Karmic in October.
. Another comment was “it works so well that the rare occasions when it can’t read my mind are annoying!”. We’re doing a lot of user testing on heavy multitaskers, developers and all-day-at-the-workstation personas for Unity in 12.04, polishing off loose ends in the experience that frustrated some in this audience in 11.04-10. If that describes you, the results should be delightful. And the HUD should be particularly empowering.
