When once you have tasted flight, you will forever walk the earth with your eyes turned skyward, for there you have been and there you will always long to return.
— Leonardo da Vinci
Multitouch is just as useful on a desktop as it is on a phone or tablet, so I’m delighted that the first cut of Canonical’s UTouch framework has landed in Maverick and will be there for its release on 10.10.10.
You’ll need 4-finger touch or better to get the most out of it, and we’re currently targeting the Dell XT2 as a development environment so the lucky folks with that machine will get the best results today. By release, we expect you’ll be able to use it with a range of devices from major manufacturers, and with addons like Apple’s Magic Trackpad.
The design team has lead the way, developing a “touch language” which goes beyond the work that we’ve seen elsewhere. Rather than single, magic gestures, we’re making it possible for basic gestures to be chained, or composed, into more sophisticated “sentences”. The basic gestures, or primitives, are like individual verbs, and stringing them together allows for richer interactions. It’s not quite the difference between banging rocks together and conducting a symphony orchestra, but it feels like a good step in the right direction
The new underlying code is published on Launchpad under the GPLv3 and LGPLv3, and of course there are quite a lot of modules for things like X and Gtk which may be under licenses preferred by those projects. There’s a PPA if you’re interested in tracking the cutting edge, or just branch / push/ merge on LP if you want to make it better. Details in the official developer announcement. The bits depend on Peter Hutterer’s recently published update to the X input protocols related to multi-touch, and add gesture processing and gesture event delivery. I’d like to thank Duncan McGreggor for his leadership of the team which implemented this design, and of course all the folks who have worked on it so far: Henrik Rydberg, Rafi Rubin, Chase Douglas, Stephen Webb at the heart of it, and many others who have expanded on their efforts.
In Maverick, quite a few Gtk applications will support gesture-based scrolling. We’ll enhance Evince to show some of the richer interactions that developers might want to add to their apps. Window management will be gesture-enabled in Unity, so 10.10 Netbook Edition users with touch screens or multi-touch pads will have sophisticated window management at their fingertips. Install Unity on your desktop for a taste of it, just apt-get install ubuntu-netbook and choose the appropriate session at login.
The roadmap beyond 10.10 will flesh out the app developer API and provide system services related to gesture processing and touch. It would be awesome to have touch-aware versions of all the major apps – browser, email, file management, chat, photo management and media playback – for 11.04, but that depends on you! So if you are interested in this, let’s work up some branches Here’s the official Canonical blog post, too.
The Ayatana Indicators work has given us a crisp, clean basis for indicators in the panel. We’ve said they will all look a particular way, and behave a particular way. And we’ve said they will be placed on the right of the panel.
But why limit indicators to the panel? Let’s make it possible for applications to use indicators themselves, for all the things that indicators are good at:
Conveying a particular state, such as whether or not the application is connected,
Providing a handle for the indicator menu, to modify that state
We’ll start with “window indicators”, or “windicators” for fun. Windicators are indicators displayed in the window title bar that behave just like the indicators in the panel: they have an icon which shows state, and clicking on the icon brings up a menu. Applications can create, update and remove window indicators using an API more or less like the AppIndicator framework first put to use in 10.04 LTS.
Window indicators, or "windicators", shown in a sample application window.
We’ve carefully placed all the panel indicators on the right, and we’ve carefully put the window controls and window title on the left. So now we have all this space on the right. As a pattern, it would fit to put the window indicators there.
Cody Russell is leading some work in Canonical around the technology which actually draws the window title bar and borders. It’s called “client side window decorations”. We are moving the rendering of the window decorations into the app itself, so that you don’t have the window manager and application drawing those pieces separately. That simplifies certain things (of course it also makes some things harder).
One of the most interesting consequences of the client-side decorations work is that it means that the application could more easily draw into the titlebar (because the application is drawing the title bar). And that makes it even more natural for the application to control the right side of the window title bar as well.
Update: Several commenters correctly pointed out that window indicators could just as easily be rendered by window managers in cases where the theme is not CSD-based. CSD provided the inspiration for giving that space to the application, it’s not essential to the implementation. It would be fantastic for window indicators to be available on
Less chrome, more content: banish the status bar
I’m on a “less is more” kick with our design efforts, and one of the things I want to banish is wasted vertical space. For netbooks, that’s particularly important. And a lot of applications have status bars at the bottom, for no good reason other than it was that way in Windows 3.1.
Typically the application status bar has:
Some status icons (“online”)
Some tools (“Yslow”)
A transient status message (“Saving draft…”)
We can replace these with a combination of windicators and temporary, overlay status bars. I really liked the Chrome browser’s use of overlay status messages, so kudos and thanks to them for the inspiration. The net result of those two steps, in apps where we can, is to save about 5% of the vertical space for your stuff – real content.
Prioritising examples for implementation
If you’re interested in this idea, please join the Ayatana mailing list and participate in the design discussions there. We’d like to develop some patterns that are generic, so that we can use a common icon and possibly also common indicator menu entries for addressing the same issue in diverse applications. Of course, applications will be free to use the mechanism for things that are unique to them.
Candidates for 10.10
It would be fantastic to implement a few of these window indicators for 10.10. Please help us choose the most useful cases! Currently on the list are:
Online / offline status indicator and toggle options for the mail client, chat program or Gwibber, the broadcast messages application.
An “unsaved” indicator, that tells people that the contents of the file they are working on have changed and potentially lets them save it or set autosave properties.
Progress indicators, which show that an action is in progress, and possibly also indicate the extent of the progress. The associated menu would enable one to pause or cancel the operation, and perhaps define the behaviour on completion of the action.
A “basket” indicator, which shows if any items have been selected for purchase,
Sharing indicators, which would show if a document is shared with multiple people, and enable one to setup such a share.
Volume indicators, which would show the loudness of application audio streams, and enable one to set the volume for that specific application.
The key thing is that these indicators are entirely application-specific, and ideally only relevant to the window that you are actually looking at.
Just like Panel Indicators…
From a visual design point of view, again the goal would be to ensure that indicators are symbolic. They would follow the same styling as Ayatana indicators:
Monochrome by default, with shape indicating the function of the indicator
Semantically colored: with red for critical problems, orange for alerts, green for positive status changes and blue for informative states that are not the default or usual state.
Integrated with the Netbook Edition Smart Panel
Last week I blogged about our decision to adopt a single, global menu for all applications, in the panel. And I also said we would explore putting the window title *and* menu into the panel, when the window is maximised. Of course, that means that we need to accommodate the window indicators in the panel as well.
So: when the window is maximised, and we are using a smart which can include both indicators and window titles, the window indicators will be inserted into the panel as well. They will appear on the right of the panel, and be the leftmost indicators. For example, here is the application, maximised (note the dodgy Ubuntu logo in the top left – that’s the panel, not the window title bar you’re looking at):
In this configuration, the system achieves “singular purpose”: the entire screen is devoted to a single application, yet the Ayatana elements continue to serve their purpose, either systemic (the battery indicator) or application specific.
When we set up Project Ayatana to improve the usability of the whole desktop, we called it Ayatana because we were focused on the “sphere of consciousness”, one’s awareness of what’s going on outside of the current application. There are two key aspects to the work:
Notifications are “awareness distilled” in the sense that you cannot interact with them at all. We designed them as ephemeral “click-transparent” messages, implemented in Notify-OSD. Their sole purpose is to notify you of transient events.
Indicator Menus combine persistent awareness of a state with a set of options for modifying that state.
In this blog I’ll outline the arc of our work on indicator menus to date, and the trajectory we expect it to follow. We’re about a year into the effort, all told, and I think it will take another 18 months before we can consider it baked. It should be done by 12.04 LTS. This is an iterative process, and things are in flux right now. I hope, when we are happy that we can commit to ABI stability, that Gnome and KDE will adopt the work too. For the moment, the rapid pace of evolution has meant that we’re depending on fantastic upstreams to keep up with us as things move.
Goals of the Ayatana Indicators
The indicators are designed to create a persistent awareness of state, or an awareness of a persistent state. They complement notifications: they are persistent, when notifications are ephemeral. You might miss a notification, but you should always be able to check your indicators. You can interact with indicators, using their menus, in contrast with the un-clickable notifications.
We value:
Support for both GNOME and KDE. Both desktop environments are important in Ubuntu. We encourage the teams to reflect a pure vision of each, but it’s also the case that users will want to run a GNOME application on Kubuntu occasionally, or vice versa. So we have to make sure the work is considered from the perspectives of developers on either side, and we have to provide APIs and libraries that work in both environments.
Accessibility. Indicators are critical elements of awareness. Whether you are connected, what the time is, whether you are online, whether your battery will last long enough for you to finish your work, whether you have messages… these are all vital to a complete computing experience. We have to make sure that visual and other disabilities can be addressed.
Familiarity and Innovation. As always, these are in tension with one another. Innovation helps us put free software at the front of the curve, but it creates the risk of breaking people’s habits and expectations.
Consistency and Usability. We want the end result to be more usable in the whole, and we are willing to lose individual nuggets if that helps make the whole more valuable.
Streamlining. There are too many indicators, that aren’t clear enough about their intent. There are also many indicators from different applications which do roughly the same thing, but in slightly different ways. The value of all the indicators is enhanced if there are fewer of them, and they are more obvious to discover and use.
Some firm decisions
Those values lead us to some anchor decisions:
D-Bus for communications. A messaging approach makes it straightforward to adopt consistent patterns across different desktop environments. We will provide wrapper libraries for both Gtk and Qt applications to access the indicator capabilities. A Qt application running on Ubuntu should “feel native” when it’s using indicators correctly. And vice-versa. The messaging approach also lets us handle accessibility in a better way: we don’t have to accommodate every possible disability visually, because we can have agents that interpret the indicator messages and handle it in ways that are appropriate for a particular disability.
Opinionated placement. We will place all indicators at the top right of the screen on GNOME. We’ll place them in a particular order, too, with the “most fundamental” indicator, which controls the overall session, in the top right. The order will not be random, but predictable between sessions and screen sizes. There will be no GUI support for users to reorder the indicators.
Constrained behaviour. All the indicators will take the form of an indicator (icon or text), and a menu. Clicking on an indicator will open its menu. Keyboard navigation will always work, and left and right arrows will translate either into submenu navigation or flipping from indicator to indicator. The whole set of indicators on the panel will be navigable as a single menu, in essence. We won’t support “right click” on indicators differently from “left click”, and there’ll be no ability for arbitrary applications to define arbitrary behaviours to arbitrary events on indicators.
Symbolic visuals. We want to pare back the visual representation of status presented by the icons. We don’t believe that visual accessibility for the disabled need drive the design of the standard icon set, as there will be both alternative icons, renderings, and entirely different options such as speech or custom devices to handle those. Colors on the indicators should have semantic purpose and be used mainly for alerts and awareness, while the shape of the icon should define its purpose.
The first part of our work was pure housekeeping. The panel in Ubuntu is very generic, it lets you put all sorts of different gadgets in all sorts of different places, and those gadgets can behave in all sorts of different ways. The result has been to stimulate innovation, but it has also made the panel very inconsistent and ultimately less useful.
We reviewed the way Ubuntu-specific applications were using the panel, and set out to clean them up. Update-manager lost its persistent notification in favour of the more direct popup window. Others will follow.
We decided to introduce a new gadget on the panel which would be a container for all the indicators which follow our new Ubuntu Ayatana pattern. And we started work on a set of indicators that would fit inside that container. Thus far, we’ve done the session, “me”, and sound indicators.
We also created a framework for applications which want to display their own indicator. That’s the AppIndicators work, which has been fantastically lead in 10.04 LTS by Jorge Castro, coordinating with many upstreams to ensure that their applications feel tightly integrated into the panel.
The icon visual design turned into a conversation about “-symbolic” icons at UDS in Dallas, and is now being realised in the ubuntu-mono icon theme in 10.04 LTS. There is work under way to make symbolic icons a more formal and rigorous construct that can be themed, and we’ll participate in that effort or offer an alternative implementation.
9 parts perspiration, 1 part innovation
The detailed design of a large set of systemic indicators, together with the work to make them all look, feel and behave in a consistent fashion, has been substantial effort involving MPT, Ted Gould, Cody Russell and many others. There’s still a lot of work to do. Conor Curran and Kalle Valo joined the team in this latest cycle. There is a great deal that remains to be done.
We also aspire to introduce some new and innovative concepts.
Category Indicators
In order to reduce the number of indicators and improve the persistence and usefulness of the indicators that remain, we’ve introduced the idea of “category indicators”. These are indicators into which multiple, similar applications can embed themselves. Instead of having a different indicator each application, we have one indicator for the whole category.
The messaging indicator, which aggregates awareness about many different types of messages from real people, is an example. Instead of having three different icons for email, IM and Identi.ca or Twitter, Ubuntu has just one messaging indicator, which can make you aware of important messages in any of those applications.
The three default applications for those lines of communication all share the same indicator. They are part of the same category. There are custom API’s for messaging applications which let them:
Insert entries in the messaging menu which are displayed even when the application is not running. Useful for helping people go straight to the activity. Instead of having to check if the email client is running, then switching to it or launching it, then going to the message composition window, I should *always* be able to compose a new message with just two clicks, regardless of whether or not the mail client is running initially.
Add custom menu entries to the messaging menu that are relevant to them. Each applications gets a “section” in the category indicator menu, and they can add custom menu entries to their section.
Register themselves as applications that should be shown in the messaging menu, or remove themselves from that menu. The default applications will show up there unless they are uninstalled or expressly configured not to use the messaging menu. Other applications will put themselves there by default when they are run by that user, who can also configure them not to display there.
Show whether they are running, a state which is indicated with a small “play” style triangle next to the application icon in the menu.
There are also some behaviours which are collective across all the applications in the category. For example, any of the applications can set the messaging indicator to an alert state, signalling that it’s worth clicking on.
Each category indicator supports a unique API that’s relevant for that category. There are some common features, for example the ability of applications to register and de-register for the indicators and the ability to add menu entries, but the details might vary substantially from one category to another.
The underlying goal is to make it clearer to users “what all of those icons are about”. There are fewer of them, and the ones that are there are more persistent – they are always there, and they always do the same sort of thing. “You’ve got a message” is useful no matter which channel the message came through. The net result is that the whole set of indicators feels tighter and better defined.
The session indicator, which displays the shutdown / restart menu, has a similar capability that replaced the “restart required” panel icon in 10.04 LTS. Since the session menu already contains the “restart” menu option, the session menu will now be set into an alert state when you need to restart. The “Restart…” menu option is changed to “Restart Required…” (though I would now prefer something like “Restart, completing updates…”).
The battery indicator shows the status of all of your batteries, from laptop to UPS to mouse and wireless keyboard. Other applications and devices which have battery information should be able to insert themselves there appropriately.
Similarly, all the calendar and alarm applications might fit into the Clock Indicator. And perhaps all the applications which have downloads might use a single category for that – there’s some discussion along those lines on the Ayatana list at the moment.
Timelines and iterations
The basic “add an indicator with a menu” capability is there now, and was used for Application Indicators in 10.04 LTS.
What complicates the picture from a delivery perspective is our evolving understanding of how best to organise the category indicators. For example, at the moment we are aggregating received messages in the messaging indicator, but the send or broadcast elements of those same communications channels are accessed through the Me menu, where we track presence. That has been controversial – sensible folks think we should perhaps restructure that to bring the elements together.
Each arrangement of category indicators involves shaping the API’s in new ways, because the categories are fundamentally different from one another, and we want to design custom indicators for each category. Not only are the individual category indicator designs changing, but the arrangement of categories themselves is subject to active debate and experimentation, which is important to getting a crisp final result.
We can’t be certain that the current configuration is the best one, and want the flexibility to continue to evolve and reshape the APIs accordingly. We expect it will take at least three iterations of Ubuntu to be certain, and that we can commit to ABI stability for 12.04 LTS onwards.
One of the driving mantras for us is “less is more”. I want us to “clean up, simplify, streamline, focus” the user experience work that we lead. The idea is to recognize the cost of every bit of chrome, every gradient or animation or line or detail or option or gconf setting. It turns out that all of those extras add some value, but they also add clutter. There’s a real cost to them – in attention, in space, in code, in QA. So we’re looking for things to strip out, as much (or more) as things to put in.
I’m not sure we’ll go as far as Microsoft has with their new Windows Phone 7 UI (links to .PPTX), which uses a design language called Metro. It’s radically pared back, and very cool work. It will be interesting to see if they’ve gone too far, or if users take to the more abstract feel of it.
It’s not hard to get people enthusiastic about the idea that less is more. However, it’s quite hard to get people to agree on which bits can be less. It turns out that one person’s clutter is another person’s most useful and valued feature.
Less, it turns out, is still less.
So, for example, consider tooltips on the panel. In bug #527458, there’s some discussion about a decision I made to deprecate tooltips on panel indicators. For quite a lot of people, that’s a little less too far.
On that particular decision, we’ll have to let time tell. For the moment, the decision stands. I’m the first to admit fallibility but I also know that it would be impossible to get consensus around a change like that. If those tooltips are, on balance, really just clutter, then unless someone is willing to take a decision that will be unpopular, they will be clutter forever. And it’s easier for me to make a decision like that in Ubuntu than for virtually anybody else. I apologise in advance for the mistakes that I will certainly make, and which others on the design team may make too, but I think it’s important to defend our willingness to pare things back and let the core, essential goodness shine through. We have to balance innovation and change with clarification and focus. We can’t *stop* innovating and changing, and we have to be willing to remove things that someone will miss.
The bug is a good place to continue the discussion about that particular issue. But I thought it would be useful to issue a call to arms, and invite suggestions from people on the Ayatana list as to what elements of the existing Ubuntu desktop can be trimmed back, on balance making the whole better.
There’s a growing awareness and excitement about the importance of design in free software. A few years ago, folks laughed out loud when it was suggested that design is a good thing for the free software community to build expertise in. And it’s been slow going, admittedly. It’s hard to bring clarity in a crowd. Or mob. We’ve been doing our part to lead that at Canonical and in the Ubuntu community, both through internal work and through public forums. If you’re interested in design and Free Software, then Ubuntu and Ayatana and related forums are great places to participate. And your participation is welcome!
Jono Bacon, Alan Pope, and many others have written, yesterday we published a new visual story and style for Ubuntu. The core design work was lead by Marcus Haslam, Otto Greenslade and Dominic Edmunds, who are the three visual artists leading our efforts in the Canonical Design team. Once we had the base ideas in place we invited some anchor members of the Ubuntu Art community to a design sprint, to test that the concept had the legs to work with the full range of forums, websites, derivatives and other pieces of this huge and wonderful project. And apparently, it does!
Here are some additional thoughts.
Embracing both Ubuntu and Canonical
One of the real challenges for us has been to find a branding and design strategy which spans the spectrum of audiences, forums and dialogues that we cover. With Ubuntu, it’s my specific dream to find a constructive blend of commercial and community interests, not only for Canonical but for other companies. That has made our design and branding work difficult – the distinctive look of Ubuntu lent itself well to pure community messaging, but it was hard to do a brochure for Canonical data center services for Ubuntu on servers. We have not only Ubuntu, but also Kubuntu and an important range of derivatives that all have a role in our ecosystem.
So we spent a lot of time trying to distill the requirements down into a set of three dimensions:
We found a set of ideas which each represent those spectrums, and which work together.
For example, we identified a palette which includes both a fresh, lively Orange, and a rich, mature Aubergine, which work together. The use of Aubergine indicates Commercial involvement of one form or another, while Orange is a signal of community engagement. The Forums will use the Orange elements more strongly, and a formal product brochure, with descriptions of supporting services, would use more of the Aubergine.
On the consumer/enterprise spectrum, we took inspiration from the aerospace industry, and identified a texture of closely spaced dots. When you see more of that, it means we’re signalling that the story is more about the enterprise, less of that, and it’s more about the consumer. Of course, there are cross-overs, for example when we are talking about the corporate desktop, where we’ll use that closely space dot texture as a boundary area, or separator. We also identified shades of Aubergine that are more consumer, or more enterprise – the darker shades mapping to a stronger emphasis on enterprise work.
And on the end-user / engineer spectrum, we took inspiration from graph paper and engineering blue prints. When you see widely spaced patterns of dots, or outline images and figures, that’s signalling that the content is more engineering-oriented than end-user oriented.
And finally, we found a number of themes which enhanced and echoed those ideas. We use a warm gray supporting colour to give shape to pages and documents, and we built on the dots and circles to create a whole style for figures, illustrations and pictograms.
The beauty of this is that we can now publish content that spans the full range, and we generally know when we start the design process what sorts of visual cues we want to be signalling. Instead of having these different mental domains fight with one another, we can now convey quite subtle collaboration between community and corporate, or work which is aimed at engineers and developers from enterprises as opposed to developers working with consumers. Time will tell how it shapes up, but for now I’m celebrating the milestone and the efforts of the team that pulled it together. There’s something there for everyone who wants to participate in the great hubbub of Ubuntuness that is our shared experience of free software.
So, for example, here’s a conference banner. The strong use of Aubergine suggests that it’s more corporate messaging (Canonical is heavily involved). Orange is used here more as a highlight. The Aubergine is darker, and there’s quite a lot of the fine dot pattern. Below the image is a set of scales showing where on those spectra this work is pitched.
As another example, here’s a brochure with an emphasis on end-users who are thinking about adopting Ubuntu’s cloud infrastructure. Again, the fine dot patterns suggests a more enterprise focus, as does the use of the dark aubergine. You can see the circle metaphor used in the quote callout.
And here’s a similar brochure, but with a more developer or engineering oriented focus: note the use of the graph-paper theme with wide spaced dots, and outline shapes.
Finally, here’s an example of a brochure and CD cover for Ubuntu:
As you can see the idea is to signal a mix of both community and Canonical involvement in the message, addressing consumer audiences with a mix of developers and end-users.
A new Ubuntu font
We have commissioned a new font to be developed both for the logo’s of Ubuntu and Canonical, and for use in the interface. The font will be called Ubuntu, and will be a modern humanist font that is optimised for screen legibility. It will be published under an open font license, and considered part of the trade dress of Ubuntu, which will limit its relevance for software interfaces outside of Ubuntu but leave it free for use across the web and in printed documents.
It will take a few months for the font to be finalised, initial elements will be final in the next week which will be sufficient for the logo and other bits and pieces, but I expect to see that font widely used in 10.10. The work has been commissioned from world-renowned fontographers Dalton Maag, who have expressed excitement at the opportunity to publish an open font and also a font that they know will be used daily by millions of people.
Initial coverage will be Western, Arabic, Hebrew and Cyrillic character sets, but over time we may be able to extend that to being a full Unicode font, with great kerning and hinting for print and screen usage globally. We are considering an internship program, to support aspiring fontographers from all corners of the world to visit London and work with Dalton Maag to extend the font to their own regional glyph set.
The critical test of the font is screen efficiency and legibility, and its character and personality are secondary to its fitness for that purpose. Nevertheless, our hope is that the font has a look that is elegant and expresses the full set of values for both Canonical and Ubuntu: adroitness, accountability, precision, reliability, freedom and collaboration. We’ll publish more as soon as we have it.
A good start
It’s been an exciting process, but I have the sense that we are just getting started. The language will get richer, we will find new things that we want to communicate, and new treatments and visual themes that resonate well with these starting points. We’ll find new ways to integrate this on the web, and on the desktop (look out for the two new themes, Radiance and Ambiance). I hope we’ll see the language being used to good effect across everything we do, both commercial and community oriented. There’s a range of expression here that should be useful to artists across the spectrum. Let me know how it works for you.
Thanks to the concerted efforts of Martin Pitt, Sebastien Bacher and several others, notify-osd and several related components landed in Jaunty last week. Thanks very much to all involved! And thanks to David Barth, Mirco Muller and Ted Gould who lead the development of notify-osd and the related messaging indicator.
Notify-OSD handles both application notifications and keyboard special keys like brightness and volume
MPT has posted an overview of the conceptual framework for “attention management” at https://wiki.ubuntu.com/NotificationDesignGuidelines, which puts ephemeral notification into context as just one of several distinct tools that applications can use when they don’t have the focus but need to make users aware of something. That’s a draft, and when it’s at 1.0 we’ll move it to a new site which will host design patterns on Canonical.com.
There is also a detailed specification for our implementation of the notification display agent, notify-osd, which can be found at https://wiki.ubuntu.com/NotifyOSD and which defines not only the expected behaviour of notify-osd but also all of the consequential updates we need to make across the packages in main an universe to ensure that those applications use notification and other techniques consistently.
There are at least 35 apps that need tweaking, and there may well be others! If you find an app that isn’t using notifications elegantly, please add it to the notification design guidelines page, and if you file a bug on the package, please tag it “notifications” so we can track these issues in a single consistent way.
Together with notify-osd, we’ve uploaded a new panel indicator which is used to provide a way to respond to messaging events, such as email and IRC pings. If someone IM’s you, then you should see an ephemeral notification, and the messaging indicator will give you a way to respond immediately. Same for email. Pidgin and Evolution are the primary focuses of the work, over time we’ll broaden that to the full complement of IM and email apps in the archive – patches welcome
There will be rough patches. Apps which don’t comply with the FreeDesktop.org spec and send actions on notifications even when the display agent says it does not support them, will have their notifications translated into alerts. That’s the primary focus of the effort now, the find and fix those apps. Also, we know there are several cases where a persistent response framework is required. The messaging indicator gets most of them, we will have additional persistent tools in place for Karmic in October.
Let’s talk about notifications! As Ryan Lortie mentioned, there was a lot of discussion across the Ubuntu, Kubuntu, GNOME, KDE and Mozilla communities represented at UDS about the proposals Canonical’s user experience design and desktop experience engineering teams have made for Ubuntu 9.04.
There are some fairly bold (read: controversial) ideas that we’d like to explore with, so the opportunity to discuss them with a broader cross-section of the community was fantastic. There were several rough edges and traps that I think we’ll avoid in the first round as a result, thanks to everyone who participated. Some of the things we work on in these teams are done directly with partners for their devices, so they don’t see this level of discussion before they ship, but it’s wonderful when we do get the opportunity to do so.
Some of these ideas are unproven, they boil down to matters of opinion, but since our commitment to them is based on a desire to learn more I think of them as constructive experiments. Experiments are just that – experiments. They may succeed and they may fail. We should judge them carefully, after we have data. We are putting new ideas into the free desktop without ego. We know those ideas could be better or worse than similar work being done in other communities, and we want to gather real user feedback to help find the best mix for everyone. The best ideas, and the best code, will ultimately form part of the digital free software commons and be shared by GNOME, KDE and every distribution. So, for those folks who were upset that we might ship something other than a GNOME or KDE default, I would ask for your patience and support – we want to contribute new ideas and new code, and that means having some delta which can be used as a basis for discussions about the future direction of upstream. In the past, we’ve had a few such delta’s in Ubuntu. Some, like the current panel layout, were widely embraced. Others, like the infamous “Ubuntu spacial mode”, were not. C’est la vie, and we all benefit from the evolution.
Experiments are also not something we should do lightly. The Ubuntu desktop is something I take very personally; I feel personally responsible for the productivity and happiness of every Ubuntu user, so when we bring new ideas and code to the desktop I believe we should do everything we can to make sure of success first time round. We should not inflict bad ideas on our users just because we’re curious or arrogant or stubborn or proud. Despite being occasionally curious, arrogant, stubborn and proud
So, what are we proposing?
First, we are focusing some attention on desktop notifications in this cycle, as part of a broader interest in the “space between applications”.
I think Canonical and Ubuntu can best help the cause of free software by focusing on the cracks between the major components of the desktop. In other words, while there are already great upstreams for individual applications in the free software desktop (Novell for Evolution, Sun for OpenOffice, Mozilla for Firefox, Red Hat for NetworkManager), we think there is a lot of productive and useful work to be done in the gaps between them. Notifications are things that many apps do, and if we can contribute new ideas there then we are helping improve the user experience of all of those applications. That’s a nice force multiplier – we’re hopefully doing work that makes the work of every other community even more valuable.
Nevertheless, expect bumps ahead. Ideas we are exploring may / will / do conflict with assumptions that are present today in various applications. We can address the relevant code in packages in main, but I’m more focused on addressing the potential social disruption that conflict can create, and that’s more a matter of conversation than code.
Notifications are interesting, subtle and complex. There are lots of different approaches on lots of different platforms. There are lots of different use cases. We’re trying to simplify and eliminate complexity, while still making it possible to meet the use cases we know about.
There has been good work in the freedesktop.org community on notifications, and even a spec that is *almost* at 1.0 in that community, with existing open source implementations. Our proposal is based on that specification, but it deprecates several capabilities and features in it. We will likely be compatible with the current API’s for sending notifications, but likely will not display all the notifications that might be sent, if they require features that we deprecate. If this experiment goes well, we would hope to help move that FD.o specification to 1.0, with or without our amendments.
The key proposals we are making are that:
There should be no actions on notifications.
Notifications should not be displayed synchronously, but may be queued. Our implementation of the notification display daemon will display only one notification at a time, others may do it differently.
That’s pretty much it. There are some subtleties and variations, but these are the key changes we are proposing, and which we will explore in a netbook device with a partner, as well as in the general Ubuntu 9.04 release, schedule gods being willing.
This work will show up as a new notification display agent, not as a fork or patch to the existing GNOME notification daemon. We don’t think the client API – libnotify – needs to be changed for this experiment, though we may not display notifications sent through that API that use capabilities we are suggesting be deprecated. We will try to ensure that packages in main are appropriately tuned, and hope MOTU will identify and update key packages in universe accordingly.
Why a completely new notification display agent? We are designing it to be built with Qt on KDE, and Gtk on GNOME. The idea is to have as much code in common as we can, but still take advantage of the appropriate text display framework on Ubuntu and Kubuntu. We hope to deliver both simultaneously, and have discussed this with both Ubuntu and Kubuntu community members. At the moment, there is some disagreement about the status of the FD.o specification between GNOME and KDE, and we hope our efforts will help build a bridge there. In Ubuntu 9.04, we would likely continue to package and publish the existing notification daemon in addition, to offer both options for users that have a particular preference. In general, where we invest in experimental new work, we plan to continue to offer a standard GNOME or KDE component / package set in the archive so that people can enjoy that experience too.
The most controversial part of the proposal is the idea that notifications should not have actions associated with them. In other words, no buttons, sliders, links, or even a dismissal [x]. When a notification pops up, you won’t be able to click on it, you won’t be able to make it go away, you won’t be able to follow it to another window, or to a web page. Are you loving this freedom? Hmmm? Madness, on the face of it, but there is method in this madness.
Our hypothesis is that the existence of ANY action creates a weighty obligation to act, or to THINK ABOUT ACTING. That make notifications turn from play into work. That makes them heavy responsibilities. That makes them an interruption, not a notification. And interruptions are a bag of hurt when you have things to do.
So, we have a three-prong line of attack.
We want to make notifications truly ephemeral. They are there, and then they are gone, and that’s life. If you are at your desktop when a notification comes by, you will sense it, and if you want you can LOOK at it, and it will be beautiful and clear and easy to parse. If you want to ignore it, you can safely do that and it will always go away without you having to dismiss it. If you miss it, that’s OK. Notifications are only for things which you can safely ignore or miss out on. If you went out for coffee and a notification flew by, you are no worse off. They don’t pile up like email, there is no journal of the ones you missed, you can’t scroll back and see them again, and therefor you are under no obligation to do so – they can’t become work while you are already busy with something else. They are gone like a mystery girl on the bus you didn’t get on, and they enrich your life in exactly the same way!
We think there should be persistent panel indicators for things which you really need to know about, even if you missed the notification because you urgently wanted that coffee. So we are making a list of those things, and plan to implement them.
Everything else should be dealt with by having a window call for attention, while staying in the background, unless it’s critical in which case that window could come to the foreground.
Since this is clearly the work of several releases, we may have glitches and inconsistencies along the way at interim checkpoints. I hope not, but it’s not unlikely, especially in the first iteration. Also, these ideas may turn out to be poor, and we should be ready to adjust our course based on feedback once we have an implementation in the wild.
We had a superb UXD and DEE (user experience design team, and desktop experience engineering team) sprint in San Francisco the week before UDS. Thanks to everyone who took part, especially those who came in from other teams. This notifications work may just be the tip of the iceberg, but it’s a very cool tip
One or more of our early-access OEM partners (companies that we work with on new desktop features) will likely ship this feature as part of a netbook product during the 9.04 cycle. At that point, we would also drop the code into a PPA for testing with a wider set of applications. There are active discussions about updating the freedesktop.org specification based on this work. I think we should be cautious, and gather real user testing feedback and hard data, but if it goes well then we would propose simplifying the spec accordingly, and submit our notification display agent to FreeDesktop.org. Long term collaboration around the code would take place on Launchpad.
The user experience and design team at Canonical includes a few folks dedicated to web technology. At the moment, there is a substantial effort under way to reshape the Launchpad UI now that we have the core capabilities for cross-project bug tracking, code publishing and translation in place. We want to make it more obvious how to get something done – especially for new users – and we want to make it feel snappy and responsive when making small changes to your project data.
In the design discussions, we spent a lot of time working on a new approach to “dialog boxes, wizards and workflows”, trying to solve a thorny problem in user interaction: how do you make it easy to do something complex? There are lots of cases in Launchpad where you need to get lots of ducks in a row before you can do something. For example, you might need to make sure there is a team with specific people in it before you subscribe that team to a bug. Or you might need to create a new milestone while triaging and scheduling work on bugs in your project.
Currently, that means jumping all around Launchpad in a way that assumes you know exactly how those pieces work. You need to go to one place to register a team, and a completely different place to setup a milestone. That means that lots of people don’t use capabilities in Launchpad, because they need to understand the whole system before they can get something small done. Every time someone bumps their head on that, we fail! And that’s the problem we set out to solve.
We came up with a nifty approach, which we call morphing dialogs, that ensures the user always has the minimum number of choices to make, and still allows for complex variations on a process in a way that feels quite natural for users.
The key ideas behind morphing dialogs are:
Only show one primary decision at a time, and make it obvious what that is. Sometimes, there are several directions you could take in order to get something done, but there is usually a single normal path for users to follow, and we always want users to be able to do the easy things easily.
Give users a sense of how far they are in the process, but don’t be too dogmatic about that, since getting one thing done often involves stepping off to the side to take care of preliminary business and those detours can also require several steps.
Here’s an example movie, which shows a person linking a blueprint to a bug. They need to search for the right blueprint, which they can do across a couple of projects simultaneously. In this mockup, they add GNOME to the list of projects that they look for the blueprint in, and when they can’t find it, they go to register a new blueprint for what they want. In the end he decides to go back and pick one from the search results. None of this involved a page load, and the round trips to the server are much cheaper than loading full pages, since we can just get what we need in highly optimized way.
You can see a couple of the key ideas coming through in the movie.
Note the “progress bar” – the green line – is not particular large or obtrusive. It’s also not obviously a progress bar, until one has done a few multi-step processes. Note also that you can have detours; you can step off to one side to get something done, like register a team or register a new blueprint, and those detours get their own progress indicator which is separate from the main one.
We had a major sprint recently that brought the whole Launchpad team together for two weeks while we did a deep dive into JavaScript and AJAX. We picked YUI 3, the next version of Yahoo’s UI toolkit for the web, as a foundational layer for this AJAX effort, and we wanted to bring everyone up to speed on the processes for designing, building and testing web client apps. It was a lot of fun.
In particular, we wanted to unify the web service API’s that we already publish with this AJAX work, so that it would be easy to write web browser code that could talk to the exact same API’s we publish for developers who are integrating with Launchpad. That’s now possible, which means that any API we use for AJAX work will also be available to developers writing their own tools to access Launchpad directly through the web services.
Thanks to the awesomeness of YUI 3, the team is now hard at work turning those ideas into reality. Given that YUI 3 is right on the cutting edge (some would say bleeding edge!) we’re focusing on pieces that don’t depend on complex widgets – those will only start to fall into place next year as YUI 3 emerges from development.
Over the next couple of months you will see pieces of this puzzle land in successive Launchpad monthly releases (or daily, if you’re on edge.launchpad.net and a beta tester). Initially, the AJAX bling will just enable inline editing. In six to nine months, the more complex pieces should have land. And by then Launchpad’s web front-end will also be open source.
With Intrepid on track to hit the wires today I thought I’d blog a little on the process we followed in designing the new user switcher, presence manager and session management experience, and lessons learned along the way. Ted has been blogging about the work he did, and it’s been mentioned in a couple of different forums (briefly earning the memorable title “the new hotness”), but since it’s one of the first pieces of work to go through the user experience design process within Canonical I thought it would be interesting to write it up.
Here is a screenshot of the work itself in action:
New FUSA applet allows you to mange your presence setting, as well as switch to a guest or other user, and logout
In one of the first user experience sessions, we looked in more detail at the way people “stop working”. We thought it interesting to try and group those actions together in a way which would feel natural to users.
We have already done some work in Ubuntu around this – for a long time we have had a button in the top-right corner of the panel which brought up a system modal dialog that gave you the usual “end your session” options of logout, restart, shutdown, hibernate, suspend and switch user. That patch was always a bit controversial and had not been accepted upstream, so we looked at ways to solve the problem differently.
We decided to use the top-right location, because it’s one of the key places in the screen that’s quick and easy to get to (you can throw your mouse into a corner of the screen very easily and accurately) and because there was a strong precedent in the old Ubuntu logout button.
One key insight was that we wanted to make “switching user” less an exercise in guesswork and more direct – we wanted to let people switch directly to the specific user they were interested in rather than have an intermediate step where they login as that other user. So we started with the Fast User Switcher applet, or FUSA, as a base fr the design. Another key idea that emerged was that we wanted to integrate the “presence setting” into the same menu, because “going offline” or “I’m busy” are similar state-of-mind-and-work decisions to “log me off the system” or “shut down”.
Menu order
We discussed at length the right order for the menu items. On the one hand, putting the “other users” at the top of the menu would mean that all the user names – yours and the ones you can switch to – would appear “in the same place” at the top of the menu. On the other, we strongly felt that things that would be used more casually and more easily should be at the top. In the end we settled on putting the presence management options at the top (Available, Away, Busy, Offline). Right next to those (in the same set) we put the “Lock screen” option, because it feels like a presence setting more than a session management setting – you are saying “Away” more than anything else.
Ted did a lot of work to make the presence menu elements work with both Pidgin and Empathy because there was some uncertainty as to which would be used by default in the release. Since it all uses dbus, it should be straightforward to make it work with KDE IM clients too.
We then put the user switching options – including the Guest Session which is a cool new feature in Intrepid that as been widely blogged (check out the YouTube demo) and which uses AppArmor to enforce security.
And finally, the session termination options – log out, suspend, hibernate, restart and shutdown are at the bottom of the menu, because you’re only ever likely to use them once in a session, by definition!
Styling The design of the menu is deliberately clean. We use very simple colours and shapes for the presence indicators, and replicate those colours and shapes in the actual GNOME panel so that you can see at a glance what your current presence setting is. Ted had to jump through some hoops, I think, to get the presence icons in the menu to line up with the current-presence-status indicator in the panel applet, but it worked out quite nicely. There’s some additional work to tighten up the layout which didn’t make it in time for the release but which might come in as a stable release update (SRU) or in Jaunty.
We decided not to put icons into the menu for each of the different statuses. Our design ethic is to aim for cleaner, less cluttered layouts with fewer icons and better choice of text. A couple of people have said that the menu looks “sparse” or “bare” but I think it sets the right direction and we’ll be continuing with this approach as we touch other parts of the system.
Upstream This work was discussed at UDS in Prague with a number of members of the GNOME community. I was also very glad to see that there’s a lot of support for a tighter, simpler panel at the GNOME hackfest, an idea that we’ve championed. The FUSA applet itself is going through a bit of a transformation upstream as it’s been merged into the new GDM codebase and the old code – on which our work is based – is more or less EOL’d. But we’ll figure out how to update this work for Jaunty and hopefully it will be easier to get it upstreamed at that point.
In Jaunty, we’ll likely do some more work on the GNOME panel, building on the GNOME user experience discussions. There was a lot of discussion about locking down the panel more tightly, which we may pursue.
Integration into Ubuntu
We realised rather late in the Ubuntu cycle that we hadn’t thought much about packaging. The Ubuntu team had kindly offered to help package and integrate the applet but we definitely learned the value of getting the packaging done earlier rather than later. We had the applet in a PPA for testing between developers fairly early, but we underestimated the difference between that and actual integration into the release.
The Ubuntu team rallied to the cause and helped to smooth the upgrade process for new users, so that we can try to get everyone onto the same footing when they start out with Intrepid whether as a new install or an upgrade. There are some challenges there, because the panel is so customisable, and we had to think hard about how we could ensure there was a consistent experience for something as important as logging out or shutting down while at the same time trying not to stomp on the preferences of folks who have customised their panels. Similarly, we were concerned that people who run different versions of Ubuntu, or different distributions entirely, with the same home directory, would have problems if those other OS’s didn’t have the same version of FUSA – we weren’t really able to address that satisfactorily.
We also realised (DOH!) that we hadn’t thought all the way through the process of integration, because we hadn’t figured out what to do with the old System menu options. It turned out that those were in a state of flux, with the Ubuntu folks having to choose between the current GNOME default which everyone said would change, the patches for the likely NEXT GNOME approach, and the old Ubuntu approach. Ted whipped up some patches to make the GNOME panel more dynamic with its menus, so that we could remove the System menu logout options when people have the same menu in the FUSA applet, but that landed too late for inclusion into Intrepid final.
All in all, I think it’s a neat piece of work and hope other distro’s find it useful too. It’s just a teaser of the work we plan to do around the desktop experience. I’m looking forward to seeing everyone at UDS Jaunty in Mountain View in December, when we can talk about the next round! Thanks and well done to Ted, Martin, Scott, Sebastien and everyone else who helped to make this a reality.
Well done to Team Ubuntu (thousands of people across Ubuntu, Debian and upstreams) who make the magic in 8.10 possible. Happy Release Day everyone!
The GNOME user experience hackfest in Boston was a great way to spend the worst week in Wall St history!
Though there wasn’t a lot of hacking, there was a LOT of discussion, and we covered a lot of ground. There were at least 7 Canonical folks there, so it was a bit of a mini-sprint and a nice opportunity to meet the team at the same time. We had great participation from a number of organisations and free spirits, there’s a widespread desire to see GNOME stay on the forefront of usability.
Neil Patel of Canonical did a few mockups to try and capture the spirit of what was discussed, but I think the most interesting piece wasn’t really possible to capture in a screenshot because it’s abstract and conceptual – file and content management. There’s a revolution coming as we throw out the old “files and folders” metaphor and leap to something new, and it would be phenomenal if free software were leading the way.
I was struck by the number of different ways this meme cropped up. We had superb presentations of “real life support problems” from a large-scale user of desktop Linux, and a persistent theme was “where the hell did that file just go?” People save an attachment they receive in email, and an hour later have no idea where to find it. They import a picture into F-spot and then have no idea how to attach it to an email. They download a PDF from the web, then want to read it offline and can’t remember where they put it. Someone else pointed out that most people find it easier to find something on the Internet – through Google – than they do on their hard drives.
The Codethink guys also showed off some prototype experience work with Wizbit, which is a single-file version control system that draws on both Git and Bazaar for ideas about how you do efficient, transparent versioning of a file for online and offline editing.
We need to rearchitect the experience of “working with your content”, and we need to do it in a way that will work with the web and shared content as easily as it does locally.
My biggest concern on this front is that it be done in a way that every desktop environment can embrace. We need a consistent experience across GNOME, KDE, OpenOffice and Firefox so that content can flow from app to app in a seamless fashion and the user’s expectations can be met no matter which app or environment they happen to use. If someone sends a file to me over Empathy, and I want to open it in Amarok, then I shouldn’t have to work with two completely different mental models of content storage. Similarly, if I’ve downloaded something from the web with Firefox, and want to edit it in OpenOffice, I shouldn’t have to be super-aware or super-smart to be able to connect the apps to the content.
So, IMO this is work that should be championed in a forum like FreeDesktop.org, where it can rise above some of the existing rivalries of desktop linux. There’s a good tradition of practical collaboration in that forum, and this is a great candidate for similar treatment.
At the end of the day, bling is less transformational than a fundamental shift in content management. Kudos to the folks who are driving this!
Update: thanks mjg59 for pointing out my thinko. The Collabora guys do great stuff, but Codethink does Wizbit.
Here’s the official Canonical blog post, too.
