Thanks to the concerted efforts of Martin Pitt, Sebastien Bacher and several others, notify-osd and several related components landed in Jaunty last week. Thanks very much to all involved! And thanks to David Barth, Mirco Muller and Ted Gould who lead the development of notify-osd and the related messaging indicator.
Notify-OSD handles both application notifications and keyboard special keys like brightness and volume
MPT has posted an overview of the conceptual framework for “attention management” at https://wiki.ubuntu.com/NotificationDesignGuidelines, which puts ephemeral notification into context as just one of several distinct tools that applications can use when they don’t have the focus but need to make users aware of something. That’s a draft, and when it’s at 1.0 we’ll move it to a new site which will host design patterns on Canonical.com.
There is also a detailed specification for our implementation of the notification display agent, notify-osd, which can be found at https://wiki.ubuntu.com/NotifyOSD and which defines not only the expected behaviour of notify-osd but also all of the consequential updates we need to make across the packages in main an universe to ensure that those applications use notification and other techniques consistently.
There are at least 35 apps that need tweaking, and there may well be others! If you find an app that isn’t using notifications elegantly, please add it to the notification design guidelines page, and if you file a bug on the package, please tag it “notifications” so we can track these issues in a single consistent way.
Together with notify-osd, we’ve uploaded a new panel indicator which is used to provide a way to respond to messaging events, such as email and IRC pings. If someone IM’s you, then you should see an ephemeral notification, and the messaging indicator will give you a way to respond immediately. Same for email. Pidgin and Evolution are the primary focuses of the work, over time we’ll broaden that to the full complement of IM and email apps in the archive – patches welcome
There will be rough patches. Apps which don’t comply with the FreeDesktop.org spec and send actions on notifications even when the display agent says it does not support them, will have their notifications translated into alerts. That’s the primary focus of the effort now, the find and fix those apps. Also, we know there are several cases where a persistent response framework is required. The messaging indicator gets most of them, we will have additional persistent tools in place for Karmic in October.
Let’s talk about notifications! As Ryan Lortie mentioned, there was a lot of discussion across the Ubuntu, Kubuntu, GNOME, KDE and Mozilla communities represented at UDS about the proposals Canonical’s user experience design and desktop experience engineering teams have made for Ubuntu 9.04.
There are some fairly bold (read: controversial) ideas that we’d like to explore with, so the opportunity to discuss them with a broader cross-section of the community was fantastic. There were several rough edges and traps that I think we’ll avoid in the first round as a result, thanks to everyone who participated. Some of the things we work on in these teams are done directly with partners for their devices, so they don’t see this level of discussion before they ship, but it’s wonderful when we do get the opportunity to do so.
Some of these ideas are unproven, they boil down to matters of opinion, but since our commitment to them is based on a desire to learn more I think of them as constructive experiments. Experiments are just that – experiments. They may succeed and they may fail. We should judge them carefully, after we have data. We are putting new ideas into the free desktop without ego. We know those ideas could be better or worse than similar work being done in other communities, and we want to gather real user feedback to help find the best mix for everyone. The best ideas, and the best code, will ultimately form part of the digital free software commons and be shared by GNOME, KDE and every distribution. So, for those folks who were upset that we might ship something other than a GNOME or KDE default, I would ask for your patience and support – we want to contribute new ideas and new code, and that means having some delta which can be used as a basis for discussions about the future direction of upstream. In the past, we’ve had a few such delta’s in Ubuntu. Some, like the current panel layout, were widely embraced. Others, like the infamous “Ubuntu spacial mode”, were not. C’est la vie, and we all benefit from the evolution.
Experiments are also not something we should do lightly. The Ubuntu desktop is something I take very personally; I feel personally responsible for the productivity and happiness of every Ubuntu user, so when we bring new ideas and code to the desktop I believe we should do everything we can to make sure of success first time round. We should not inflict bad ideas on our users just because we’re curious or arrogant or stubborn or proud. Despite being occasionally curious, arrogant, stubborn and proud
So, what are we proposing?
First, we are focusing some attention on desktop notifications in this cycle, as part of a broader interest in the “space between applications”.
I think Canonical and Ubuntu can best help the cause of free software by focusing on the cracks between the major components of the desktop. In other words, while there are already great upstreams for individual applications in the free software desktop (Novell for Evolution, Sun for OpenOffice, Mozilla for Firefox, Red Hat for NetworkManager), we think there is a lot of productive and useful work to be done in the gaps between them. Notifications are things that many apps do, and if we can contribute new ideas there then we are helping improve the user experience of all of those applications. That’s a nice force multiplier – we’re hopefully doing work that makes the work of every other community even more valuable.
Nevertheless, expect bumps ahead. Ideas we are exploring may / will / do conflict with assumptions that are present today in various applications. We can address the relevant code in packages in main, but I’m more focused on addressing the potential social disruption that conflict can create, and that’s more a matter of conversation than code.
Notifications are interesting, subtle and complex. There are lots of different approaches on lots of different platforms. There are lots of different use cases. We’re trying to simplify and eliminate complexity, while still making it possible to meet the use cases we know about.
There has been good work in the freedesktop.org community on notifications, and even a spec that is *almost* at 1.0 in that community, with existing open source implementations. Our proposal is based on that specification, but it deprecates several capabilities and features in it. We will likely be compatible with the current API’s for sending notifications, but likely will not display all the notifications that might be sent, if they require features that we deprecate. If this experiment goes well, we would hope to help move that FD.o specification to 1.0, with or without our amendments.
The key proposals we are making are that:
There should be no actions on notifications.
Notifications should not be displayed synchronously, but may be queued. Our implementation of the notification display daemon will display only one notification at a time, others may do it differently.
That’s pretty much it. There are some subtleties and variations, but these are the key changes we are proposing, and which we will explore in a netbook device with a partner, as well as in the general Ubuntu 9.04 release, schedule gods being willing.
This work will show up as a new notification display agent, not as a fork or patch to the existing GNOME notification daemon. We don’t think the client API – libnotify – needs to be changed for this experiment, though we may not display notifications sent through that API that use capabilities we are suggesting be deprecated. We will try to ensure that packages in main are appropriately tuned, and hope MOTU will identify and update key packages in universe accordingly.
Why a completely new notification display agent? We are designing it to be built with Qt on KDE, and Gtk on GNOME. The idea is to have as much code in common as we can, but still take advantage of the appropriate text display framework on Ubuntu and Kubuntu. We hope to deliver both simultaneously, and have discussed this with both Ubuntu and Kubuntu community members. At the moment, there is some disagreement about the status of the FD.o specification between GNOME and KDE, and we hope our efforts will help build a bridge there. In Ubuntu 9.04, we would likely continue to package and publish the existing notification daemon in addition, to offer both options for users that have a particular preference. In general, where we invest in experimental new work, we plan to continue to offer a standard GNOME or KDE component / package set in the archive so that people can enjoy that experience too.
The most controversial part of the proposal is the idea that notifications should not have actions associated with them. In other words, no buttons, sliders, links, or even a dismissal [x]. When a notification pops up, you won’t be able to click on it, you won’t be able to make it go away, you won’t be able to follow it to another window, or to a web page. Are you loving this freedom? Hmmm? Madness, on the face of it, but there is method in this madness.
Our hypothesis is that the existence of ANY action creates a weighty obligation to act, or to THINK ABOUT ACTING. That make notifications turn from play into work. That makes them heavy responsibilities. That makes them an interruption, not a notification. And interruptions are a bag of hurt when you have things to do.
So, we have a three-prong line of attack.
We want to make notifications truly ephemeral. They are there, and then they are gone, and that’s life. If you are at your desktop when a notification comes by, you will sense it, and if you want you can LOOK at it, and it will be beautiful and clear and easy to parse. If you want to ignore it, you can safely do that and it will always go away without you having to dismiss it. If you miss it, that’s OK. Notifications are only for things which you can safely ignore or miss out on. If you went out for coffee and a notification flew by, you are no worse off. They don’t pile up like email, there is no journal of the ones you missed, you can’t scroll back and see them again, and therefor you are under no obligation to do so – they can’t become work while you are already busy with something else. They are gone like a mystery girl on the bus you didn’t get on, and they enrich your life in exactly the same way!
We think there should be persistent panel indicators for things which you really need to know about, even if you missed the notification because you urgently wanted that coffee. So we are making a list of those things, and plan to implement them.
Everything else should be dealt with by having a window call for attention, while staying in the background, unless it’s critical in which case that window could come to the foreground.
Since this is clearly the work of several releases, we may have glitches and inconsistencies along the way at interim checkpoints. I hope not, but it’s not unlikely, especially in the first iteration. Also, these ideas may turn out to be poor, and we should be ready to adjust our course based on feedback once we have an implementation in the wild.
We had a superb UXD and DEE (user experience design team, and desktop experience engineering team) sprint in San Francisco the week before UDS. Thanks to everyone who took part, especially those who came in from other teams. This notifications work may just be the tip of the iceberg, but it’s a very cool tip
One or more of our early-access OEM partners (companies that we work with on new desktop features) will likely ship this feature as part of a netbook product during the 9.04 cycle. At that point, we would also drop the code into a PPA for testing with a wider set of applications. There are active discussions about updating the freedesktop.org specification based on this work. I think we should be cautious, and gather real user testing feedback and hard data, but if it goes well then we would propose simplifying the spec accordingly, and submit our notification display agent to FreeDesktop.org. Long term collaboration around the code would take place on Launchpad.
The user experience and design team at Canonical includes a few folks dedicated to web technology. At the moment, there is a substantial effort under way to reshape the Launchpad UI now that we have the core capabilities for cross-project bug tracking, code publishing and translation in place. We want to make it more obvious how to get something done – especially for new users – and we want to make it feel snappy and responsive when making small changes to your project data.
In the design discussions, we spent a lot of time working on a new approach to “dialog boxes, wizards and workflows”, trying to solve a thorny problem in user interaction: how do you make it easy to do something complex? There are lots of cases in Launchpad where you need to get lots of ducks in a row before you can do something. For example, you might need to make sure there is a team with specific people in it before you subscribe that team to a bug. Or you might need to create a new milestone while triaging and scheduling work on bugs in your project.
Currently, that means jumping all around Launchpad in a way that assumes you know exactly how those pieces work. You need to go to one place to register a team, and a completely different place to setup a milestone. That means that lots of people don’t use capabilities in Launchpad, because they need to understand the whole system before they can get something small done. Every time someone bumps their head on that, we fail! And that’s the problem we set out to solve.
We came up with a nifty approach, which we call morphing dialogs, that ensures the user always has the minimum number of choices to make, and still allows for complex variations on a process in a way that feels quite natural for users.
The key ideas behind morphing dialogs are:
Only show one primary decision at a time, and make it obvious what that is. Sometimes, there are several directions you could take in order to get something done, but there is usually a single normal path for users to follow, and we always want users to be able to do the easy things easily.
Give users a sense of how far they are in the process, but don’t be too dogmatic about that, since getting one thing done often involves stepping off to the side to take care of preliminary business and those detours can also require several steps.
Here’s an example movie, which shows a person linking a blueprint to a bug. They need to search for the right blueprint, which they can do across a couple of projects simultaneously. In this mockup, they add GNOME to the list of projects that they look for the blueprint in, and when they can’t find it, they go to register a new blueprint for what they want. In the end he decides to go back and pick one from the search results. None of this involved a page load, and the round trips to the server are much cheaper than loading full pages, since we can just get what we need in highly optimized way.
You can see a couple of the key ideas coming through in the movie.
Note the “progress bar” – the green line – is not particular large or obtrusive. It’s also not obviously a progress bar, until one has done a few multi-step processes. Note also that you can have detours; you can step off to one side to get something done, like register a team or register a new blueprint, and those detours get their own progress indicator which is separate from the main one.
In particular, we wanted to unify the web service API’s that we already publish with this AJAX work, so that it would be easy to write web browser code that could talk to the exact same API’s we publish for developers who are integrating with Launchpad. That’s now possible, which means that any API we use for AJAX work will also be available to developers writing their own tools to access Launchpad directly through the web services.
Thanks to the awesomeness of YUI 3, the team is now hard at work turning those ideas into reality. Given that YUI 3 is right on the cutting edge (some would say bleeding edge!) we’re focusing on pieces that don’t depend on complex widgets – those will only start to fall into place next year as YUI 3 emerges from development.
Over the next couple of months you will see pieces of this puzzle land in successive Launchpad monthly releases (or daily, if you’re on edge.launchpad.net and a beta tester). Initially, the AJAX bling will just enable inline editing. In six to nine months, the more complex pieces should have land. And by then Launchpad’s web front-end will also be open source.
With Intrepid on track to hit the wires today I thought I’d blog a little on the process we followed in designing the new user switcher, presence manager and session management experience, and lessons learned along the way. Ted has been blogging about the work he did, and it’s been mentioned in a couple of different forums (briefly earning the memorable title “the new hotness”), but since it’s one of the first pieces of work to go through the user experience design process within Canonical I thought it would be interesting to write it up.
Here is a screenshot of the work itself in action:
New FUSA applet allows you to mange your presence setting, as well as switch to a guest or other user, and logout
In one of the first user experience sessions, we looked in more detail at the way people “stop working”. We thought it interesting to try and group those actions together in a way which would feel natural to users.
We have already done some work in Ubuntu around this – for a long time we have had a button in the top-right corner of the panel which brought up a system modal dialog that gave you the usual “end your session” options of logout, restart, shutdown, hibernate, suspend and switch user. That patch was always a bit controversial and had not been accepted upstream, so we looked at ways to solve the problem differently.
We decided to use the top-right location, because it’s one of the key places in the screen that’s quick and easy to get to (you can throw your mouse into a corner of the screen very easily and accurately) and because there was a strong precedent in the old Ubuntu logout button.
One key insight was that we wanted to make “switching user” less an exercise in guesswork and more direct – we wanted to let people switch directly to the specific user they were interested in rather than have an intermediate step where they login as that other user. So we started with the Fast User Switcher applet, or FUSA, as a base fr the design. Another key idea that emerged was that we wanted to integrate the “presence setting” into the same menu, because “going offline” or “I’m busy” are similar state-of-mind-and-work decisions to “log me off the system” or “shut down”.
We discussed at length the right order for the menu items. On the one hand, putting the “other users” at the top of the menu would mean that all the user names – yours and the ones you can switch to – would appear “in the same place” at the top of the menu. On the other, we strongly felt that things that would be used more casually and more easily should be at the top. In the end we settled on putting the presence management options at the top (Available, Away, Busy, Offline). Right next to those (in the same set) we put the “Lock screen” option, because it feels like a presence setting more than a session management setting – you are saying “Away” more than anything else.
Ted did a lot of work to make the presence menu elements work with both Pidgin and Empathy because there was some uncertainty as to which would be used by default in the release. Since it all uses dbus, it should be straightforward to make it work with KDE IM clients too.
We then put the user switching options – including the Guest Session which is a cool new feature in Intrepid that as been widely blogged (check out the YouTube demo) and which uses AppArmor to enforce security.
And finally, the session termination options – log out, suspend, hibernate, restart and shutdown are at the bottom of the menu, because you’re only ever likely to use them once in a session, by definition!
Styling The design of the menu is deliberately clean. We use very simple colours and shapes for the presence indicators, and replicate those colours and shapes in the actual GNOME panel so that you can see at a glance what your current presence setting is. Ted had to jump through some hoops, I think, to get the presence icons in the menu to line up with the current-presence-status indicator in the panel applet, but it worked out quite nicely. There’s some additional work to tighten up the layout which didn’t make it in time for the release but which might come in as a stable release update (SRU) or in Jaunty.
We decided not to put icons into the menu for each of the different statuses. Our design ethic is to aim for cleaner, less cluttered layouts with fewer icons and better choice of text. A couple of people have said that the menu looks “sparse” or “bare” but I think it sets the right direction and we’ll be continuing with this approach as we touch other parts of the system.
Upstream This work was discussed at UDS in Prague with a number of members of the GNOME community. I was also very glad to see that there’s a lot of support for a tighter, simpler panel at the GNOME hackfest, an idea that we’ve championed. The FUSA applet itself is going through a bit of a transformation upstream as it’s been merged into the new GDM codebase and the old code – on which our work is based – is more or less EOL’d. But we’ll figure out how to update this work for Jaunty and hopefully it will be easier to get it upstreamed at that point.
In Jaunty, we’ll likely do some more work on the GNOME panel, building on the GNOME user experience discussions. There was a lot of discussion about locking down the panel more tightly, which we may pursue.
Integration into Ubuntu
We realised rather late in the Ubuntu cycle that we hadn’t thought much about packaging. The Ubuntu team had kindly offered to help package and integrate the applet but we definitely learned the value of getting the packaging done earlier rather than later. We had the applet in a PPA for testing between developers fairly early, but we underestimated the difference between that and actual integration into the release.
The Ubuntu team rallied to the cause and helped to smooth the upgrade process for new users, so that we can try to get everyone onto the same footing when they start out with Intrepid whether as a new install or an upgrade. There are some challenges there, because the panel is so customisable, and we had to think hard about how we could ensure there was a consistent experience for something as important as logging out or shutting down while at the same time trying not to stomp on the preferences of folks who have customised their panels. Similarly, we were concerned that people who run different versions of Ubuntu, or different distributions entirely, with the same home directory, would have problems if those other OS’s didn’t have the same version of FUSA – we weren’t really able to address that satisfactorily.
We also realised (DOH!) that we hadn’t thought all the way through the process of integration, because we hadn’t figured out what to do with the old System menu options. It turned out that those were in a state of flux, with the Ubuntu folks having to choose between the current GNOME default which everyone said would change, the patches for the likely NEXT GNOME approach, and the old Ubuntu approach. Ted whipped up some patches to make the GNOME panel more dynamic with its menus, so that we could remove the System menu logout options when people have the same menu in the FUSA applet, but that landed too late for inclusion into Intrepid final.
All in all, I think it’s a neat piece of work and hope other distro’s find it useful too. It’s just a teaser of the work we plan to do around the desktop experience. I’m looking forward to seeing everyone at UDS Jaunty in Mountain View in December, when we can talk about the next round! Thanks and well done to Ted, Martin, Scott, Sebastien and everyone else who helped to make this a reality.
Well done to Team Ubuntu (thousands of people across Ubuntu, Debian and upstreams) who make the magic in 8.10 possible. Happy Release Day everyone!
The GNOME user experience hackfest in Boston was a great way to spend the worst week in Wall St history!
Though there wasn’t a lot of hacking, there was a LOT of discussion, and we covered a lot of ground. There were at least 7 Canonical folks there, so it was a bit of a mini-sprint and a nice opportunity to meet the team at the same time. We had great participation from a number of organisations and free spirits, there’s a widespread desire to see GNOME stay on the forefront of usability.
Neil Patel of Canonical did a few mockups to try and capture the spirit of what was discussed, but I think the most interesting piece wasn’t really possible to capture in a screenshot because it’s abstract and conceptual – file and content management. There’s a revolution coming as we throw out the old “files and folders” metaphor and leap to something new, and it would be phenomenal if free software were leading the way.
I was struck by the number of different ways this meme cropped up. We had superb presentations of “real life support problems” from a large-scale user of desktop Linux, and a persistent theme was “where the hell did that file just go?” People save an attachment they receive in email, and an hour later have no idea where to find it. They import a picture into F-spot and then have no idea how to attach it to an email. They download a PDF from the web, then want to read it offline and can’t remember where they put it. Someone else pointed out that most people find it easier to find something on the Internet – through Google – than they do on their hard drives.
The Codethink guys also showed off some prototype experience work with Wizbit, which is a single-file version control system that draws on both Git and Bazaar for ideas about how you do efficient, transparent versioning of a file for online and offline editing.
We need to rearchitect the experience of “working with your content”, and we need to do it in a way that will work with the web and shared content as easily as it does locally.
My biggest concern on this front is that it be done in a way that every desktop environment can embrace. We need a consistent experience across GNOME, KDE, OpenOffice and Firefox so that content can flow from app to app in a seamless fashion and the user’s expectations can be met no matter which app or environment they happen to use. If someone sends a file to me over Empathy, and I want to open it in Amarok, then I shouldn’t have to work with two completely different mental models of content storage. Similarly, if I’ve downloaded something from the web with Firefox, and want to edit it in OpenOffice, I shouldn’t have to be super-aware or super-smart to be able to connect the apps to the content.
So, IMO this is work that should be championed in a forum like FreeDesktop.org, where it can rise above some of the existing rivalries of desktop linux. There’s a good tradition of practical collaboration in that forum, and this is a great candidate for similar treatment.
At the end of the day, bling is less transformational than a fundamental shift in content management. Kudos to the folks who are driving this!
Update: thanks mjg59 for pointing out my thinko. The Collabora guys do great stuff, but Codethink does Wizbit.
When you present yourself on the web, you have 15 seconds to make an impression, so aspiring champions of the web 2.0 industry have converged on a good recipe for success:
Make your site visually appealing,
Do something different and do it very, very well,
Call users to action and give them an immediate, rewarding experience.
We need the same urgency, immediacy and elegance as part of the free software desktop experience, and that’s is an area where Canonical will, I hope, make a significant contribution. We are hiring designers, user experience champions and interaction design visionaries and challenging them to lead not only Canonical’s distinctive projects but also to participate in GNOME, KDE and other upstream efforts to improve FLOSS usability.
Fortunately, we won’t be working in a vacuum. This is an idea that is already being widely explored. It’s great to see that communities like GNOME and KDE have embraced user experience as a powerful driver of evolution in their platforms. Partly because of the web-2.0 phenomenon and the iPhone, there’s a widely held desire to see FLOSS leap forward in usability and design. We want to participate and help drive that forward.
There’s also recognition for the scale of the challenge that faces us. When I laid out the goal of “delivering a user experience that can compete with Apple in two years” at OSCON, I had many questions afterwards about how on earth we could achieve that. “Everyone scratches their own itch, how can you possibly make the UI consistent?” was a common theme. And it’s true – the free software desktop is often patchy and inconsistent. But I see the lack of consistency as both a weakness (GNOME, OpenOffice and Firefox all have different UI toolkits, and it’s very difficult to make them seamless) and as a strength – people are free to innovate, and the results are world-leading. Our challenge is to get the best of both of those worlds.
I don’t have answers to all of those questions. I do, however, have a deep belief in the power of the free software process to solve seemingly intractable problems, especially in the long tail. If we articulate a comprehensive design ethic, a next-generation HIG, we can harness the wisdom of crowds to find corner cases and inconsistencies across a much broader portfolio of applications than one person or company could do alone. That’s why it’s so important to me that Canonical’s design and user experience team also participate in upstream projects across the board.
In Ubuntu we have in general considered upstream to be “our ROCK”, by which we mean that we want upstream to be happy with the way we express their ideas and their work. More than happy – we want upstream to be delighted! We focus most of our effort on integration. Our competitors turn that into “Canonical doesn’t contribute” but it’s more accurate to say we measure our contribution in the effectiveness with which we get the latest stable work of upstream, with security maintenance, to the widest possible audience for testing and love. To my mind, that’s a huge contribution.
Increasingly, though, Canonical is in a position to drive real change in the software that is part of Ubuntu. If we just showed up with pictures and prototypes and asked people to shape their projects differently, I can’t imagine that being well received! So we are also hiring a team who will work on X, OpenGL, Gtk, Qt, GNOME and KDE, with a view to doing some of the heavy lifting required to turn those desktop experience ideas into reality. Those teams will publish their Bzr branches in Launchpad and of course submit their work upstream, and participate in upstream sprints and events. Some of the folks we have hired into those positions are familiar contributors in the FLOSS world, others will be developers with relevant technical expertise from other industries.
One strong meme we want to preserve is the idea that Ubuntu, the platform team, is still primarily focused on integration and distribution. We will keep that team and the upstream work distinct to minimise the conflict of interest inherent in choosing the patches and the changes and the applications that actually ship each six months as part of an Ubuntu release.
Of course, there’s a risk to participation, because you can’t easily participate without expressing opinions, visions, desires, goals, and those can clash with other participants. It’s hard to drive change, even when people agree that change is needed. I hope we can find ways to explore and experiment with new ideas without blocking on consensus across diverse and distributed teams. We have to play to our strengths, which include the ability to diverge for experimental purposes to see what really works before we commit everyone to a course of action. It will be a challenge, but I think it’s achievable.
All of this has me tapdancing to work in the mornings, because we’re sketching out really interesting ideas for user interaction in Launchpad and in the desktop. The team has come together very nicely, and I’m thoroughly enjoying the processes, brainstorming and prototyping. I can’t wait to see those ideas landing in production!