Mark Shuttleworth | here be dragons

Jono Bacon, Alan Pope, and many others have written, yesterday we published a new visual story and style for Ubuntu. The core design work was lead by Marcus Haslam, Otto Greenslade and Dominic Edmunds, who are the three visual artists leading our efforts in the Canonical Design team. Once we had the base ideas in place we invited some anchor members of the Ubuntu Art community to a design sprint, to test that the concept had the legs to work with the full range of forums, websites, derivatives and other pieces of this huge and wonderful project. And apparently, it does!

Here are some additional thoughts.

Embracing both Ubuntu and Canonical

One of the real challenges for us has been to find a branding and design strategy which spans the spectrum of audiences, forums and dialogues that we cover.  With Ubuntu, it’s my specific dream to find a constructive blend of commercial and community interests, not only for Canonical but for other companies. That has made our design and branding work difficult – the distinctive look of Ubuntu lent itself well to pure community messaging, but it was hard to do a brochure for Canonical data center services for Ubuntu on servers. We have not only Ubuntu, but also Kubuntu and an important range of derivatives that all have a role in our ecosystem.

So we spent a lot of time trying to distill the requirements down into a set of three dimensions:

We found a set of ideas which each represent those spectrums, and which work together.

For example, we identified a palette which includes both a fresh, lively Orange, and a rich, mature Aubergine, which work together. The use of Aubergine indicates Commercial involvement of one form or another, while Orange is a signal of community engagement. The Forums will use the Orange elements more strongly, and a formal product brochure, with descriptions of supporting services, would use more of the Aubergine.

On the consumer/enterprise spectrum, we took inspiration from the aerospace industry, and identified a texture of closely spaced dots. When you see more of that, it means we’re signalling that the story is more about the enterprise, less of that, and it’s more about the consumer. Of course, there are cross-overs, for example when we are talking about the corporate desktop, where we’ll use that closely space dot texture as a boundary area, or separator. We also identified shades of Aubergine that are more consumer, or more enterprise – the darker shades mapping to a stronger emphasis on enterprise work.

And on the end-user / engineer spectrum, we took inspiration from graph paper and engineering blue prints. When you see widely spaced patterns of dots, or outline images and figures, that’s signalling that the content is more engineering-oriented than end-user oriented.

And finally, we found a number of themes which enhanced and echoed those ideas. We use a warm gray supporting colour to give shape to pages and documents, and we built on the dots and circles to create a whole style for figures, illustrations and pictograms.

The beauty of this is that we can now publish content that spans the full range, and we generally know when we start the design process what sorts of visual cues we want to be signalling. Instead of having these different mental domains fight with one another, we can now convey quite subtle collaboration between community and corporate, or work which is aimed at engineers and developers from enterprises as opposed to developers working with consumers. Time will tell how it shapes up, but for now I’m celebrating the milestone and the efforts of the team that pulled it together. There’s something there for everyone who wants to participate in the great hubbub of Ubuntuness that is our shared experience of free software.

So, for example, here’s a conference banner. The strong use of Aubergine suggests that it’s more corporate messaging (Canonical is heavily involved). Orange is used here more as a highlight. The Aubergine is darker, and there’s quite a lot of the fine dot pattern. Below the image is a set of scales showing where on those spectra this work is pitched.

As another example, here’s a brochure with an emphasis on end-users who are thinking about adopting Ubuntu’s cloud infrastructure. Again, the fine dot patterns suggests a more enterprise focus, as does the use of the dark aubergine. You can see the circle metaphor used in the quote callout.

And here’s a similar brochure, but with a more developer or engineering oriented focus: note the use of the graph-paper theme with wide spaced dots, and outline shapes.

Finally, here’s an example of a brochure and CD cover for Ubuntu:

As you can see the idea is to signal a mix of both community and Canonical involvement in the message, addressing consumer audiences with a mix of developers and end-users.

A new Ubuntu font

We have commissioned a new font to be developed both for the logo’s of Ubuntu and Canonical, and for use in the interface. The font will be called Ubuntu, and will be a modern humanist font that is optimised for screen legibility. It will be published under an open font license, and considered part of the trade dress of Ubuntu, which will limit its relevance for software interfaces outside of Ubuntu but leave it free for use across the web and in printed documents.

It will take a few months for the font to be finalised, initial elements will be final in the next week which will be sufficient for the logo and other bits and pieces, but I expect to see that font widely used in 10.10. The work has been commissioned from world-renowned fontographers Dalton Maag, who have expressed excitement at the opportunity to publish an open font and also a font that they know will be used daily by millions of people.

Initial coverage will be Western, Arabic, Hebrew and Cyrillic character sets, but over time we may be able to extend that to being a full Unicode font, with great kerning and hinting for print and screen usage globally.  We are considering an internship program, to support aspiring fontographers from all corners of the world to visit London and work with Dalton Maag to extend the font to their own regional glyph set.

The critical test of the font is screen efficiency and legibility, and its character and personality are secondary to its fitness for that purpose. Nevertheless, our hope is that the font has a look that is elegant and expresses the full set of values for both Canonical and Ubuntu: adroitness, accountability, precision, reliability, freedom and collaboration. We’ll publish more as soon as we have it.

A good start

It’s been an exciting process, but I have the sense that we are just getting started. The language will get richer, we will find new things that we want to communicate, and new treatments and visual themes that resonate well with these starting points. We’ll find new ways to integrate this on the web, and on the desktop (look out for the two new themes, Radiance and Ambiance).  I hope we’ll see the language being used to good effect across everything we do, both commercial and community oriented. There’s a range of expression here that should be useful to artists across the spectrum. Let me know how it works for you.

From March next year, I’ll focus my Canonical energy on product design, partnerships and customers. Those are the areas that I enjoy most and also the areas where I can best shape the impact we have on open source and the technology market. I’m able to do this because Jane Silber, who has been COO at Canonical virtually from the beginning, will take on the job of CEO.

Since Jane joined the company, she and I have shared the load of coordinating between the leaders of all the key teams that make up Canonical. We’ve been through various permutations as new initiatives needed different kinds of attention; Jane currently leads the Ubuntu One effort, for example.

I’ve become very passionate about design and quality, and want to spend more time figuring out how we harness the collaborative process to build better, more insightful products. I can’t think of a more interesting challenge, and luckily I couldn’t think of a better person to take over my formal management and leadership responsibilities at Canonical than Jane. We’ve worked together long enough, and closely enough, that I can be confident of continuity in the pieces I most care about and also excited about the ways in which I think Jane will raise the bar for the senior team. As a former VP at General Dynamics, Jane has more experience of large customers and large organisational leadership, which I see as essential for Canonical over the next five years. We are being welcomed as a partner and supplier to ever-larger businesses, and I want to make sure we are a robust answer to their needs.

Many folks in the community will know Jane from Ubuntu Developer Summits, and of course she’s well established as a leader at Canonical. In order to focus on the new role, we’ll be hiring for a COO and a new lead for Ubuntu One (both positions will be advertised publicly as well as within Canonical). There’s no rush, so we plan to coordinate things carefully and I expect I’ll be focused on my new role by March.

Six-month cycles are great. Now let’s talk about meta-cycles: broader release cycles for major work. I’m very interested in a cross-community conversation about this, so will sketch out some ideas and then encourage people from as many different free software communities as possible to comment here. I’ll summarise those comments in a follow-up post, which will no doubt be a lot wiser and more insightful than this one

Background: building on the best practice of cadence

The practice of regular releases, and now time-based releases, is becoming widespread within the free software community. From the kernel, to GNOME and KDE, to X, and distributions like Ubuntu, Fedora, the idea of a regular, predictable cycle is now better understood and widely embraced. Many smarter folks than me have articulated the benefits of such a cadence: energising the whole community, REALLY releasing early and often, shaking out good and bad code, rapid course correction.

There has been some experimentation with different cycles. I’m involved in projects that have 1 month, 3 month and 6 month cycles, for different reasons. They all work well.

..but addressing the needs of the longer term

But there are also weaknesses to the six-month cycle:

• It’s hard to communicate to your users that you have made some definitive, significant change,
• It’s hard to know what to support for how long, you obviously can’t support every release indefinitely.

I think there is growing insight into this, on both sides of the original “cadence” debate.

A tale of two philosophies, perhaps with a unifying theory

A few years back, at AKademy in Glasgow, I was in the middle of a great discussion about six month cycles. I was a passionate advocate of the six month cycle, and interested in the arguments against it. The strongest one was the challenge of making “big bold moves”.

“You just can’t do some things in six months” was the common refrain. “You need to be able to take a longer view, and you need a plan for the big change.” There was a lot of criticism of GNOME for having “stagnated” due to the inability to make tough choices inside a six month cycle (and with perpetual backward compatibility guarantees). Such discussions often become ideological, with folks on one side saying “you can evolve anything incrementally” and others saying “you need to make a clean break”.

At the time of course, KDE was gearing up for KDE 4.0, a significant and bold move indeed. And GNOME was quite happily making its regular releases. When the KDE release arrived, it was beautiful, but it had real issues. Somewhat predictably, the regular-release crowd said “see, we told you, BIG releases don’t work”. But since then KDE has knuckled down with regular, well managed, incremental improvements, and KDE is looking fantastic. Suddenly, the big bold move comes into focus, and the benefits become clear. Well done KDE

On the other side of the fence, GNOME is now more aware of the limitations of indefinite regular releases. I’m very excited by the zest and spirit with which the “user experience MATTERS” campaign is being taken up in Gnome, there’s a real desire to deliver breakthrough changes. This kicked off at the excellent Gnome usability summit last year, which I enjoyed and which quite a few of the Canonical usability and design folks participated in, and the fruits of that are shaping up in things like the new Activities shell.

But it’s become clear that a change like this represents a definitive break with the past, and might take more than a single six month release to achieve. And most important of all, that this is an opportunity to make other, significant, distinctive changes. A break with the past. A big bold move. And so there’s been a series of conversations about how to “do a 3.0″, in effect, how to break with the tradition of incremental change, in order to make this vision possible.

It strikes me that both projects are converging on a common set of ideas:

• Rapid, predictable releases are super for keeping energy high and code evolving cleanly and efficiently, they keep people out of a deathmarch scenario, they tighten things up and they allow for a shakeout of good and bad ideas in a coordinated, managed fashion.

• Big releases are energising too. They are motivational, they make people feel like it’s possible to change anything, they release a lot of creative energy and generate a lot of healthy discussion. But they can be a bit messy, things can break on the way, and that’s a healthy thing.

Anecdotally, there are other interesting stories that feed into this.

Recently, the Python community decided that Python 3.0 will be a shorter cycle than the usual Python release. The 3.0 release is serving to shake out the ideas and code for 3.x, but it won’t be heavily adopted itself so it doesn’t really make sense to put a lot of effort into maintaining it – get it out there, have a short cycle, and then invest in quality for the next cycle because 3.x will be much more heavily used than 3.0. This reminds me a lot of KDE 4.0.

So, I’m interesting in gathering opinions, challenges, ideas, commitments, hypotheses etc about the idea of meta-cycles and how we could organise ourselves to make the most of this. I suspect that we can define a best practice, which includes regular releases for continuous improvement on a predictable schedule, and ALSO defines a good practice for how MAJOR releases fit into that cadence, in a well structured and manageable fashion. I think we can draw on the experiences in both GNOME and KDE, and other projects, to shape that thinking.

This is important for distributions, too

The major distributions tend to have big releases, as well as more frequent releases. RHEL has Fedora, Ubuntu makes LTS releases, Debian takes cadence to its logical continuous integration extreme with Sid and Testing .

When we did Ubuntu 6.06 LTS we said we’d do another LTS in “2 to 3 years”. When we did 8.04 LTS we said that the benefits of predictability for LTS’s are such that it would be good to say in advance when the next LTS would be. I said I would like that to be 10.04 LTS, a major cycle of 2 years, unless the opportunity came up to coordinate major releases with one or two other major distributions – Debian, Suse or Red Hat.

I’ve spoken with folks at Novell, and it doesn’t look like there’s an opportunity to coordinate for the moment. In conversations with Steve McIntyre, the current Debian Project Leader, we’ve identified an interesting opportunity to collaborate. Debian is aiming for an 18 month cycle, which would put their next release around October 2010, which would be the same time as the Ubuntu 10.10 release. Potentially, then, we could defer the Ubuntu LTS till 10.10, coordinating and collaborating with the Debian project for a release with very similar choices of core infrastructure. That would make sharing patches a lot easier, a benefit both ways. Since there will be a lot of folks from Ubuntu at Debconf, and hopefully a number of Debian developers at UDS in Barcelona in May, we will have good opportunities to examine this opportunity in detail. If there is goodwill, excitement and broad commitment to such an idea from Debian, I would be willing to promote the idea of deferring the LTS from 10.04 to 10.10 LTS.

Questions and options

So, what would the “best practices” of a meta-cycle be? What sorts of things should be considered in planning for these meta-cycles? What problems do they cause, and how are those best addressed? How do short term (3 month, 6 month) cycles fit into a broader meta-cycle? Asking these questions across multiple communities will help test the ideas and generate better ones.

What’s a good name for such a meta-cycle? Meta-cycle seems…. very meta.

Is it true that the “first release of the major cycle” (KDE 4.0, Python 3.0) is best done as a short cycle that does not get long term attention? Are there counter-examples, or better examples, of this?

Which release in the major cycle is best for long term support? Is it the last of the releases before major new changes begin (Python 2.6? GNOME 2.28?) or is it the result of a couple of quick iterations on the X.0 release (KDE 4.2? GNOME 3.2?) Does it matter? I do believe that it’s worthwhile for upstreams to support an occasional release for a longer time than usual, because that’s what large organisations want.

Is a whole-year cycle beneficial? For example, is 2.5 years a good idea? Personally, I think not. I think conferences and holidays tend to happen at the same time of the year every year and it’s much, much easier to think in terms of whole number of year cycles. But in informal conversations about this, some people have said 18 months, others have said 30 months (2.5 years) might suit them. I think they’re craaaazy, what do you think?

If it’s 2 years or 3 years, which is better for you? Hardware guys tend to say “2 years!” to get the benefit of new hardware, sooner. Software guys say “3 years!” so that they have less change to deal with. Personally, I am in the 2 years camp, but I think it’s more important to be aligned with the pulse of the community, and if GNOME / KDE / Kernel wanted 3 years, I’d be happy to go with it.

How do the meta-cycles of different projects come together? Does it make sense to have low-level, hardware-related things on a different cycle to high-level, user visible things? Or does it make more sense to have a rhythm of life that’s shared from the top to the bottom of the stack?

Would it make more sense to stagger long term releases based on how they depend on one another, like GCC then X then OpenOffice? Or would it make more sense to have them all follow the same meta-cycle, so that we get big breakage across the stack at times, and big stability across the stack at others?

Are any projects out there already doing this?

Is there any established theory or practice for this?

A cross-community conversation

If you’ve read this far, thank you! Please do comment, and if you are interested then please do take up these questions in the communities that you care about, and bring the results of those discussions back here as comments. I’m pretty sure that we can take the art of software to a whole new level if we take advantage of the fact that we are NOT proprietary, and this is one of the key ways we can do it.

Thanks to the concerted efforts of Martin Pitt, Sebastien Bacher and several others, notify-osd and several related components landed in Jaunty last week. Thanks very much to all involved! And thanks to David Barth, Mirco Muller and Ted Gould who lead the development of notify-osd and the related messaging indicator.

Notify-OSD handles both application notifications and keyboard special keys like brightness and volume

MPT has posted an overview of the conceptual framework for “attention management” at https://wiki.ubuntu.com/NotificationDesignGuidelines, which puts ephemeral notification into context as just one of several distinct tools that applications can use when they don’t have the focus but need to make users aware of something. That’s a draft, and when it’s at 1.0 we’ll move it to a new site which will host design patterns on Canonical.com.

There is also a detailed specification for our implementation of the notification display agent, notify-osd, which can be found at https://wiki.ubuntu.com/NotifyOSD and which defines not only the expected behaviour of notify-osd but also all of the consequential updates we need to make across the packages in main an universe to ensure that those applications use notification and other techniques consistently.

There are at least 35 apps that need tweaking, and there may well be others! If you find an app that isn’t using notifications elegantly, please add it to the notification design guidelines page, and if you file a bug on the package, please tag it “notifications” so we can track these issues in a single consistent way.

Together with notify-osd, we’ve uploaded a new panel indicator which is used to provide a way to respond to messaging events, such as email and IRC pings. If someone IM’s you, then you should see an ephemeral notification, and the messaging indicator will give you a way to respond immediately. Same for email. Pidgin and Evolution are the primary focuses of the work, over time we’ll broaden that to the full complement of IM and email apps in the archive – patches welcome

There will be rough patches. Apps which don’t comply with the FreeDesktop.org spec and send actions on notifications even when the display agent says it does not support them, will have their notifications translated into alerts. That’s the primary focus of the effort now, the find and fix those apps. Also, we know there are several cases where a persistent response framework is required. The messaging indicator gets most of them, we will have additional persistent tools in place for Karmic in October.

As widely reported, Russia has closed commercial public access to Soyuz seats for flights after the US shuttle is retired.

Now that the ISS has the capacity for a larger full-time crew, the seats are more likely to be devoted to long-duration ISS crew rotation than short-term ISS visits, whether visits by professional EU / US astronauts or folks flying privately. I’ve no doubt that there are economics attached to the Russian seats that are similar for both cases – the EU and US have to pay for the lift just like us ordinary folks.

There are a couple of interesting twists to the story.

One is that, when the Shuttle is retired, the Russians will have the only manned access to orbital flight in the ISS partnership. Russia and China will be the only nations with manned orbital capabilities, and the US huffishly refuses to welcome China into the ISS club. Expect the price of a seat to rise substantially while that’s the case.

Another is the EU’s plan to evolve their autonomous cargo vessel (ATV) into a manned capability, something that’s perfectly feasible and quite sensible IMO.

And the third twist is that the Russians have long been open to commercial offers for a long-duration flight (six month ISS crew rotation). That woudl require substantially more training (12-18 months minimum depending on who you ask) but would certainly include the Soyuz lift to get there and back.

Let’s talk about notifications! As Ryan Lortie mentioned, there was a lot of discussion across the Ubuntu, Kubuntu, GNOME, KDE and Mozilla communities represented at UDS about the proposals Canonical’s user experience design and desktop experience engineering teams have made for Ubuntu 9.04.

See the mockup as a Flash movie.

There are some fairly bold (read: controversial) ideas that we’d like to explore with, so the opportunity to discuss them with a broader cross-section of the community was fantastic. There were several rough edges and traps that I think we’ll avoid in the first round as a result, thanks to everyone who participated. Some of the things we work on in these teams are done directly with partners for their devices, so they don’t see this level of discussion before they ship, but it’s wonderful when we do get the opportunity to do so.

Some of these ideas are unproven, they boil down to matters of opinion, but since our commitment to them is based on a desire to learn more I think of them as constructive experiments. Experiments are just that – experiments. They may succeed and they may fail. We should judge them carefully, after we have data. We are putting new ideas into the free desktop without ego. We know those ideas could be better or worse than similar work being done in other communities, and we want to gather real user feedback to help find the best mix for everyone. The best ideas, and the best code, will ultimately form part of the digital free software commons and be shared by GNOME, KDE and every distribution. So, for those folks who were upset that we might ship something other than a GNOME or KDE default, I would ask for your patience and support – we want to contribute new ideas and new code, and that means having some delta which can be used as a basis for discussions about the future direction of upstream. In the past, we’ve had a few such delta’s in Ubuntu. Some, like the current panel layout, were widely embraced. Others, like the infamous “Ubuntu spacial mode”, were not. C’est la vie, and we all benefit from the evolution.

Experiments are also not something we should do lightly. The Ubuntu desktop is something I take very personally; I feel personally responsible for the productivity and happiness of every Ubuntu user, so when we bring new ideas and code to the desktop I believe we should do everything we can to make sure of success first time round. We should not inflict bad ideas on our users just because we’re curious or arrogant or stubborn or proud. Despite being occasionally curious, arrogant, stubborn and proud

So, what are we proposing?

First, we are focusing some attention on desktop notifications in this cycle, as part of a broader interest in the “space between applications”.

I think Canonical and Ubuntu can best help the cause of free software by focusing on the cracks between the major components of the desktop. In other words, while there are already great upstreams for individual applications in the free software desktop (Novell for Evolution, Sun for OpenOffice, Mozilla for Firefox, Red Hat for NetworkManager), we think there is a lot of productive and useful work to be done in the gaps between them. Notifications are things that many apps do, and if we can contribute new ideas there then we are helping improve the user experience of all of those applications. That’s a nice force multiplier – we’re hopefully doing work that makes the work of every other community even more valuable.

Nevertheless, expect bumps ahead. Ideas we are exploring may / will / do conflict with assumptions that are present today in various applications. We can address the relevant code in packages in main, but I’m more focused on addressing the potential social disruption that conflict can create, and that’s more a matter of conversation than code.

Notifications are interesting, subtle and complex. There are lots of different approaches on lots of different platforms. There are lots of different use cases. We’re trying to simplify and eliminate complexity, while still making it possible to meet the use cases we know about.

There has been good work in the freedesktop.org community on notifications, and even a spec that is *almost* at 1.0 in that community, with existing open source implementations. Our proposal is based on that specification, but it deprecates several capabilities and features in it. We will likely be compatible with the current API’s for sending notifications, but likely will not display all the notifications that might be sent, if they require features that we deprecate. If this experiment goes well, we would hope to help move that FD.o specification to 1.0, with or without our amendments.

The key proposals we are making are that:

• There should be no actions on notifications.
• Notifications should not be displayed synchronously, but may be queued. Our implementation of the notification display daemon will display only one notification at a time, others may do it differently.

That’s pretty much it. There are some subtleties and variations, but these are the key changes we are proposing, and which we will explore in a netbook device with a partner, as well as in the general Ubuntu 9.04 release, schedule gods being willing.

This work will show up as a new notification display agent, not as a fork or patch to the existing GNOME notification daemon. We don’t think the client API – libnotify – needs to be changed for this experiment, though we may not display notifications sent through that API that use capabilities we are suggesting be deprecated. We will try to ensure that packages in main are appropriately tuned, and hope MOTU will identify and update key packages in universe accordingly.

Why a completely new notification display agent? We are designing it to be built with Qt on KDE, and Gtk on GNOME. The idea is to have as much code in common as we can, but still take advantage of the appropriate text display framework on Ubuntu and Kubuntu. We hope to deliver both simultaneously, and have discussed this with both Ubuntu and Kubuntu community members. At the moment, there is some disagreement about the status of the FD.o specification between GNOME and KDE, and we hope our efforts will help build a bridge there. In Ubuntu 9.04, we would likely continue to package and publish the existing notification daemon in addition, to offer both options for users that have a particular preference. In general, where we invest in experimental new work, we plan to continue to offer a standard GNOME or KDE component / package set in the archive so that people can enjoy that experience too.

The most controversial part of the proposal is the idea that notifications should not have actions associated with them. In other words, no buttons, sliders, links, or even a dismissal [x]. When a notification pops up, you won’t be able to click on it, you won’t be able to make it go away, you won’t be able to follow it to another window, or to a web page. Are you loving this freedom? Hmmm? Madness, on the face of it, but there is method in this madness.

Our hypothesis is that the existence of ANY action creates a weighty obligation to act, or to THINK ABOUT ACTING. That make notifications turn from play into work. That makes them heavy responsibilities. That makes them an interruption, not a notification. And interruptions are a bag of hurt when you have things to do.

So, we have a three-prong line of attack.

1. We want to make notifications truly ephemeral. They are there, and then they are gone, and that’s life. If you are at your desktop when a notification comes by, you will sense it, and if you want you can LOOK at it, and it will be beautiful and clear and easy to parse. If you want to ignore it, you can safely do that and it will always go away without you having to dismiss it. If you miss it, that’s OK. Notifications are only for things which you can safely ignore or miss out on. If you went out for coffee and a notification flew by, you are no worse off. They don’t pile up like email, there is no journal of the ones you missed, you can’t scroll back and see them again, and therefor you are under no obligation to do so – they can’t become work while you are already busy with something else. They are gone like a mystery girl on the bus you didn’t get on, and they enrich your life in exactly the same way!
2. We think there should be persistent panel indicators for things which you really need to know about, even if you missed the notification because you urgently wanted that coffee. So we are making a list of those things, and plan to implement them.
3. Everything else should be dealt with by having a window call for attention, while staying in the background, unless it’s critical in which case that window could come to the foreground.

Since this is clearly the work of several releases, we may have glitches and inconsistencies along the way at interim checkpoints. I hope not, but it’s not unlikely, especially in the first iteration. Also, these ideas may turn out to be poor, and we should be ready to adjust our course based on feedback once we have an implementation in the wild.

We had a superb UXD and DEE (user experience design team, and desktop experience engineering team) sprint in San Francisco the week before UDS. Thanks to everyone who took part, especially those who came in from other teams. This notifications work may just be the tip of the iceberg, but it’s a very cool tip

One or more of our early-access OEM partners (companies that we work with on new desktop features) will likely ship this feature as part of a netbook product during the 9.04 cycle. At that point, we would also drop the code into a PPA for testing with a wider set of applications. There are active discussions about updating the freedesktop.org specification based on this work. I think we should be cautious, and gather real user testing feedback and hard data, but if it goes well then we would propose simplifying the spec accordingly, and submit our notification display agent to FreeDesktop.org. Long term collaboration around the code would take place on Launchpad.

The user experience and design team at Canonical includes a few folks dedicated to web technology. At the moment, there is a substantial effort under way to reshape the Launchpad UI now that we have the core capabilities for cross-project bug tracking, code publishing and translation in place. We want to make it more obvious how to get something done – especially for new users – and we want to make it feel snappy and responsive when making small changes to your project data.

In the design discussions, we spent a lot of time working on a new approach to “dialog boxes, wizards and workflows”, trying to solve a thorny problem in user interaction: how do you make it easy to do something complex? There are lots of cases in Launchpad where you need to get lots of ducks in a row before you can do something. For example, you might need to make sure there is a team with specific people in it before you subscribe that team to a bug. Or you might need to create a new milestone while triaging and scheduling work on bugs in your project.

Currently, that means jumping all around Launchpad in a way that assumes you know exactly how those pieces work. You need to go to one place to register a team, and a completely different place to setup a milestone. That means that lots of people don’t use capabilities in Launchpad, because they need to understand the whole system before they can get something small done. Every time someone bumps their head on that, we fail! And that’s the problem we set out to solve.

We came up with a nifty approach, which we call morphing dialogs, that ensures the user always has the minimum number of choices to make, and still allows for complex variations on a process in a way that feels quite natural for users.

The key ideas behind morphing dialogs are:

• Only show one primary decision at a time, and make it obvious what that is. Sometimes, there are several directions you could take in order to get something done, but there is usually a single normal path for users to follow, and we always want users to be able to do the easy things easily.
• Give users a sense of how far they are in the process, but don’t be too dogmatic about that, since getting one thing done often involves stepping off to the side to take care of preliminary business and those detours can also require several steps.

Here’s an example movie, which shows a person linking a blueprint to a bug. They need to search for the right blueprint, which they can do across a couple of projects simultaneously. In this mockup, they add GNOME to the list of projects that they look for the blueprint in, and when they can’t find it, they go to register a new blueprint for what they want. In the end he decides to go back and pick one from the search results. None of this involved a page load, and the round trips to the server are much cheaper than loading full pages, since we can just get what we need in highly optimized way.

You can see a couple of the key ideas coming through in the movie.

Note the “progress bar” – the green line – is not particular large or obtrusive. It’s also not obviously a progress bar, until one has done a few multi-step processes. Note also that you can have detours; you can step off to one side to get something done, like register a team or register a new blueprint, and those detours get their own progress indicator which is separate from the main one.

We had a major sprint recently that brought the whole Launchpad team together for two weeks while we did a deep dive into JavaScript and AJAX. We picked YUI 3, the next version of Yahoo’s UI toolkit for the web, as a foundational layer for this AJAX effort, and we wanted to bring everyone up to speed on the processes for designing, building and testing web client apps. It was a lot of fun.

In particular, we wanted to unify the web service API’s that we already publish with this AJAX work, so that it would be easy to write web browser code that could talk to the exact same API’s we publish for developers who are integrating with Launchpad. That’s now possible, which means that any API we use for AJAX work will also be available to developers writing their own tools to access Launchpad directly through the web services.

Thanks to the awesomeness of YUI 3, the team is now hard at work turning those ideas into reality. Given that YUI 3 is right on the cutting edge (some would say bleeding edge!) we’re focusing on pieces that don’t depend on complex widgets – those will only start to fall into place next year as YUI 3 emerges from development.

Over the next couple of months you will see pieces of this puzzle land in successive Launchpad monthly releases (or daily, if you’re on edge.launchpad.net and a beta tester). Initially, the AJAX bling will just enable inline editing. In six to nine months, the more complex pieces should have land. And by then Launchpad’s web front-end will also be open source.

Some of the comments on my last post on the economic unwinding of 2008 suggested that people think we are witnessing the end of capitalism and the beginning of a new socialist era.

I certainly hope not.

I think a world without regulated capitalism would be a bleak one indeed. I had the great privilege to spend a year living in Russia in 2001/2002, and the visible evidence of the destruction wrought by central planning was still very much present. We are all ultimately human, with human failings, whether we work for a state planning agency or a private company, and those failings have consequences either way. To think that moving all private enterprise into state hands will somehow create a panacea of efficiency and sustainability is to ignore the stark lessons of the 20th century.

The leaders and decision makers in a centrally-planned economy are just as fallible as those in a capitalist one – they would probably be the same people! But state enterprises lack the forces of evolution that apply in a capitalist economy – state enterprises are rarely if ever allowed to fail. And hence bad ideas are perpetuated indefinitely, and an economy becomes dysfunctional to the point of systemic collapse. It is the fact that private enterprises fail which keeps industries vibrant. The tension between the imperative to innovate and the consequences of failure drives capitalist economies to evolve quickly. Despite all of the nasty consequences that we have seen, and those we have yet to see, of capitalism gone wrong, I am still firmly of the view that society must tap into its capitalist strengths if it wants to move forward.

But I chose my words carefully when I said “regulated capitalism”. I used to be a fan of Adam Smith’s invisible hand, and great admirer of Ayn Rand’s vision. Now, I feel differently. Left to it’s own devices, the market will tend to reinforce the position of those who were successful in the past, at the exclusion of those who might create future successes. We see evidence of this all the time. The heavyweights that define an industry tend to do everything in their power to prevent innovation from changing the rules that enrich them.

A classic example of that is the RIAA’s behaviour – in the name of “saving the music industry” they have spent the past ten years desperately trying to keep it in the analog era to save their members, with DRM and morally unjustifiable special-interest lobbying around copyright rules that affect the whole of society.

Similarly, patent rules tend to evolve to suit the companies that hold many patents, rather than the people who might generate the NEXT set of innovative ideas. Of course, the lobbying is dressed up in language that describes it as being “in the interests of innovation”, but at heart it is really aimed at preserving the privileged position of the incumbent.

In South Africa, the incumbent monopoly telco, which was a state enterprise until it was partially privatized in 1996, has systematically delayed, interfered, challenged and obstructed the natural process of deregulation and the creation of a healthy competitive sector. Private interests act in their own interest, by definition, so powerful private interests tend to drive the system in ways that make THEM healthier rather than ways that make society healthier.

Left to their own devices, private companies will tend to gobble one another up, and create monopolies. Those monopolies will then undermine every potential new entrant, using whatever tactics they can dream up, from FUD to lobbying to thuggery.

So, I’m a fan of regulated capitalism.

We need regulation to ensure that society’s broader needs, like environmental sustainability, are met while private companies pursue their profits. We also need regulation to ensure that those who manage national and international infrastructure, whether it’s railways or power stations or financial systems, don’t cook the books in a way that lets them declare fat profits and fatter bonuses while driving those systems into crisis.

But effective regulation is not the same as state management and supervision. I would much rather have private companies managing power stations competitively, than state agencies doing so as part of a complacent government monopoly.

Good regulation is very hard. Over the years I’ve interacted with a few different regulatory authorities, and I sympathise with the problems they encounter.

First, to be an effective regulator, you need superb talent. And for that you need to pay – talent follows the money and the lights, whether we like it or not, so to design a system on other assumptions is to design it for failure. My ideal regulator is an insightful genius working for the common good, but since I’m never likely to meet that person, a practical goal is to encourage regulators to be small but very well funded, with key salaries and performance measures that are just behind the industries they are supposed to regulate. Regulators must be able to be fired – no sense in offering someone a private sector salary and public sector accountability. Unfortunately, most regulators end up going the other way, hiring more and more people of average competence, that they become both expensive and ineffective.

Second, a great regulator needs to be independent. You’re the guy who tells people to stop doing what will hurt society; it’s very hard to do that to your friends. A regulatory job is a lonely job, which is why you hear so many stories of regulators being wined and dined by the industries they regulate only to make sure they don’t look too hard in the back room. A great regulator needs to know a lot about an industry, but be independent of that industry. Again, my ideal is someone who has made a good living in a sector, knows it backwards, can justify their high price, but wants to make a contribution to society.

Third, a great regulator needs to have teeth and muscle. It has been very frustrating for me to watch the South African telecomms regulator get tied up in court by Telkom, and stymied by government department inadequacy. Regulators need to be able to drive things forward, they need to be able to change the way companies behave, and they cannot rely on moral suasion to do so.

And fourth, a regulator has to make very tough decisions about innovation, which amount to venture capital decisions – to make them well, you have to be able to tell the future. For example, when an industry changes, as all industries change, how should the rules evolve? When a new need for society is identified, like the need to address climate change early and systemically, how should the rules evolve? Regulators need to move forward as fast as the industries they regulate, and they need to make decisions about things we don’t yet understand. And even when you regulate, you may not be able to stop an impending crisis. It’s very easy to criticize Greenspan for his light touch regulation on hedge funds and derivatives today, but it’s not at all clear to me that regulation would have made a difference, I think it would simply have moved the shadow global financial system offshore.

So regulation is extremely difficult, but also very much worth investing in if you are trying to run a healthy, vibrant, capitalist society.

Coming back to the original suggestion that sparked this blog – I’m sure we will see a lot of failed capitalists in the future. Hell, I might join their ranks, I wouldn’t be the first . But that doesn’t spell the end of capitalism, only the opportunity to start again – smarter.

With Intrepid on track to hit the wires today I thought I’d blog a little on the process we followed in designing the new user switcher, presence manager and session management experience, and lessons learned along the way. Ted has been blogging about the work he did, and it’s been mentioned in a couple of different forums (briefly earning the memorable title “the new hotness”), but since it’s one of the first pieces of work to go through the user experience design process within Canonical I thought it would be interesting to write it up.

Here is a screenshot of the work itself in action:

New FUSA applet allows you to mange your presence setting, as well as switch to a guest or other user, and logout

In one of the first user experience sessions, we looked in more detail at the way people “stop working”. We thought it interesting to try and group those actions together in a way which would feel natural to users.

We have already done some work in Ubuntu around this – for a long time we have had a button in the top-right corner of the panel which brought up a system modal dialog that gave you the usual “end your session” options of logout, restart, shutdown, hibernate, suspend and switch user. That patch was always a bit controversial and had not been accepted upstream, so we looked at ways to solve the problem differently.

We decided to use the top-right location, because it’s one of the key places in the screen that’s quick and easy to get to (you can throw your mouse into a corner of the screen very easily and accurately) and because there was a strong precedent in the old Ubuntu logout button.

One key insight was that we wanted to make “switching user” less an exercise in guesswork and more direct – we wanted to let people switch directly to the specific user they were interested in rather than have an intermediate step where they login as that other user. So we started with the Fast User Switcher applet, or FUSA, as a base fr the design. Another key idea that emerged was that we wanted to integrate the “presence setting” into the same menu, because “going offline” or “I’m busy” are similar state-of-mind-and-work decisions to “log me off the system” or “shut down”.

Menu order
We discussed at length the right order for the menu items. On the one hand, putting the “other users” at the top of the menu would mean that all the user names – yours and the ones you can switch to – would appear “in the same place” at the top of the menu. On the other, we strongly felt that things that would be used more casually and more easily should be at the top. In the end we settled on putting the presence management options at the top (Available, Away, Busy, Offline). Right next to those (in the same set) we put the “Lock screen” option, because it feels like a presence setting more than a session management setting – you are saying “Away” more than anything else.

Ted did a lot of work to make the presence menu elements work with both Pidgin and Empathy because there was some uncertainty as to which would be used by default in the release. Since it all uses dbus, it should be straightforward to make it work with KDE IM clients too.

We then put the user switching options – including the Guest Session which is a cool new feature in Intrepid that as been widely blogged (check out the YouTube demo) and which uses AppArmor to enforce security.

And finally, the session termination options – log out, suspend, hibernate, restart and shutdown are at the bottom of the menu, because you’re only ever likely to use them once in a session, by definition!

Styling
The design of the menu is deliberately clean. We use very simple colours and shapes for the presence indicators, and replicate those colours and shapes in the actual GNOME panel so that you can see at a glance what your current presence setting is. Ted had to jump through some hoops, I think, to get the presence icons in the menu to line up with the current-presence-status indicator in the panel applet, but it worked out quite nicely. There’s some additional work to tighten up the layout which didn’t make it in time for the release but which might come in as a stable release update (SRU) or in Jaunty.

We decided not to put icons into the menu for each of the different statuses. Our design ethic is to aim for cleaner, less cluttered layouts with fewer icons and better choice of text. A couple of people have said that the menu looks “sparse” or “bare” but I think it sets the right direction and we’ll be continuing with this approach as we touch other parts of the system.

Upstream
This work was discussed at UDS in Prague with a number of members of the GNOME community. I was also very glad to see that there’s a lot of support for a tighter, simpler panel at the GNOME hackfest, an idea that we’ve championed. The FUSA applet itself is going through a bit of a transformation upstream as it’s been merged into the new GDM codebase and the old code – on which our work is based – is more or less EOL’d. But we’ll figure out how to update this work for Jaunty and hopefully it will be easier to get it upstreamed at that point.

In Jaunty, we’ll likely do some more work on the GNOME panel, building on the GNOME user experience discussions. There was a lot of discussion about locking down the panel more tightly, which we may pursue.

Integration into Ubuntu
We realised rather late in the Ubuntu cycle that we hadn’t thought much about packaging. The Ubuntu team had kindly offered to help package and integrate the applet but we definitely learned the value of getting the packaging done earlier rather than later. We had the applet in a PPA for testing between developers fairly early, but we underestimated the difference between that and actual integration into the release.

The Ubuntu team rallied to the cause and helped to smooth the upgrade process for new users, so that we can try to get everyone onto the same footing when they start out with Intrepid whether as a new install or an upgrade. There are some challenges there, because the panel is so customisable, and we had to think hard about how we could ensure there was a consistent experience for something as important as logging out or shutting down while at the same time trying not to stomp on the preferences of folks who have customised their panels. Similarly, we were concerned that people who run different versions of Ubuntu, or different distributions entirely, with the same home directory, would have problems if those other OS’s didn’t have the same version of FUSA – we weren’t really able to address that satisfactorily.

We also realised (DOH!) that we hadn’t thought all the way through the process of integration, because we hadn’t figured out what to do with the old System menu options. It turned out that those were in a state of flux, with the Ubuntu folks having to choose between the current GNOME default which everyone said would change, the patches for the likely NEXT GNOME approach, and the old Ubuntu approach. Ted whipped up some patches to make the GNOME panel more dynamic with its menus, so that we could remove the System menu logout options when people have the same menu in the FUSA applet, but that landed too late for inclusion into Intrepid final.

All in all, I think it’s a neat piece of work and hope other distro’s find it useful too. It’s just a teaser of the work we plan to do around the desktop experience. I’m looking forward to seeing everyone at UDS Jaunty in Mountain View in December, when we can talk about the next round! Thanks and well done to Ted, Martin, Scott, Sebastien and everyone else who helped to make this a reality.

Well done to Team Ubuntu (thousands of people across Ubuntu, Debian and upstreams) who make the magic in 8.10 possible. Happy Release Day everyone!

The GNOME user experience hackfest in Boston was a great way to spend the worst week in Wall St history!

Though there wasn’t a lot of hacking, there was a LOT of discussion, and we covered a lot of ground. There were at least 7 Canonical folks there, so it was a bit of a mini-sprint and a nice opportunity to meet the team at the same time. We had great participation from a number of organisations and free spirits, there’s a widespread desire to see GNOME stay on the forefront of usability.

Neil Patel of Canonical did a few mockups to try and capture the spirit of what was discussed, but I think the most interesting piece wasn’t really possible to capture in a screenshot because it’s abstract and conceptual – file and content management. There’s a revolution coming as we throw out the old “files and folders” metaphor and leap to something new, and it would be phenomenal if free software were leading the way.

I was struck by the number of different ways this meme cropped up. We had superb presentations of “real life support problems” from a large-scale user of desktop Linux, and a persistent theme was “where the hell did that file just go?” People save an attachment they receive in email, and an hour later have no idea where to find it. They import a picture into F-spot and then have no idea how to attach it to an email. They download a PDF from the web, then want to read it offline and can’t remember where they put it. Someone else pointed out that most people find it easier to find something on the Internet – through Google – than they do on their hard drives.

The Codethink guys also showed off some prototype experience work with Wizbit, which is a single-file version control system that draws on both Git and Bazaar for ideas about how you do efficient, transparent versioning of a file for online and offline editing.

We need to rearchitect the experience of “working with your content”, and we need to do it in a way that will work with the web and shared content as easily as it does locally.

My biggest concern on this front is that it be done in a way that every desktop environment can embrace. We need a consistent experience across GNOME, KDE, OpenOffice and Firefox so that content can flow from app to app in a seamless fashion and the user’s expectations can be met no matter which app or environment they happen to use. If someone sends a file to me over Empathy, and I want to open it in Amarok, then I shouldn’t have to work with two completely different mental models of content storage. Similarly, if I’ve downloaded something from the web with Firefox, and want to edit it in OpenOffice, I shouldn’t have to be super-aware or super-smart to be able to connect the apps to the content.

So, IMO this is work that should be championed in a forum like FreeDesktop.org, where it can rise above some of the existing rivalries of desktop linux. There’s a good tradition of practical collaboration in that forum, and this is a great candidate for similar treatment.

At the end of the day, bling is less transformational than a fundamental shift in content management. Kudos to the folks who are driving this!

Update: thanks mjg59 for pointing out my thinko. The Collabora guys do great stuff, but Codethink does Wizbit.