Mark Shuttleworth | here be dragons

Thanks to the concerted efforts of Martin Pitt, Sebastien Bacher and several others, notify-osd and several related components landed in Jaunty last week. Thanks very much to all involved! And thanks to David Barth, Mirco Muller and Ted Gould who lead the development of notify-osd and the related messaging indicator.

Notify-OSD handles both application notifications and keyboard special keys like brightness and volume

MPT has posted an overview of the conceptual framework for “attention management” at https://wiki.ubuntu.com/NotificationDesignGuidelines, which puts ephemeral notification into context as just one of several distinct tools that applications can use when they don’t have the focus but need to make users aware of something. That’s a draft, and when it’s at 1.0 we’ll move it to a new site which will host design patterns on Canonical.com.

There is also a detailed specification for our implementation of the notification display agent, notify-osd, which can be found at https://wiki.ubuntu.com/NotifyOSD and which defines not only the expected behaviour of notify-osd but also all of the consequential updates we need to make across the packages in main an universe to ensure that those applications use notification and other techniques consistently.

There are at least 35 apps that need tweaking, and there may well be others! If you find an app that isn’t using notifications elegantly, please add it to the notification design guidelines page, and if you file a bug on the package, please tag it “notifications” so we can track these issues in a single consistent way.

Together with notify-osd, we’ve uploaded a new panel indicator which is used to provide a way to respond to messaging events, such as email and IRC pings. If someone IM’s you, then you should see an ephemeral notification, and the messaging indicator will give you a way to respond immediately. Same for email. Pidgin and Evolution are the primary focuses of the work, over time we’ll broaden that to the full complement of IM and email apps in the archive – patches welcome

There will be rough patches. Apps which don’t comply with the FreeDesktop.org spec and send actions on notifications even when the display agent says it does not support them, will have their notifications translated into alerts. That’s the primary focus of the effort now, the find and fix those apps. Also, we know there are several cases where a persistent response framework is required. The messaging indicator gets most of them, we will have additional persistent tools in place for Karmic in October.

Let’s talk about notifications! As Ryan Lortie mentioned, there was a lot of discussion across the Ubuntu, Kubuntu, GNOME, KDE and Mozilla communities represented at UDS about the proposals Canonical’s user experience design and desktop experience engineering teams have made for Ubuntu 9.04.

See the mockup as a Flash movie.

There are some fairly bold (read: controversial) ideas that we’d like to explore with, so the opportunity to discuss them with a broader cross-section of the community was fantastic. There were several rough edges and traps that I think we’ll avoid in the first round as a result, thanks to everyone who participated. Some of the things we work on in these teams are done directly with partners for their devices, so they don’t see this level of discussion before they ship, but it’s wonderful when we do get the opportunity to do so.

Some of these ideas are unproven, they boil down to matters of opinion, but since our commitment to them is based on a desire to learn more I think of them as constructive experiments. Experiments are just that – experiments. They may succeed and they may fail. We should judge them carefully, after we have data. We are putting new ideas into the free desktop without ego. We know those ideas could be better or worse than similar work being done in other communities, and we want to gather real user feedback to help find the best mix for everyone. The best ideas, and the best code, will ultimately form part of the digital free software commons and be shared by GNOME, KDE and every distribution. So, for those folks who were upset that we might ship something other than a GNOME or KDE default, I would ask for your patience and support – we want to contribute new ideas and new code, and that means having some delta which can be used as a basis for discussions about the future direction of upstream. In the past, we’ve had a few such delta’s in Ubuntu. Some, like the current panel layout, were widely embraced. Others, like the infamous “Ubuntu spacial mode”, were not. C’est la vie, and we all benefit from the evolution.

Experiments are also not something we should do lightly. The Ubuntu desktop is something I take very personally; I feel personally responsible for the productivity and happiness of every Ubuntu user, so when we bring new ideas and code to the desktop I believe we should do everything we can to make sure of success first time round. We should not inflict bad ideas on our users just because we’re curious or arrogant or stubborn or proud. Despite being occasionally curious, arrogant, stubborn and proud

So, what are we proposing?

First, we are focusing some attention on desktop notifications in this cycle, as part of a broader interest in the “space between applications”.

I think Canonical and Ubuntu can best help the cause of free software by focusing on the cracks between the major components of the desktop. In other words, while there are already great upstreams for individual applications in the free software desktop (Novell for Evolution, Sun for OpenOffice, Mozilla for Firefox, Red Hat for NetworkManager), we think there is a lot of productive and useful work to be done in the gaps between them. Notifications are things that many apps do, and if we can contribute new ideas there then we are helping improve the user experience of all of those applications. That’s a nice force multiplier – we’re hopefully doing work that makes the work of every other community even more valuable.

Nevertheless, expect bumps ahead. Ideas we are exploring may / will / do conflict with assumptions that are present today in various applications. We can address the relevant code in packages in main, but I’m more focused on addressing the potential social disruption that conflict can create, and that’s more a matter of conversation than code.

Notifications are interesting, subtle and complex. There are lots of different approaches on lots of different platforms. There are lots of different use cases. We’re trying to simplify and eliminate complexity, while still making it possible to meet the use cases we know about.

There has been good work in the freedesktop.org community on notifications, and even a spec that is *almost* at 1.0 in that community, with existing open source implementations. Our proposal is based on that specification, but it deprecates several capabilities and features in it. We will likely be compatible with the current API’s for sending notifications, but likely will not display all the notifications that might be sent, if they require features that we deprecate. If this experiment goes well, we would hope to help move that FD.o specification to 1.0, with or without our amendments.

The key proposals we are making are that:

• There should be no actions on notifications.
• Notifications should not be displayed synchronously, but may be queued. Our implementation of the notification display daemon will display only one notification at a time, others may do it differently.

That’s pretty much it. There are some subtleties and variations, but these are the key changes we are proposing, and which we will explore in a netbook device with a partner, as well as in the general Ubuntu 9.04 release, schedule gods being willing.

This work will show up as a new notification display agent, not as a fork or patch to the existing GNOME notification daemon. We don’t think the client API – libnotify – needs to be changed for this experiment, though we may not display notifications sent through that API that use capabilities we are suggesting be deprecated. We will try to ensure that packages in main are appropriately tuned, and hope MOTU will identify and update key packages in universe accordingly.

Why a completely new notification display agent? We are designing it to be built with Qt on KDE, and Gtk on GNOME. The idea is to have as much code in common as we can, but still take advantage of the appropriate text display framework on Ubuntu and Kubuntu. We hope to deliver both simultaneously, and have discussed this with both Ubuntu and Kubuntu community members. At the moment, there is some disagreement about the status of the FD.o specification between GNOME and KDE, and we hope our efforts will help build a bridge there. In Ubuntu 9.04, we would likely continue to package and publish the existing notification daemon in addition, to offer both options for users that have a particular preference. In general, where we invest in experimental new work, we plan to continue to offer a standard GNOME or KDE component / package set in the archive so that people can enjoy that experience too.

The most controversial part of the proposal is the idea that notifications should not have actions associated with them. In other words, no buttons, sliders, links, or even a dismissal [x]. When a notification pops up, you won’t be able to click on it, you won’t be able to make it go away, you won’t be able to follow it to another window, or to a web page. Are you loving this freedom? Hmmm? Madness, on the face of it, but there is method in this madness.

Our hypothesis is that the existence of ANY action creates a weighty obligation to act, or to THINK ABOUT ACTING. That make notifications turn from play into work. That makes them heavy responsibilities. That makes them an interruption, not a notification. And interruptions are a bag of hurt when you have things to do.

So, we have a three-prong line of attack.

1. We want to make notifications truly ephemeral. They are there, and then they are gone, and that’s life. If you are at your desktop when a notification comes by, you will sense it, and if you want you can LOOK at it, and it will be beautiful and clear and easy to parse. If you want to ignore it, you can safely do that and it will always go away without you having to dismiss it. If you miss it, that’s OK. Notifications are only for things which you can safely ignore or miss out on. If you went out for coffee and a notification flew by, you are no worse off. They don’t pile up like email, there is no journal of the ones you missed, you can’t scroll back and see them again, and therefor you are under no obligation to do so – they can’t become work while you are already busy with something else. They are gone like a mystery girl on the bus you didn’t get on, and they enrich your life in exactly the same way!
2. We think there should be persistent panel indicators for things which you really need to know about, even if you missed the notification because you urgently wanted that coffee. So we are making a list of those things, and plan to implement them.
3. Everything else should be dealt with by having a window call for attention, while staying in the background, unless it’s critical in which case that window could come to the foreground.

Since this is clearly the work of several releases, we may have glitches and inconsistencies along the way at interim checkpoints. I hope not, but it’s not unlikely, especially in the first iteration. Also, these ideas may turn out to be poor, and we should be ready to adjust our course based on feedback once we have an implementation in the wild.

We had a superb UXD and DEE (user experience design team, and desktop experience engineering team) sprint in San Francisco the week before UDS. Thanks to everyone who took part, especially those who came in from other teams. This notifications work may just be the tip of the iceberg, but it’s a very cool tip

One or more of our early-access OEM partners (companies that we work with on new desktop features) will likely ship this feature as part of a netbook product during the 9.04 cycle. At that point, we would also drop the code into a PPA for testing with a wider set of applications. There are active discussions about updating the freedesktop.org specification based on this work. I think we should be cautious, and gather real user testing feedback and hard data, but if it goes well then we would propose simplifying the spec accordingly, and submit our notification display agent to FreeDesktop.org. Long term collaboration around the code would take place on Launchpad.

With Intrepid on track to hit the wires today I thought I’d blog a little on the process we followed in designing the new user switcher, presence manager and session management experience, and lessons learned along the way. Ted has been blogging about the work he did, and it’s been mentioned in a couple of different forums (briefly earning the memorable title “the new hotness”), but since it’s one of the first pieces of work to go through the user experience design process within Canonical I thought it would be interesting to write it up.

Here is a screenshot of the work itself in action:

New FUSA applet allows you to mange your presence setting, as well as switch to a guest or other user, and logout

In one of the first user experience sessions, we looked in more detail at the way people “stop working”. We thought it interesting to try and group those actions together in a way which would feel natural to users.

We have already done some work in Ubuntu around this – for a long time we have had a button in the top-right corner of the panel which brought up a system modal dialog that gave you the usual “end your session” options of logout, restart, shutdown, hibernate, suspend and switch user. That patch was always a bit controversial and had not been accepted upstream, so we looked at ways to solve the problem differently.

We decided to use the top-right location, because it’s one of the key places in the screen that’s quick and easy to get to (you can throw your mouse into a corner of the screen very easily and accurately) and because there was a strong precedent in the old Ubuntu logout button.

One key insight was that we wanted to make “switching user” less an exercise in guesswork and more direct – we wanted to let people switch directly to the specific user they were interested in rather than have an intermediate step where they login as that other user. So we started with the Fast User Switcher applet, or FUSA, as a base fr the design. Another key idea that emerged was that we wanted to integrate the “presence setting” into the same menu, because “going offline” or “I’m busy” are similar state-of-mind-and-work decisions to “log me off the system” or “shut down”.

Menu order
We discussed at length the right order for the menu items. On the one hand, putting the “other users” at the top of the menu would mean that all the user names – yours and the ones you can switch to – would appear “in the same place” at the top of the menu. On the other, we strongly felt that things that would be used more casually and more easily should be at the top. In the end we settled on putting the presence management options at the top (Available, Away, Busy, Offline). Right next to those (in the same set) we put the “Lock screen” option, because it feels like a presence setting more than a session management setting – you are saying “Away” more than anything else.

Ted did a lot of work to make the presence menu elements work with both Pidgin and Empathy because there was some uncertainty as to which would be used by default in the release. Since it all uses dbus, it should be straightforward to make it work with KDE IM clients too.

We then put the user switching options – including the Guest Session which is a cool new feature in Intrepid that as been widely blogged (check out the YouTube demo) and which uses AppArmor to enforce security.

And finally, the session termination options – log out, suspend, hibernate, restart and shutdown are at the bottom of the menu, because you’re only ever likely to use them once in a session, by definition!

Styling
The design of the menu is deliberately clean. We use very simple colours and shapes for the presence indicators, and replicate those colours and shapes in the actual GNOME panel so that you can see at a glance what your current presence setting is. Ted had to jump through some hoops, I think, to get the presence icons in the menu to line up with the current-presence-status indicator in the panel applet, but it worked out quite nicely. There’s some additional work to tighten up the layout which didn’t make it in time for the release but which might come in as a stable release update (SRU) or in Jaunty.

We decided not to put icons into the menu for each of the different statuses. Our design ethic is to aim for cleaner, less cluttered layouts with fewer icons and better choice of text. A couple of people have said that the menu looks “sparse” or “bare” but I think it sets the right direction and we’ll be continuing with this approach as we touch other parts of the system.

Upstream
This work was discussed at UDS in Prague with a number of members of the GNOME community. I was also very glad to see that there’s a lot of support for a tighter, simpler panel at the GNOME hackfest, an idea that we’ve championed. The FUSA applet itself is going through a bit of a transformation upstream as it’s been merged into the new GDM codebase and the old code – on which our work is based – is more or less EOL’d. But we’ll figure out how to update this work for Jaunty and hopefully it will be easier to get it upstreamed at that point.

In Jaunty, we’ll likely do some more work on the GNOME panel, building on the GNOME user experience discussions. There was a lot of discussion about locking down the panel more tightly, which we may pursue.

Integration into Ubuntu
We realised rather late in the Ubuntu cycle that we hadn’t thought much about packaging. The Ubuntu team had kindly offered to help package and integrate the applet but we definitely learned the value of getting the packaging done earlier rather than later. We had the applet in a PPA for testing between developers fairly early, but we underestimated the difference between that and actual integration into the release.

The Ubuntu team rallied to the cause and helped to smooth the upgrade process for new users, so that we can try to get everyone onto the same footing when they start out with Intrepid whether as a new install or an upgrade. There are some challenges there, because the panel is so customisable, and we had to think hard about how we could ensure there was a consistent experience for something as important as logging out or shutting down while at the same time trying not to stomp on the preferences of folks who have customised their panels. Similarly, we were concerned that people who run different versions of Ubuntu, or different distributions entirely, with the same home directory, would have problems if those other OS’s didn’t have the same version of FUSA – we weren’t really able to address that satisfactorily.

We also realised (DOH!) that we hadn’t thought all the way through the process of integration, because we hadn’t figured out what to do with the old System menu options. It turned out that those were in a state of flux, with the Ubuntu folks having to choose between the current GNOME default which everyone said would change, the patches for the likely NEXT GNOME approach, and the old Ubuntu approach. Ted whipped up some patches to make the GNOME panel more dynamic with its menus, so that we could remove the System menu logout options when people have the same menu in the FUSA applet, but that landed too late for inclusion into Intrepid final.

All in all, I think it’s a neat piece of work and hope other distro’s find it useful too. It’s just a teaser of the work we plan to do around the desktop experience. I’m looking forward to seeing everyone at UDS Jaunty in Mountain View in December, when we can talk about the next round! Thanks and well done to Ted, Martin, Scott, Sebastien and everyone else who helped to make this a reality.

Well done to Team Ubuntu (thousands of people across Ubuntu, Debian and upstreams) who make the magic in 8.10 possible. Happy Release Day everyone!

I had the opportunity to present at the Linux Symposium on Friday, and talked further about my hope that we can improve the coordination and cadence of the entire free software stack. I tried to present both the obvious benefits and the controversies the idea has thrown up.

Afterwards, a number of people came up to talk about it further, with generally positive feedback.

Christopher Curtis, for example, emailed to say that the idea of economic clustering in the motor car industry goes far further than the location of car dealerships. He writes:

Firstly, every car maker releases their new models at about the same time. Each car maker has similar products – economy, sedan, light truck. They copy each other prolifically. Eventually, they all adopt a certain baseline – seatbelts, bumpers, airbags, anti-lock brakes. Yet they compete fiercely (OnStar from GM; Microsoft Sync from Ford) and people remain brand loyal. This isn’t going to change in the Linux world. Even better, relations like Debian->Ubuntu match car maker relations like Toyota->Lexus.

I agree with him wholeheartedly. Linux distributions and car manufacturers are very similar: we’re selling products that reach the same basic audience (there are niche specialists in real-time or embedded or regional markets) with a similar range (desktop, workstation, server, mobile), and we use many of the same components just as the motor industry uses common suppliers. That commonality and coordination benefits the motor industry, and yet individual brands and products retain their identity.

Let’s do a small thought experiment. Can you name, for the last major enterprise release of your favourite distribution, the specific major versions of kernel, gcc, X, GNOME, KDE, OpenOffice.org or Mozilla that were shipped? And can you say whether those major versions were the same or different to any of the enterprise releases of Ubuntu, SLES, Debian, or RHEL which shipped at roughly the same time? I’m willing to bet that any particular customer would say that they can’t remember either which versions were involved, or how those stacked up against the competition, and don’t care either. So looking backwards, differences in versions weren’t a customer-differentiating item.  We can do the same thought experiment looking forwards. WHAT IF you knew that the next long-term supported releases of Ubuntu, Debian, Red Hat and Novell Linux would all have the same major versions of kernel, GCC, X, GNOME, KDE, OO.o and Mozilla. Would that make a major difference for you? I’m willing to bet not – that from a customer view, folks who prefer X will still prefer X. A person who prefers Red Hat will stick with Red Hat. But from a developer view, would that make it easier to collaborate? Dramatically so.

Another member of the audience came up to talk about the fashion industry. That’s also converged on a highly coordinated model – fabrics and technologies “release” first, then designers introduce their work simultaneously at fashion shows around the world. “Spring 2009″ sees new collections from all the major houses, many re-using similar ideas or components. That hasn’t hurt their industry, rather it helps to build awareness amongst the potential audience.

The ultimate laboratory, nature, has also adopted release coordination. Anil Somayaji, who was in the audience for the keynote, subsequently emailed this:

Basically, trees of a given species will synchronize their seed releases in time and in amount, potentially to overwhelm predators and to coordinate with environmental conditions. In effect, synchronized seed releases is a strategy for competitors to work together to ensure that they all have the best chance of succeeding. In a similar fashion, if free software were to “release its seeds” in a synchronized fashion (with similar types of software or distributions having coordinated schedules, but software in different niches having different schedules), it might maximize the chances of all of their survival and prosperity.

There’s no doubt in my mind that the stronger the “pulse” we are able to create, by coordinating the freezes and releases of major pieces of the free software stack, the stronger our impact on the global software market will be, and the better for all companies – from MySQL to Alfresco, from Zimbra to OBM, from Red Hat to Ubuntu.

A distribution occupies a very specific niche in the free software ecosystem. Among other things, we need to accept some responsibility for ALL the software defects (“bugs”) that users actually experience across the entire stack. Most users don’t install their apps from upstream source tarballs, they install them from the packages provided by their distribution. So when they experience a bug, they don’t know if it’s a bug introduced by that distribution, or a bug in the underlying upstream code. They don’t know, they don’t care, and they shouldn’t have to. More often than not they will report the issue to their distribution, and the way we respond to it is important, because it represents an opportunity to make the whole ecosystem more robust.

I had a lecturer who was very opposed to the use of the term “bugs”. He said that the term “bug” was a cute-sification for “nasty biting insect”, and similarly, software defects have potentially serious consequences, so we shouldn’t treat them lightly. Bug work is serious work, and it’s one of the most important forms of contribution to the digital commons that Ubuntu can make, so I’d like to salute the extraordinary efforts of the Ubuntu Quality Assurance Team and Bug Squad. Initiatives like five-a-day are already making a huge difference to our users. As Henrik Omma says, effective bug reporting requires a diligent and professional approach, and I’ve noticed a real improvement in our community. Hopefully, we can bring the benefits of that competence to the broader free software ecosystem.

Ubuntu gets as many bugs reported against it as OpenOffice, Mozilla, Gnome, and KDE combined.The vast majority of those bugs are issues that exist in upstream tarball releases, or in Debian. Our primary goals should be to ensure that fixes we produce, and information we generate in the QA process, make their way upstream where they will benefit the broadest cross-section of the community. Separately, we want to ensure that each Ubuntu release ships without major issues, regardless of where those issues originated. We are responsible for the user experience of every line of code, even though we don’t produce every line of code.

In the month of April 2008, I found the following bug counts for large FLOSS projects:

With hindsight, April was possibly a bad choice, because it was an Ubuntu release month so there’s usually a small spike in the number of bugs filed. It would be interesting to see the stats for other distributions, and projects, over a full year. But the general picture is clear – within our family of distributions, Ubuntu carries the brunt of the load w.r.t. bug tracking, triage and patch management – not only for our users, but for a broad cross-section of the open source stack.

When I delved into the data to see how we do with pushing bugs upstream, I found a somewhat mixed picture. In many cases, we do very well indeed. We have a very good relationship with GNOME, for example, with a very high percentage of bugs appropriately forwarded to the relevant upstream bug tracker. In other projects, it’s harder to make a definitive statement. The percentage varies based on whether the Ubuntu team members have good relationships upstream, or whether there’s a person acting as an ambassador from Ubuntu to upstream (this is a great way to make a difference if you care about a specific application in Ubuntu!) or whether upstream themselves have taken an interest.

We need to improve the tools that support these kinds of cross-project conversations. Launchpad does currently allow us to track the status of a bug in many different bug trackers, and there are quite a few distributions and upstreams that are now either using Launchpad directly or exchanging data efficiently. We’ll keep working to improve the quality of exchange across the whole ecosystem, including those projects that don’t use Launchpad themselves

Vincenzo Ciaglia from Linux-Magazine Italia sent me a few questions related to the release of 8.04 LTS. Since he was going to translate the conversation into Italian this week, he was happy for me to blog the English version here.

1) Hi Mark, thank you for your availability. Some simple questions to introduce you to our readers to start. What’s your role at Ubuntu/Canonical and what do you do in your spare time? What are your hobbies?
My favourite sport is snowboarding, and I enjoy travel to tropical spots. But ultimately I’m happiest when I’m being a geek, reading, playing games or relaxing with friends.

2) You’re the founder of Canonical, the company behind Ubuntu. Why do you decided to invest a lot of dollars (10$ million) to start up the company? In which fields does it work to make its business? How do you make the company sustainable?
The vision of Ubuntu and Canonical is a symbiotic one. We believe that Linux has grown to the point where it is possible to build the platform at a low enough cost to make it sustainable purely though services around it, rather than through licensing the platform. In other words, we think that support, training, online services, and professional engineering for people who want to adapt Ubuntu commercially will earn enough money to pay for Ubuntu itself.

That means that we can fundamentally change the business model of the OS industry. Of course, it till take time to prove that we can achieve this, but we have a superb team and now that Ubuntu is well established we see increasing demand for services from Canonical, which is positive.

3) Ok, let’s talk about the latest Ubuntu 8.04. In an interview you said that “Hardy Heron is your most significant release ever”. Well, can you talk about the main improvements of this release?
First, this is an LTS (“Long Term Support”) release that was delivered on a very precise schedule. Six months ago we committed to shipping 8.04 LTS on April 24th, and we did exactly that. As far as I know, nobody has ever shipped an “enterprise class” OS release on a schedule that precise. And not only did we do that, but we have now committed to ship the next LTS in April 2010, it will be 10.04 LTS, and we’ll set the exact date six months in advance like we did with this one. It is thanks to Debian and the free software community that it is possible for us to do this. So 8.04 LTS has proven our ability to deliver not just 18-month-supported releases on time, but also LTS releases on time. We very much hope that other distributions will follow our lead on the LTS cycle with their enterprise releases, because that will make it easier for us all to collaborate, and make all the major Linux distributions better.

Second, there are very significant new developments for Ubuntu itself. On the server, we worked with HP on their Proliant range, and with Dell on their PowerEdge range, to ensure that 8.04 LTS will be compatible with their popular x86 servers. We’re not yet certified, but we are sure that it will “Just Work”. Sun Microsystems has gone further, and has actually certified 8.04 LTS on a range of their x86 servers. This is a major step forward for Ubuntu on the server. We see an amazing amount of usage now for Ubuntu on the server – it’s the most popular server platform for several ISV’s. So it’s important that we work with server vendors, and server solution vendors. We’ve also put a lot of work into the use of KVM and VMWare virtualisation, because we see people building hundreds of virtual appliances on Ubuntu.

On the desktop, we have focused on making it easier to install Ubuntu, especially on a machine which already has Windows, where you can now install Ubuntu into a file on the Windows partition instead of having to resize your Windows partition to make a new partition for Ubuntu. That makes it much easier for people to test out Ubuntu, and hence to get a taste of free software. We have also worked on many of the common things that people want to do with their PC, such as work with photos and music, and started to improve the user experience there.

4) There are still some hardware issues (especially with some wireless devices) in Ubuntu. How do you think to solve these kind of problems and improve hardware support in the next releases?
We are increasingly able to work directly with the hardware manufacturers, to try and convince them to develop free software drivers for their hardware. Our relationships with different PC companies mean that we can lobby strongly for people to embrace Linux, properly.

We also work very hard to tie together all the ugly pieces of string that are needed to make the user experience of Linux on your hardware a pleasant one. Unfortunately, for example, there are multiple different wireless stacks, for example, with different capabilities. And Ubuntu spends a lot of time integrating and debugging them to try and create a harmonious, standardised experience for end-users.

5) What kind of improvements Ubuntu 8.04 bring to server and virtualization solutions?
Ubuntu Server Edition brings all the wonderful characteristics of Debian to the front – it’s modular, efficient, has a huge package selection, and is easy to install and manage. In addition, we’ve done a lot of work with server manufacturers to ensure compatibility with their popular hardware, and have started certifying with some of them.

Our virtualisation offering is based around KVM and VMWare. Out of the box, Ubuntu should give you the best possible experience with both of them. It is optimised and rigorously tested, and Ubuntu is certified on VMWare’s ESX Server platform. KVM, the free software virtualisation option we prefer, is built in to our standard server kernel, so you can get started with virtualisation immediately. There is also a Xen kernel for folks who prefer Xen.

We have done a lot of work around the integration of Ubuntu servers and Windows networks, particularly in the field of Active Directory and SMB file sharing servers. We worked with a company called Likewise to make sure that there is a smooth process to join an Active Directory network, and can even manage Linux machines through AD using Likewise’ professional tools. All the capabilities to do the basic stuff are free software and built in to Ubuntu.

6) A dirty question from our readers: Ubuntu is really a giant now, are you trying to kill the Debian project?
Absolutely not. I’m a Debian Developer myself, and very proud of what Debian has achieved, and also proud of everything that Ubuntu contributes to the broader Debian project. We consider Ubuntu to be a member of the Debian family, that’s just purely focused on the specific use cases and platforms that our customers want.

Much of what we do in Ubuntu contributes directly to Debian. We lead the packaging of many important pieces of the desktop, and server, and toolchain, and contribute that work directly to Debian. As a result, Debian is updated much faster these days than it used to be without Ubuntu. We have lead many key transitions and always try to collaborate with the relevant people upstream AND in Debian to ensure that the work flows smoothly into those projects. Most DD’s are very happy to collaborate, but some view Ubuntu as a threat, and refuse to collaborate, or make unreasonable demands on Ubuntu because they think “you have money” when in fact most of Ubuntu is volunteer driven.

My vision is that Debian and Ubuntu both grow stronger through good collaboration. I’m trying to have a keynote accepted at DebConf to help make that vision a reality, but so far have had no luck in getting approval. Hopefully, the leadership of Debian will start to come around tot he idea that Ubuntu’s success is very good for Debian.

7) You’re working with embedded devices and electronics company. In which way? What is you work in the tight partnership with Intel?
Linux is increasingly used by embedded solution providers, and many of them want to use Ubuntu. So we are working with Intel to make sure that Ubuntu fully supports their low-power hardware (cpu’s, chipsets, graphics and so on).

Ubuntu surely is the most used and appreciated GNU/Linux distribution in the world. But, do you think that Ubuntu Linux will reach – one day – the success of other operating systems like OS X and Windows? In which way do you think to accomplish a similar goal?
I do believe that free software will come to be widely recognised, trusted and used by everyday computer users, as opposed to being limited to specialists as it is today. Hopefully Ubuntu will play a part in that, but I don’t think one platform will dominate that free software era like Windows dominated the proprietary software era. Ubuntu is focused on specific needs, and there are other versions of Linux or BSD that meet others.

In order to break out from the pack, we need to deliver a desktop experience that is exciting, that is easy to use, and which people are confident will be compatible with their future needs and with those of their colleagues.

9) Everybody talk about GNU/Linux but seems that not so many peoples trust Linux for now (some statistics talk about 0.xx% of the Linux world usage). What is the problem? Nevertheless Ubuntu is really a great operating system. Do you think it’s just a matter of marketing or because the lack of game packages?
I think it takes time to change the habits of hundreds of millions of people! I also think we need to deliver an experience that is simply better than the alternatives. Projects like Firefox don’t define their goal as being “a good browser”, they say “we want to be the best browser on any platform, period”, and as a result they are very popular even on Windows. We need that winning attitude everywhere.

10) Canonical was the first GNU/Linux company to make a deal with a computer vendor, Dell. How is going your business now? The curious thing is that on IdeaStorm a lot of users asked for Linux Computers but after some months seems that the sales aren’t so good, or at least not as expected. Why? Maybe GNU/Linux is not ready for the consumer market yet?
I agree that the more people actually buy systems with Linux pre-installed, the faster things like hardware support will be addressed. Fortunately, we see millions of units being shipped with Linux already, just not at the high-end of the PC business. The low-end, especially in countries like Brazil and China, is very active. Slowly, Linux is becoming a volume player.

11) Are you working with other computer vendor to sell other Ubuntu-based desktops and laptops? We’re Italians and the Ubuntu-Dell computers never arrived in our country. Why Dell, and other vendor, is so shy to sell GNU/Linux computers world wide?
That’s a simple matter of demand and the cost to meet it. Any PC manufacturer will take a firm view of the economics of an opportunity, and it’s healthy that they do. Until folks in Italy are really willing to buy computers with Linux pre-installed, there will not be a real market for them. I’m sure there are local providers who build good computers who will pre-install Ubuntu for you. You need to help them become big enough that the Dell’s and HP’s and Lenovo’s of the world are confident there is an opportunity that is worthwhile for them.

12) What is your point of view about the the Novell-Microsoft controversial deal?
There are some good intentions, and there are some bad intentions, and unfortunately they are all mixed up in that deal. On the positive side, it’s good to see Microsoft acknowledge the need for both Linux and Windows, and the need for interoperability. On the negative side, the deal only works financially because Novell and Microsoft have the same business model – licensing software for a certain price per seat.

Microsoft is in an awkward position. They very much want to stop the free use of Linux, and they would like to use patents to do so, which is why they structured the deal as a notional “IP license”. But they also know that free software engineers could probably avoid any patents they raise, so they have been unwilling to state which patents they think justify such a deal.

13) In a interview (http://mybroadband.co.za/nephp/?m=show&id=6672) you declared that “you’d love to work with Microsoft”. Do you want to make another deal following the Novell one or what?
I am very happy to work with Microsoft, or any other company, to improve the state of free software and the software industry as a whole. There are many things that we can collaborate on where we have shared interests – encouraging good telecommunications policy, for example.

But I will not agree to a deal like the Novell one, because I don’t think there is any IP issue in fact, and until Microsoft actually states what patents it is concerned about there is no need for us to take any action. Unfortunately for Novell, I think they have done a deal which gives them a short-term boost, at a very high long term cost. Time will tell.

14) And what do you think about the OOXML standard and the Microsoft Open Promise?
I don’t believe that ISO’s declaration of OOXML as a standard will actually deliver any benefit to users of Microsoft Office. They will still be using a big, bloated piece of software with no competition, that is not-quite-standards-compliant. That’s a pity. Microsoft’s customers had an opportunity to ignite real innovation in the office document space, by encouraging Microsoft to support and existing, open, well-defined document standard in ODF. But they didn’t – Microsoft managed to push enough partners and resellers into the standards process that the ISO decision did not really reflect anything other than Microsoft’s commercial interests.

15) We’re getting a GNU/Linux ultramobile-lowcost-laptop everyday. What do you think about Eee PC Linux revolution?
I think they are great!

16) Acer, HP, MSI, Asus and much more want to join the Linux-powered UMPC market. Are you making some deals to port Ubuntu on some of these laptop?
Lots of people are installing Ubuntu onto their UMPC’s, so I think it’s reasonable that some manufacturers may choose to pre-install it. It’s their decision! If you think that would be popular, then it would probably be worth encouraging them to do so.

17) A lot of analysts talks about a GNU/Linux conquer on mobile market in the next few years. From smartphone to UMPC: the future is Linux. Can you talk about Ubuntu Mobile, its concept, the present and the future?
Yes! Intel is driving a project called Moblin, which aims to produce a mobile software platform for handheld devices, and we’re basing Ubuntu Mobile on that. The first versions are out already, and the roadmap looks very exciting.

Traditionally, it was very expensive to produce software for consumer electronic devices, because they were all specialised hardware with specialised operating systems and application development environments. We are aiming to change that – to make it so that you can build a simple .deb on x86 which can be installed on any piece of consumer electronics that uses this platform. That should greatly increase the amount of innovation we see in the mobile space.

18) What do you think about your competitors? Fedora/Red Hat, openSUSE and Mandriva are doing good work as well as Ubuntu. What GNU/Linux distribution do you prefer if you couldn’t use Ubuntu?
Yes, all of the distributions make contributions to the art and industry of free software. I’m very glad that lots of companies continue to invest in Linux, it makes it a much healthier and more vibrant ecosystem than it would be if just one company dominated it. So I’m very happy with the competition. If Ubuntu didn’t exist, I would use Debian.

19) And what is your feeling about the latest Sun acquisition (MySQL)? Are you working with Sun to port the OpenJDK on Ubuntu?
MySQL is a great company and a very good fit for Sun. I hope they are happy together and that the company will continue to produce a superb free software database.

Yes, OpenJDK is part of Ubuntu 8.04 (though it is not yet at the core, and not yet the default Java environment). We hope to have 8.04 LTS fully TCK-certified in due course. And most of all we are grateful to the Sun folks for letting us package OpenJDK as a proper Ubuntu package, neatly integrated with the rest of the OS. We are aiming for a result which feels “all Java, all Ubuntu”, and I would encourage your users to try it out! Make sure “universe” is enabled on your Ubuntu machine, then type “sudo apt-get install openjdk-6-jdk”.

20) What are the next Canonical plans? Are there any interesting initiatives in progress?
Of course, but this is not the right place for a press release

21) Finally, do you think that GNU/Linux is “really” ready for the desktop users? In which way could be improved?
Yes, I believe it is ready for SOME desktop users. If you really want a desktop that is web-oriented, then Linux is an excellent choice, with either Gnome or KDE (I’m really impressed with the work going on as part of KDE4, by the way). We know that there are millions of people using Linux today. And we are focused on solving the problems that prevent more and more people from adopting it.

Free software is intrinsically a better way to build software, I believe. But we should not plan to be judged on our morals, we should expect to be judged on our software. We have to deliver something that LOOKS and FEELS better, then we can expect people to embrace it fully. And once people realise they can have something that is better AND sustainable AND comes with many freedoms, the world will be a fundamentally different place. That is our goal.

22) Our interview seems to be completed. Do you have something to add for our readers? Thank you for your time and keep up the excellent work!
Please participate! There are lots of ways to get involved with upstream projects or with Ubuntu. Help spread the word, or fix a bug, or translate something from the desktop into Italian!

The Canonical OEM team has been approached by a number of OEM’s who want to sell netbooks (small, low-cost laptops with an emphasis on the web) based on Ubuntu. Almost universally, they’ve asked for standard Ubuntu packages and updates, with an app launcher that’s more suited to new users and has the feeling of a “device” more than a PC.

There are some very cool launchers out there – AWN is a current favourite of mine – but people seem to prefer the more 2-dimensional tabbed approach, so the OEM team implemented a lightweight but still very classy launcher for this use case. The work received a detailed review in Ars Technica and has been covered in Free Software Magazine and elsewhere.

The aim was to do something very simple that could be tested easily, work with touch devices and made shippable very quickly. It also needed to be efficient on lower-power devices, and work well with Intel hardware, which seems to be the preferred platform for this generation of devices and allows us to slip a few nice effects in that would be hard without the right hardware support. Here’s a screenshot of a recent version:

The new launcher is free software – so far, everything Canonical has funded, written and published for general public use on Ubuntu has been under the GPL. Currently we use GPLv3. You can grab the relevant packages from a public PPA, just add the following entry to your /etc/apt/sources.list:

deb http://ppa.launchpad.net/netbook-remix-team/ubuntu hardy main


The PPA contains a number of packages for the launcher, some GNOME panel applets, window manager tweaks and themes. These bits and pieces are small but improve the experience of Ubuntu run with the netbook launcher on screens with lower vertical resolution. There’s also some code in there specific to the Intel netbook hardware platforms, don’t install ume-config-netbook unless you are on the right hardware! This is all code produced by Canonical and published on Launchpad under free software licenses:

https://launchpad.net/netbook-remix
https://launchpad.net/netbook-remix-launcher

I’m particularly happy with the way it gives you more screen space for web browsing, which is probably the major use case on these form factors:

There are still plenty of interesting corner cases, Ars calls out issues with the GiMP’s palettes, for example, so please do take the opportunity to test it with the apps you think you’d run on a small laptop (or as El Reg would say, laptot).  And feel free to push up and submit for inclusion a branch or two if you’re up to a bit of Clutter hackery!

For the rest, the netbook remix uses standard ubuntu packages from the standard ubuntu archive, with standard security updates. So it meets all of our usual commitments around security and compatibility. You can recreate the netbook remix just by installing 8.04, adding the PPA to your list of repositories, fetching the packages and configuring them appropriately for your system.

The netbook remix is not part of the “official Ubuntu editions”, it’s not like Kubuntu or Ubuntu or Ubuntu Server. It’s a separate remix published by the Canonical OEM team. It will probably get revved in October when Ubuntu 8.10 is released, but that’s up to the Canonical OEM team and their customers, and not the responsibility of the Ubuntu project team.

In working with manufacturers, the OEM team creates custom install images which are specific to hardware from those OEM’s. They have the free software packages I’ve described, and they may also include third-party software selected by OEM’s which Canonical cannot redistribute, so we can’t publish the custom installers that are produced under contract. Those images typically are hand-customised for a faster boot time, which means they will only work on the particular device for which they were intended, unlike standard Ubuntu which should auto-detect and configure itself for whatever hardware it is being booted on.

We specifically wanted to do this project as an Ubuntu Remix – based on standard Ubuntu 8.04 packages, with modified package selection and some additional code, but leaving the core platform packages unmodified. In terms of the trademark guidelines for an Ubuntu Remix companies cannot call their platform Ubuntu if they have modified packages (especially the kernel and desktop packages) but they can if they are just re-arranging standard Ubuntu packages. Canonical is in a privileged position as the Ubuntu trademark owner – we can certify a custom kernel if we believe it has been done in an appropriate way that won’t conflict with standard Ubuntu maintenance processes, and if we can keep the custom kernel up to date to the same standard as the normal Ubuntu kernel. So these are certified Ubuntu devices from Canonical, even though they are more customized than other people can within the Remix guidelines.

We’re also working with two companies that want more radical user interface innovation. Canonical is participating directly in the design and implementation of one of those UI’s, and we’re integrating someone else’s UI on an Ubuntu base for the second project. I haven’t seen either of those UI’s, for confidentiality reasons, but I’m told that the teams working on them think they have great ideas that will elevate, in different ways, the state of the art. All in all it will be exciting to see how the netbook era stimulates innovation in the Linux user experience, because there are a lot of companies wanting to build differentiated UI’s on a standard Linux base. And directly or indirectly Canonical will help to bring that innovation to KDE and GNOME and hence to the wider Linux ecosystem.

An update on the long term plans for Ubuntu release management. 8.04 LTS represented a very significant step forward in our release management thinking. To the best of my knowledge there has never been an “enterprise platform” release delivered exactly on schedule, to the day, in any proprietary or Linux OS. Not only did it prove that we could execute an LTS release in the standard 6-month timeframe, but it showed that we could commit to such an LTS the cycle beforehand. Kudos to the technical decision-makers, the release managers, and the whole community who aligned our efforts with that goal.

As a result, we can commit that the next LTS release of Ubuntu will be 10.04 LTS, in April 2010.

This represents one of the most extraordinary, and to me somewhat unexpected, benefits of free software to those who deploy it. Most people would assume that precise release management would depend on having total control of all the moving parts – and hence only be possible in a proprietary setting. Microsoft writes (almost) every line of code in Windows, so you would think they would be able to set, and hit, a precise target date for delivery. But in fact the reverse is true -  free software distributions or OSV’s can provide much better assurances with regard to delivery dates than proprietary OSV’s, because we can focus on the critical role of component selection, integration, testing, patch management and distribution rather than the pieces which upstream projects are better able to handle – core component feature development. This is in my mind a very compelling reason for distributions to focus on distribution – that’s the one thing they do which the upstreams don’t, so they need to invest heavily in that in order to serve as the most efficient conduit of upstream’s work.

We also committed, for the first time, to a regular set of point releases for 8.04 LTS. These will start three months after the LTS, and be repeated every six months until the next LTS is out. These point releases will include support for new hardware as well as rolling up all the updates published in that series to date. So a fresh install of a point release will work on newer hardware and will also not require a big download of additional updates.

Gerry Carr at Canonical put together this diagram which describes the release management plan very nicely:

The Ubuntu team does an amazing job of ensuring that one can update from release to release, and from LTS release to LTS release directly, too. I’m very proud to be part of this community! With the addition of some capability to support newer hardware in LTS releases, I think we are doing our part in the free software community – helping to deliver the excellent work of thousands of other teams, from kernel.org to GNOME and KDE, safely to a huge audience.

There’s one thing that could convince me to change the date of the next Ubuntu LTS: the opportunity to collaborate with the other, large distributions on a coordinated major / minor release cycle. If two out of three of Red Hat (RHEL), Novell (SLES) and Debian are willing to agree in advance on a date to the nearest month, and thereby on a combination of kernel, compiler toolchain, GNOME/KDE, X and OpenOffice versions, and agree to a six-month and 2-3 year long term cycle, then I would happily realign Ubuntu’s short and long-term cycles around that. I think the benefits of this sort of alignment to users, upstreams and the distributions themselves would be enormous. I’ll write more about this idea in due course, for now let’s just call it my dream of true free software syncronicity.

Hearty congratulations to the entire Ubuntu community on the successful launch of 8.04 LTS. This was our best release cycle ever, from the planning at UDS-Boston last year, at which we had many different teams and companies, to the beta process which attracted so much in the way of testing and patches. I think we can be justifiably proud of the quality of 8.04 LTS. From the code to the documentation, from translations to advocacy, this has been a team effort with the shared goal of delivering the very best free software experience to the very widest possible audience. May Hardy be both enduring and endearing.

I’m very conscious of the fact that Ubuntu is the pointy edge of a very large wedge – we are the conduit, but we exist only because of the extraordinary dedication and effort of thousands of other communities and projects. We all owe a great deal to the team who make Debian’s “unstable” repository possible, and of course to the upstream projects from GNOME and KDE through to the Linux kernel. We hope you will be proud of the condition in which we have carried your excellent work through to the users of Ubuntu.

So, well done everybody! I hope that friends, family, colleagues and others will have the opportunity to try it out and understand why we have all devoted so much to this project. Our work is deeply important – we are helping to bring free software to a new level of acceptance and adoption in the wider world.  Ubuntu’s success adds to the success of free software. So as much as it is fun, challenging, the opportunity of a lifetime, a profession for some and a passion for others, it’s also changing the world. I don’t exactly want to shout “Save the Cheerleader, Save the World” but to me you are all Heroes.

Mark

Windows is a very important platform, and our justifiable pride in Linux and the GNU stack shouldn’t blind us to the importance of delivering software that is widely useful. I believe in bringing free software to people in a way that is exciting and empowering to them, and one of the key ways to do that is to show them amazing free software running on their familiar platform, whether that’s Windows or the MacOS.

Firefox, for example, is an inspiring free software success story, and I’m certain that a key driver of that success is their excellent support for the Windows environment. It’s a quick download and an easy install that Just Works, after which people can actually FEEL that free software delivers an innovative and powerful browsing experience that is plainly better than the proprietary alternatives. I’ve noticed that many of the best free software projects have a good Windows story. MySQL and PostgreSQL both do. Bazaar works well too. And users love it – users that may then be willing to take a step closer to living in the GNU world entirely.

So, I was absolutely delighted with the way Agostino Russo and Evan Dandrea steered the Windows-native installer for Ubuntu into 8.04 LTS. What I think is really classy about it is the way it uses the Windows Boot Manager sensibly to offer you the Ubuntu option. If I was a Windows user who was intrigued but nervous about Linux, this would be a really great way to get a taste of it, at low risk. Being able to install and uninstall a Linux OS as if it were a Windows app is a brilliant innovation. Kudos to Agostino and Evan, and of course also to the guys who pioneered this sort of thinking (it’s been done in a number of different ways). It looks crisp, clean and very professional:

I’m a little daunted at something as new as WUBI being the very first experience that people have of Linux, free software and Ubuntu, but initial reports are positive.  I did have a question from the media that started with “it didn’t work for me but…” which makes me a wee bit nervous.

So – yesterday I suggested folks hammer on the Heron for servers, today, here’s a call for folks who have a Windows machine and would like to see WUBI in action to test it out and let the developers know if there are any last-minute gotchas. Happy hunting!