Mark Shuttleworth | here be dragons

The discussion in blogs and comments on collaboration and standards is really important. It’s also easy to cast as “GNOME vs Canonical vs KDE”, and that would be incorrect. My critique here is not of the body of GNOME developers, it’s of specific decisions and processes, which in my view have let GNOME down.

The reason I care is, to state the obvious, a well-functioning GNOME is important to Ubuntu and Canonical. And I don’t think we’re there. Alternatively, a well-functioning FreeDesktop.org is important, and we’re not there either.

Dave Neary, who to his credit has been trying to understand how matters reached this point, blogged his conclusions to date. They warrant a response.

He summarises:

FreeDesktop.org is broken as a standards body

That may be true today, but I think we should fix it. With Meego around, and other efforts that are lower profile, there are now even more reasons to have well specified standards for desktop interoperability. They won’t all work, but they deserve better respect and quality than they have today. So my response to Dave is: let’s fix that, rather than pretending “it’s broke so it don’t matter”.

Mark Shuttleworth doesn’t understand how GNOME works

Fortunately I’m apparently in good company because his next conclusion is GNOME is not easy to understand. Perhaps a more accurate summation would be “Gnome is not self-consistent, or deterministic, so it can often come to two quite contradictory conclusions at the same time, and Mr Shuttleworth didn’t understand the one we’d now like to promote.”

Dave mailed me to say that he’d got a “definitive” perspective on how the appindicator API’s came to be rejected. It includes pointers as to the requirements for submitting external dependencies. These are “make the case for the dependency, which should be a few sentences or so, and wait a short while for people to check it out (e.g. making sure it builds)”. Now, a reading of the correspondence around the API’s suggests that Ted and others did a lot more than the “few sentences and make sure it builds”, yet the proposal was rejected.

In addition, Dave got commentary from two members of the Gnome release team, who make these decisions. The two views disagree with one another.

I’m not sure what to think, then, about Dave’s assertion that I don’t understand Gnome. Seems the follow-on conclusion is closer to the truth.

Dave says:

[…] to get things done in GNOME, you need to talk to the right people. That means, defining your problem, and identifying the stakeholders who are also interested in that problem, and working out a solution with them (am I repeating myself?). Mark seems to want GNOME to behave like a company, so that he can get “his people” to talk to “our people” and make it happen. I think that this misunderstanding of how to wield influence within the GNOME project is a key problem.

But then again, over the years I have heard similar feedback from GNOME Mobile participants, and people in Nokia – so it’s not all Mark’s fault. As Jono says here: GNOME does have a reputation of being hard to work with for companies – no point in denying it (then again, so does the kernel, and they seem to get along fine).

Hold on a sec. There’s been ample documentation of conversations. Dave can’t even point to two stakeholder who agree with each other in the release team!

I also understand that there is an interest in putting on a good face, and not airing your dirty laundry in public (ironic, eh?) – for the past few years, the party line in Canonical has been “We love GNOME, we’re a GNOME shop” while behind the scenes there have been heartfelt conversations about the various problems which exist in GNOME & how to address them. The problem is, because these discussioons happen behind the scenes, they stay there. We never get beyond discussions, agreeing there is a problem, but never working together on a solution.

Yes, the stated line from Canonical has been exactly what Dave describes – we’ve worked hard to stay within the Gnome umbrella, even where we’ve felt that the deck was stacked against us. It’s tiring. After a year or so of being the public whipping boy for cutting commentary from competitors under the Gnome banner, a franker line is needed.

Owen Taylor, desktop lead at Red Hat and the person to whom Jon McCann referred the app indicators API decision, then weighed in. He suggests several things:

Mark argues that GNOME should be a place where we have internal competition. But his idea of internal competition seems to be competition between different end-user experiences. His entrant into the competition is Unity, an environment with a user experience designed completely in isolation from GNOME. The other entrant would, I suppose, be the GNOME 3 desktop that GNOME has created.

This competition doesn’t make sense to me: what would be left of GNOME if Unity “won” that competition? Not even the libraries are left, because every decision that is made about what goes into library should be driven by that same question “what does the user see?” No widget should go into GTK+ unless it makes sense in a GNOME application. GNOME cannot cede the user experience and still work as a project.

This is exactly why we proposed the app indicator API’s as *external* dependencies. They are capabilities which GNOME apps can take advantage of if they are around, but which are not essential if they are not there. Unity could quite easily move to the fore in GNOME, if it won this competition, just like lots of other ideas and pieces of code have started outside the core of GNOME but become essential to it.

Owen’s argument reinforces the idea (which is in my view broken) that the only idea that matter are the ones generated internally to the GNOME project (defined very tightly by folks who maintain core modules or have core responsibilities). It’s precisely this inward view that I think is leading GNOME astray.

Owen’s point that “no widget should go into Gtk if it is not needed by a GNOME application” is unlikely to be comforting to the XFCE folk, or other desktop environments which build on GNOME. If anything, it will make them feel that things in “core GNOME” are likely to be difficult to adopt and collaborate with, because their needs, apparently don’t matter.

He also says “But I’ve never seen Canonical make the leap and realize that they could actually dive in and make GNOME itself better.”.. which is rather insulting to all the people from Canonical who spend a lot of their day doing exactly that.

He talks about App Indicators, saying that “They didn’t even propose changes to core GNOME components to support application indicators.” Actually, we did, and those changes required App Indicators to be an external dependency. So we proposed that, and it was rejected. Repeat ad absurdum.

In the end, in comments, Owen says that “[It] is a common misperception [that Gnome Shell and Gnome 3 can be separated]. That somehow the work we’ve done on GNOME Shell is somehow separable from the rest of GNOME 3. The work on the GTK+ theme and the window manager theme is done on together. The work on GNOME Shell is done together work work on System Settings and on the internal gnome-settings-daemon and gnome-session components.” Well, that’s convenient. Define one piece – your piece – as critical, therefor making it above competition. I expect that Ubuntu will ship Gnome 3 perfectly well with Unity.

Also in comments, Owen points out that the work Shell developers did around messaging was done as an update to an FD.o spec. An update that AFAICT was not discussed, just amended and pushed. He says, in a triumph of understatement, “We certainly haven’t done a good job discussing the small additions to the specification we needed”.

Finally, Owen concludes that having Unity and Gnome Shell as separate desktops may be the only way forward. Seems like he’s worked hard to ensure that’s the case.

Next up, Jeff Waugh is writing a set of related blog posts. One of them walks through the app indicators timeline, and is relatively comprehensive in doing so.

Jeff draws a conclusion that we’re mistaken in feeling that App Indicators were deliberately blocked because Unity presented a risk of competition with Gnome Shell; but he draws that purely based on the timing of the conversation proposing App Indicators as an external dependency, which was four days *before* the name Unity was introduced.

Yes, that’s true. But Unity was simply the new name for work which has been ongoing since 2007: The Ubuntu Netbook Remix interface. That work was very much in the frame throughout, and it’s been suggested that it was that work which catalysed Gnome Shell in the first place.

So even though Jeff is right on the claim that app indicators were discussed *before* the Unity name was revealed, that’s not in any way material to a discussion of the motives of those who rejected app indicators. This was an API from Canonical, related to work in the Ubuntu Netbook interface, and it got rejected for reasons which are unlike any other response to a proposed external dependency.

Perhaps, in the light of changed circumstances, Jeff will change his opinion. Good commentators do.

Jeff also goes on to talk about Ted and Aurelien, who were proposing the app indicators work in GNOME and KDE respectively. KDE apps worked smoothly, Gnome rejected Ted’s proposal. Jeff says “I don’t believe the key difference here is between GNOME and KDE, it is in Canonical’s approach to engagement, and its commitment of developers to the task.” It’s worth pointing out that Ted and Aurelien were both working for the same manager under the same guidance with the same goals. Jeff draws the conclusion that Canonical could have done things differently. I would have drawn the conclusion that Gnome was less open to collaboration than KDE.

Finally, Jeff looks at the requirements for dependencies, and notes that Canonical didn’t need to engage at all, though he (and we recognised the same) says that would have caused other problems. He concludes:

Canonical barely made an effort to have libappindicator “accepted into GNOME” (which, in the context of his criticism, Mark believed was important or necessary)

As I’ve shown above, the stated requirements are a very low bar. We did that, and more, yet the App Indicator API’s were rejected. As I’ve said before, it’s bizarre that such a different standard was applied to this API, and not other API’s. The only rational explanation is that the decision is nothing to do with the API’s, and everything to do with politics. Those politics are bad for Gnome in the long run. I want Gnome to be healthy in the long run, ergo, my critique.

It did not even need to go through this process anyway, because it did not need to be an “external dependency” in order to achieve Mark’s stated goals (ie. application adoption of the API)

Unfortunately for Jeff, we’d been told in no uncertain terms that module owners and core apps were under pressure about these API’s. They wanted to see the external dependency blessed. So we proposed it. Owen says we “didn’t try to propose changes to core apps”… we did, and the external dependency was the blocker.

So the rejection of libappindicator, for all the sturm und drang, is essentially meaningless — it had no bearing on the opportunity for collaboration between Canonical and GNOME

In fact, it’s what’s left that collaboration in limbo. What to do with all the patches produced for GNOME apps that make them work with app indicators? Hmm… that would be collaboration, but the uncertainty created by the rejection as an external dependency creates a barrier to that collaboration. As Jeff says, those patches can land without any problems. But to land such a patch, after the refusal, takes some guts on the part of the maintainer. Lots have done it (thank you!) but some have said they are concerned they will be criticised for doing so.

Unity did not exist in the public sphere when libappindicator was declared irrelevant to GNOME Shell, and was not ever the reason why: (a) there wasn’t much interest in libappindicator, or (b) GNOME Shell developers decided they were on a different path

Correct, *Unity* was not the public name of the work at the time, Ubuntu Netbook Remix was.

Not proven in this part of the series, but worth noting: the person Mark specifically chose to attack, Jon McCann, did not decide to exclude libappindicator because — being a design contributor to GNOME Shell — he felt threatened by Unity. In fact, he had no part in the decision, and didn’t know Unity existed!

Jon certainly knew a great deal of work on interfaces was being done. That became branded Unity later, but the timing of the change in name is irrelevant.

Phew.

There are good faith efforts being made to bridge divides all over the show, for which I’m grateful and to which we’re contributing. My comments here are to address what I see as convenient papering over, which will not stand the test of time. It’s important – to me, to the members of the community working on Unity and Ubuntu (and there are substantial communities in both) that simplistic accusations against us are not left to stand unchallenged.

The goal – for everyone, I think – is great free software. I know we’re committed to that, and doing what we think is needed to achieve it.

Competition is tough on the contestants, but it gets great results for everyone else. We like competitive markets, competitive technologies, competitive sports, because we feel the end result for consumers or the audience is as good as it possibly could be.

In larger organisations, you get an interesting effect. You get *internal* competition as well as external competition. And that’s healthy too. You get individuals competing for responsibility, and of course you want to make sure that the people who will make the wisest choices carry the responsibilities that demand wisdom, while those who have the most energy carry the responsibilities for which that’s most needed. You get teams competing with each other too – for resources, for attention, for support elsewhere, for moral authority, for publicity. And THAT’s the hardest kind of competition to manage, because it can be destructive to the organisation as a whole.

Even though it’s difficult to manage, internal competition is extremely important, and should not be avoided out of fear. The up side is that you get to keep the best ideas because you allowed them to compete internally. If you try and avoid it, you crowd out new ideas, and end up having to catch up to them. Usually, what goes wrong is that one group gets control of the organisation’s thinking, and takes the view that any ideas which do not come from within that group are a threat, and should be stopped. That’s very dangerous – it’s how great civilisations crash; they fail to embrace new ideas which are not generated at the core.

In Ubuntu, we have a lot of internal competition. Ubuntu and Kubuntu and Xubuntu and Edubuntu and *buntu-at-large have to collaborate and also, to a certain extent, compete. We handle that very well, I think, though occasionally some muppet calls Kubuntu the blue-headed-stepchild etc etc. It’s absolutely clear to everyone, though, that we have a shared interest in delivering ALL these experiences together with as much shared vision and commonality as possible. I consider the competition between these teams healthy and constructive and worth maintaining, even though it requires some fancy footwork and causes occasional strains.

The challenge for Gnome leadership

The sound and fury writ large in blog comments this week is all about how competition is managed.

Gnome is, or should be, bigger than any of the contributing individuals or companies. Gnome leadership should be in a position to harness competition effectively for the good of the project. That does, however, require very firm leadership, and very gutsy decisions. And it requires vigilance against inward thinking. For example, I’ve seen the meme reiterated multiple times that “one should not expect Gnome to embrace ideas which were not generated and hosted purely within Gnome”. That’s chronic inward thinking. Think of all the amazing bits of goodness in the free software stack which were NOT invented in Gnome but are a part of it today. Think how much better it is when goodness is adopted across multiple desktop environments, and how much harder it is to achieve that when something is branded “K” or “G”.

When we articulated our vision for Unity, we were very clear that we wanted to deliver it under the umbrella of Gnome. We picked Gnome-friendly technologies by and large, and where we felt we needed to do something different, that decision required substantial review. We described Unity as “a shell for Gnome” from the beginning, and we have been sincere in that view. We have worked successfully and happily with many, many Gnome projects to integrate Unity API’s into their codebase.

This is because we wanted to be sure that whatever competitive dynamics arose were *internal* to Gnome, and thus contributing to a better result overall in Gnome in the long term.

We’ve failed.

Much of the language, and much of the decision making I’ve observed within Gnome, is based on the idea that Unity is competition WITH Gnome, rather than WITHIN Gnome.

The key example of that is the rejection of Unity’s indicator API’s as external dependencies. That was the opportunity to say “let’s host this competition inside Gnome”. Even now, there’s a lack of clarity as to what was intended by that rejection, with some saying “it was just a reflection of the fact that the API’s were new and not used in any apps”. If that were the case, there would be no need for prior approval as an external dependency; the rejection was clearly an attempt to prevent Gnome applications from engaging around these API’s. It’s substantially failed, as many apps have happily done the work to blend in beautifully in the Unity environment, but there has been a clear attempt to prevent that by those who feel that Unity is a threat to Gnome rather than an opportunity for it.

Dave Neary has to his credit started to ask “what’s really going on here”?

In his blog post, he quoted the rationale given for the rejection of Canonical’s indicator API’s, which I’ll re-quote here and analyze in this light:

it doesn’t integrate with gnome-shell

That’s it – right there. Remember, this was a proposal for the indicator API’s to be an *external* dependency for Gnome. That means, Gnome apps can use those API’s *optionally* when they are being run on a platform where they are useful. It has NOTHING to do with the core Gnome vision. External API’s exist precisely BECAUSE it’s useful to encourage people to use Gnome apps on all sorts of platforms, including proprietary ones like Windows and MacOS and Solaris, and they should shine there too.

So the premier reason given for the rejection of these API’s is a reason that, as best we can tell, has never been used against an external dependency proposal before: “it’s different to Gnome”. At the heart of this statement is something deeper: “it’s competition with an idea someone in Gnome wants to pursue”.

What made this single statement heartbreaking for me to see was that it spoke clearly to the end of one of Gnome’s core values: code talks. Here we had API’s which were real, tested code, with patches to many Gnome apps available, that implemented a spec that had been extensively discussed on FreeDesktop.org. This was real code. Yet it was blocked because someone – a Gnome Shell designer – wanted to explore other ideas, ideas which at the time were not working code at all. There’s been a lot of commentary on that decision. Most recently, Aaron Seigo pointed out that this decision was as much a rejection of cross-desktop standards as it was a rejection of Canonical’s code.

Now, I can tell you that I was pretty disgusted with this result.

We had described the work we wanted to do (cleaning up the panel, turning panel icons into menus) to the Gnome Shell designers at the 2008 UX hackfest. McCann denies knowledge today, but it was a clear decision on our part to talk about this work with him at the time, it was reported to me that the conversation had happened, and that we’d received the assurance that such work would be “a valued contribution to the shell”. Clearly, by the time it was delivered, McCann had decided that such assurances were not binding, and that his interest in an alternative panel story trumped both that assurance and the now-extant FreeDesktop.org discussions and spec.

But that’s not the focus of this blog. My focus here is on the management of healthy competition. And external dependencies are the perfect way to do so: they signal that there is a core strategy (in this case whatever Jon McCann wants to do with the panel) and yet there are also other, valid approaches which Gnome apps can embrace. This decision failed to grab that opportunity with both hands. It said “we don’t want this competition WITHIN Gnome”. But the decision cannot remove the competitive force. What that means is that the balance was shifted to competition WITH Gnome.

probably depends on GtkApplication, and would need integration in GTK+ itself

Clearly, both of these positions are flawed. The architecture of the indicator work was designed both for backward compatibility with the systray at the time, and for easy adoption. We have lots of apps using the API’s without either of these points being the case.

we wished there was some constructive discussion around it, pushed by the libappindicator developers; but it didn’t happen

We made the proposal, it was rejected. I can tell you that the people who worked on the proposal consider themselves Gnome people, and they feel they did what was required, and stopped when it was clear they were not going to be accepted. I’ve had people point to this bullet and say “you should have pushed harder”. But proposing an *external* dependency is not the same as trying to convince Shell to adopt something as the mainstream effort. It’s saying “hey, here’s a valid set of API’s apps might want to embrace, let’s let them do so”.

there’s nothing in GNOME needing it

This is a very interesting comment. It’s saying “no Gnome apps have used these API’s”. But the Gnome apps in question were looking to this very process for approval of their desire to use the API’s. You cannot have a process to pre-approve API’s, then decline to do so because “nobody has used the API’s which are not yet approved”. You’re either saying “we just rubber stamp stuff here, go ahead and use whatever you want”, or you’re being asinine.

It’s also saying that Unity is not “in GNOME”. Clearly, a lot of Unity work depends on the adoption of these API’s for a smooth and well-designed panel experience. So once again, we have a statement that Unity is “competition with Gnome” and not “competition within Gnome”.

And finally, it’s predicating this decision on the idea being “in Gnome” is the sole criterion of goodness. There is a cross-desktop specification which defines the appindicator work clearly. The fact that KDE apps Just Work on Unity is thanks to the work done to make this a standard. Gnome devs participated in the process, but appeared not to stick with it. Many or most of the issues they raised were either addressed in the spec or in the implementations of it. They say now that they were not taken seriously, but a reading of the mailing list threads suggests otherwise.

It’s my view that cross-desktop standards are really important. We host both Kubuntu and Ubuntu under our banner, and without such collaboration, that would be virtually impossible. I want Banshee to work as well under Kubuntu as Amarok can under Ubuntu.

What can be done?

This is a critical juncture for the leadership of Gnome. I’ll state plainly that I feel the long tail of good-hearted contributors to Gnome and Gnome applications are being let down by a decision-making process that has let competitive dynamics diminish the scope of Gnome itself. Ideas that are not generated “at the core” have to fight incredibly and unnecessarily hard to get oxygen. Ask the Zeitgeist team. Federico is a hero, but getting room for ideas to be explored should not feel like a frontal assault on a machine gun post.

This is no way to lead a project. This is a recipe for a project that loses great people to environments that are more open to different ways of seeing the world. Elementary. Unity.

Embracing those other ideas and allowing them to compete happily and healthily is the only way to keep the innovation they bring inside your brand. Otherwise, you’re doomed to watching them innovate and then having to “relayout” your own efforts to keep up, badmouthing them in the process.

We started this with a strong, clear statement: Unity is a shell for Gnome. Now Gnome leadership have to decide if they want the fruit of that competition to be an asset to Gnome, or not.

A blessing in disguise

Aaron’s blog post made me think that the right way forward might be to bolster and strengthen the forum for cross-desktop collaboration: FreeDesktop.org.

I have little optimism that the internal code dynamics of Gnome can be fixed – I have seen too many cases where a patch which implements something needed by Unity is dissed, then reimplemented differently, or simply left to rot, to believe that the maintainers in Gnome who have a competitive interest on one side or the other will provide a level playing field for this competition.

However, we have shown a good ability to collaborate around FD.o with KDE and other projects. Perhaps we could strengthen FreeDesktop.org and focus our efforts at collaboration around the definition of standards there. Gnome has failed to take that forum seriously, as evidenced by the frustrations expressed elsewhere. But perhaps if we had both Unity and KDE working well there, Gnome might take a different view. And that would be very good for the free software desktop.

The GNOME user experience hackfest in Boston was a great way to spend the worst week in Wall St history!

Though there wasn’t a lot of hacking, there was a LOT of discussion, and we covered a lot of ground. There were at least 7 Canonical folks there, so it was a bit of a mini-sprint and a nice opportunity to meet the team at the same time. We had great participation from a number of organisations and free spirits, there’s a widespread desire to see GNOME stay on the forefront of usability.

Neil Patel of Canonical did a few mockups to try and capture the spirit of what was discussed, but I think the most interesting piece wasn’t really possible to capture in a screenshot because it’s abstract and conceptual – file and content management. There’s a revolution coming as we throw out the old “files and folders” metaphor and leap to something new, and it would be phenomenal if free software were leading the way.

I was struck by the number of different ways this meme cropped up. We had superb presentations of “real life support problems” from a large-scale user of desktop Linux, and a persistent theme was “where the hell did that file just go?” People save an attachment they receive in email, and an hour later have no idea where to find it. They import a picture into F-spot and then have no idea how to attach it to an email. They download a PDF from the web, then want to read it offline and can’t remember where they put it. Someone else pointed out that most people find it easier to find something on the Internet – through Google – than they do on their hard drives.

The Codethink guys also showed off some prototype experience work with Wizbit, which is a single-file version control system that draws on both Git and Bazaar for ideas about how you do efficient, transparent versioning of a file for online and offline editing.

We need to rearchitect the experience of “working with your content”, and we need to do it in a way that will work with the web and shared content as easily as it does locally.

My biggest concern on this front is that it be done in a way that every desktop environment can embrace. We need a consistent experience across GNOME, KDE, OpenOffice and Firefox so that content can flow from app to app in a seamless fashion and the user’s expectations can be met no matter which app or environment they happen to use. If someone sends a file to me over Empathy, and I want to open it in Amarok, then I shouldn’t have to work with two completely different mental models of content storage. Similarly, if I’ve downloaded something from the web with Firefox, and want to edit it in OpenOffice, I shouldn’t have to be super-aware or super-smart to be able to connect the apps to the content.

So, IMO this is work that should be championed in a forum like FreeDesktop.org, where it can rise above some of the existing rivalries of desktop linux. There’s a good tradition of practical collaboration in that forum, and this is a great candidate for similar treatment.

At the end of the day, bling is less transformational than a fundamental shift in content management. Kudos to the folks who are driving this!

Update: thanks mjg59 for pointing out my thinko. The Collabora guys do great stuff, but Codethink does Wizbit.