Mark Shuttleworth | here be dragons

I’m absolutely thrilled to see this chart of untriaged bugs in Inkscape since the project moved to Launchpad:

As you can see, the Inkscape community has been busy triaging and closing bugs, radically reducing the “new and unknown” bug count and giving the developers a tighter, more focused idea of where the important issues are that need to be addressed.

A lot of my personal interest in free software is motivated by the idea that we can be more efficient if we collaborate better. If we want free software to be the norm for personal computing software, then we have to show, among other things, that the open, free software approach taps into the global talent pool in a healthier, more dynamic way than the old proprietary approach to building software does. We don’t have money on our side, but we do have the power of collaboration.

I put a lot of personal effort into Launchpad because I love the idea that it can help lead the way to better collaboration across the whole ecosystem of free software development. I look for the practices which the best-run projects follow, and encourage the Launchpad guys to make it easy for everyone to do those things. These improvements and efficiencies will help each project individually, but it also helps every Linux distribution as well. This sort of picture gives me a sense of real accomplishment in that regard.

Bryce Harrington, who happens to work for Canonical and is a member of the Inkscape team, told me about this and blogged the experience. I’ve asked a few other Inkscape folks, and they seem genuinely thrilled at the result. I’m delighted. Thank you!

Congratulations to the Launchpad team on today’s public beta!

Launchpad is built for Ubuntu, but it’s great to see other projects adopting it too, most notably in recent weeks Zope and Silva. I hope those projects find it much easier to collaborate with one another, and with other projects too.

If you’re curious about Launchpad, this new guide to Launchpad’s feature highlights is a quick read (with pictures :-)) and gives you a good sense of how best to use it. It covers all the major applications that make up the service - translation, community support, planning, bug tracking, code hosting and team management.

And if you’re keen to test new features and willing to file bugs, you can join the Launchpad Beta Tester’s team, and live on the bleeding edge with nightly builds of the system. We’ll reopen the beta server (beta.launchpad.net) later in the week to test some additional new functionality.

My latest (small) contribution to the code was to help Salgado with the system that lets you set a custom logo for every page in the system that belongs to you or your project. Have fun decorating! Here’s the official announcement of the beta.

One of the Blueprint hacks I got around to this weekend was to add the idea of “essential” subscribers to a feature specification in Launchpad. Since we use Launchpad to plan our meetings, its useful to know when we are arranging the discussion sessions who has to be present when a topic is discussed, and who is simply interested in the discussion.

There’s a brief doc describing the feature further on help.launchpad.net and I think it will first be used for Edgy+1 planning in November.

PS - thanks to Google for the star image, artistic improvements welcome!

The inevitable Web 2.0-ism has landed in Launchpad… TAGS. Congrats to Brad Bollenbach and Bjorn Tillenius on the landing. In this first cut, only the bug tracker supports them, but if you can suggest interesting other applications for tags in Launchpad I’m sure the guys on #launchpad would be keen to listen.

There’s a demo server with a full list of Python’s bugs from Sourceforge imported into a Launchpad instance, with tags, that you can check out for a taste of bug tagging goodness. That’s not the usual, production LP, it’s a demo setup we use to show new features or to import sample data.

Personally, I’m something of a sceptic. Tags are certainly a useful way to do short term organisation of things in a project, but they can’t compete with rigorous structure for ongoing management. Essentially, a wiki is FULL of tags, but organising things in wiki’s is fraught with challenges. We used to organise the feature planning for Ubuntu in the wiki, but it drove me nuts - to the point that I had to sit down and write the feature planning system in Launchpad. Folksonomies are powerful in the right places, but they are no magic bullet.
Still, I’m intrigued to see how projects use them. You can see how the LP team themselves are coordinating their tags-onomy - I like the idea of having agreed, and proposed, tags. Of course, you can tag anything with anything, but *some* structure or convention makes the whole endeavour much more useful.

Hmm… time to add that to the Malone Highlights doc I wrote up over the w/e.

I realised that Blueprint has very little in the way of documentation to help people figure it out. Despite this, usage of Blueprint seems to be taking off, with nearly 700 specifications in the system and quite a lot of Launchpad karma getting dished out daily. But documentation will help, especially since Blueprint is starting to take shape and will be at the heart of our planning for Dapper+1.

So without further ado, I give you:

Official Blueprint Docs (v0.1)

Enjoy, and comments, updates and clarifications welcome! Note, some of those features discussed have landed in the code but not yet moved to production, so you can find them at http://staging.ubuntu.com/ rather than http://launchpad.net/ for the moment.

Phew, big Blueprint landing just went in to LP, I guess it will be in production some time this week. This gets the spec tracker into roughly the right shape for us to get Dapper+1 smoothly planned and executed. It also involved quite a lot of groundwork for the 1.0 UI I am driving towards for Launchpad. So perhaps I should blog about those plans and solicit some feedback.

The high-level story is this:

• Applications on subdomains. We will move the functionality of the different facets of Launchpad into subdomains. So rosetta.launchpad.net will be where you will find translation pages, malone.launchpad.net will have bugs, blueprint.launchpad.net will have feature tracking. These applications will share a navigation system so if you are used to using the one app, and moving around Launchpad objects (upstream, ubuntu and derivatives, people and so on) then you will easily feel comfortable moving around in a different app. The big advantage of this is that it keeps you “in the zone” as you move around Launchpad. So say I am working on the spec system, and I click on the link to Carlos’ homepage in Launchpad. Typically, when I do that, it’s because I want to see what Carlos has on his plate in terms of feature development. Using subdomains gives us this for free, because I will be jumping, say, from http://blueprint.launchpad.net/distros/ubuntu/dapper/ to http://blueprint.launchpad.net/people/carlos/ and as you can see I’m automatically still in the “specs” mode. I’ve watched people using Launchpad, and usually when we jump from thing to thing we still want to keep focused on the same “facet” of that thing.
• A better menu system. A lot of the navigation will shift to the top-bar menu and breadcrumbs system. Right now you can see an (ugly) first step towards that, if you hold your mouse over the Launchpad in the top left corner you see a dropdown menu that takes you to distros, products, projects, people, meetings… now imagine you could continue navigating in to the specific product or distro or package you are looking for. Less clicking for everybody. The breadcrumbs will also become part of the menu system, so say you are looking at the 1.5.0.2 release of Firefox (two thumbs up btw guys) and you want to see what features are planned for 2.0, you should be able to get there just by hovering over Firefox in the breadcrumbs, then see the 2.0 series, and go straight to it.
• Better switching between apps. At the moment, we confuse two different kinds of switching. You can switch between objects that you are viewing, and you can switch between the aspect or facet of the object too. Sometimes, clicking on “Bugs” in the actions menu does BOTH kinds of switch, and that’s really bad. Its unpredictable. So we will work towards ensuring that any action link has a predictable result - it either is an action in the current application on the current object, or it changes the app but keeps you on the current object, or it changes the application and takes you to that app home page (”I’m switching now to think about translations, take me to the Rosetta home page”).
• Canonical object names. We track quite a few kinds of objects in Launchpad. The main ones are distributions (and their structure, releases, packages), upstreams (in the form of overarching projects, as well as individual products and their releases) and people. In order to disambiguate them we’ve kept separate namespaces for them, and used the URL structure to indicate what you are looking at. Hence you can have http://launchpad.net/products/mozilla/ and http://launchpad.net/projects/mozilla/ if you want. Now we want to switch to a different approach, where we canonicalise the names of those three major kinds of object. So there can only be one “ubuntu” thing, and that’s the distro. And only one “mozilla” thing, and that’s a project. And only one “firefox” thing, which is a product. This will let us simplify the URL’s quite a bit, down to http://blueprint.launchpad.net/firefox/ for the specifications for Firefox, we can drop the /products/ bit because we have Canonical names. This also gives us a “teleport” facility, where typing the name of the thing teleports you straight there. An email address will take you straight to the home page of the person you are thinking of. A product name will take you straight to that products Launchpad page in the app you are working on.

As you can imagine, there have been lots of dicussions, debates and fairly intense arguments about the UI for Launchpad. It’s a huge piece of infrastructure, I think it’s quite a bit richer in scope than SourceForge, which would be a reasonable comparison point, so it’s taken a long time for us to get a handle on how most efficiently to present it. I’m pretty confident in this plan, and keen to see it in place in the next three months, perhaps even in time for Dapper.

Further down the line I think we can use AJAX tools to speed up the interaction processes, but I don’t want to jam those into the infrastructure until the core plumbing is robust and mature. Right now the world of AJAZ is very much in flux, and so is the underlying set of frameworks we use in Launchpad: Zope 3 and SQLObject. We have a lot of work to do just to get the core app framework robust for taking Launchpad to scale, and that’s where we are focusing the resources. We’ve published pretty much all the Zope and SQLObject changes we are making, and I think I would like to make more of that work. Once the AJAX stuff matures, we can fold it into the app framework in a general and graceful fashion.

I wrote most of the spec tracker in Launchpad (click here if you want to see it in action) and it needs a little more love before the next Ubuntu developer summit. So I’ve been hacking on it in preparation for the Launchpad sprint. There’s still another round of updates coming, but for now please enjoy:
Proper implementation tracking: a proper implementation status dial, which lets people say how they are making progress with the actual coding of the feature. JaneW has been asking for that for a while now, since she compiles the weekly Dapper development summary and it should make that process a LOT simpler.

Safe targeting to releases and series: a reasonable way to target a spec to a distro release (”dapper”) or a product series (”1.2″). Anybody that is related to the spec can propose it for a release or series, but only the series managers can accept or decline the feature. So it behaves a bit like a wiki, with only the drivers of the upstream product or the distro able to give the go-ahead for a feature to be an official goal for a specific release.

Along the way I fixed a bunch of embarrassing bugs in the menus for product series. I blogged separately about how product series are *supposed* to work and why they are important in Launchpad. I’m working on making ProductSeries an essential part of the system, since a lot of the data model issues get cleaner when you assume that series is always there.
These changes should all be in production in already at launchpad.net. As usual, these features work equally well for upstreams using Launchpad as they do for the distro teams.

Martin Meredith blogs that it would be good for other distros also to adopt Malone. Most of the larger distros have established bugzilla’s or their own custom bug trackers, though, so we specifically designed Malone so that it can link to those external bug trackers rather than requiring them to adopt Malone.

Right now, it still requires opening a bug in each of the upstream trackers, which is a pain.

In the future, I think we should see if we can get permission from the upstream bug tracker operators for Launchpad to be able to open bugs directly in their trackers. So, from Launchpad, if we had permission from Mozilla, you should be able to say “this bug in Ubuntu’s Firefox package is also in upstream, please open a bug report for it”. And that should report the bug in bugzilla.mozilla.org, then link it in all the right ways in Malone.

The problem will definitely be the risk of filing unnecessary dups in the upstream tracker. Bugzilla is getting good at getting people to check that the bug they are reporting has not already been reported. We need that same level of defense in Launchpad (though with the ticket tracker slowly improving, we should see more problems starting out there, being handled by the community, and graduating to Malone only once they are verified). And we need a way to make sure that Launchpad does not get a bad reputation for being used to poke dups into upstream bug trackers.

In Launchpad, we have a thing called a “product series” that represents a series of releases from the same branch. For example, Firefox 1.5, 1.5.0.1, 1.5.1 etc would all belong to the “1.5″ series. The name “product series” sucks horribly, and I’d really like to get a better name. Suggestions welcome, on #launchpad or on the launchpad-users mailing list.

So, why do we have these?

Very simply, because many (if not most) of the things that developers care about are related to work in progress. Think about it. We don’t really edit the translations of the 1.5.0.1 release of Firefox - because those are in the tarball, they are done and dusted. If we are editing translations for 1.5.0.1 its because we actually want those translations published along with 1.5.1… the next release in the series. So we should track the translations of the series, not the release.

Similarly, if you want to fix a bug in an upstream product, and do it for say 1.5 of Firefox, then once again you are “targeting this fix to the next release in the series” of the branch you are landing it on.

So, in Launchpad, both translations and bugs are tightly coupled to upstream product series.

Of course, a product series is very similar to a special branch of that product. And in fact, in Launchpad you can associate a particular branch with a series. This then means that we can monitor that branch for special events. In future, for example, we will be able to know that you have fixed a bug in a particular series when we see a commit message to that effect on the branch. Pretty cool vapourware

The last few evenings have been devoted to the part of Launchpad called Soyuz, which tracks all the distributions and packages that are registered in Launchpad. It’s the heart of the “distro management” capability of Launchpad, that allows customers to manage their own Ubuntu derivative through the web. And of course we will be dogfooding it constantly, but managing Ubuntu itself in it.

Wednesday, all going well, we will open up the Dapper archive based on Launchpad, which will be the first time that we will have this stuff in production. Expect delays for the first few days as we iron out the kinks.

My own work has been the first cut of the pages that display which package is published where in the archive at any given time. That sounds very straightforward, until you realise how fiendishly complicated a distro package archive can be. They are very much structures that have evolved, and so a lot of the complexity comes from having to deal with the badness of reality.

For example, there is no guarantee that a newly upload package will build on all the supported architectures. So say I upload Apache 2.1.3-1 source to Ubuntu, and it builds on all architectures. You would expect to have those binaries published on all architectures in short order. Now, we upload 2.1.3-2, which builds on all architectures except for PPC. This is where the fun starts. The “current” version of Apache is now different, depending on which architecture you are looking at.

One resolution might be to say “build everywhere and reject if it fails anywhere”. But we are constantly adding new architectures, and as those become more esoteric the chances of a build failure rises dramatically. We can’t hold back the distro when a package fails to build on ARM. So the system has to be able to handle this divergence between architectures.

But it gets even more interesting. There is no rule, for example, that a package called foo on i386 should have the same description, or dependencies, as a package with the same name (and even the same version) on amd64. They are just entirely different beasts. So to do this justice, someone has to drill all the way down to the lowest level before they see stuff that’s human readable. Even though 99.9% of the time, pacakges with the same name and version on different architectures WILL have the same details.

So I’ve put quite a lot of work into trying to balance ease of user interface and technical correctness. At the lowest level, you should see exactly what’s published, which versions, when they were built (and more detail on those than you could have imagined you really, really wanted). At a higher level, however, you should be able to search and find the “obvious answers”.

I’m sure there are bugs, and improvements to come, but it’s nice finally to see this code going into production. Try following the link above to the Ubuntu pages in Launchpad, then searching for packages, and send me your suggestions on how we can improve things.