Mark Shuttleworth | here be dragons

Today we announced that Dapper will include full support for SPARC, and in particular for the UlstraSPARC T1 (the “Niagara” architecture) in all its multicore multithread glory.

I would credit the Linux/SPARC community (and David Miller in particular), and the OpenSPARC community, with the speed of this port moving from “first code” to production supportable. When I first saw David speaking about Niagara support at LCA in Dunedin in January, we all thought that Dapper could support traditional SPARC at release but then get Niagara support some months later in an update.

But the fervour with which the community at large under David’s leadership attacked the problem has meant that Linux on Niagara has progressed far faster than we expected – so much so that the first SPARC CD release of Dapper (which will be uploaded a little after the other architectures when we make the Dapper release) will support most UltraSPARC T1 machines out of the box. David – wow! Thanks for that leadership. I guess we have to add your SPARC work to the list of things for which the free software community have to thank Red Hat 😉

I should also thank Matthias Klose, Ben Collins, Jeff Bailey, and the unstoppable Fabio di Nitto for integrating that community work into Dapper and for making sure that the installer and userspace bits are all in good shape. I hope that the Linux-on-SPARC community will experience a bit of a resurgence now that these officially-supported options are out there and in production use.

The other aspect has been the OpenSPARC process, which essentially means that everything you need (all the documentation, the source code of hypervisors etc, and even the RTL defining the processor itself) is published under free software licences. So this is the first time that Linux has been ported to a TOTALLY open architecture. My hope is that this will allow Linux to keep up with the proprietary alternatives in supporting the very latest and greatest hardware features on SPARC on the day the machines first roll on out of the factory.

Now, if only more hardware manufacturers took that approach, we wouldn’t have to deal with the frustrations and shortcomings of binary (or worse – entirely unavailable) drivers on Linux.

Joi ito has had a few stern looks from the US INS regarding visa waiver forms.

I can relate.

I have a UK passport by virtue of the fact that my father was born in the UK (mostly by accident – another fun story). So I also know about the visa waiver program – it used to cover me too. Until one day I flew into the US briefly, on my own plane, to visit friends in DC as part of a long trip. When we arrived at Dulles, the immigration officer said there was a small problem. The operator of my plane had never signed the visa-waiver treaty, and so despite the fact that I had entered the US 27 times previously on that same passport, without a visa, they would now have to decline me entry.

But before doing that they would:

• take me in for questioning
• search me (I objected to the strip search, they relented)
• fingerprint me and send those fingerprints off around the world (no, Mossad is not looking for me, yet)
• examine for obvious tattoos and other distinguishing features
• ask me to sign a statement of wrongdoing (I declined)
• terminate my visa waiver access – from then on I need a visa

A complication was that, because they did not have records of all the times I left the USA, they believed I had previously stayed for longer than the 90 days. Fortunately I was able to get copies of all my inbound and outbound tickets faxed to them, so I think they eventually came to believe that I had not actually overstayed the visa program ever.
Then they let me back on the plane, we flew to Ottawa, the US embassy kindly gave me a visa, and we returned to the USA.

Now, flying into the USA I am ALWAYS sent off for extra questions and paperwork. And on applying for a new visa, I have to fill out the form for “people with a criminal record” (cross out the criminal record part, write in “visa waiver declined”, I kid you not). It’s a joyless process.

Hello, land of the free, knock knock.

I fell in love with the USA once. It was built on beautiful principles. Alas, it appears to have forsaken those in the name of security and expediency. As a result, I think the world is looking for a new source of inspiration – a new country where the most interesting people of the world can arrive, feel welcome, and feel free. Joi, best you be sure to hand that little green form back, every time.

I got struck down upon my return from Mexico with a fever. Other than that, it was a fantastic conference – well done to the organisers for picking such a cool venue. Highlights this year (for me, anyhow):

• The 10m jump. Bonzai.
• Meeting Warren of MEPIS fame in person.
• Getting to do not one but TWO Ubuntu Q & A sessions. Thanks, Don.
• The flurry of activity around the Java licence changes. Still not settled, but still, well done to all involved for at least making good progress so far.
• The curious geodesic-ish dome, which it turns out covers a sulphur spring (why? no idea. but cool nonetheless)
• Sinchronizada’s. Especially in two’s. Asynchronously.
• Mexico City Margarita’s (alas, not much to be said for those in Oaxtepec)
• Oaxtepec Negro Modelo’s (forever to be known as “The Black Hunk”)
• The bandwidth if you woke up before 9am
• Tango lessons
• Dodging the Pool Police for late night skinny-dipping. You KNOW I love skinny-dipping.
• The mountains on the drive up from Mexico City.
• The volcano to the east of the complex.
• Some positive feedback from Joey Hess (along with some constructive criticism, but hey, it’s a start)
• Lots of DD’s thanking me for Ubuntu. Don’t thank me. Thank them. And thank yourselves 🙂 And while you’re at it, THANK UPSTREAM! But nonetheless, thank you for saying thank you, because it’s a labour of love.
• The view of Mexico City from the plane, coming and going. And coming. And going.

So, all in all, I had a great time.

I’m pleased that there was good discussion about the Ubuntu and Debian relationship. Overall I think we’re past the “sky is falling in” phase and into the “practical ways to improve the flow” phase, which is thanks to hard work on both fronts. I note far fewer Ubuntu bugs spuriously ending up in the debian BTS, and a lot of coordinated tracking of bugs across Ubuntu, Debian and upstream. From our side, at least, Launchpad makes it a lot easier for Ubuntu to be smart about that. Raphael and Enrico made a list of things we can do on all fronts that will improve that coordination – we’ll get to quite a few of those this year, I think. Some folks apparently grumbled in private but never showed up to the Ubuntu Q & A’s – you know who you are. Hello, folks, you don’t get to party unless you show up! To those who did – thanks for the Q’s. I hope the A’s were satisfactory – ping me on irc.freenode.net if you have more.

But mostly, it was just good fun. I’m glad that the rain and the flooding didn’t bring out the scorpions, as it sometimes does in tropical environments. My Spanish may suck but I never felt like a stranger in Oaxtepec. Or, better put, I never felt like a lone stranger. We’re all a bit touched, really, by local standards. Manyana…

There was a bit of grumbling about Red Hat at Debconf this year (along with plenty of grumbling about Ubuntu too). While I agree that the Red Hat Network legalese effectively makes RHEL a proprietary product, I think it’s important to give credit to Red Hat for the role they have played and continue to play in bringing Linux, GNU and the free software stack to the wider world.

Red Hat was, and remains, essential to the free software and linux ecosystem.

Most of the world’s computer users have never seen or touched a Linux environment directly. Yes, of course we all use it in the form of Google or Akamai or a WiFi base station that we installed ourselves last weekend… but in terms of actually staring at the keyboard and typing into a Linux console or X-Term, adoption is still very much in the low-single-digits. Which means that less than 1-in-20 people who consider themselves IT people have hands-on experience with free software. For most of those people, Red Hat “speaks their language”. Which means that they are far more likely to get their first taste of free software via Red Hat than they are, say, compiling their personal kernels from scratch.

And we all win, when Red Hat has a win.

Every Red Hat server installed at UBS is a win for Free Software.

Every Fedora Core 5 installation at a school is a win for Linus Torvalds, for me, for Ubuntu, for Debian and even for Richard Stallman.

Every time Red Hat gets a mention on CNBC or that stupid Cramer show that really should have died with the dot com bomb, it’s a win for Free Software.

Why?

Because people learn by taking single steps. They learn by tasting new ideas little by little and not getting burned. They learn that “Linux” works, is reliable, is predictable, and that not running Microsoft Office is potentially a feature because it also means that it does not run all those pesky macro viruses.

And they learn that “normal”, which they used to define as “Windows”, is just part of the full universe of options they have at their disposal.

That of course inevitably touches their curiousity bone… and who knows where they might actually end up – in Gentoo, in Ubuntu, in Debian, perhaps even in OpenSolaris-land. It doesn’t matter – it’s all GNU goodness. Even if they stick with Red Hat.
In the free software community, we are as likely to turn viciously on one another as we are to stand united. And that’s a big weakness for us, as a community. It’s too easy to get free software advocates, who agree on 99% of things, to shred one another over the remaining 1%. That just makes us a confusing place for new users – and a disturbing place for more corporate adopters. It makes it easy for proprietary software companies to divide and conquer the free software world. We divide spontaneously.

Instead, we should strongly affirm the things in which we all agree.

• Software CAN be Free (using Richard’s terminology) and therefor we believe it will ALL END UP FREE. And we’re committed to reinventing everything we need until the free software stack is a genuinely complete computing universe. We’re already pretty far along.
• Free software is not just cheaper. It’s BETTER. It’s produced using a better process attracting better talent and it evolves faster, resulting in better innovation. All of that adds up to great value.

If we keep reminding ourselves that we agree on all of that, then our disagreements come into perspective. Instead of criticism for Red Hat, I think it would be more constructive to remind ourselves of the things that Red Hat does for the free software community first – and then perhaps to talk about why we prefer another system, for whatever personal reasons.

Just off the top of my head, here are a couple of things for which all of us free software advocates have Red Hat to thank:

• When the EU was voting on software patents, it was the surprising sight of Red Hat and SUN jointly appealing for clearer thinking that tipped the scales in favour of the defeat of the motion.
• Red Hat has in many ways been the public vehicle of IBM’s major Linux initiative. Without Red Hat, much of that work would have had less “punch”, because Red Hat was able to encapsulate it into a platform that could be presented to traditional IT people. We’ve all benefitted from that punch.
• Red Hat was a leading proponent of GNOME, and to date has put far more active resources into the GNOME desktop even than Ubuntu. I do intent to match that as soon as Ubuntu stands on its own two feet, but until that time, hats off (har) to Red Hat.
• The NSA and Red Hat teamed up to make SE Linux feasible. Even though that hasn’t yet become widely adopted, it still was a crucial step in getting the Fed’s to treat Linux seriously in the data center – and that’s in turn brought a rush of hardware vendors and ISV’s on board.

There’s a danger in something we love to hate. It’s that we can forget to love it.

Daniel writes about the new X.org blue spark hotness. So how about some Edgy packages next Friday, Daniel? And if 7.2 can come forward a month, it’s also Edgy material…

I have to admit I was a little edgy, facing 15,000 people at the Java One event in the Moscone Center on Tuesday. Being on stage like that makes me break out in incomprehensible Swahili-nglish, every time. So I was grateful when Jonathan Schwartz gently pointed out that the licencing change we were discussing meant we could make *Java* easily available to free software desktop users, not Linux, as I said in my nervousness. Doh. Most free software desktop users already have Linux :-). Some day maybe it will be easier to be in the spotlight. Till then, I can only hope for hosts that gracious.

So, down to the nitty gritty.

Even though this was not the announcement we were all hoping for (a complete shift to free software Java) I was pleased to be part of the “Distro Licence for Java” announcement. As best I can tell, the new leadership at SUN clearly recognises the importance of the free software model AND the role of the community. That’s a big step forward and important to the progress of free software. If my being there could help accelerate the date when we really do have a free software licence for Java, then I was happy to take the time.

The new licence does not mean that we can include Java in Ubuntu by default. It does not yet meet our criteria for free software in order to get into “main”. But it DOES mean we can put it in the Multiverse or Commercial repositories, and people who want it can trivially get it after they have installed Ubuntu on a desktop or a server. Three clicks, and you’re done. Also, it can certainly be part of the default install of distros like MEPIS which leverage Ubuntu but add interesting and to many people useful proprietary bits.

So it’s a constructive step.

I wouldn’t expect a “big bang” conversion at SUN – it takes a long time to build consensus throughout a large organisation and it also takes time to straighten out the details of potentially thousands of other legal commitments. But the quote of the day was certainly Rich Green saying that the question is not “if” Java should be open source, but “how”. SUN wants to live in a free software world but also wants to continue to deliver on the promise of Java as a “compatible everywhere” platform. Even if there are already warts on the universal compatibility front (“write once, debug everywhere”) we’re all better off for the fact that Java is basically Java is basically Java. Developers don’t want to have to deal with “extended double remix” versions from a multitude of vendors. Especially not from vendors that specialise in “embrace, extend and extinguish” lines of attack.

So I thought I would ponder a little the strategic options and hypothesize a game plan that might do what SUN needs, while remaining firmly in the free software world. Here it is, inspired by the view of the Pacific coast as I make my way back down from San Francisco to Mexico City and Debconf.

1. Pick a copyleft licence, like the GPL
   If SUN were to make Java available under a free software licence it would benefit from having it under a copyleft licence, which means that folks taking it freely would also have to contribute their work back. Following the lead of the Free Software Foundation and requiring copyright assignment would preserve SUN’s undisputed ability to defend the ownership and licence of Java while broadening the base of talent that can help make Java perform on a wider variety of platforms. For many companies, particularly those who want to embed Java in their own infrastructure, a copyleft licence will not do (it would force them to make their own applications Free) and there will thus be plenty of incentive for them to strike a reasonable agreement on commercial terms for a custom licence to the Java code. This dual-licensing approach has worked for companies like MySQL. Java is both a client and a server technology, so the reach of such a strategy is even wider than server-only platforms like MySQL.
2. Manage the trademark very, very well
   Copyright is only one leg of the intellectual property table. The trademark is the piece MOST associated with compatibility. SUN gets to say who can use that trademark even if it makes the code itself available under the GPL. Of course, use of the trademark should be conditional upon the maintainance of compatibility. Those test suites will have to get even better. If SUN is smart it can deploy the trademark as free advertising on all the platforms which are not in any event revenue-generating (like, for example, the free software desktop) while at the same time benefitting from royalties in places where it can reasonably claim a share of the revenue flow (the proprietary desktop and the embedded marketplace).
3. Keep the patent option open
   I don’t believe software patents are a good thing (they’re a no-win deal for society, we shouldn’t grant a monopoly in exchange for a disclosure of something that could never be kept a secret anyway). That said, since the US allows them, SUN has them and SUN’s competitors have them too, it’s smart to tie Java-related patent licences to good behaviour on the part of licencees. The GPLv3 touches on this – in short, SUN can leverage the popularity of Java in order to provide itself with some protection from patent suits.

All of us at some stage or another have dreamed of having a brilliant idea then resting on our laurels while the cash flowed in. Truth is, life is never that easy. While Java was and still is a brilliant idea, there is plenty of competition out there and the field is hotting up, especially amongst the free software alternatives. I’d say there are no more than two years between today, and the day that a free software widget comes along that has all the characteristics of Java and *doesn’t* come from SUN. So, much as SUN would like the world + dog to rush to licence Java, that isn’t happening and won’t happen. More to the point, when that other contender exists it will rapidly become ubiquitous, at which point SUN will have lost the opportunity to lead that platform – forever.

Today’s markets are created by tools and standards that become pervasive and ubiquitous. Apache became pervasive because it is free software, and Java could do the same. This opportunity is SUN’s to lose, but everything I’ve seen suggests that they do want to grasp it with both hands. I can only applaud that commitment.

As a small incentive, I’ll close with a thought on the value of being the platform where innovation happens.

Right now, there’s plenty of development on “Java on Linux”. Think of every major company that deploys heavyweight J2EE infrastructure on Linux today. But that sort of development is not what drives innovation. When I look at the Java code that’s out there it’s industrial, but not exciting. The really exciting stuff tends to happen at the fringes, in student dorm rooms and tiny companies. And they tend to use the tools that are most immediately accessible to them.

• If Java had been free five years ago, PHP and LAMP might never have come into existence, because it would have been possible to deliver that vision using a lightweight integration of Java, scripting and the database.
• If Java had been free software four years ago, then it might have less of the reputation as a “corporate overkill platform” that it’s at risk of being saddled with right now
• If Java had been free software three years ago, then it would have even higher penetration on cellphones and mobile platforms, and there would not be competing free osftware implementations of the mobile Java VM and API’s
• If Java had been free two years ago, then Ruby-on-Rails might have been “Java on Rails” (though truth be told I’d rather it was Python on Rails!).
• If Java was free today, more of the AJAX stuff that is popping up all over the show would be done in Java (on the server side), because every student with a Linux box would have Java installed by default

A big enterpise-focused company like SUN will always convince big enterprises like Deutsche Bank that they should buy technology and services from SUN. Fair enough. But they can’t convince a smart Comp.Sci. student to envision the world using their tools – unless they make those tools pervasively and freely available. And it’s that smart Comp.Sci. student who writes the magic that will be headline grabbing material three years from now. Innovation starts in unexpected, lightweight sorts of places. So you need to ensure that your tools can reach into those lightweight sorts of places. When Java is free software, it will flow further out into the network – and that can only be a good thing for SUN.

You know, some days London just isn’t so bad after all.

Update: Of course. The fact that I am not actually allowed INTO that lovely garden is another matter altogether.

We’ve published a few more job descriptions in the “Jobs@Ubuntu” department this week. Jane and I went through our notes recently and realised we want to hire 24 people in the next few months! Fortunately, I enjoy meeting new candidates :-).
The scope of the jobs we have in mind for people is definitely widening from the pure development focus (though we need a few more of you too :-)) to the full range of skills that it takes to manage a project of this scale – and the relationships that it brings. As we’ve grown and as we start to shape Ubuntu into something that works well for the community and for commercial deployment, we need a whole new set of talent that speaks both the traditional language of management and marketing, as well as the new language of participation and community.
I enjoyed Mitchell Baker’s blog recently on the need for new roles in free software projects which I think says it better than I can. Except I would have said “free software” rather than “open source” ;-).

The challenge I think in hiring for these new positions is that community driven organisation is not something that “young executive tigers” are going to be familiar with. While I enjoy helping other companies wrap their heads around free software and the dynamic of the community, I sort of hope we can find a team of executives and operatives that have already made that leap. Rare people indeed. If you know someone who fits the bill, point them in my direction!

I believe distributed revision control is essential for healthy free software community development and I’ve been funding some R&D in that regard, which has turned into “bzr” (short for Bazaar-NG Version Control System). The latest release, bzr 0.8, has all the core functionality that we wanted and performs pretty darn well too for something that has had no optimisation and is written in Python with no C extensions.

The new release will be in Dapper, and in fact they hope to get a 0.8.1 out the door before Dapper ships so please test 0.8 and if you do find any issues let them know.
We’ve put a lot of effort into the “feel” of the system. Version control has to be like Goldilocks porridge – “just right” – but it’s difficult to describe exactly what just right is all about other than that it should be a smooth experience from start to finish. I think with 0.8 the Bzr guys are nearly there.

It’s trivial to start versioning a directory or tree of directories, and then to start passing those versions around to collaborate with friends. It has some unique capabilities, like the ability to work on a shared branch (like subversion) and then trivially to split off and work on your own branch, then merge back to the shared branch. This ability to be online and collaborative, and offline when I need to be, is what really makes me more productive. The new version supports a more efficient file storage format, called knits, which has radically improved the speed of network pushes and merges.
Here’s the short version of how to use bzr:

1. Start versioning files in a directory with bzr init
2. Make changes, and commit locally with bzr commit
3. Get status on that tree with bzr status
4. Push the branch to an sftp server with bzr push sftp://where.com/here/
5. Merge from another branch with bzr merge http://there.com/thatbranch/

In short, it’s fast, easy to use and “just works” the way you expect it to. I’m really pleased with the work that Martin and the bzr team have done. The community is shaping up to be nice and vibrant too. There are about 40 regular participants in discussions about bzr implementation, and there’s a nice plugin model with lots of interesting plugins coming from the community to do cool stuff like graphical representations of trees and merges.

It does a really fast annotate, so if A changes a line, and B merges that change from A, and C merges that change from B, and you merge from C, you can still quickly see that the line was changed by A. That still blows my mind. Gets and pushes are not as fast as git but the guys want to focus on network latency optimisation and local performance in the next major version, for now the emphasis is on the “revision control experience”.
To test Bzr you can:

• Download the latest release
• On Ubuntu Dapper, apt-get install bzr bzrtools

And of course it’s all under the GPL. Now, if only we could merge some of the good work being done in the Mercurial project into Bzr we would have one kickass Python VCS… I like their single-tree performance but far prefer our user experience.