Dapper has now been out two months, and in general the feedback from the release has been wonderful. I feel somewhat relieved – it was a challenging set of goals we set ourselves, but it seems that (a) most users prefer Dapper to any previous release and (b) most users are very happy with the combination of features and stability. Phew.
However, there’s always room for improvement, so we have been evaluating the release and figuring out what we can do next time to have a closer alignment of expectations and reality.
Matt Zimmerman put it best in a private mail which he’s given me permission to republish:
Matt Zimmerman wrote:
I've been thinking some lately about a disconnect in terms of the
community's expectations for Dapper. We've had more negative reviews (and
forums comments) of Dapper than for previous releases, and while it's not
clear to me whether this is proportional to the apparent increase in user
base, I'm concerned by what I find when analyzing the criticism.
Many are criticizing shortcomings in Ubuntu which have existed for years
now, or deplore the lack of eye candy and other superficial features, as
justification for an overall negative impression of the release. In
particular, I see repeated mentions of:
- Lack of 3D support out of the box on nVidia chipsets: [0]
- Need to use the command line for certain administrative tasks (including
the above) [0] [5]
- Lack of out-of-the-box support for Java, Flash, MP3, DVD, etc.
(RestrictedFormats) [0] [4]
- Lack of 3D accelerated desktop effects and other eye candy (e.g.
Xgl/AIGLX, prettier usplash) [1] [5]
- Lack of support for a particular hardware component (e.g., wireless card
or printer) [1] [2] [4] [6]
- Ubuntu not being easy enough for the typical user [0]
- Lack of availability of development tools in the default install [0] [2]
- Manual partitioning is clunky [0] [3]
None of these are new problems, but they are pointed out as examples of
major shortcomings by these reviews. It's notable that in some cases, we're
being compared with Windows, rather than other Linux distributions, which is
a much higher bar, but overall my impression is that there has been a
disconnect between the expectations of the community and what we delivered
with Dapper. In particular, I see indications that users expected Dapper:
- to be better-looking ("polished")
- to have more long-standing feature wishes implemented ("polished")
- to have no regressions from Breezy ("polished")
- to have fewer bugs than a typical Ubuntu release ("polished")
There have been a number of comments on bugs in the less-frequently-used
portions of Ubiquity, but we expected this when rolling a new installer for
Dapper. The only other genuine quality issue I've ascertained is a
collection of printing-related regressions, which are still being analyzed
and will be fixed via updates as we are able. Apart from those, the harsh
criticisms mostly fall into the category of unmet expectations.
Another disconcerting trend is that a number of the articles mention having
a negative experience interacting with the Ubuntu community for support.
It's possible that we're starting to see a decline in overall coherence
resulting from such rapid growth of the community: new people are getting
involved faster than they can learn proper behaviour from the existing
members. I'm not sure how to address this, but perhaps it should be
discussed at the next CC meeting?
I had a dialogue with a representative from the forums where he expressed
surprise at the Edgy feature list, because he thought the features which
went into Dapper were more "edgy" in comparison. He thought that things
like the revamped boot process infrastructure and 3D eye candy were "polish"
that he would have expected in Dapper, whereas from a developer perspective,
they are risky and disruptive features. It is this illustrative example
which led me to notice this disconnect, and do this analysis of the reviews.
So, what have we learned? Perhaps that even if we meet our goals in our own
eyes, we may be considered a failure by some if they have a different
interpretation of our intentions.
Should we be less high-level and more explicit about our goals for a
release? That's risky, too. Our Edgy announcement[7] went into detail
about specific features which fit the theme, but of the five specific
examples, it's quite likely that no more than one or two will actually be
implemented in Edgy. Users who latched onto the examples and built
excitement around them will be disappointed. The announcement explained
that the development team would be given the freedom to bring in risky new
technologies, but also foretold an increase in "bling" as a result, while it
turns out that our development team isn't very interested in that at all.
Users who expect Edgy to be MovieOS will feel misled.
My conclusion is that we should be rather more careful about how we announce
our plans, because these predictions can have disproportionate repercussions
on how our release is received by the world. We should be specific, but
only where we can be accurate. We should express the high-level vision for
the release, but guide the user on its interpretation as well.
We gained our position by meeting and exceeding the expectations of users,
not merely our own, and I think it's important that we continue to do so.
[0] http://www.ourtweaks.com/articles/linux.html
[1] http://linuxguy.bur.st/?p=12
[2] http://www.tectonic.co.za/view.php?id=1026
[3] http://www.nuxified.org/the_ubuntu_dapper_experience
[4] http://www.linuxforums.org/reviews/ubuntu_6.06_review.html
[5] http://www.tuxmachines.org/node/7300
[6] http://distrowatch.com/weekly.php?issue=20060612#opinion
[7] https://lists.ubuntu.com/archives/ubuntu-announce/2006-April/000064.html
So, what can we do better next time?
First, I think we need to realise that any heightened publicity will create impossible expectactions. Dapper was by far the “biggest” release the project has produced to date, we were all excited about it, there was a lot of buzz, and as a result there’s a temptation for each person to hope that this release will have the set of things that THEY think is essentia.
Second, I think the word “polish” is misleading, so I will try to stop using it. The problem is that it says different things to different people. To “polish” a release means to fix lots of little things that are irritating but not essential, and of course, everyone has their own list of bugbears. So saying the release will be “more polished” is about as bad as saying it will be “more better” :-). In future, I think we should articulate specifically where we will be investing the effort. In the case of Dapper, it was (a) the kernel, (b) the installer, (c) the server, and (d) the graphics theme and icons. Of course there was a ton of other work done, but that was what *I* meant by “polish”. So we need to communicate much more clearly EXACTLY where the investment and the results will be found.
Third, we should be honoured to be in head-to-head comparisons with Windows. Some of the toughest criticism of Dapper I have seen comes from people who have never used Linux before. And so they quite rightly stack us up against the OS they use every day. That is something of a success already – we can at least claim to be in the ballpark. Now we need to look PAST where Windows is today, and deliver an experience to those users that is so compelling in some places that they are willing to forgive a few of the warts.