#5: Real real-time collaboration

Wednesday, December 20th, 2006

This is one post in a series, describing challenges we need to overcome to make free software ubiquitous on the desktop.

Collaboration is the key ingredient in free software – the fact that developers can collaborate despite geographical and cultural differences between them is what has made it all possible. And our tools for collaboration are pretty good. I maintain you need three things before you get an explosion in collaboration: you need a common format, you need revision control (so you know who changed what, when) and you need a transport layer. In the case of free software, it was text files, CVS and email that underpinned much of the growth in the developer community.

Recently, wiki’s have shown the benefits of direct collaboration  for content other than source code – and wiki’s also have those three ingredients – a format, revision control, and a transport layer.
We could do with some improved tools (I think Bazaar has all the ingredients for a next-gen version control system, for a start, take a look at it if you’ve not done so recently) in the free software community. But this post isn’t about that – it’s about bringing collaboration to chattering classes.

See – people who work with word processors and spreadsheets have rights too! And they could benefit dramatically from much better collaboration. Think about it – they often email documents around so that other people can edit something or review something. And keeping track of those documents and the changes is a bit of a black art – involving large amounts of what lawyers call “redlines” – Word documents with changes highlighted.

Wouldn’t it be better if they could just collaborate in the documents directly? And what if we could make that collaboration real-time, much as Gobby makes text file editing and collaboration real-time? If you’ve seen Croquet, you’ve likely seen a demo of £d virtualised collaboration between avatars… so this “real time collaboration” idea and the sensory immersion idea (and of course the presence challenge) are all related.

I’m pretty sure the proprietary software world is going to do this – so it’s an opportunity for us to be a few years ahead of the state of their art. Real, real-time collaboration on all sorts of file types will radically change the way people think of working on their PC’s.

22 comments:

  1. Vincent says: (permalink)
    December 20th, 2006 at 10:40 am

    The problem with this is that my fellow students don’t want to. At least, as long as it doesn’t work perfectly (even though MS Word does not either). I’ve tried to introduce them to Google Documents or Zoho Writer but they were very reluctant. Some hesitatingly use it, but not whole-heartedly. What I’m trying to say is that it must really work perfectly for (normal) people to really use it.

  2. Janne Kaasalainen says: (permalink)
    December 20th, 2006 at 11:13 am

    There already is projects that go to this direction, even if not perhaps to the extent you suggest. Autodesk does similar things with programs such as Toxik (composition and video) and Avid has Alienbrain (CVS style system for images and media formats). Might wish to take a look at these.

  3. Paul says: (permalink)
    December 20th, 2006 at 11:37 am

    I think one of the key issues would be to develop a secure and private platform for people to colllaborate on. This probably won’t require massive innovation but when you consider having parties to a negotiation editing and working on a draft agreement remotely using something like a wiki then it needs to be secure and hidden from public view. Once you get that sorted out the next task is to get people to buy into that sort of collaboration. We have become accumstomed to emailing documents, working on them in the privacy of our own computers and mailing it back. Collaborating in realtime on a document formatted as a wiki page requires a bit of a mindset shift.

  4. James says: (permalink)
    December 20th, 2006 at 12:32 pm

    I haven’t tried real-time collaboration, but to me it sounds like it will just make collaboration more complicated. The big problem with current collaboration tools (based on my limited experience working with them) is conflicts and conflict resolution. I suppose if you could see what other people were doing in real-time, conflicts wouldn’t arise, but is full real-time collaboration really the answer?

    What’s wrong with only checking for conflicts in real-time to ensure that they do not occur, but otherwise leaving individuals to choose when they want to do a commit?

  5. Søren Hansen says: (permalink)
    December 20th, 2006 at 12:54 pm

    Google Docs is already doing this. They allow importing and exporting of roughly the same formats that OpenOffice supports, and several people can be editing the same document at a time kind of like Gobby only with a few seconds delay. When I first saw that, it got me thinking along the same lines as you.

    There should definitely be a free software version of this. We’ve already got all the building blocks.

    A lot would be gained by basing the document format on HTML. A realtime collaborative web thing could be built on some sort of AJAXy version of the design mode of both Firefox/Epiphany/whatever or IE like TinyMCE or FCKEditor. OpenOffice can be scripted to convert MS Office or ODT documents to HTML and vice versa, but of course HTML is somewhat limited compared to ODT or OpenXML or whatever. I’m sure a clever solution could be found (maybe a subset of LaTeX HTML conversion).

    If the backend for revision control was based on Bazaar one could branch the document when going offline or something and merge it when you get back.

    The framework could probably in time be extended to do spreadsheets, presentations and all the other whizzbang that comprises a full office suite.

    So the recipe could be something like:
    Ingredients:
    2 spoonfulls of Bazaar
    1 pinch of AJAX
    2 slices of TinyMCE
    1 cup of LaTeX

    Put it all in a big pot and stir like a mad man.

    Serve early, serve often. Enjoy.

    Soo many cool things to do, so little time. :-)

  6. Luis Arias says: (permalink)
    December 20th, 2006 at 1:19 pm

    I wish to point out that at XWiki, we are working on the topic of large scale collaboration in our advanced wiki environment. In XWiki we can collaborate on content but also on source code because we have Velocity and Groovy as high-level scripting languages integrated in the wiki. This is all done in open source under an LGPL licence. An online example of this that is in an early stage of deployment is the Curriki.org website which in its next phase will leverage wiki style collaboration to provide a complete authoring system aimed towards curricula. Our current research and development agenda will be taking our XWiki platform in the direction of P2P and offline usage and also integration of semantic web concepts and features.

  7. Huygens says: (permalink)
    December 20th, 2006 at 1:24 pm

    There are already open source project walking on those tracks and pretty advanced :-)
    One project from a French university and released under open source match your criteria:
    * PengYou: http://www.pengyou-project.info/en/index.php (watch the demo in the About section)

    Another one also from a French university, but the license is yet unknown, is:
    * ShareIt: http://www.shareit.fr/en/default.aspx (still under development, but there are some demo already in the download section)

    Huygens

  8. Jonathan says: (permalink)
    December 20th, 2006 at 1:31 pm

    Mark,
    As much as I hate to say, but with what MS is doing and pushing with Sharepoint it appears they “get” collaboration better then some people do. While Sharepoint is probally not the best piece of web software standards wise, it is easy to setup and integrates greatly into MS Office. Until I see a product that integrates so tightly into OOO or perhaps KOffice as it improves, Office and Sharepoint is still the best combination I have seen. When you can get rid of a file share, a company drive where everyone dumps their docs on and replace it with a “intranet” type of software that includes versioning control, and the ability to check-out items as easiely as Sharepoint, then you are behind the game.

  9. Henri Bergius says: (permalink)
    December 20th, 2006 at 4:53 pm

    Not only should collaboration happen inside the desktop applications, but free software apps should also communicate with their server-side equivalents. There are loads of good content management systems, project management systems and others in the open source sphere, but the only way to use them is via web.

    As to collaboration, the OLPC human interface guidelines has some good ideas:
    http://wiki.laptop.org/go/OLPC_Human_Interface_Guidelines/The_Laptop_Experience/Zoom_Metaphor

    In an optimal world all collaboration would happen seamlessly over ZeroConf, just like it does with apps like SubEthaEdit in the Mac world. For more remote collaboration we could then have either P2P networking or collaboration via a free software server-based app.

  10. lefty.crupps says: (permalink)
    December 20th, 2006 at 6:49 pm

    In somewhat of the same thought-frame, if technically very different, is real-time remote support. The kind where one use can call another and, from across the world, can share control of a desktop. Windows has had this for years, being built into their MSN chat program. Kopete says its out of their scope, and other programs (VNC etc) don’t seem to have any push to implement this (NAT traversal, which in my eyes requires a chat-like connection to start with).

    Not until average users can collaborate on fixing their errors will they have interest in collaborating on documents with a ‘broken’ OS (‘broken’ in their eyes, maybe just mis-configured for a knowledgeable user). If a user cannot find their document because of whatever reason, but knows how to do so in MSWindows, which are they going to use? The one they know, or the one for which they can find no real-time support?

  11. Phil Hagelberg says: (permalink)
    December 20th, 2006 at 8:45 pm

    I’m looking forward to seeing Gobby support in more applications. Gobby isn’t just an editor, it’s a protocol for implementing collaborative editing. Let’s see this stuff get more widespread! (I’ve already hacked rudimentary emacs support, but I don’t think anyone has done any other implementations.)

  12. SixDays says: (permalink)
    December 20th, 2006 at 9:02 pm

    http://www.synchroedit.com/ is a project is exactly this…

  13. Timo Aaltonen says: (permalink)
    December 20th, 2006 at 10:24 pm

    There’s already O3Spaces that works with both Openoffice.org/Staroffice and MS Office. Don’t know if it’s any good.
    http://o3spaces.org/

    It is true that similar funcionality as OSS is quite desirable

  14. liquidat says: (permalink)
    December 21st, 2006 at 1:53 am

    It would be nice to have the ability to collaborate on stuff. One way I would *love* to see is to have a jabber account with jingle support.
    The jingle p2p system could take over the data exchange part (maybe with keys to have company secrets secured) and could make sure whom you are working together with, and the app developers could concentrate on the app.

    Also, with jingle we would have already a telephone and a video conferencing software (ok, some bits not ready yet, but that can be done).

    A good start would be to first have a good jabber/jingle client (maybe in telepathy with an example client to not disgruntle gaim and kopete) with full audio, video and file sharing support. The next step would be to add a collaborating sketch board (Coccinella is a client which can do this already, I think, there should be some code usable; also the kdevelop guys develop something like that for their development suite) to be able to collaborate on simple drawings and sketches. After that a simple text editor and a simple wiki should follow.
    With the experiences gained from these projects the next big step would be full integration into OpenOffice/KOffice – and together with this integration the project could settle on stable APIs which other app developers could use to also integrate this new awesome app with their apps.

    Last but not least there would follow a collaboration management server for collaborational work.

    I think that this sounds like a cool plan – the only thing left now are some developers and some people with a realy vision to drive the project and the developers into the right direction. ;)

  15. Andy Hadfield says: (permalink)
    December 21st, 2006 at 7:24 pm

    Collaboration is key. In fact, we came up with an interesting idea the other day. Have been playing with Ubuntu for quite a while now – particularly impressed with stability and ability to run of low spec machines. In SA, I’m sure people (well, I hope people) are aware that the medical fraternity is under huge IT pressure. They just don’t have the infrastructure, machines, support or software needed to store patient data, get lab results across distances and collaborate on diagnoses and management methods. Already thinking Wiki? Well, the seed of the idea is to get gov sponsorship or other to roll out an Ubuntu network to all hospitals. Stable. Locked down. Network perhaps over dialup. Hospitals prob get MS licenses for free – but the machines keep crashing. Baragwanath hospital for instance, (largest hospital in the world, in Joburg) – has FIVE working terminals. That’s ridiculous. Wondering what people think? And if there’s any takeup to the idea? Link to the article is on my name above…

  16. laserjock.us :: REVU Sprint Success says: (permalink)
    December 23rd, 2006 at 6:44 pm

    [...] The REVU Sprint was a big success for us and the community. Giving the Ubuntu community a chance to contribute to their OS is one of the biggest strengths of our distro. One of the keys to generating contributions is quick feedback and acknowledgment of work done by contributors. As Mark has blogged recently, collaboration is a key element of free software and we are utilizing IRC, REVU, and in the future bzr/Launchpad to make collaborative contribution of software to Ubuntu relatively easy, timely, and fun. [...]

  17. Mike’s Thoughts » Linux on the Desktop - Different Ideas and Theorem says: (permalink)
    December 24th, 2006 at 3:54 am

    [...] As you all probably know by reading my puny blog, I link to Mark Shuttleworth’s blog.  Mark has been running this interesting commentary on making open source software more prevelant on the desktop.  I’ve read over his posts with a great degree of interest especially this one.  If you remember awhile ago,  Asa Dotzler published on his blog some reasons why Linux was not moving closer to the desktop.  Compare the two sets of ideas there and see what Mark is saying regarding the tools, the approaches, the validation and the reasons why Asa says that it was not ready in 2005.   I’d like to see Mark address the areas that Asa listed in 2005 and see his take on where Ubuntu Linux has brought us in the original areas and how his primary areas of making open source more prevalent and ubiquitious address the issues or at least dent them in that Asa discussed.  But the few reasons in my conversation were more at the street level and perhaps if you read Ian Murdock’s blog of late, you’ll see another set of reasons regarding package management and installation of software.  [...]

  18. Meriblog: Meri Williams’ Weblog » links for 2006-12-24 says: (permalink)
    December 24th, 2006 at 11:45 pm

    [...] Mark Shuttleworth » Blog Archive » #5: Real real-time collaboration “See – people who work with word processors and spreadsheets have rights too! And they could benefit dramatically from much better collaboration.” Mark nails it again – a great challenge for OSS to attack (tags: community opensource markshuttleworth opensourcesoftware linux ubuntu collaboration) [...]

  19. Mo says: (permalink)
    December 26th, 2006 at 2:38 pm

    I remember the online music jamming community Res Rocket, there was a collaborative music sequencer program that had virtual studios where MIDI instrument owners layed down their lines and could edit others; it was way ahead of it’s time and a great success ’til it folded. It’d be great if that sort of thing was part of the Ubuntu Multimedia/Music distro. With current advances this could be superior to Res Rocket.

  20. Thorsten Wilms says: (permalink)
    February 5th, 2007 at 11:46 am

    I think (realtime) collaboration shouldn’t be tackled application by application. Instead there should be a framework to make it a feature of the system (ideally on a layer below desktop environments). That’s the only way to make it widely available and consistent.

    For transport, both p2p and client/server should be possible. I guess each participating application would need to understand the concept of Edits for the pieces of information that are exchanged as fast as possible. So the transport layer sends/receives Edits as opaque elements and the apps have to make sense of them.

    Edits should be viewable in a log format like IRC chat. In collaborative editing, the same flow from synchronous to asynchronous should be expected.

    There needs to be versioning, branching, merging and annotations. Edits should be tracked with information Who did it When.

    Users should be able to ‘share’ (make available, expose) on the version level (private/public versions …)

    There should be a central ‘Person’ or Contact type of object to bundle address information, to represent state (availability) and for associating Edits/documents/activities.

    [Mark: I touched alot of this in my "Rethinking Desktop Computing" document, send to Ms Newman]

  21. Dustin Harriman says: (permalink)
    July 27th, 2007 at 1:36 am

    I can’t believe nobody has mentioned Alfresco. There is a straight-up explanation of it at Wikipedia also.

    Alfresco is very “industrial strength”, but it does everything asked for here (plus a whole bunch of other stuff). They have a commercially supported version, and a GPL’ed “community version”, which has all the same functionality as the commercially-supported version. A key strength is that it has CIFS/SMB and WebDAV access. This lets you easily copy documents in and out of the Alfresco, with no need for any special “thin client” (in both Linux and Windows clients). You can even “map a drive letter” to the Alfresco server in Windows.

    It has lots of Open Source technologies under the hood, check out their “technical specs”

    It is probably overly complex for most small groups’ needs: it does all kinds of fancy workflow management, enforcing of fine-grained permissions, etc. It’s very “all-singing, all-dancing”. Their target market is large enterprises.

    I too hope someday there emerges a sort of “Alfresco-Lite” that just does ODF document collaboration in a wiki-like fashion.

  22. Olo says: (permalink)
    August 1st, 2007 at 6:42 pm

    I can see that some people (e.g. suggesting that O3Spaces has the features) aren’t quite grasping the concept.

    For those, this Wikipedia article should make it more clear: http://en.wikipedia.org/wiki/Collaborative_real-time_editor