Writing a book collaboratively?

Wednesday, October 4th, 2006

Someone joked about giving Jane the title of “Chief Wiki Officer”. We have about 7 wiki’s internally and there’s no end in sight! I think we could consolidate them and use access controls to keep various communities distinct but – for those of us who have to roam across them – integrated. This got me thinking about the limits of current wiki technology. Is there any comprehensive list of wiki platforms?

At the same time at the Foundation in Cape Town we are busy gearing up to commit about R30m (about $4.5m in today’s terms) to create a complete set of school books for all subjects in the South African curriculum in wiki format. I got to wondering which format we should use. There’s a lot of action around MediaWiki (used by Wikipedia) but one of our goals is to be able to print low cost books – real books with tables of contents etc on real paper – for those schools which don’t have bandwidth or PC’s. Can MediaWiki produce DocBook output? Or should we be using a different collaborative platform altogether? I remember hearing about a DocBook wiki system, but can’t find any references to that now.

So here’s the question – if you wanted to write a book collaboratively, with rigorous access control and revision control but also with chapters and sections, and an index and tables of contents etc – what platform would you use (free software greatly preferred, and open format’s required, of course :-)).

86 comments:

  1. Yannick says: (permalink)
    October 4th, 2006 at 7:13 pm

    Hi Mark,

    Maybe you heard about “DocBook Wiki” (http://doc-book.sf.net) ? (3rd suggestion of Google for DocBook wiki system. :-)

  2. Bruce D'Arcus says: (permalink)
    October 4th, 2006 at 7:15 pm

    I don’t have a good answer for you, as I’m interested in the same thing. But it seems to me from looking at this page that it shouldn’t be hard to export MediaWiki content to structured XML. Just dump it to its XML export format and then run a script convert the wiki markup (could even map it directly to TeX, which can — in the right hands — generate really gorgeous output).

    In general, I think the wiki world needs to see the possibility of using wikis as hybrid-authoring tools of this sort. Writely is just a wiki after all, but they have good support for export (including of OpenDocument). It’s kind of embarrassing that at the ODF TC (where I am a member) we can’t export our wiki content as ODF!

    In any case, please do an update if you find a good answer.

  3. Neale Pickett says: (permalink)
    October 4th, 2006 at 7:18 pm

    Drupal has a “book” module that more or less does this (a collaborative book with chapters, etc.). I think people need to be authenticated to edit anything. I believe Plone has something similar.

    I’m not a real big fan of either, but those are the ones I know about.

  4. Andreas Dalsgaard says: (permalink)
    October 4th, 2006 at 7:25 pm

    You might take a look at DocBookWiki http://doc-book.sourceforge.net/homepage/

    From the homepage:

    The features of DocBookWiki can be summarized like this:

    * Can display a DocBook document online.
    * Can display several documents at once (a list of books).
    * Can display each of them in several languages.
    * Allows to edit a certain section of a certain book in a certain language.
    * Editing can be done in several modes, like text, html, xml, texi, latex etc.
    * The basic format is always DocBook (XML), no matter how it is displayed or edited.
    * Each document (in each language) can be converted automatically into other formats (like PDF, RTF, LaTeX, etc.) for downloading.
    * All the history of modifications is kept (in CVS) and any previous versions of a document can be recovered (by tag or by date) by the admin of the site.
    * Authentication of editors can be enabled and admin can assign different access rights and permissions to the editors.

    In short, DocBookWiki will be like a wikiwiki which saves the content in XML (DocBook) format.

    Regards
    Andreas Dalsgaard

  5. Neil Blakey-Milner says: (permalink)
    October 4th, 2006 at 7:30 pm

    I spent a while chatting to Brad about this exact problem about a year ago. My take is that you want a DocBook-equivalent backend format, since using HTML tags like em/strong/&c. to mean application/classname/constant/literal obviously loses a lot of information. The problem is that we have these wonderful WYSI(almost)WYG htmlarea/TinyMCE/Kupu/&c. applications that allow non-techies to use presentation markup, and nothing like it for context markup for something like DocBook (which is the obvious choice for documentation).

    So, you make an HTML dialect that uses class attributes on your HTML tags to indicate that this particular em (in HTML) is actually representing a classname (in DocBook), and you make those little buttons in the WYSIAWYG editors generate those tags, and not what they do now. It’s valid HTML, and can be converted into DocBook for use beyond plain HTML as well.

    So, no existing solutions, but a general plan on how one could attack the problem. And it can relatively easily be retrofitted onto whatever solution provides the access and revision control.

    One advantage Mediawiki has is that it allows you to edit sections of a larger document, and obviously this would make it a lot less intimidating for non-techies to make changes. As much as I’d prefer a Python solution, it may be best to go with the MediaWiki in terms of being more accessible for contributors.

    Neil

  6. Joshua K says: (permalink)
    October 4th, 2006 at 7:46 pm

    We’ve been investigating this as well, as I am doing documentation for a Master’s Project via a wiki, but want to be able to print it. PmWiki (http://www.pmwiki.org) with the WikiPublisher plugin (http://www.pmwiki.org/wiki/Cookbook/PublishPDF) produces some very pretty output. They have demos at the above site. By default, it uses a remote typesetting service, but you can download all the software and instructions to locate your typesetting locally. Something they’d probably appreciate if you’re doing 500+ page texts. :)

  7. no one in particular says: (permalink)
    October 4th, 2006 at 7:51 pm

    Is there any comprehensive list of wiki platforms?

    If there is, where do you think it would be? That’s right: wikipedia has one! I don’t know if it’s comprehensive, but I’d imagine it covers all the major ones. I certainly haven’t heard of most of the ones listed.

  8. TobiasStrauß says: (permalink)
    October 4th, 2006 at 7:54 pm

    i can’t believe that latex didn’t get called. Hm. It fits all your criteria. Managed by rsc (svn,cvs,blub) and splitted each chaper a file. Simple, plaintext , approved, flexible ???
    it would be my choice.

    greets

    Tobias

  9. Estama says: (permalink)
    October 4th, 2006 at 8:00 pm

    What you are looking for is a wiki using a markup language somewhat like this:
    Asciidoc: http://www.methods.co.nz/asciidoc/
    or
    Deplate: http://deplate.sourceforge.net/index.php
    or
    Restructured text: http://docutils.sourceforge.net/rst.html

    Asciidoc is geared toward docbook export but it doesn’t look that great in its text representation.
    Restructured text looks great in its text represantation but the docbook export is a work in progress.
    Deplate is in the middle.

  10. Mike Lovegrove says: (permalink)
    October 4th, 2006 at 8:13 pm

    It sounds like the software you described in your post is this http://doc-book.sourceforge.net/homepage/

    I haven’t had chance to check it out properly yet, but it looks like development has stalled and there is no real action on the mailing lists.

    Still, in terms of what you’re looking for it seems to be reasonably close to the mark.

  11. paa.kwesi says: (permalink)
    October 4th, 2006 at 8:17 pm

    You might want to take a look at Drupal, it’s core book module and the contributed docbook module documented here http://www.puregin.org/node/688 and http://drupal.org/node/38757.

    It does quite a decent job.

  12. Joe Terranova says: (permalink)
    October 4th, 2006 at 8:22 pm

    http://en.wikibooks.org/wiki/Main_Page uses the wikimedia format, and allows exporting as pdf, and obviously sections and chapters. May or may not have what you’re looking for.

  13. Rick says: (permalink)
    October 4th, 2006 at 8:24 pm

    I’ve not played with a ton of the wiki packages out there, but we use DokuWiki which has one great feature. Each page is a text file in a directory structure matching your Wiki paths. It seems it might be possible to create some extra code to walk the txt files and do some conversion of wiki/docbook markup?

  14. Dave says: (permalink)
    October 4th, 2006 at 8:43 pm

    Just saw today in fact that the MoinMoin wiki has the ability to output DocBook. No idea how good it is though… Also, while not directly wiki, there is a markup system called AsciiDoc that uses wiki like syntax to produce DocBook. Writing a parser for an existing wiki that interprets asciidoc shouldn’t be too difficult.

  15. Jeff Waugh says: (permalink)
    October 4th, 2006 at 8:54 pm

    Not FLOSS, but you might want to chat to O’Reilly about their publishing system. Perhaps they could turn it into a public web application for projects such as this. It might even tickle Tim’s Web 2.0 fancy, too.

  16. John Stowers says: (permalink)
    October 4th, 2006 at 9:03 pm

    I would suggest looking at MoinMoin. Its Docbook support is currently the best of the bunch, there was a Google SOC project on improving the Docbook this summer.

    http://fedoraproject.org/wiki/SummerOfCode/2006

    I know both Fedora and GNOME are looking into moving their docs over to a wiki based format and using the Docbook export functionality.

    If you want to do other fancy automated built type things MoinMoin also has an XML-RPC interface which may fit into some sort of publishing workflow.

  17. khaled says: (permalink)
    October 4th, 2006 at 9:42 pm

    We’re currently working on a wiki for the office and my partner in crime and I have been giving all of this a lot of thought. One wiki system that has caught our eye, although we’re still thinking about it is Dokuwiki(http://en.wikipedia.org/wiki/DokuWiki). A good summary of this can be found here: http://www.sitepoint.com/blogs/2004/10/11/pick-of-the-wikis-dokuwiki/. Although from reading that article it also mentions that mediawiki does have the option to export to docbook format…

  18. LaserJock says: (permalink)
    October 4th, 2006 at 10:21 pm

    Mark,
    You might also want to talk with the doc team. We had a Summer of Code project for a Moin wiki -> docbook converter that seems to work pretty well. I’m not sure if it would handle a whole book, depending on the size, but I’ve used it for a article/small chapter size chunk. We’ve also done docbook -> pdf -> lulu.com with the Ubuntu documentation.

    -LaserJock

  19. Mary says: (permalink)
    October 4th, 2006 at 10:46 pm

    You may be thinking of Apache Lenya, which the Ubuntu documentation team was considering using for a time. I haven’t been on the doc team for ages and never evaluated it myself, but some of people still there might be able to comment.

  20. Simon Willison says: (permalink)
    October 4th, 2006 at 11:56 pm

    LaTeX would be ideally suited if it didn’t suffer from (by current standards) relatively obscure syntax. It produces beautiful documents and has a great HTML converter, but being macro based it unfortunately has a pretty steep learning curve.

  21. labiculum says: (permalink)
    October 4th, 2006 at 11:59 pm

    Maybe you guys should have a look at wikimatrix.org where you can compare wikis based on numerous criteria. Personally, I prefer DokuWIki – which also has an optional DocBook plugin. Hope that helps.

    labiculum

  22. Alexandre Vassalotti says: (permalink)
    October 5th, 2006 at 1:40 am

    MoinMoin. A Moin to DocBook plugin was one the projects of this year Summer of Code. [http://fedoraproject.org/wiki/MoinDocBookProject] You should talk with Matthew East. He know more about it than me.

  23. Tim Butler says: (permalink)
    October 5th, 2006 at 1:58 am

    Not too sure how they do it, but PediaPress (http://pediapress.com/) have a very dynamic solution to printing wikipedia pages. I haven’t used them yet personally but I have created a demo book and it appears to work quite well. It’d certainly be worth talking to them as they certainly have a very powerful system in place.

  24. Yasser El-Hawari says: (permalink)
    October 5th, 2006 at 2:23 am

    I am not a specialist in this topic but interrested in a solution since a long time. In my opinion this problem should be solved “from scratch” as it is very important/fundamental to easily produce/work on web content on the one hand and to be able to put out something beautiful like LaTeX does that is worth to be printed on the other hand.

    I can’t believe there is no proper solution for this!:-/

  25. somecanuckchick says: (permalink)
    October 5th, 2006 at 3:12 am

    The Wiki format, i.e. Wikipedia, in my not so humble (contributing) opinion – is one of the most comprehensive and searchable forms of ‘pedia information… and then some.

    Had the likes of Wiki existed back in the days now known as the formative… life would have been so different.

    With respect to using wiki technology to write a book… from the tech writers perspective, I think that acquiring feedback – from anywhere, anytime, etc. – is essential and effective in 2006+. It makes sense. Especially if the same principles of editing that most wikis are used in the process…

    It is, by far, one of the most simplest, and savvy ways to write a book that requires a number of sources, footnotes, etc.

    my 2 cents,
    :) SC

  26. Frank says: (permalink)
    October 5th, 2006 at 3:56 am

    Some resources that may be of use…

    http://www.wikimatrix.org/ ( feature comparisons of various wiki’s)

    MoinMoin appears to have a “render as Docbook” action.

    “MoinMoin 1.3.5 supports DocBook parsing and generation.” from http://moinmoin.wikiwikiweb.de/DocBook

    “DocBookWiki can also be used to edit a DocBook document online, from the web.” from http://doc-book.sourceforge.net/homepage/

  27. silwol says: (permalink)
    October 5th, 2006 at 4:03 am

    I just found this one:
    http://doc-book.sourceforge.net/homepage/

    Never tried it, but what is written on the homepage sounds quite promising for your requirements.

  28. Anmar Oueja says: (permalink)
    October 5th, 2006 at 4:08 am

    I have an interest in this particular subject as I have been trying to find a convenient way for our developers to collaborate on writing the Spec, Design and test documents for all the software projects we work on. Now my ideas might not fit your requirements perfectely, but the crossover might help fill in some pieces of the puzzle.

    Components of the system are:

    - MoinMoin engine as a command line tool (http://moinmoin.wikiwikiweb.de/) – require development to strip out the engine from the wiki program
    - Use dokuwiki syntax with a few changes (http://wiki.splitbrain.org/wiki:dokuwiki). Dokuwiki’s syntax is the most logical and sane one I have seen of many wikis
    - Use Prince (commercial tool unfortunately) to generate the PDFs of the final docs (http://www.princexml.com/)
    - Use version control system to manage the changes. For your needs a wiki interface might make more sense.

    Prince is probably the biggest component you might consider. You will be able to generate a professional looking PDF docs that rival those generated by DocBook or LaTex.

    Why not use DocBook instead? I need the guys to work on the docs collaboratively and mergin XML is a nightmare. Using Wiki tags is much easier to learn and decipher and most importantly merge. Providing professional looking documents/books in PDF is a Prince task. Now using a wiki or a command line tool to parse the wiki tags is a work flow choice.

    Why not LaTex? you don’t wanna go there ;)

    Hope this helps

    Salam

  29. Koos says: (permalink)
    October 5th, 2006 at 5:28 am

    Is there any comprehensive list of wiki platforms?
    http://www.wikimatrix.org/

  30. George says: (permalink)
    October 5th, 2006 at 6:01 am

    ask the doc team, been through this 100 times.
    Last i knew of nothing was upto scratch yet.

    maybe its time to fund one!

  31. Brandon Barry says: (permalink)
    October 5th, 2006 at 7:15 am

    MoinMoin may be able to meet your needs. See Mikko Virkkilä’s (completed) Summer of Code project, which “aims to improve and facilitate the use of the DocBook format, inside a MoinMoin-wiki”.

    I’ve exported individual pages as DocBook articles, but I haven’t exported an entire book. It seems a recent MoinMoin change has broken the latter option. Thomas Waldmann is working on a fix.

    http://fedoraproject.org/wiki/MoinDocBookProject

  32. Florian Hackenberger says: (permalink)
    October 5th, 2006 at 7:19 am

    There is http://www.wikimatrix.org/ which lists a LOT of WIKIs and has a comparison chart.

    Regards,
    Flo

  33. Reto Mueller says: (permalink)
    October 5th, 2006 at 7:31 am

    There is actually a (German) Company an Project which publishes paper books form wikipedia contents.
    The project has a webpage on http://de.wikipedia.org/wiki/Wikipedia:WikiPress (sadly only in German) and http://www.wikipress.de.
    I do not know in detail how the process works but to me this would be the place to go and ask for more information.

  34. Diwaker says: (permalink)
    October 5th, 2006 at 8:01 am

    Have you looked at Code2? http://codebook.jot.com/WikiHome — they’re trying to revise the book entirely on the Wiki. Clearly Jot doesn’t meet the requirements of being open source, but its a start. I think it should be possible to build something around any Wiki which stores files in plain text — like MoinMoin or DokuWiki.

  35. Reto Mueller says: (permalink)
    October 5th, 2006 at 8:06 am

    I knew ther would be something similar in English!
    http://en.wikipedia.org/wiki/Wikipedia:WikiReader put only 1-or to have actually been printed.
    yours andreas

  36. Donn Ingle says: (permalink)
    October 5th, 2006 at 8:19 am

    I am thinking mainly about the printing end of the equation. I am thinking about lulu.com (where there are ubuntu books on sale).
    I am thinking that if a printer (physical machine) like the one lulu uses (they don’t own it, but I can’t recall the name of the company that does) could accept open-source input (like an HTML file) then it would be a powerful way to cross the digital divide.
    At the moment (on Lulu) one has to produce content in a very specific PDF format. They actually require Adobe products and they recommend Windows as a platform. This is anathema to Free software et al.

    I am not technical enough, but I cannot see why an HTML page (esp with a print style sheet, or a limited repertoire of formatting) as generated by MediaWiki cannot be sent to a script to chop it up into the correct dimensions for output to an on-demand printer (POD) which can accept that input.

    Yes – it’s all about the owning and possibly the creating of said printer, but why not? Why leave the hegemony in the hands of Adobe and Co.? There must be lots of people out there who want an open POD solution. I do!

    Perhaps it’s easier to think of a: MediaWiki (via css) –> script –> PDF output –> POD output.

    That’s my R0.1

    D.

  37. hummingbird says: (permalink)
    October 5th, 2006 at 8:44 am

    DocBook or YACXS (Yet Another Custom Xml Schema) + Hack in support to Kupu or whatever WYSIAWYG (with thanks to Neil for articulating this :-) editor.

    Shouldn’t be a huge amount of work to extend the editor to support other tag lists than HTML and the actual rendering can be handled with XSLT.

  38. Murray says: (permalink)
    October 5th, 2006 at 9:27 am

    You may already know of it, but via John Naughton’s online diary I came across

    http://www.ikiw.org/2006/09/23/wikisym-2006-proceedings/

    which claims to be a collection of papers about using Wiki’s for education. Including one by wiki father Ward Cunningham.

  39. Vincent says: (permalink)
    October 5th, 2006 at 9:31 am

    How about just contributing to Wikibooks?

    http://en.wikibooks.org/wiki/Main_Page

  40. housetier says: (permalink)
    October 5th, 2006 at 9:35 am

    I would use Drupal (http://drupal.org) with textile input filter (http://drupal.org/handbook/modules/textile) or ReST input filter (once it is available).

    Not only *would* I use it, I *do* use it. Drupal has been extremely robust against all my attempts at tweaking and tuning. I make changes and drupal continues to function.

    This book (http://nrrd.de/dasbuch/die_1337_kammern_des_irc) is in German and collaboratively written by me and a friend.

  41. Stefan Hinz says: (permalink)
    October 5th, 2006 at 10:09 am

    @Vincent: I don’t think Wikibooks is what Mark is after. Here’s why:
    “Wikibooks is not paper. Thus, Wikibooks has no size limits, can include links, etc. It also means that the style and length of writing appropriate for paper may not be appropriate here. The authors of a Wikibook module don’t worry about an event occuring tomorrow that makes all the large, expensive paper copies outdated, as the Wikibook will change as well.”
    http://en.wikibooks.org/wiki/Wikibooks:What_is_Wikibooks#Wikibooks_is_not_paper

  42. anonymous says: (permalink)
    October 5th, 2006 at 10:35 am

    Have you had a look at this?

    http://doc-book.sourceforge.net/homepage/

  43. Gord Allott says: (permalink)
    October 5th, 2006 at 10:36 am

    I would have to say that the ultimate in collaborative book writing has to be a combination of Lyx and a revision control system like svn (or something newer if you like that kinda thing). Lyx is just simple what you see is what you mean system where you type what you want and mark it as say, a line that states a title, or author or chapter or whatever you want and it formats it all for you. it then produces LaTeX output which is great for the ‘written word’ and can automatically generate things like Contents and that stuff.

  44. Jeffrey Elkner says: (permalink)
    October 5th, 2006 at 10:44 am

    I’ve been grappling with this problem for 7 years in the Open Book Project, and no adequate solution yet exists to my knowledge. It is encouraging to see a clear statement of the problem from one who may have the where-with-all to actually do something about it.

    Neil Blakey-Milner’s comments are right on the mark. I’ve settled on using the lore markup system from the Python Twisted framework for the following reasons:

    1. It is simple, easy to learn and easy to use.
    2. It generates nice looking XHTML, particularly for Python code examples.
    3. It autogenerates a nice index file (thanks to the work of George Paci at my prodding ;-)
    4. It offeres the promise of producing Doc-Book and/or LaTeX output (though the features George added for indexing do yet support this).

    No one would be happier than I would if a generally excepted and well supported solution to this very important problem where found. Sign me up to use it!

    jeff elkner
    open book project
    http://ibiblio.org/obp

  45. Reto Mueller says: (permalink)
    October 5th, 2006 at 11:02 am

    please combine these into one coment
    many thanks, reto

    everybody wants to do it, see the following links:
    http://www.hula-project.org/Wiki_Conversion
    and
    http://de.wikipedia.org/wiki/Wikipedia:PDF-Generator#Weblinks

    best implementation so far (my opinion)
    http://www.wikipublisher.org

  46. barkholt says: (permalink)
    October 5th, 2006 at 1:13 pm

    How about checking out LyX?

  47. Stefan Hinz says: (permalink)
    October 5th, 2006 at 1:59 pm

    The “general plan” Neil suggests for converting Wiki format to DocBook is kind of “DocBook XSL stylesheets reversed”. When generating HTML from DocBook XML (using Norman Walsh’s DocBook XSL stylesheets) XML elements are mapped to HTML classes (that can, in turn, be formatted using CSS). Here’s an example:

    DocBook XML: This is what I mean.
    HTML: This is what I mean.

    Since Wiki is just a simplified way of creating HTML, what you would need to create a variant of HTML that could be converted into DocBook is a set of markup instructions that create HTML classes as the one outline above (class=”emphasis”).

    However, that would leave the problem that DocBook XML is much more complex than HTML. In fact, even HTML is far more complex than the subset of HTML you’d typically want to create using a Wiki. Look, for example, at the toolbar for editing Wikipedia articles (http://en.wikipedia.org/w/index.php?title=DocBook&action=edit&section=4). It has 22 buttons, and advanced users may enter a few more markups “manually”. Compare this set of about 30 elements with DocBook (http://docbook.org/tdg/en/html/part2.html): The latter has 417 elements. Even if you’d decide to use just a subset you’d end up with maybe 100 elements. Converting these into Wiki formatting buttons would make the learning curve for Wiki editing much steeper.

    Having said this, there’s still the possibility of using a very reduced subset of DocBook that is targeted at printed books only (or at least mainly). The reason is that printed books use a very flat kind of markup: That “markup” uses serif font, monospaced font, indentations, and so forth. For example, in printed books you normally won’t see any difference between this-is-a-file, this-is-a-program, and this-is-a-function, to name three popular DocBook elements. So one could decide to use just one element for all of these: this-is-a-file, this-is-a-program, and this-is-a-function. That kind of markup would be close to the presentational markup used by both Wiki and HTML.

    At MySQL, we’re using DocBook XML at the base format for creating all kinds of output, from the obvious ones that you can generate using the DocBook XSL style sheets (HTML, PDF, CHM) to exotic formats like Unix man pages, the Info format, TexInfo (!), and — printed books! We’ve successfully created a number of MySQL Press books from DocBook, where our publishers can either feed what we deliver directly into Adobe FrameMaker (Germany) or into MS Word (USA). (Yes, they use MS Word for finalising the layout. ;-))

  48. Earl Norris says: (permalink)
    October 5th, 2006 at 2:41 pm

    I was thinking about something similar in 2002 when people write how to’s and managing them. My reason is that if you wrote a 50 page manual and a new version of the program came out you could just edit the manual here and there to update it. However the many users who downloaded the first version should just be able to download the sycn version and pressto you have the latest version of the doc. And in case you printed the first doc out go to print diff and it should only print the pages that have changed. Many times when following the manual you find things missing so you add them in and send it to the owner. the owner reviews it and desides to add it in and pressto your work it on the latest version.
    If you all the doc’s in one folder and each different book has it own folder and populated by .html sheets. So the update server would have an apache server with a MySQL DB. So when you run a update to the parent folder they check the relivent server. the server checks the version, versions gets html update and removes older html files.
    Doc’s should be written in pain text or open office starndard.

  49. Caroline Ford says: (permalink)
    October 5th, 2006 at 2:47 pm

    I think Wikibooks (from the wikimedia foundation, a sister project of wikipedia) does a lot of what you want and runs on mediawiki.

    http://en.wikibooks.org/wiki/Main_Page

    I think all the ubuntu wikis are moin moin which is less featured than mediawiki, but probably less resource hungry.

    There are mediawiki scripts to convert to xml, I am told that you can convert from xml ->docbook.

    http://svn.wikimedia.org/viewvc/mediawiki/trunk/wiki2xml/

  50. Alper says: (permalink)
    October 5th, 2006 at 3:19 pm

    I had a similar problem when I was writing a report in my personal wiki and my professor needed to see it in decent print format.

    Wikitext has a fair bit of markup semantics a lot of which in principle could be converted to something like LaTeX directly. I searched around for some time but most of the tools and projects around this process were either abandoned or very immature.

    I got the best results using wt2db to convert the wikitext into docbook. Xmllint to debug the errors from the resulting XML and then fop to create a PDF.

    The results were still not satisfactory and I abandoned the wiki format altogether but if you had the time and inclination this process could very well be refined. Each of the steps are open source and pretty straight forward and once the document is in XML you can transform it all you like.

  51. Dean Sas says: (permalink)
    October 5th, 2006 at 3:42 pm

    There’s a fedora sponsored project to convert moin moin markup to docbook (http://fedoraproject.org/wiki/MoinDocBookProject)

    I think Andreas Lloyd (lloydinho) used it for some of the contributing to ubuntu docs, I don’t know how well it went though.

  52. meneame.net says: (permalink)
    October 5th, 2006 at 3:43 pm

    Libros colaborativos en formato wiki…

    Mark Shuttleworth explica en su blog que se van a hacer libros escolares para todas las asignaturas, en Sudáfrica. La peculiaridad es que se tratará de libros colaborativos en formato wiki, y pregunta por un gestor de wikis que permita exportar a for…

  53. glo says: (permalink)
    October 5th, 2006 at 5:14 pm

    Dokuwiki! http://wiki.splitbrain.org/
    This gizmo has built in access controls.
    Access can be controlled for namespaces (segments of the wiki) with fine grained controll for read, write, view access by group, by user.

    Although it has been mentioned by others, have a look at http://www.wikimatrix.org … a good site for comparing wikis.

  54. Andy Lester says: (permalink)
    October 5th, 2006 at 6:01 pm

    Mark, I’m a developer working with Socialtext, and our Socialtext Open wiki is freely available from Sourceforge. I’d be glad to work with you to see if it’s what fits your needs.

  55. jcwinnie says: (permalink)
    October 5th, 2006 at 9:24 pm

    In a quick scan, I saw no mention of the Portland Pattern Repository, “Fie,” I say, “woe betide a Chief Wiki Officer ignorant of Cunningham & Cunningham.”

  56. Matthew East says: (permalink)
    October 6th, 2006 at 7:34 am

    Wiki formats tend to use inherently lossy formats, and docbook is incredibly rich and precise, so obviously a tool for moving between the two is never going to be perfect. However, as others have mentioned, the summer of code tools for MoinMoin are extremely good.

    I wrote about it in this blog post: Using a wiki in open source documentation projects.

    Another option would be to write your docbook using gobby.

    While you’re thinking about tools, it’s also worth asking yourself “why docbook”? Maybe you have already, I dunno.

    Matt

  57. Un libro su Ubuntu Edgy Eft, scritto in collaborazione! « Ubuntista says: (permalink)
    October 6th, 2006 at 9:47 am

    [...] Oggi Mark Shuttleworth anticipa la mia ricerca in questo suo post, e devo dire che tra i commenti relativi sono venute fuori diverse idee. La questione è: voglio inserire testo e immagini e link che siano ben impaginati in un wiki, ma anche trasformabili in un formato per la stampa. [...]

  58. Marc Portier says: (permalink)
    October 6th, 2006 at 12:15 pm

    Sounds like Daisy CMS http://daisycms.org might fit the glove quite naturally.

    The platform consists of a repository and a seperate front-end. Best is of course to take a look for yourselves, upfront I think this grab from the feature-list might be interesting to you:

    * seperate repository and front-end application, using an open REST based API in between
    (js and Java bindings to that API are available OOTB) this allows to programmatically interact with your content (not unlike you do with a normal database)
    (sample python code is there somewhere as well, and a gsoc student made a php client to the system)
    it also means you can easily upload content into the system from other sources (think databases, legacy repositories you want to replace, or even from parsing of code-with-annotations for generating API-references)

    * sql like query language
    it encorporates full text search and content specific functions alowing to search/sort on typical content aspects (including semantical fields, administrative metadata, links to other docs, …)
    being a central feature of the system, the results of these queries are used to automatically build navigation trees and (or parts thereof), document inclusions, access control, rss feeds and notifications
    (as such it allows to support different workflows of content management targetted to the organization)
    it also has a configurable facet-browser on top.

    * free document structuring:
    * inter-doc: storage structure is completely flat, multiple hierarchies (even quiery based ones) can be layed on top of it providing different structured views on that information
    this allows for maximum flexibility and optimal content repurposing (i.e. reuse in different contexts, being different websites or different book-publications)
    * doc itself: multiple document-types can be created, each with own sets of semantical tags. The standard front-end will automatically produce specific editors for each type.

    * special book-publishing features: (well I really think these set us appart from the rest out there)
    * a specific book-structure document-type that allows to specify the skeleton of your Table-Of-Contents in function of both directed documents in the repository or automated query results, it also allows to specify some common publication variables
    * in the editor we allow absolute image positioning, index-entries and cross references to other docs images and
    * during the publication process we triangulate the cross refs to actual page and image numbers, produce lists of images, of tables, an index at the back coordinated different headers/footers, and do proper heading-indentation depending on the level at which a certain doc was hooked up in the book-structure… etc etc
    * OOTB there is support for creating the book-publication in page-chuncked html, one single html and pdf. Own types (e.g. to docbook or even latex could be added.)

    * more
    * built-in rigid html cleaning of the input (which allows to ensure the wysiwyg editor produces decent reusable xhtml)
    * svg inclusion
    * clear and well document extension points (for both editor, book publication and repository)
    * backup/restore utility
    * import/export tool to transfer (selected) docs between multi-repository-setups

    Only min point I see is that it’s complete enough to take some time to grasp it all. There is however extensive documentation available here:
    More information to be found here: http://cocoondev.org/books/
    (those are in fact published books from our own documentation-wiki)

    Oh, and there is in current svn work in progress to build your own ubuntu-ready package.
    It will depend on java-5 and mysql (other dbs are possible, postgress has proven to work)

  59. Markus Z. says: (permalink)
    October 6th, 2006 at 1:59 pm

    Hello

    This thing is so important.
    And I find it very good that you by one addressed.
    Paper is somewhat simpler to distribute than an online medium.

    Each day I visit your Blog with tension around all information.

    Greeting Markus

  60. Marc Portier says: (permalink)
    October 6th, 2006 at 2:19 pm

    comments on comments from others:
    (with the danger of abusing this blog as a forum)

    @Neil // http://www.markshuttleworth.com/archives/59#comment-6925

    Yep: the trick is exactly how Daisy CMS extends the htmlarea to produce rigid xhtml that is semantically rich enough to be converted in a dedicated xml format later on.

    So actually there _is_ an existing solution that implements your plan already ;-)

    @somecanunchick // http://www.somecanuckchick.com/

    I agree that the required collaborative features call for something with the ease of use of a wiki. That’s why our default included front-end behaves like a wiki.

    The real challenge in Mark’s request however (IMHO) is being able to repurpose the typically scattered and cross-referenced wiki-entries in a consistent paper version with crisp layout and a lot of typical type-setting features (headers, footers, paragraph renumbering, index creation yadayada… ) It’s thus a bigger challenge then only having some entry-to-pdf: aggregation, restructuring and indexing, prividing the hooks for type-setting features… etc etc

    Also note that some of these issues are not entirely seperated from the editing environment: e.g. for introducing footnotes, image positioning hints (that could be different for paper versus online viewing) and index-entries you’ll be missing the hooks to enter those in the typical wiki app…

    @Rick // http://www.markshuttleworth.com/archives/59#comment-6940

    Traversing a predefined file-system organization of your documents towards a Table Of Contents has some limitations. If you want to produce a different publication based on the same content-repository you’ll need to create some alternative directory structure composed of smart symlinks for each of those.

    Note that some course material often gets reused between the 2h/week and the 5h/week syllabus on a certain topic.

    Also: you’ll typically want control of per section header/footer introduction per different publication. So in the end (having a desire for more or less aggressive content reuse) defining your publication becomes largely a job that is maybe 90% orthogonal to content creation and management.

    Note that content-reuse keeps paying back: not only at initial creation, but again and again at recuring revisions of content and publication.

    @ George // http://www.markshuttleworth.com/archives/59#comment-6998

    Where is the doc-team? I don’t understand how we staid under their radar :-)

    @ Earl Norris // http://www.markshuttleworth.com/archives/59#comment-7046

    The issue you refer to (with the various parallel releases) is typically refered to in content management context with the term ‘variants’. The goal is to have somewhat of an identity relation between different docs in the repository that allow those to be handled, controled, used, refered in similar ways where it is appropriate and in different ways where should be the case.

    DaisyCMS has support for variants that are composed of two dimensions: ‘language’ and ‘branch’. The latter can take any semantics you’ld want it to, the most use we’ve seen has exactly been ‘release’.
    You can see this at use in our online documentation: check the menu ‘variants’ on http://cocoondev.org/daisydocs-1_5/154.html

    @ Mark

    Some things I forgot to mention earlier that might be quite useful in the South African context:

    1/ (still 11 official languages down there?) is support for both i18n of the environment as well as the managed content, allowing to reuse what is common (like settings and usefull structure) between the languages and override/translate what is different. (see previous remark on variants)

    2/ A second gsoc student this year created a ‘detachment’ client that would allow editing content offline in any html editor to be uploaded/synced with the repo at a later time. This might be a usefull bonus over the common wiki approach in areas where continued and high bandwidth access is not a given.

    @all

    Rereading I see I haven’t been exactly typically flemishly modest :-(
    Euh, yeah, we’re proud of what we achieved so far and I honestly think Daisy CMS deserves your consideration in this.
    And I don’t think I’ve been overselling our existing solution here.

    So let’s still mak the obligatory disclaimer: while I think the threshold is reasonable, I’m quite sure that optimal usage of some of the (more cms) features mentioned here will require more thinking/planning then is the case with the typical wiki setup.

    Anyways while we hope to be already usefull, we’re not ready yet , there is (always) more work to do (2.0 is underway)
    Hope to meet some of you on our mailing-list… looking forward to reading more challenging use cases…

  61. Flavio says: (permalink)
    October 6th, 2006 at 4:04 pm

    Mark, just use DocBook (or Latex) along with Subversion (or CVS). That’s it. TobiasStrauß already said the same thing, but since so many people are coming up with fancy web-based solutions, I thought the concept should be reiterated.

    To everyone else: Remember, there’s life outside the browser!

  62. Mark Horner says: (permalink)
    October 6th, 2006 at 5:14 pm

    Hi Mark

    I just wanted to post a quick comment here that we’ve already started doing what you describe using a modified Drupal installation (see http://www.fhsst.org). We found that WikiBooks and other wikis were missing a lot of the necessary structure and permissions.

    Starting with the Drupal book module we’ve implemented workflow (using the workflow and actions modules) for sections allowing us to have sections available as assignments for volunteers, then editors and finally book coordinators all with different permissions For the first 3 books we actually use LaTeX so that we can render properly formatted printable books without any problems. We arrived at this solution after trying a CVS repository with .tex files or WikiBooks but both have serious shortcomings for a general community and for the development of a proper, focused document.

    We are currently editing the books and things are working reasonably well, as always we have a list of improvements to implement but then thats always the case in development.

    Cheers

    Mark

  63. Ben Francis says: (permalink)
    October 6th, 2006 at 5:30 pm

    There are definitely a lot of of people who want to do this, I wrote a proposal for the Google Summer of Code to implement it for Mediawiki (unfortunately it got turned down). I was hoping to be able to export my MediaWiki as DocBook and ODF and even support native wiki storage as DocBook.

    Wikis have proved hugely useful to me and I have installed them in organisations I have worked in. The problem is that once the data is in the wiki it is kind of stuck there, there needs to be a way to view wiki pages (or an entire wiki) in different formats using content negotiation. I think that wikis should become a part of a wider document management system where people can access and collaborate on the same documents using desktop applications and web applications via a web service API.

  64. Enrique Ocaña González says: (permalink)
    October 9th, 2006 at 1:05 pm

    Have you had a look to “parsewiki”? It’s a tool to parse texts in MediaWiki format and output them in html, xhtml, docbook o latex format.

    Maybe a plugin to integrate that tool into your current wiki system could be written.

  65. Markus Majer says: (permalink)
    October 9th, 2006 at 8:00 pm

    Hi Mark.

    The wiki you are using at wiki.ubuntu.com, the MoinMoin Wiki, also can do DocBook output – so why search anything other when things are already there? ;)

  66. Sanjoy Mahajan says: (permalink)
    October 10th, 2006 at 4:24 am

    For producing documents I like ConTeXt. It is a TeX variant suited for publishing books, especially physics and mathematics textbooks. You can specify page layouts, place figures in the margins, use PDF bookmarks, color, configure section headings, get automatic table of contents (and easy indexing), and it’s well integrated with my favorite diagram-drawing language (MetaPost). See http://contextgarden.net/

    I would hate to use anything but TeX or a variant for documents with equations, although masochists of my acquaintance use OpenOffice — but the results look ghastly.

    For revision control, I like mercurial (‘hg’), which is simple and fast and distributed. See http://www.selenic.com/mercurial

    Both projects are free software (GPL) and have friendly, active user and developer communities.

    A collaborative book-writing platform, still to be written!, could use hg for revision control of the ConTeXt source, with a ConTeXt middle-end that generates PDF for the browser to display.

  67. Luis Arias says: (permalink)
    October 10th, 2006 at 11:28 am

    Hi Mark,

    Sorry I did not see your post sooner. I think you will be interested in what we are doing with GELC – http://www.gelc.org. We are basically building an XWiki based platform (LGPL) that will provide a wiki approach to educational content publishing, mixing and distribution. It would be best for you to get in touch with Joshua Marks at GELC directly, current development plans may meet your timeframe and of course I think Joshua or Bobbi would be thrilled to discuss this with you. Please let me know if I can provide you with any assistance with this.

    Sincerely,
    Luis

  68. Sean Wheller says: (permalink)
    October 11th, 2006 at 4:45 pm

    The best concept of a wiki-like Docbook XML editing environment is demonstrated in docbook::collab [http://forge.joomla.org/sf/sfmain/do/viewProject/projects.docbook_collab].

    docbook::collab enables through-the-web editing of sdocbook (simplified docbook). docbook::collab is an extension to Joomla [http://www.joomla.org]. So it inherits much of the power of Joomla.

    Even at version 1.0.2 docbook::collab has perfromed very well with internal tests. We have made some customizations to the basic, but nothing worth putting back into the project. For example we do not use the BitFlux XML editor, we just use plain text views. We have appended the full docbook backend to our system. In future we want to place the content storage outside of MySQL and into SVN filesystem.

    Overall the benefit is that we do no round tripping between wiki and docbook. All target presentation formats are transformed using the traditional docbook toolchain. Authors get a reasonably easy way to author, of course they know something about docbook which helps.

    Looking forward. For collab to be able to service the needs described in your post it needs some development. However, the investment therein would produce a system that is far closer to the collaborative model you seek than that of wiki.

    Hope this helps.

  69. Joshua Marks says: (permalink)
    October 11th, 2006 at 5:11 pm

    Mark et al,

    Much of what I would have said is already captured in the posts on this very important thread. I will add that the format of the content (Docbook, enhanced Wiki Syntax, LaTeX, etc.) is a secondary issue when one is considering solutions for collaborative content development in long form. The requirements for collaborative book development are varied and often steep, including the need for role based access privileges, workflow and life cycles states, and the ability to create and manage educational/topical metadata. One can define the document structure in a number if ways and transform as needed at render time. In fact, this is why DocBook and its accessible variant NIMAS exist, to abstract structure from presentation style. However, enabling the rational and manageable editorial process necessary to realize a complete and coherent book is a different problem and one that lives more in the realm of content management platforms then a free-for-all wiki environment.

    The Global Education and Learning Community (GELC) has just started its “Curriki.org” open source development project to build a collaborative textbook creation tool set on top of the xWiki platform. The xWiki platform already supports page level role-based access privileges as well as PDF and XML export of pages or entire page structures. It is also one of the few wikis that supports JSR-168 and will eventually support a JSR-170 (Java content repository) back-end. We are working with a number of open curriculum and textbook projects to understand and support their requirements. We are also leveraging XWiki’s built-in scripting tools, Velocity and Groovy to enable automated assembly and rendering of the content into both printable forms (PDF and potentially DocBook as well), and eventually as IMS content packages (Hopefully supporting the Common Cartridge spec for real LMS interoperability.) So in short, you are asking for what we are trying to build. We would love to find ways to work together.

    Best,

    Joshua Marks
    CTO, GELC

  70. Michael Vorburger says: (permalink)
    October 16th, 2006 at 10:51 am

    I am sure you have seen that OLPC is working on something in this area… have a look at
    http://dev.laptop.org/git.do?p=users/krstic/docformat;a=blob;h=d191a5a9f8beb6a052b5e8d30f7a4da99f67a07a;hb=a811c70ced7a02adfc35f66453c7548f8095af6f;f=document-format.txt (http://dev.laptop.org/git.do?p=users/krstic/docformat;a=summary)

  71. Tonie Putter says: (permalink)
    October 18th, 2006 at 9:56 am

    An inquiry into the ‘limits of current wiki technology’ is a good idea. During the period 1997-99 my conclusion was that the headlong rush into wikis was too much of an ‘either-or’ strategy: either a RDBMS or a flat wiki. So I dediced to invent a hybrid, what I now call a ‘structured wiki’ which combines and integrates the power of both approaches.

    After all, if we think of a wiki as ‘just a way of communicating’ we can write a few lines of code that will make any RDBMS wiki-capable. Its a bagatelle to make a procedure ‘smart’ enough to process text in a database field that has been marked up in wiki formats such as [[text]flag / operand].

    What wikis won’t do are all the things that RDBMS and OOP approaches do so very well. And, in my hybrid approach, it is possible to combine two kinds of scaling strategies, the usual factorial scaling that is a troublesome and costly attribute of RDMS factorial sets of ‘treatments’ on the one hand, and the wonderful fractal nature of scaling that is possible in a wiki that has been set up in a smart way on the other.

    As far as I can tell, my inquiry into, and combination of factorial- and fractal-scaling (‘relationality’ & ‘relations’, respectively), has not been done anywhere, but that is not my main interest or concern.

    In any case, our structured wiki has been up and working since January 2000 (wikipedia went public in April 2000). The square bracket syntax I invented in 1996, while I was working for the United nations, was placed in teh public domain in 1998. Inter alia, EcoPort can do all that you seem to require of a wiki textbook platform for the Thutong/DoE enterprise in South Africa. I have made my case in a wiki/hypertext email message that I sent to Claire Davis, asking her to forward it to you.

    I would like an opportunity to see a list of functional specifications that you deem essential and desirable for the WikiTextBook task. I say this, because I am totally confident that our structured wiki called EcoPort, would probably be able to meet most, if not all your needs and be a seamless driver behind the seams within Thutong. (To us, wiki capabiloity is simply an attribute of any information object / knowledge resource we send out / create. Much like the colour of a garment is an intrinsic attribute of the garment. In so many ways, wikis are not a ‘big deal’….)

    Sincerely,
    Tonie Putter
    CEO EcoPort Foundation under the patronage of Nelson Mandela and Edward O. Wilson.

  72. My personal Blog says: (permalink)
    October 18th, 2006 at 3:55 pm

    [...] Mark Shuttleworth » Blog Archive » Writing a book collaboratively?: Mark Shuttleworth | here be dragons « Ubuntu road warrior tips Writing a book collaboratively? Someone joked about giving Jane the title of “Chief Wiki Officer”. We have about 7 wiki’s internally and there’s no end in sight! I think we could consolidate them and use access controls to keep various communities distinct but – for those of us who have to roam across them – integrated. This got me thinking about the limits of current wiki technology. Is there any comprehensive list of wiki platforms? At the same time at the Foundation in Cape Town we are busy gearing up to commit about R30m (about $4.5m in today’s terms) to create a complete set of school books for all subjects in the South African curriculum in wiki format. I got to wondering which format we should use. There’s a lot of action around MediaWiki (used by Wikipedia) but one of our goals is to be able to print low cost books – real books with tables of contents etc on real paper – for those schools which don’t have bandwidth or PC’s. Can MediaWiki produce DocBook output? Or should we be using a different collaborative platform altogether? I remember hearing about a DocBook wiki system, but can’t find any references to that now. So here’s the question – if you wanted to write a book collaboratively, with rigorous access control and revision control but also with chapters and sections, and an index and tables of contents etc – what platform would you use (free software greatly preferred, and open format’s required, of course :-) ). [...]

  73. Danilo Segan says: (permalink)
    October 25th, 2006 at 11:43 am

    Ah, the reappearing topic. My 2005 Google SoC project, unfortunately still unfinished, presents framework for WYSIWYG live documentation editor with all revision control and interactivity features you need. http://live.gnome.org/LiveDocumentationEditing and a slow demo on http://kvota.net/sarma/.

    I really hope to be able to finish it up, but just check the demo for the wysiwyg docbook editting sweetness :)

  74. Alan Levin says: (permalink)
    October 25th, 2006 at 12:25 pm

    Having used a few of the wiki softwares, I vote for Mediawiki as the project to enahnce for this purpose. I also suggest that you’d wan’t to start those enhancements/plugins (as Neil – nbm – hinted to above) before you get too far with the book writing.

  75. Johannes Henn says: (permalink)
    November 2nd, 2006 at 1:53 pm

    Mark

    Your concept of creating a complete set of subject handbooks in electronic form for South African Schools is interesting. I say interesting, as I believe you are trying to address the severe shortage of text books in South African schools. I used to be a Science and Maths teacher, so I am quite aware of the poor quality of books available at that time. I don’t think anything has changed. To put together a decent curriculum, one had to use about 5 – 10 text books to get enough content to be able to address and teach the outcomes. And most of the time additional resources had to provide the really usable content.

    I remember getting anything from 10 – 20 “new” text books to review for possible purchase by the school after the new curriculum was introduced. To say the least, I seldom even considered using a text book received in such a manner for my own planning. When I left teaching at the end of 2003, very few text books were utilized in the primary school where I taught. All material was prepared and concised by the teachers.

    Which brings me to your collaborative digital text book effort. You want to spend a lot of money on an effort that might not give you the desired result. I am not shooting the idea down, someone has to start somewhere with addressing the mess regarding text books in South Africa. It’s the demographics of South Africa that will be your biggest obstacle. A Science text book for example, cannot be universal for the whole country. I know this as I recall the Science text books I received for evaluation. Some were completely unsuitable for children living in cities and vice versa. I am sure that it would probably suffice to say that every learning area will have the same problem. You could end up with a number of text books for every learning area for many “regions” in South Africa.

    Personally I believe there is a better solution regarding the provision of text books electronically. I would love to tell you what it is I refer to, but I will be shooting myself in the foot! I have been fortunate to study during my teaching years and acquire a degree in Computer Science. I am using my years of experience in teaching now in the development of a revolutionary system that will address the very problem of regioning in South Africa with regards to the provision of suitable teaching material. And more! It will not cost R30 million… But then maybe you shouldn’t display this comment either!

  76. John Rankin says: (permalink)
    November 14th, 2006 at 3:50 am

    The wikipublisher server is designed for just this purpose. It takes “wikibook xml” and transforms it into LaTeX for typesetting. This gives all the book-type features one might need: table of contents, chapters, list of figures, list of tables, etc, etc. To see an example, go to

    http://www.wikipublisher.org/wiki/index.php?n=Wikipublisher.UserGuide

    and press “Typeset book”.

    Currently, there is a plug-in for the PmWiki engine. This teaches the wiki to output wikibook xml instead of xhtml. It includes options for mathematical equations, replacing low resolution screen images with high resolution print images, and other print-friendly features. The typesetting server can work with any wiki able to deliver wikibook xml. The effort required to create a plug-in for a particular wiki engine depends on a number of factors, of course.

    For more information, you can contact me from the site.

  77. Ev says: (permalink)
    November 15th, 2006 at 6:13 pm

    How did you finally resolve the problem Mark?

  78. Rudd-O says: (permalink)
    November 23rd, 2006 at 9:39 am

    Mark,

    I wrote a book using MediaWiki, for my thesis. I developed software that consolidates MediaWiki pages into a single HTML (Boom! book microformat) page with the images inlined in the book. I found it a breeze to work with MediaWiki and then consolidate the book at any time when anyone wanted a “snapshot” of the work in progress, and it was also easy to generate the final printed output.

    http://software-libre.rudd-o.com is the address. MediaBook is the software. Feel free to contact me or have anyone in your staff contact me if you want help on how to use it.

  79. Riccardo Murri says: (permalink)
    January 16th, 2007 at 2:31 pm

    I think that the main issue in writing a book collaboratively through
    a Wiki is how to build an open and active community of contributors.
    Building a community of contributors entails lowering the threshold to
    editing, so that people aren’t scared off by technical and
    typographical details, and can focus on the *content* they add.
    Failing to build that, there’s no advantage over the usual editor+CVS
    model of collaborative writing.

    I was able to find very little on the net about this human-interface
    side of “writing a book collaboratively”, and very little support from
    current software. (I’d be pleased to hear about any thought on the
    topic.) It might be that this is a very new topic, and solutions are
    only beginning to be drafted.

    I have been setting up a Wiki for collaborative writing and editing of
    a tutorial book on Grid Computing ( http://www.egrid.it/doc/gme ); we
    use it as an online reference resource, and print copies to hand out
    to participants at training events. We also would like participants
    at training events to edit the content to clarify points that were
    written in initiated jargon, share their view, add their own examples
    and experiences — in short, have the text reflect a community
    experience.

    Here comes the problem. Just putting the text into a Wiki isn’t
    sufficient:

    1- Every contributor has to learn the markup language used by the
    Wiki system.[1]

    2- More important, every contributor has to agree to some stylistic
    and typographical conventions (how to format computer output and
    examples, usage of acronyms, etc.), or you end up with
    inconsistently-formatted text.

    It is *very important* to keep the effort needed as light as possible,
    or you loose contributors, that is, active community members. (In our
    case, it’s scientific researchers that perceive the time spent in
    learning the details of some computer program as time wasted in not
    doing useful research — even if it could save them some later on…)

    I believe that item 2 above is the most important one: coherent
    typographical rendering is *very* important to novice readers, who
    often cannot tell apart the different elements from context alone.[2]
    Coherent typographical conventions convey “action” clues (e.g.,
    monospaced text is for “type on keyboard exactly as written in text”)
    but they *need to stay the same throughout the text*.

    In the end, here’s a few technical details on how we are trying to do
    it:

    – We use reStructuredText for Wiki markup; most reST markup is just
    formatting a plain text document for better reading. It helps to
    keep the page sources readable even after many edits, and could
    reduce the learning time and effort required by prospective
    contributors.

    – We require authors to comply with some (hopefully simple and
    natural) text formatting conventions. Of course, the conventions
    are stated on a Wiki page, so they can be discussed and changed -
    establishing the conventions is part of book writing itself!

    – From time to time, we dump the page *sources*, assemble them, and
    produce a PDF booklet (reST -> LaTeX -> PDF), that we can print
    and hand out.

    .. footnotes:

    .. [1] WYSIWYG editors might lighten the effort required here,
    although I think that most WYSIWYG editors are harder to use than
    well-designed lightweight text markup, for anything other than
    casual editing of short text.

    .. [2] Think of GNU/Linux novices learning bash command-line usage:
    if the book lists an example line reading “bash$ grep ‘myregex’
    myfile.txt”, will they know that they should not type the “bash$”
    part if it’s not written in a different font or color? will all of
    them understand that they should substitute their own values for
    “myregex” and “myfile.txt” without any visual clue?

  80. Holger Rauch says: (permalink)
    January 22nd, 2007 at 12:07 pm

    Does anybody know of a Wiki supporting the syntax used by Asciidoc (http://www.methods.co.nz/asciidoc/)? If a Wiki does not support the Asciidoc syntax directly, is there some way to make it “understand” (parse and render) Asciidoc markup?

    Thanks in advance for any hints/pointers!

    Kind regards,

    Holger

  81. The Net Duck says: (permalink)
    April 10th, 2007 at 10:05 pm

    hi,
    I was interested in the post because a while back I had tried to start a little website that people could write books together and got some activity, but wasn’t able to keep it running. I think it’s a great Idea especially for education. The site was bookmaster.proboards7.com and was fun.
    So not to seem like I am trying to spam, I think that the best platform would be python because it is like java (just better) because it’s multi platform based.I guess that kind of seems obvious.

    The Net Duck

  82. Marbux says: (permalink)
    August 19th, 2007 at 9:26 pm

    Is there a particular reason for selecting Docbook as the output format? I would suspect that OpenDocument ODT format would be preferred, since Edubuntu ships with OpenOffice.org.

    I recommend that you seriously investigate two apps designed for the purpose of creating books. One is SiSU, http://www.jus.uio.no/sisu/ (;) the other has already been mentioned, the Daisy platform. http://cocoondev.org/daisy/index.html (.)

    SiSU outputs, among many formats, ODT, LaTeX, PDF, HTML, and structured XML. It is a mature publishing framework for creating lengthy articles and books, using a wiki-like markup in plain ASCII files as input, with the same imput file outputting to multiple formats. Features for academic writing are particularly strong, given SiSU’s origin as a publishing tool for legal texts. However, I’m unsure of its support for mathematical symbols, having never used them in SiSU, so it might be necessary to add them in LaTeX. The SiSU markup is far more mnemonic than most wiki syntaxes, and for me was easier to learn. (I prefer writing in XHTML markup myself.)

    SiSU has outstanding support for numbering of all supported object types, e.g., chapter, page, paragraph, line, footnotes. The SiSU site has numerous sample books. It will also automagically generate a concordance for indexing purposes.

    Downsides: No WYSIWYG editor, lacks strong management features for documents, versions, and variants. Has Linux/Unix dependencies.

    I won’t add much to what’s already been said about Daisy. Daisy does have a WYSIWYG editor and very strong management features for documents, versions, and variants. It also has strong transclusion capabilities, providing a single editing point for recurring content, which IMHO is a critical feature when producing documents with overlapping content.

    Supports ODT and several other formats out of the box, but apparently not Docbook. But with the underlying Cocoon componentry framework, adding additional output formats should be fairly trivial. See generally http://cocoon.apache.org/2.1/features.html (.) And with Cocoon as the framework, it should also be feasible in Daisy to use a WebDAV client such as OOo as the “live” input editor.

    Creating cross-referencing links in Daisy is particularly easy, with a search dialog built into the link forming system.

    Between the two systems, I left SiSU for Daisy because of users’ strong preference for working with a WYSIWYG editor, Daisy’s far stronger document management system, and stronger collaboration features. But that came at the cost of the far more robust document formatting capabilities in SiSU.

  83. 暴走外人CEO奮闘日記 » Blog Archive » Wikiで本・書籍・マニュアルを書く方法 says: (permalink)
    October 20th, 2007 at 12:45 am

    [...]  まず調べたら、海外ではこのブログ(http://www.markshuttleworth.com/archives/59)で過去に議論がある。というか、ブログに書き込んだだけで80人ぐらいコメントあるっていうのはすげーな!こいつなにもの? [...]

  84. Lucas says: (permalink)
    March 17th, 2008 at 8:47 pm

    Debian uses moinmoin to convert moinmoin pages to docbook to pdf/html/ other…
    check it out:

    http://wiki.debian.org/DebianReference

  85. Un libro su Ubuntu Edgy Eft, scritto in collaborazione! says: (permalink)
    November 22nd, 2008 at 5:44 pm

    [...] Mark Shuttleworth anticipa la mia ricerca in questo suo post, e devo dire che tra i commenti relativi sono venute fuori diverse idee. La questione è: voglio [...]

  86. Tools for collaborative authoring and editing: wikis vs. Wordpress vs. SharePoint says: (permalink)
    May 19th, 2011 at 11:29 am

    [...] Mark Shuttleworth blog archives: Writing a book collaboratively? [...]