Building clouds for fun and profit

Monday, September 19th, 2011

So you’d like to spin up an internal cloud for hadoop or general development, shifting workloads from AWS to your own infrastructure or prototyping some new cloud services?

Call Canonical’s cloud infrastructure design and consulting team.

There are a couple of scenarios that we’re focused on at the moment, where we can offer standardised engagements:

  • Telco’s building out cloud infrastructures for public cloud services. These are aiming for specific markets based on geography or network topology – they have existing customers and existing networks and a competitive advantage in handling outsourced infrastructure for companies that are well connected to them, as well as a jurisdictional advantage over the global public cloud providers.
  • Cloud infrastructure prototypes at a division or department level. These are mostly folk who want the elasticity and dynamic provisioning of AWS in a private environment, often to work on products that will go public on Rackspace or AWS in due course, or to demonstrate and evaluate the benefits of this sort of architecture internally.
  • Cloud-style legacy deployments. These are folk building out HPC-type clusters running dedicated workloads that are horizontally scaled but not elastic. Big Hadoop deployments, or Condor deployments, fall into this category.

Cloud has become something of a unifying theme in many of our enterprise and server-oriented conversations in the past six months. While not everyone is necessarily ready to shift their workloads to a dynamic substrate like Ubuntu Cloud Infrastructure (powered by OpenStack) it seems that most large-scale IT deployments are embracing cloud-style design and service architectures, even when they are deploying on the metal. So we’ve put some work into tools which can be used in both cloud and large-scale-metal environments, for provisioning and coordination.

With 12.04 LTS on the horizon, OpenStack exploding into the wider consciousness of cloud-savvy admins, and projects like Ceph and CloudFoundry growing in stature and capability, it’s proving to be a very dynamic time for IT managers and architects. Much as the early days of the web presented a great deal of hype and complexity and options, only to settle down into a few key standard practices and platforms, cloud infrastructure today presents a wealth of options and a paucity of clarity; from NoSQL choices, through IAAS choices, through PAAS choices. Over the next couple of months I’ll outline how we think the cloud stack will shape up. Our goal is to make that “clean, crisp, obvious” deployment Just Work, bringing simplicity to the cloud much as we strive to bring it on the desktop.

For the moment, though, it’s necessary to roll up sleeves and get hands a little dirty, so the team I mentioned previously has been busy bringing some distilled wisdom to customers embarking on their cloud adventures in a hurry. Most of these engagements started out as custom consulting and contract efforts, but there are now sufficient patterns that the team has identified a set of common practices and templates that help to accelerate the build-out for those typical scenarios, and packaged those up as a range of standard cloud building offerings.

 

18 comments:

  1. foo says: (permalink)
    September 19th, 2011 at 3:28 pm

    Mark,

    Have you tested Windows 8 yet?

    As a Linux and Mac user, I must say that I’m impressed. (I’m not sure if it will become a hit or a flop, since people don’t like change — but the Metro interface has some really nice ideas in it.)

    There is one particular feature that I think could be adopted by Ubuntu:

    Windows 8 seems to have abolished the status bar! Do you want to see the clock, or the battery status? Have some live tiles showing the time, battery, wi-fi, etc.

    Microsoft nailed it!

    Linux and Mac have had widgets for ages; but they are always hidden behind the usual interface. Microsoft made it central, and removed distractions: you must use them, and you’ll get used to them. In fact, I already like them!

    What do you think about Windows 8 and Metro? Do you like it? Do you see some obvious problems? Are there any ideas that could be adopted?

    Cheers,

  2. roman says: (permalink)
    September 19th, 2011 at 5:17 pm

    happy birthday Mark) You are doing the great deal!)

  3. Benjamin Kerensa says: (permalink)
    September 20th, 2011 at 1:45 am

    Glad that Canonical is sponsoring PuppetConf which is all about Cloud…. Ubuntu Oregon will have a booth and two of our LoCo members will have an opportunity to learn more about the cloud which they can take back to the community and share even more.

  4. Jef Spaleta says: (permalink)
    September 20th, 2011 at 4:37 pm

    Hey Benjamin,

    Is Ubuntu Oregon’s Loco team prepared to field questions as to why Canonical is not integrating puppet based technologies in its cloud deployment story? And instead is building ensemble…err I mean juju… as an in-house Canonical built product and making it a core part of the Ubuntu cloud services stack to displace the puppet usage? Because I would imagine there are going to be a lot of people at _PuppetConf_ who would be very interested in answers to questions along those lines.

    -jef

  5. 2 cents worth says: (permalink)
    September 20th, 2011 at 8:06 pm

    Happy (belated) birthday

  6. mark says: (permalink)
    September 20th, 2011 at 8:49 pm

    Hi jspaleta@fedoraproject.org

    Juju and puppet do different things, so it’s perfectly reasonable for to support PuppetConf and also support juju. Juju charms can use puppet for configuration management, for example. Juju is about service orchestration. Puppet is about configuration management; they work at different levels in the stack. If you haven’t tried Juju, give it a whirl, it makes defining cloud deployments a lot of fun.

    Mark

  7. Jef Spaleta says: (permalink)
    September 20th, 2011 at 9:01 pm

    Mark,
    I’d love to try juju! But I can’t seem to find it available for Debian. Can you point me to the juju for Debian stable packages? I’d very much appreciate the pointer. In fact it would be great if Canonical would show some leadership and have their employees target Debian as a primary package submission for all Canonical upstream projects and then synced into Ubuntu from Debian, as certain individuals are currently proposing external Ubuntu contributors do as a best practice.

    -jef

  8. Jef Spaleta says: (permalink)
    September 20th, 2011 at 9:23 pm

    Mark,
    And your absolute correct. juju doesn’t compete with puppet. Sorry for not being clearer. Let me rephrase.

    Juju competes with mcollective. And at PuppetConf there is a track called Puppet 201 which is going to spend some time talking about to use mcollective with a puppet data source to do orchestration. Its unforunate that track is named Puppet 201 even though its talking about orchestration. No way on earth is that particular choice of track naming going to confuse attendees about exxactly where puppet lives in the orchestration stack…nope.

    Is Juju going to layer over puppet like mcollective does? So far I haven’t read over any “charm” logic which suggest that its envisioned to do that. In fact I thought part of the bullet point salespitch was juju’s streamlined way of doing away with the multiple pluggable data provider model that mcollective uses to layer over puppet or chef.

    Regardless of my unfortunate incorrect view of what role juju plays and how it layers with other tools, I definitely expect people at PuppetConf to have questions about it and considering its being stood up as an Ubuntu integrated/differentiated feature compared to all other linux based OSes, I fully expect that Oregon Loco team is going to be fielding questions about it as they’ll be the most obvious people to ask. Hopefully Canonical will have someone standing by at the Loco booth who will be prepared to provide factually correct information. Because it would be very unfortunate indeed if well-meaning individuals provided factually incorrect answers about Canonical’s juju at the live event where mcollective layer over puppet based orchestration was being demo’d as a featured track. Very unfortunate indeed. Don’t worry, even though Alaska Air is a sponsor, I don’t have a plane ticket…yet.

    -jef

  9. mark says: (permalink)
    September 20th, 2011 at 9:40 pm

    @Jef

    What are you waiting for, get packaging!

    Mark

  10. Jef Spaleta says: (permalink)
    September 22nd, 2011 at 6:54 pm

    Hi Jef,

    Ubuntu Oregon LoCo is here at PuppetConf is you have any questions about puppet and Ubuntu we would be happy to discuss it with you although decisions like the one you presented are not something that Ubuntu Oregon is involved in making. Ubuntu Users have varying preferences for their Ubuntu Servers and Desktop and when it comes to customizing Ubuntu to meet your needs the options are nearly unlimited. Notably the PuppetLabs people are very supportive of local community to include Ubuntu Oregon and other FOSS UG’s.

  11. Andrew says: (permalink)
    September 23rd, 2011 at 5:00 am

    Hi

    Probably something you have thought of and or done already, but.. with the new cables we should start seeing substantial price decreases in costs of internet usage, from a users viewpoint there have been some rumblings but no real WOW offers. Someone has to blink to get the local suupliers out of their comfort zones and start getting some real savings across to the consumers, this is something you could immediately impact possibly by simply rolling out a ‘plan’ which would cause the market to try to pre-empt you. Regards and thanks for making a difference!

  12. Vikram Buzz says: (permalink)
    September 24th, 2011 at 4:57 am

    Here is an interesting quiz on Oneiric Ocelot. Crack it!

    Ubuntu 11.10 Oneiric Ocelot Quiz

    fyi.

  13. Jose says: (permalink)
    September 24th, 2011 at 8:20 pm

    @foo: Ubuntu is not free to make a Windows 8 equivalent. You need hardware acceleration for doing what Microsoft and Apple does, and those companies have good 3d acceleration support linux has not(Microsoft because games depend on windows and Apple because they create their hardware with good 3d capabilities always).

    Meanwhile, nouveau is slowly getting gpu fan speed control now, and it will take a while for them to support OpenCL, video decoding and other essential stuff people has got used to. By the time it works, Apple and Microsoft would have improved their Oses, so it seems like eternal catch up.

    Microsoft is also winning on the DirectX front with the main 3d CAD software and all games supporting it as it makes them money, first because windows is where users are and second because DirectX improves and is simplified fast, making cheaper to support it that the obsolete mammoth OpenGL is (designed by committee and no party willing to give up obsolete functions for API cleanup and simplification). DirectX asserts quality too with makes companies tremendous QA savings.

    Few companies could make software for Ubuntu as the almost infinite configurations will make support cost to skyrocket. Android is getting into phones and tablets because it works.

    Ubuntu is too big to be on phones, but tablets are a very good space for it if we could find a way to change the UI for controlling standard menus and buttons with big capacitive fingers.

    Nobody had solved the problem yet,(although MS is trying with ribbon menus and “intelligent focus”).

    This is something that we could do better and innovate instead of just copying what works from Apple again.

  14. Benjamin Kerensa says: (permalink)
    September 25th, 2011 at 4:57 am

    @Jef: I had commented in response to your comment but Mark’s nice Akismet filtering decided to moderate that comment and for some reason it was never put through. Basically, I stated that such decisions are up to Canonical and LoCo’s do not really play a role in decision making when it comes to Ubuntu.

    Ideally, I would like to see Ubuntu Members play more of a role in having input without a requirement of being at UDS but that is likely wishful thinking. I do encourage you to continue to reach out to Canonical or ask questions wherever most appropriate for a proper response and thanks for your continued advocacy of FOSS.

  15. Barbie says: (permalink)
    September 28th, 2011 at 6:10 pm

    #occupywallstreet Mark Shuttleworth is a 1%. Canonical is in it for the profit! What more do you need to know! He’s using the 99% ‘for fun’ volunteers all to his own end. Start your own truely ‘free’ OS projects! #occupywallstreet

  16. now don't cha know you sure won't be no fool says: (permalink)
    September 30th, 2011 at 11:15 pm

    @Barbie

    Geez, I’m not even sure which cause you’ve diminished more :(

    Pro tip: Whilst trolling, it’s best to depart from your usual username.
    P.S. Ya want more?, take it to /r/linux.

  17. 2 cents worth says: (permalink)
    October 4th, 2011 at 5:34 pm

    Oh sour grapes!

  18. 2 cents worth says: (permalink)
    October 5th, 2011 at 11:51 am

    Well, as Jonathan Carter once said,”…Billionaire…who knew…”