Cloudy prognosis for mainframes

Monday, October 24th, 2011

The death of the mainframe is about as elusive as the year of the Linux desktop. But cloud computing might finally present a terminal opportunity, so to speak, to those stalwarts of big business computing, by providing a compelling answer to the twin stories of reliability and throughput that have always been highlights of the big iron pitch.

Advocates of big iron talk about reliability. But with public clouds, we’re learning how to build services that achieve very high levels of reliability despite having low individual node reliability. It doesn’t matter if a single node in the cloud fails – cloud-style architectures route around that damage and keep the overall service available. Just as we dial storage reliability up or down by designing RAID arrays for the right balance of performance and resilience to failure, you can dial service reliability up or down in the cloud by allowing for redundancy. That comes at a price, of course, but the price of an extra 9 is substantially lower when you tackle it cloud-style than when you try and achieve it on a single piece of hardware.

The other big strength of big iron was always throughput. Customers will pay for it, so mainframe vendors were always happy to oblige them. But again, it’s hard to beat the throughput of a Hadoop cluster, and even harder to scale the throughput of a mainframe as cost-effectively as one can scale a private cloud infrastructure underneath Hadoop.

I’m not suggesting insurance companies will throw away their mainframes. They’re working, they’re paid for, so they’ll stick around. But the rapid adoption of cloud-based architectures is going to make it very difficult to consolidate future IT onto mainframes (something that happened in every prior generation) and is also going to reduce the incentive for doing so in the first place. After 20 years of imminent irrelevance, there’s finally a real reason to think their time is up.

31 comments:

  1. Andrew Ampers Taylor says: (permalink)
    October 24th, 2011 at 8:38 am

    Good! I’ll have a lot more room in my bedroom when I get rid of my mainframe!

    Seriously though, I remember when I first came to England. The Chairman of IBM, when giving a talk (1955) said that the world would never need more than five computers. He was slightly out.

    I think mainframes will continue to be sold for the foreseeable future and the reason doesn’t knock your prognosis, I agree, there is no real need for mainframes now. However, mindsets have to change first. In addition to this, executives in senior positions in mainframe departments of corporates will fight for all their worth to keep the status quo.

  2. IGnatius T Foobar says: (permalink)
    October 24th, 2011 at 11:31 am

    You’re still missing the point. Canonical is alienating existing Ubuntu users by forcing Unity down our throats. We want out computers to look and act like computers, not like overgrown smartphones.

    Until you fix this big big big problem, any “cloud” stuff you do is tantamount to rearranging the deck chairs on the Titanic.

  3. Chauncellor says: (permalink)
    October 24th, 2011 at 12:09 pm

    @IGnatius:

    sudo apt-get install kubuntu-desktop gnome-shell xubuntu-desktop lubuntu-desktop fluxbox icewm.

  4. IGnatius T Foobar says: (permalink)
    October 24th, 2011 at 1:32 pm

    @Chauncellor yes I’m aware of all of that, and xubuntu-desktop is my current choice, but the purpose of this exercise is to exert pressure to downplay Unity’s place in the current Ubuntu experience. Until the problem of alienating the existing user base is corrected, new efforts such as cloud computing are misdirected.

  5. Andrew Ampers Taylor says: (permalink)
    October 24th, 2011 at 1:57 pm

    @IGnatiusTFoobar

    I think someone who made £580,000,000, in the computer software business, by the time they reached 24, may have a better idea on Unity than yourself, Sir.

    If I shared your views on Unity I would, personally, move to another Distro as I would know, instinctively, that trying to persuade the Ubuntu Council to move back to the original desktop, would be a total waste of my time, effort and energies and that there are more important things to concern me with.

    But it is your life and if you want to waste it fighting Unity, then – of course – this is entirely up to you. But do think seriously on this as I hate to see people so consumed waste their time and energies.

  6. Jef Spaleta says: (permalink)
    October 24th, 2011 at 4:36 pm

    Mark,

    Would you care to respond to this press article?

    http://www.computerworlduk.com/news/infrastructure/3303178/cloud-popularity-and-mobile-usage-driving-increase-in-mainframe-capacity/

    The argument being put forward there, and supported by survey data, is that private cloud architecture will increase mainframe infrastructure investment in direct contradiction to what you seem to be saying.

    There’s nothing incompatible with the concept of selling mainframes to be the hardware over which the cloud infrastructure model runs. And IBM seems more than happy to sell you mainframes for cloud workloads:
    http://www-03.ibm.com/systems/z/solutions/cloud/index.html

    -jef

  7. Fish says: (permalink)
    October 24th, 2011 at 4:43 pm

    Replace main-frames with cloud computing?

    One word: Gmail

    Second word: Amazon

  8. anomie says: (permalink)
    October 24th, 2011 at 5:09 pm

    Cloud is a candidate to replace mainframes on the basis of availability and throughput. OK. What of secure data storage?

  9. lee says: (permalink)
    October 24th, 2011 at 5:21 pm

    we only load the LTS versions (thus have not tried unity yet (will wait for reviews of 12.04)). however we already notice a number of issues we probably won’t like very much. first up is the cost of big touchscreen monitors (and the likelihood of daily cleaning). secondly is… just what advantage do they really have (most of us can click a mouse button and select a menu item just as fast (if not faster) than using a touchscreen)?

    so other than owning the latest expensive “toy” why would anyone leave convention?

  10. Nick says: (permalink)
    October 24th, 2011 at 6:18 pm

    Right. Because corporations are so well known for getting rid of antiquated technologies, and wholesale rewriting their internal systems.

  11. YetAnotherBob says: (permalink)
    October 24th, 2011 at 6:26 pm

    Yet another “Mainframe is dead” article based on the “cloud”.

    The Mainframe has “died” at least 50 times in the last 50 years, yet there are more of them than ever right now. What will these “Cloud” systems run on? It will take more processing power than my little obsolete Android phone. Oh, that’s right, it will be on someone elses mainframe.

    Get real, the “Cloud” is just a network connection to another computer on the network. There will still be a powerful computer on the other end, whether in the next room, or the next continent. True, the power of the mainframes of 50 years ago now resides in my shirt pocket, but, the amount of power I want has increased vastly too. No, mainframes aren’t going away. They are just changing forms, and getting easier to connect to.

    Still, it remains true that if you want the data that defines and continues your business, you will want to have physical control over the storage and distribution of it, and that means physical control over the machine that runs the processes. In short, a Mainframe.

    Less sensitive information may be more cheaply run on someone elses machine, but those who believe that this can run ALL their business will have an expensive lesson to learn.

    The old adage is still true. Good, Cheap, Fast, pick any two.

  12. mainframe truth says: (permalink)
    October 24th, 2011 at 6:32 pm

    While ubuntu is a compelling user desktop focused linux environment, Mr. Shuttleworth displays his complete lack of knowledge and understanding of mainframe features and benefits- especially his casual dismissal of the internal reliability of mainframe equipment, as well as the raw transaction performance in a single complex. When it comes to computing focused on business transactions and finance, mainframes have inherent advantages when you have a sufficiently high enough volume of work to justify them. Were IBM to adopt more reasonable pricing approaches, mainframes would be even more compelling to a broader range of businesses. Even so, if Mr. Shuttleworth knew what he was talking about, he would explain how he compared a Unix/Linux-based distributed approach and a mainframe approach based on some combination of traditional mainframe environments like TPF, VM, and/or zOS. I seriously doubt Mr. Shuttleworth even knows what TPF actually is, much less other transaction software such as CICS, IMS, and so on.

    Mainframes continue to be the computing foundation for the most critical transactional aspects of business in the plurality of companies with more than $50M in annual revenue(turnover). There’s got to be a reason for this besides the customary bromides of migration cost and organizational lethargy/inertia.

  13. Roland says: (permalink)
    October 24th, 2011 at 6:37 pm

    Mainframes still win in IO. If you have to print a few million {electric,gas,insurance,bank,other} bills/statements every month, you need a mainframe with IO channels and printers to match. They will die when paperwork dies.

  14. mark says: (permalink)
    October 24th, 2011 at 6:49 pm

    @Roland

    Interesting use case; I think a bank of printers, addressable from the cloud, could quite happily compete in the scenario you describe.

  15. mark says: (permalink)
    October 24th, 2011 at 6:57 pm

    BMC sells mainframe software, so has an interest in the growth of the mainframe market. Takeouts from me from http://www.computerworlduk.com/news/infrastructure/3303178/cloud-popularity-and-mobile-usage-driving-increase-in-mainframe-capacity/ are the number of people surveyed who are concerned about cost and interested in private cloud.

    Cloud architectures are fault-tolerant by software architecture, only an idiot would pay for the same fault tolerance twice. Therefore, no matter how hard IBM tries to sell the idea of mainframes being the hardware for cloud, I don’t see it panning out that way. The whole point of the work that’s brought us cloud is to be able to do very large, very reliable services on low-cost, unreliable hardware.

  16. mark says: (permalink)
    October 24th, 2011 at 6:58 pm

    @anomie

    Security and access control are perfectly achievable in cloud environments.

  17. Jef Spaleta says: (permalink)
    October 24th, 2011 at 7:53 pm

    Mark,

    replacing unreliable hardware still comes at a cost. You do have to replace the “unreliable” hardware on a more frequent basis even if you do have additional hardware redundancy on-premises. Hardware redundancy only continues to work if you keep replacing the “unreliable” hardware as it fails. The moment you stop paying to replace the failures, its the moment the abstracted resiliance of the cloud model falls down.

    On a private on-premises cloud does that continual upkeep of “unreliable” hardware give you cost performance to offset the upfront cost of purchasing “reliable” mainframe hardware? I don’t think you’re in a position to actually argue that as you aren’t in the hardware business. And I haven’t seen anyone actually in the business of selling iron making the case that “unreliable” hardware will be cost effective for on-premise cloud needs. But please, feel free to point me to any vendor who is pushing “unreliable” hardware to power the cloud based data center approach to reliable services. Please.

    -jef

  18. Jo-Erlend Schinstad says: (permalink)
    October 24th, 2011 at 10:21 pm

    “There is strength in numbers”. Many businesses had their reliable systems, upon which they relied, in a reliable environment known as the World Trade Center. If I have a choice between having one reliable computer and having two reliable clusters of unreliable hardware, placed in different places, acting well together, I think I might want to not place all my eggs in one basket.

    I think we’re going to see more examples of highly distributed businesses, with many small offices in different countries. It might make more sense to have small clusters in each office, with clients that prefer the closest service, but can easily switch to another when that’s useful. Software is being designed differently, splitting the job into smaller processes, and the smallest computers are getting very powerful. It is useful to look at the past when you try to understand the future, but it shouldn’t be the only thing you consider. For example, I would agree that HD video is nicer than DVD, but will I ever be in a hurry to get 4K video? I don’t think so. When the screens outperform my eyes, then there’s little to attract. I don’t think any one argument for the cloud will persuade anyone that the mainframe is going away, but if they consider all the arguments, they just might.

    Jef: you ask about costs. Well, if you invest in a mainframe and your business expects organizational growth, then you have to prepare for that from the very start. If instead you build a cloud, then it’ll be easier to expand as the requirement grows. I won’t pretend to have the answer to that question, but it’s more complicated than what you portray. There might be differences in human resources as well, and that’s a significant part of the equation. By the way, haven’t I read that both Google and Amazon base their clouds on unreliable computers? I think I have.

  19. Jef Spaleta says: (permalink)
    October 24th, 2011 at 10:51 pm

    Jo-Elrand,

    Google and Amazon are vastly different economies of scale that what can be expected to exist as operational private clouds. Google supports its own hardware and operating system…the whole stack..internally. Is that the future for every private cloud? All support provided in house with custom hardware and custom operating system and custom application stack? If that’s the future, Canonical has no chance of selling support services in that future.

    But that’s not the future. Google and Amazon are at one end of the curve. They are massively large. So large they actually sell their extra capacity to externals. That’s made possible by the scale at which they operate. Once you get very very big and you are expending support dollars internally to deal with some sort of hardware failure that is occurring on the order of minutes, then you build so much redundant capacity that you sell some of it off. But that’s not what inhouse private clouds are going to look like. Not by a long shot.

    I would love Canonical to document the administration of a in-production private cloud that they are running internally. Since they aren’t a cloud provider themselves, watching them dogfood the private cloud model for their own needs would be very interesting. Not public cloud guest instances running on rackspace or AWS. A fully functional, private cloud for production use (not development or prototyping) with secure data storage requirements demands that prevent it from being moved into a public cloud entirely. Distributed globally at multiple locations as you describe. I’d like to see the administration of such a multiple-location private cloud documented over the course of a year or two including hardware failure response and recovery using nothing but “unreliable” hardware.

    -jef

  20. mainframe truth says: (permalink)
    October 24th, 2011 at 11:47 pm

    One other interesting note for the mainframe-unaware- IBM has had a facility known as Geographically Distributed Parallel Sysplex as a component of os/390 and its later version zOS for at least a couple
    of decades. This allows for distributed fault tolerance in mainframe systems.

  21. steve says: (permalink)
    October 25th, 2011 at 1:19 am

    I think you missed something. With the cloud, you have services and VIRTUAL MACHINES that do the work. A cloud service provider will want to maximize their hardware reliability and throughput and minimize hardware floor space and power usage. Sounds like a mainframe to me.

    There is an opportunity here for a new kind of mainframe.

  22. I like black and white... says: (permalink)
    October 25th, 2011 at 2:25 am

    I dare you to make a post about your garden. Firstly because it would be a nice to read an actual personal blog post on your personal blog again, but mostly because I’m curious how the always entertaining Jef will find a way to attack it ;)

  23. paul says: (permalink)
    October 25th, 2011 at 3:02 am

    Mainframes are not just useful for MVS operating types systems, but can also run up to and perhaps exceeding 1000 virtual machines running Linux and/or Unix.

    With the mainframes record on reliability, redundancy, and throughput, they are actually a very good buy for large data centers.

  24. Justin Andrusk says: (permalink)
    October 25th, 2011 at 3:17 pm

    I would be interested to know if there are plans for adopting a security architecture framework for the cloud such as SABSA.

  25. Jef Spaleta says: (permalink)
    October 25th, 2011 at 5:01 pm

    paul,

    Reliability aside. what’s the power consumption usage footprint look like for a mainframe powered cloud versus commodity hardware power cloud? If we are going to talk about on going costs, power consumption has to be factored in.

    -jef

  26. Claudio says: (permalink)
    October 25th, 2011 at 11:33 pm

    I know is off-topic but Mark please listen to me. A great application like Remastersys has died in the last days. I’m sure you know this application and how is easy to use. Why don’t integrate a so simple and easy application in Ubuntu repositories? I know there are clones but Remastersys remains the best. If i was you i’d contact the developer. I’m sure he would be very pleased. Sorry for my bad English and keep up the good job. Ah, i saw what is supposed to be the new iconset for Ubuntu 12.04. Yellowicon sounds familiar to you? ;) You are great and i bet Ubuntu will be as great as you. Bye!

  27. mainframe truth says: (permalink)
    October 26th, 2011 at 4:25 am

    Linux on the mainframe is a poor example of how to best exploit mainframe strengths.

    MVS, VM, and TPF offer much better solutions to high volume transaction-centered
    business processing on the mainframe than Linux can ever hope to achieve.

    This all being said, if you do want to run linux on the mainframe, you save
    tons of money in consolidation costs associated with administration, power,
    physical space, and cabling. Assuming your linux workloads are not compute-bound,
    a mainframe-based consolidation solution can cost considerably less for commodity
    hardware to operate, provided you have tolerable performance in each virtual
    instance.

    Even so, the best use of a mainframe cannot happen with Linux/Unix, because
    the architectural model of Linux/Unix operating systems is very far away from
    mainframe strengths. Linux/Unix also proves to be very compute-bound on the
    mainframe because the i/o cannot be optimized for channel-based implementation
    to the same degree traditional mainframe environments encourage.

  28. steve says: (permalink)
    October 26th, 2011 at 2:29 pm

    Not all mainframes are IBM (But most are). UNISYS has an X86 based mainframe that, with a little engineering, could work nicely for a CLOUD service provider.

  29. IGnatius T Foobar says: (permalink)
    October 26th, 2011 at 4:38 pm

    Running lots of virtual Linux images on a mainframe is a form of consolidation. Running Lots of virtual Linux images on a relatively smaller number of physical hosts (call it a cloud if you want to) is another, different form of consolidation.

    They really don’t have anything to do with each other. Pursuing them as an integrated strategy would be a quite foolish effort.

    Another quite foolish effort is Canonical’s insistence on turning computer desktops into overgrown smartphones via the awful “Unity” UI. This major mistake which is alienating the existing user base will continue to be brought up until it is corrected.

  30. steve says: (permalink)
    October 26th, 2011 at 8:37 pm

    Maybe I’m too old. I’ve heard this all before. This is a rehash of the “Glass house” vs “Distributed” discussion circa 1976, 1984, 1989, 1993, 1999, 2011. the bottom line is everyone has the same goals:
    1) Lots of computing power.
    2) No IT staff to run it.
    3) All the control of everything is centralized.
    4) It costs near nothing.

    These myths are always assumed:
    1) IT staff knows best.
    2) End users are the root of all trouble.
    3) It takes a visionary like ????? to figure it out.(Fill in the blank with “Steve Jobs” or “Mark Shuttleworth” or “Gene Amdahl” or )

    The bottom line is, if you virtualize everything (Storage, computers, networks, displays, phones etc) then you can do whatever you want.

    Are mainframes dead? No!
    Why? Because they virtualize everything.
    Why is that important?
    1) Money. Putting a lot of stuff in a small box is cheaper than putting a little in a lot of boxes at some point. Even GOOGLE is discovering this truth.
    2) It’s easy to a fix virtual computer; even a computer can do it itself. Virtual errors can be programmed away.
    3) You can change the hardware without changing the virtual. (Look at the AS400 of old).

    As for UNITY, Check

  31. Rob says: (permalink)
    October 26th, 2011 at 8:47 pm

    Interesting to read this post on one screen while logged on to TSO at work on the other. I’d love to see it happen, but having been at the mainframe command line for a few years and steeped in that cubicle culture I fully agree with your caveat about insurance companies and the like. Switching away from this platform is still quite a few years off for our needs.