News Stay informed about the latest enterprise technology news and product updates.

An inside look at Massachusetts' grand open cloud experiment

Through the Massachusetts Open Cloud initiative, Boston University professor Orran Krieger is adamant on reshaping the public cloud market and spurring big data innovation.

While public cloud adoption continues to grow in the enterprise, some markets -- such as life sciences, academia and HPC -- crave a different kind of cloud. And that's exactly what Orran Krieger, research professor and director of the cloud computing initiative at Boston University's Hariri Institute, wants to create.

Alongside other members of the academic community and the state of Massachusetts, Krieger is launching the Massachusetts Open Cloud (MOC), a new flavor of public cloud based on an open and customizable model Krieger says contrasts sharply with the leading public cloud platforms today.

Dubbed as Massachusetts' first Open Cloud eXchange (OCX), the MOC was introduced in April 2014. After a proof of concept last year, Krieger and his team hope to stand up a production environment over the next several months. The MOC, at first, will be used primarily by researchers and students across the five universities supporting it, and then become commercially available.

SearchCloudComputing recently spoke with Krieger -- who spent five years at VMware, helping to shape vCloud, before joining Boston University (BU) -- about how MOC differs from other public clouds, the innovative uses he anticipates with MOC and why open source is here to stay.

Why did you see a need for the MOC?

Orran Krieger: When Jonathan [Appavoo, assistant professor of computer science at Boston University] and I were brainstorming about what cloud could be, we had sort of a vision of what the future would be. And that vision included the ability to offer a highly differentiated, specialized cloud [environment] to innovate.

Instead of having to sell it to500 large enterprises and build a large sales force or sell to another company… our vision was that you could actually go and do it yourself and expose it to the whole world. You could do something that improved performance by an order or magnitude, or security, or even bring it to a niche marketplace. But if you could expose it to the whole world, you could drive the utilization of it up. We believed that cloud would just radically increase innovation.

If you looked at what cloud was actually becoming, it was becoming a small number -- essentially an oligopoly -- of very large corporations that had a huge amount of automation and complexity [in terms] of how they manage their infrastructure. And it was [driving] almost the opposite of innovation. Instead of at least having several hundred companies, you had only three or four major companies.

If you believe that cloud is the future of computing – which I think there is a strong trend that indicates it probably will be – I think it's very dangerous if cloud computing actually becomes an oligopoly that stifles innovation. ...Cloud will never actually achieve its potential of being the future of computing if it doesn't become a place where a very broad community can innovate in it.

We are trying to create a [new model] and, if we can demonstrate this, then I think a lot of big cloud providers will also follow suit.

How did you get the MOC off the ground?

Krieger: When we started this, the five major universities – BU, Northeastern, MIT, Harvard and UMass [Amherst] – had sort of already gotten together and built the $100 million MGHPCC [Massachusetts Green High Performance Computing Center] data center in Western Massachusetts, where there was low power costs...It was being used essentially as a colo facility where these universities each had their own massive amounts of research computing they would do. And a lot people had a vision that this could be a place where we could build something shared, but that wasn't part of the original mandate.

It was a [part] of what I saw as a critical need for the future of the computing industry, but the other point is that, to build something, you have to have an initial captive marketplace. You have to have a place to grow up. And we had both of those. We were actually able to convince all five universities to participate in this and build a shared cloud for the universities. Then, we were able to convince the Commonwealth that has a major focus on big data that this was a really important vehicle of change.

So what exactly is the infrastructure behind the MOC?

Krieger: It consists of servers, computers, networking gear, storage, external network connectivity -- all sort of provided by our different partners. The key partners on this are Intel, Red Hat, Cisco, Lenovo, Brocade [and] Two Sigma. Each of these partners is providing, in most cases, infrastructure that they are donating to the effort to get it off the ground.

And the Massachusetts Technology Collaborative has also provided $3 million in compute planning to get us off the ground. It's had a really, really big amount of donations and involvement from industry partners.

MOC is based on OpenStack, as well, right? Why did you choose to go open source?

Krieger: Yes. It's based around OpenStack….It was critical for us to work with open source, so researchers and partners can evolve the software.

There is a lot of perception that there are big winners and big losers [in cloud]. You look at how successful particular companies have been in this space, but I think it's all up for grabs.
Orran Kriegerresearch professor and director of the cloud computing initiative at Boston University's Hariri Institute

What kind of innovative uses do you see on the MOC?

Krieger: It crosses a very wide spectrum. We have users in the high performance computing community that are wrapping their projects around it; we have users in the big data community, we have a large number of computer scientists that are just developing various applications, and then there are life sciences users. It's also folks that are extremely concerned about security. They maybe have hosted things on a variety of other infrastructures, but they don't have the visibility into what providers are doing internally. They don't have the ability to audit across all the different layers. They may know that these providers have very strong security policies and mechanisms, but they don't actually have the visibility into it and they don't have control over it.

We have some really innovative applications. ...One [is] medical imaging for fetuses. Because the fetus moves, [its image] isn't readable, so nobody does MRI for fetuses. If you could do 3D reconstruction and motion correction, then you actually can make a readable image out of it, but it takes a massive amount of compute time, which makes it clinically irrelevant, because you can't wait ten hours to diagnose a fetus. To build a prototype that's a thousand nodes could end up doing that in a few seconds. That would be a radical change. And hospitals can't afford to buy that amount of infrastructure themselves. It only becomes cost effective if the person only has to pay for seconds of use, and that's the type of thing that a public cloud today, which is looking to satisfy the majority of applications, isn't going to build, even though it could end up being a radically important market.

We talked about another use with smart cities. You might have companies that want to put in infrastructure that controls the traffic lights. But they want to do it in an environment where they can actually merge the information from different municipalities to, for example, send a police car from Newton to Brookline to Boston without ever hitting a traffic light. You can imagine doing these things, but you have to draw information from many different municipalities, so cloud makes sense. But on the other hand, whoever controls the traffic lights isn't going to want to -- or isn't going to have the ability to -- put their infrastructure onto Amazon's cloud, for example. 

[Another use case] is a company that wants to evaluate different products. You have Brocade and Juniper and Intel and Lenovo and IBM and HP and all these different companies participating in it, [so] then people can pick and choose from different [technologies].

Why wouldn't some organizations have the ability to put their infrastructure on Amazon, like you mention above?

Krieger: In general, in the big public clouds, only the cloud provider has the capability of introducing infrastructure into the cloud. Integrating into the automation, management [and] billing of the cloud is not something third parties can easily do. 

Why else might a company choose MOC over a public cloud like Amazon or Microsoft?

Krieger: There are a whole variety of reasons. Amazon and Microsoft are incredible innovation platforms… they are doing awesome stuff. But every time the universities, for example, evaluate [those platforms], it's always been clear that it's not cost effective.

[Public cloud] is incredibly cost effective today for somebody that hasn't built computing infrastructure or doesn't have skills to build it. And it's incredibly cost effective if you're a startup and you have aspirations that next week you may need 10,000 servers, but this week you only need two, and you need to scale up and scale down.

But for us it made no sense… [we wanted to] build a place where we can share data sets and it becomes far more cost-effective for us than to just use one of the public clouds.

Open source is at heart of MOC. Do you think open source is the future of cloud?

Krieger: Open source has largely won today. I mean, if you look at Microsoft, they are deploying Linux … this is what people run their applications on -- open source software. So open source and the ability for a broad community to innovate has sort of fundamentally won, certainly in the server space today and in the cloud space, and that's not going to change. We are not at all alone in terms of using open source for what we are doing. Google's stuff is all based on open source hypervisors, Amazon's is, Rackspace's is… But today, open source isn't enough because people have to be able to deploy their innovation at scale and people have to be able to add new hardware to something if that's where their innovation is. We are taking open source to the next level by creating an open cloud. But whether or not we are successful, it's clearly the case that open source is going to be successful. 

Now, there's going to be roles for non-open source things, and that's going to continue. If you look at what my team and I built at VMware and what they continue to work very hard at… it's a much easier to manage environment than OpenStack. OpenStack is a complicated platform, and there is always going to be a tradeoff between the people who have an appetite to manage something complex versus wanting a turnkey solution that just solves their problems.

Any other trends you think we'll see in the cloud computing market?

Krieger: I think the rate of innovation is increasing. Kubnernetes, Mesos, containers of various sorts – things that have been relatively static in terms of our assumptions about what an operating system is and how workloads use it are changing at an ever-increasing rate it seems. That's going to continue for a while. And that's sort of the fundamental challenge we all face. Large communities can sometimes be really good and agile at picking up on these things, and large communities can also, in some cases, not have the ability to turn rapidly on a dime. It's going to be a shakeout. …

There is a lot of perception that there are big winners and big losers. You look at how successful particular companies have been in this space, but I think it's all up for grabs. I think we're in a really exciting time where the industry is changing, and changing evermore rapidly.

Kristin Knapp is site editor for SearchCloudComputing. Contact her at [email protected] or follow @kknapp86 on Twitter.

Dig Deeper on Public cloud and other cloud deployment models