Cloud testing tool gets leg up with Verizon iPhone launchDate: Feb 03, 2011
In anticipation of heavy Web traffic following Verizon's iPhone launch, the carrier turned to an application load testing tool that runs on Amazon Web Services.
In this interview with Tom Lounibos, president and CEO of Soasta, find out why Verizon chose his company's cloud tool to help manage any iPhone-related surges.
For a full look at this cloud testing tool in action, check out our video demonstration of CloudTest with Soasta's vice president of engineering.
Read the full transcript from this video below:
Cloud testing tool gets leg up with Verizon iPhone launch
Jo Maitland: Hi, I'm Jo Maitland. I'm here with Tom Lounibos,
who's CEO of Soasta.
Hey, Tom, how are you doing?
Tom Lounibos: Good. How are you doing, Jo?
Jo Maitland: Good. So we're here because this week Soasta is launching a new
version of its product, which I'm actually going to let Tom give you the
high level on what this is, and then in the second part of this interview
we'll actually talk to Tal, who's the VP of engineering here at Soasta, and
he does a deep dive on the actual product and how it works. But for the
high level, we're going to ask Tom what the product is, and who's using it
Tom Lounibos: Sure. Thanks, Jo. Thanks for being here, too. So, on Monday what
we're going to be releasing is, we're unleashing, what we would say, the
power of the cloud. And the view that we have is we're applying an
application towards cloud computing that is fairly unique, versus cloud
computing looking for applications. And this is in the area of performance
and load testing, and we become one of the stars of cloud computing in that
our application, on any given day, may be provisioning 10,000-20,000
servers around the globe for the benefit of our customers to test their
So, what we do is we simulate users, virtual users, whether it be a Netflix
user downloading a movie, or a Verizon user ordering a new phone, or a
TurboTax user actually filing their tax forms. But to create virtual users,
we're using the cloud to simulate an enormous amount of traffic hitting a
website, and it's fairly unique. It's kind of the killer application for
cloud computing. We'll provision, at any given point in time, hundreds if
not thousands of servers to simulate 100,000 users or a million users.
Jo Maitland: And so Verizon's particularly interested in this right now, as
they're about to launch the iPhone.
Tom Lounibos: Yes. You know anybody that has a consumer facing website, whether it's
a large company like Verizon or TurboTax, or a very small company like M-
Dot Networks, we're all moving more and more toward the web as our sales
and distribution channel. And you don't want to spend a lot of money to
push a product like Apple with iPhone 4 just last year, to a website where
they spent about $100 million on a marketing campaign, and then push
everybody to a site like AT&T to sign up and pre-order these phones, and
then have the site crash. And that's what happened with AT&T. And it's not
to say AT&T did a bad job; it's just that you want the performance to the
site to be the best possible, and sometimes in today's market, you could get
millions of users hitting a site all at once. And so it's a very complex
Jo Maitland: So why is Verizon leaving it to the last minute to do this kind of
testing? Don't companies want to test a little bit earlier? Or is it that
they want to test as close to when the actual launch happens?
Tom Lounibos: Well, I think they've been testing for a while here, so it's not like
they're just waiting until the last minute, but it is hard to get these
complex networks up and running, and get prepared. And this has been the
problem that we've tried to solve. In the past, and kind of the dirty
little secret within web application development, is it was nearly
impossible to do any kind of production level testing because it was so
expensive, or you didn't have the hardware to actually produce the user
experiences. And so that combination caused small companies to basically
not test. So 90% of the applications, and there are hundreds of thousands
of applications now that are out there, are not actually performance tested
before they go live, and so all of us consumers become -- and I think
everybody has felt that with Twitter and Facebook -- we're a bit like
digital crash test dummies. We're the testers for these sites.
Our belief was, because we were building web applications, that there
should be a better way. There should be an easier, more scalable, and fast
way of doing it that's affordable for everybody, not just enterprises, but
small companies as well. So we created this category we call cloud testing,
and we created this product called Cloud Test, and tomorrow, or next week,
what we'll be launching here is, we're unleashing the power of this cloud
community that we've pulled together. The product is called Cloud Test Pro,
and it is installed inside your firewall, or in a cloud, whatever you may
Jo Maitland: As opposed to the last version, which was SAS based.
Tom Lounibos: SAS based, where it was pretty much all on demand. And so from an on-
demand standpoint, our engineers would build a test, and then run the test,
and such. That was important for us, for a lot of reasons, to maintain that
model because there is so much complexity in cloud computing, that it would
have been very difficult for individuals themselves to provision hundreds
of servers in the cloud, or in multiple cloud providers. And so we've
worked really hard on automating the processes that you'll see in a demo
here in a few minutes with Tal, so that you can deploy these servers and
these virtual user environments within minutes, whether or not you're using
Amazon's cloud services, or you're using IBM's cloud services, or
Microsoft's, or if you're using private cloud services from VMware, or from
Eucalyptus or Cloud.com.
Now you're able to provision users, virtual users, both internally, using
private clouds or static servers, or externally using vendors like Amazon
and such. And it enables you to have complete power and control, and
probably the most significant thing to us is the data itself, the results
and the integrated analytics that are produced, and Tal will show you in
the demonstration. That's one of the biggest problem areas for testers.
It's one thing to be able to build a test. It's another thing to be able to
really analyze the data and come up with actionable intelligence. In other
words, "I need to upgrade my load balancers," "I need to redesign my
architecture," or, "I may need to change a flash file that's running for 26
seconds down to 18 seconds to allow more people to come in." So we help
customers not only get through a deployment of their app, but in many ways,
build better apps in the future because they understand scale a little bit
Jo Maitland: But now, if I want to buy the product myself, the new version, the
Pro version, do I need to have my own relationships with AWS, with
Microsoft Axure, IBM Test and Dev Cloud? How does that piece tie together?
Tom Lounibos: No. We handle it all together. You'll maintain your own accounts, so
you'll have to have an account set up, but then we'll automate the process
of all the provisioning into those different environments. Basically, what
customers will be able to do is create their own private global test cloud.
Now, that private test cloud, if you're TurboTax, is probably only going to
be in the United States, because it's for testing an application where only
U. S. citizens are filing taxes. But if you're Hallmark, you may want to
have locations in the UK, or even in Asia, as people send cards. So it's
really dependent on your application and where your customers are coming
from, but you may want to generate load from London, or Hong Kong, or
Milan, or Germany, as well as the West Coast and East Coast of the United
States, or in the middle of the United States.
Jo Maitland: So I can control that?
Tom Lounibos: You control all of it. So you have a large variety of locations, but
you can create your own private cloud, whether it's for internal testing,
like one customer of ours wanted to test Google Apps, specifically Google
Mail. So they had 100,000 employees, and they wanted to test what the
response time would be for those employees.
Jo Maitland: Who was that?
Tom Lounibos: I'm not allowed, actually, to tell you that, but it's a very large
biotech firm. And the idea here was, before they shifted over, they wanted
to make sure that Google Mail actually worked for them. So we were able to
replicate a large amount of users coming from internal out to the cloud. So
I think when you're testing an application, you want to test externally,
you want to test internally, and you want to have the power to do both. And
we're now giving the power to our customers to be able to have what we
consider to be the largest test cloud ever built. Or the largest test
platform ever built for both internal and external testing.
Jo Maitland: So, it sounds to me like you have a pretty hot product here. Has EA
come knocking? Is HP banging on the door yet? What's your read on that
Tom Lounibos: You know, we're known for a couple of different things. The big
testing tool product for the last 25 years has been Mercury Interactive.
Jo Maitland: Right.
Tom Lounibos: Mercury Interactive was acquired by HP for $4.5 billion just four or
five years ago. And we're kind of known as Mercury of the Cloud. And as we
build new, more modern web applications, there is a requirement for new
technologies because the problems that we're... As an example, the tests
that we used to do ten years ago on client, server apps might have generated
a few megabytes of data, of test data. The tests we do now, on a daily
basis -- tonight, as an example -- will generate probably ten terabytes of
Jo Maitland: Is that the Verizon test?
Tom Lounibos: That's one of the tests we're doing tonight. And in the future we'll
be actually generating test data sets that are petabytes of data. So you've
got to be able to analyze that data in different ways, and the complexity
of these tests are that problems come from a cascading series of events;
it's not a single, one-dimensional problem. So with these problem sets,
it's multi-dimensional, and you need a multidimensional technology and tool-
set to be able to solve that and a platform to solve that, and that's what
we're unleashing on Monday. And we're very excited about it.