WavebreakmediaMicro - Fotolia

Start early to get the most out of embedded software testing

Cloud Expo session preview: From smartphones to cars, the devices we depend on daily demand that embedded software testing gets an early start from cooperative developers and QA.

This is the fifth in a continuing series of stories previewing sessions of importance to cloud application developers at the Cloud Expo conference, which takes place June 7 to 9 at the Javits Center in New York City.

John Paliotta is co-founder and CTO of Vector Software Inc., a provider of testing products for safety and business-critical embedded software based in East Greenwich, R.I. The company's VectorCAST test automation platform leverages test-driven development, continuous integration and change-based testing, with the goal of helping software development teams automate complex testing tasks. Paliotta's Cloud Expo session, "The Industrial Internet of Things and Software Quality: Why It Matters Now," is scheduled for Wednesday, June 8, at 5:40 p.m.

Why is embedded software testing so difficult and problematic?

John Paliotta: Avionics was the first industry that had a thorough methodology for building in quality during, not after, the software development process. That's because when a plane goes down, it's on the front page of every newspaper and people want answers. Our smartphones, smart thermostats or other devices are different -- they all freeze up or crash, but it's not because developers or testers are lazy or are doing a bad job. It takes a lot of time to test thoroughly. If you're in an industry that doesn't mandate thorough testing, or if there is no pain in having lower quality, then testing doesn't get done. Yes, everybody cares about quality, but to do it thoroughly and continuously is expensive and time-consuming.

Doesn't this lack of deep testing become problematic as apps get older, undergo dozens or hundreds of revisions and suffer from inevitable code bloat?

Paliotta: I worked on a Navy submarine system that had been written in assembly code. We rewrote it in a higher-level language because assembly was hard to maintain. Later, it was rewritten from scratch -- again -- this time, in the Ada language. Today, we can't do that, because there isn't enough money or time to completely retool and modernize. The code bases stick around for decades, get bigger and keep getting modified. Testing for this is incredibly complex.

What impact does size have on embedded software testing?

Paliotta: The Apollo lunar lander's guidance computer had just 2,000 bytes of its memory available for programs. [Editor's note: That's 0.0019 MB.] Today, even cars have hundreds of thousands of lines of embedded code. All of that code has to be tested as it's developed; it's a safety issue.

Cloud Expo conference coverage

How virtualized apps and IoT mesh

IoT firmware updates are challenging for developers

Repairing HVACs with software development

How cognitive computing apps refocus dev skills

Can't we simply reboot when software goes bad?

Paliotta: That is another issue. The longer a program runs, the more states it has. The Joint Strike Force fighter jet's navigation system has to be rebooted every three hours.

You've painted a dire picture of software testing. What is the underlying cause?

Paliotta: The problem is that feedback loops span days and weeks. If a developer is working on a completely new project every couple of weeks, it's difficult to remember the code a month later when someone comes in with a bug. Another difficulty is that when quality assurance reports a bug, the developers can't reproduce it. That can quickly lead to an adversarial relationship. You need to eliminate that pingpong.

If you have the inmates running the asylum, where anyone can make code changes, you'll have a code base that gets out of control very quickly.
John PaliottaCTO, Vector Software

What's the answer that you'll be discussing in your presentation?

Paliotta: The first thing is to keep tight control over the code. If you have the inmates running the asylum, where anyone can make code changes, you'll have a code base that gets out of control very quickly. You also need to build quality testing in at every step of the project, not just during a test phase after coding is completed. The cost of not doing testing and having a major problem, especially with software that is embedded in devices, could be much more than if you had done proper testing throughout the project's development lifecycle. This is true for embedded software testing or applications.

What is the impact on developers?

Paliotta: You're not going to change the mentality of developers overnight; they've been allowed to have things their way for so long. If you put new burdens on them, they can walk across the street and probably find a better job. It's up to the CIO to strike a balance between engineering requirements and the need to keep developers' job reasonable.

Next Steps

Testing embedded software? Demand extra scrutiny.

Are smart devices really stupid?

Understand embedded software's unique security requirements.

Dig Deeper on Cloud application development