News Stay informed about the latest enterprise technology news and product updates.

IoT data: Handling a raging torrent in the cloud

As Internet of Things sensors appear in more places, it is cloud applications that must handle the data packets they generate. Depending on the use, the data could be anything from a trickle to a raging torrent. Are the applications you’re building capable of handling such volumes? And in real time? Is your cloud infrastructure or provider service agreement robust enough to handle enormous volumes and instantly scale when those volumes spike? Hope so.

Gartner research vice president Kyle Hilgendorf looks at IoT in a unique way. One IoT sensor creates a miniscule amount of data. Think of it as a single raindrop. But, put enough raindrops together and you get a downpour. That’s where we’re headed, he says.

Consider one scenario. If a sensor is monitoring the temperature inside some device — refrigerator, toaster oven, NAS unit, industrial furnace, airliner jet engine, or nuclear power plant — and reporting the temperature every 15 seconds, the volume of data your application needs to handle is tiny. But, if your application tracks temperature reports for every jet engine that a major airline carrier has in the air at any point in time, that single raindrop of data has now become part of a massive torrent. Your application needs to handle that, and storage must be capable of keeping up with both the collective amount of data and its incoming speed.

But wait, there’s more. Your application also has to decide which sensor data to keep and which can be ignored. Maybe you should write every temperature reading to a file or maybe just the ones that are outside of a pre-defined acceptable range — good old exception reporting. And what about that span? A 10-degree temperature variance in a toaster oven is not a big deal, but for a jet engine that variance could spell disaster. Then there are decisions about sounding the alarms in real time or simply presenting aggregated reports periodically. As always, there is no right or wrong answer.

How are the applications you’re building processing, leveraging, and storing incoming IoT data? Share your strategies with us; we’d like to hear from you.