Log management tools have been a mainstay in IT infrastructures for years. Log data helps admins monitor system events and identify any potential issues with performance, security and more. A web server log, for example, allows admins to see all of the files users have requested to access on a website, along with other usage patterns.
In the cloud, log management tools play an equally important role as they do on premises. In general, cloud log management tools fall into two categories: those that monitor cloud applications and those that play more of a "forensics" role. The second category helps cloud admins pinpoint the cause of downtime or sluggish performance, for example, said Dave Bartoletti, principal analyst at Forrester Research Inc., an analyst firm in Cambridge, Mass.
Because of that first category of tools -- those that help track cloud app performance in real time -- cloud log management tools are now finding a home with development teams, as well.
"More and more, these tools try to serve multiple consumers," Bartoletti said. "It's not just IT organizations anymore that want to look at log information."
Many major public cloud providers offer their own tools to help users track and manage log files. AWS, for example, has AWS CloudWatch Logs, while Google has Stackdriver Logging. There are also third-party log management tools, from companies such as Splunk and Loggly, which can work in the cloud, as well as open source log management tools such as Elastic Logstash.
While some of these open source cloud computing tools offer greater flexibility, they also present unique challenges, especially when it comes to support, said Alex Witherspoon, vice president of platform engineering at FlightStats, a Portland, Oregon-based global data service company in the aviation space, which uses LosStash.
"As your needs grow … [the tool can be] expensive to operate, especially if you don't have dedicated engineering talent to focus on it," Witherspoon said.
Overall, most organizations use a cloud log management tool that is specific to a certain cloud platform or vendor. Some tools offer cross-cloud or multicloud log management features, but, according to Bartoletti, it's still early days for that functionality.
"I think most companies now are solving the problem for specific clouds, specific infrastructure -- there isn't really an uber log management solution [for different cloud platforms]," Bartoletti said.