A case for centralising and correlating event and log data

What's happening to my systems?

It's a question that I'm sure many fellow IT people out there have been faced with. Whether you're on ground zero doing systems administration and a server dies or wondering why all of a sudden orders on your website have slumped. It could literally be anything; a faulty device, software crash... maybe you got hacked? How do you know? Worse still, maybe something bad is happening right now and you haven't even realised yet!

How many of you have been faced with this situation and thought, "I'll just check the logs" and go from there, only to find that they either don't exist or are building up so quickly that those you want to see have already been deleted. Frustrating right? Those issues are just the tip of the iceberg and unfortunately this is still the reality for many organisations. Logging, monitoring, alerts and incident management are often neglected or given little to no priority.
Hmm.. I can hear the crickets starting to chirp already! Okay, so this isn't the most thrilling topic so let me jump straight to some of the benefits of a well thought out solution to make the most of your log data.

The Single Pane of Glass

There's no doubt that many point solutions already exist that do logging and monitoring for specialist purposes very well (e.g. network performance monitoring, antivirus logging/alerts, host performance monitoring, etc.). The trouble with this is that it's more difficult to get across all the systems to look at data. Not being able to look at it all at once can mean that things will be missed. A solution to centralise all the events/logs being generated can greatly alleviate this problem. Think about the time and effort saved if you only had one place to login to look at everything.

Operational and Security Monitoring

If you don't already have something in place to capture logs and events why not? The benefits with respect to operational and security monitoring will vary from one organisation to another. Let me give you a couple of examples to get you thinking.

• You run a website that processes a high volume of transactions. Performance and availability are absolutely critical because if it's not running well you're losing $$$'s. Typical items that could be monitored here are system performance logs, process availability, web server logs and database server logs. This will provide more than enough data to detect a decline in performance and generate alerts. You could also provide some really cool data to other departments within the business. Take marketing for instance. Data you're already colleting from the database and web server could be used to provide some nice statistics on the types and quantities of products sold, which in turn can be used to tailor advertising campaigns or even help guide spending on product R&D.

• Security is paramount within your organisation to protect sensitive assets. You've got all the usual controls in place to prevent anything going pear-shaped but if it does will you know? Monitoring for suspicious activity can be the difference between stopping something before it's too late or having a nasty incident to clean up. Sure you have firewall logs, intrusion detection logs and system security logs but are you REALLY looking at them? Let's face it, nobody is going to look through all this stuff manually AND regularly. Logging and monitoring needs to be taken a step up to centralise all this data and then correlate and aggregate events to provide useful alerts (think SIEM). Having a system like this in place can also greatly simplify any internal and/or external compliance reporting requirements.

Incident Response

Pure Hacking has worked with numerous organisations in an incident investigation and response capacity. More often than not, establishing a timeline of what happened and how it happened is difficult due to lack of logs from relevant systems. Time can often be wasted looking in the wrong places because no decent leads are available thereby turning the investigation into a costly exercise. Having the right logging and monitoring in place can help avoid this problem. Remediation efforts can be better focused on actual issues rather than on areas where there was only suspicion of a problem and business can return to normal quicker.

I hope some of those examples provided food for thought. Let's face it, I could simply have said you need a centralised logging solution for the sake of compliance; think PCI requirement 11. Whilst this is a necessity for those working with cardholder data, the benefits of a well-planned and executed logging and monitoring solution can be realised for many other businesses. Think about your most valuable business data… wouldn't you want to know if somebody was trying to get at it right now?