You are viewing 'log management'
How to use DNS logs
Over the last several months, there has been a lot of interest about Domain Name System (DNS) logging and what can be done with DNS logs. I discussed parts of this topic in my last blog, Finding the Culprit, and will continue to expand on some of those ideas. Many people ask if ActiveGuard® supports DNS logging. While it is not supported at this moment in time, there is a larger discussion to have around the topic.
This larger discussion starts with the number of logs produced by DNS servers. Let’s say an organization of 15,000 employees decides to log all the requests and responses for DNS. This organization would produce approximately 100 logs per second, or 8.6M logs a day. On average, these logs are 750bytes in size, so we will need 6GB per day uncompressed to store these logs. This is not too bad of a number, but you have to remember how your log collection capability... read more >
How to Use ELK to Solve Your One-off Log Analysis Problems
Performing log analysis with divergent data sets can be the stuff nightmares are made of. If you are lucky, your organization may have only a few dozen different log types throughout your environment. If you perform log analysis as a service, forget about it. There are many fantastic log management solutions on the market today, including our own ActiveGuard service. These solutions have robust log collection, analysis, and search capability. For a comprehensive, enterprise log analysis solution they are ideal, however they require substantial implementation and tuning for your specific environment and are intended for long term log aggregation and monitoring.
It is not always feasible to stand up one of these solutions on short notice or for a one-off project.
So where does that leave you? Manual log normalization and analysis? Manual techniques do have their... read more >
Why human analysis is so important
Lately, there has been significant discussion comparing log analytics to human observation for monitoring events and alerts created by security devices. Determining how and when to use which method (analytics or human) is critical to understanding the root cause behind any given analytical issue. Notifications from devices often need a professional human evaluation to correctly analyze the data and put it into the right context. This can be monotonous and time consuming for the engineers and/or analysts who are looking at the different alerts to determine legitimacy. This ultimately creates a significant dilemma for IT organizations on how to properly handle automation issues — should they use log analytics to save time, or continue with the more thorough but tedious human observation?
To solve this issue, let’s begin by putting the problem in context. Log analytics is used to autonomously monitor a device or devices by utilizing a set... read more >
Collecting, Monitoring and Retaining Critical Log Data for Compliance
Are you considering a managed security services provider (MSSP) as a part of your security management program?
If so, you probably have a good idea of how systems and application event logs can detect problems and provide valuable information about what is happening in your environment.
When log generation is configured correctly, and logs are properly used, the data can be the canary in the coal mine that alerts you to danger; the shining path you can follow, showing you where an attacker has been and the damage created. The data can serve as evidence, sometimes giving you a warm feeling of satisfaction that a problem has been solved or the realization that a villain has met justice. Beyond that, these logs can be an important part of meeting regulatory and compliance standards.
Discussion about... read more >
12 Log Data Sources for Incident Response
When the Solutionary Security Engineering Research Team (SERT) gets involved in a critical incident response, it’s fairly common for the organization we’re helping not to have centralized logging in place. It’s also common to conduct response efforts in network areas that have little logging or visibility.
These are significant and yet common challenges, and have a negative impact on anyone’s ability to piece together what happened. That does not mean, however, that we cannot do any incident research. It’s not ideal, but a partial picture can be created given enough data from a wide range of sources.
There’s also a common misconception that the logs needed for continuous security... read more >