Using ELK to detect network threats
Thousands of organizations all around the world use Elastic Stack, also known as ELK Stack (which stands for Elasticsearch, Logstash, Kibana), to manage, monitor and analyze logs. The open source tools are flexible and can be applied to multiple different use cases. In this post, we will highlight one such application: Elastic Stack for SIEM.
What is SIEM and why do you need it?
SIEM stands for Security Information and Event Management. In a nutshell, SIEM is a combination of technologies that give an overall look at a system’s infrastructure as well as analysis (and more), in order to keep the system safe.
If your company handles sensitive data, you need to have an SIEM. Companies employ SIEM tools for compliance, certifications, log management and monitoring, case management, and enforcing and identifying policy violations.
Elastic Stack + SIEM
To get a real-world example, we went directly to an expert. Chris Rock is the CEO and founder of Kustodian and SIEMonster. SIEMonster is an SIEM tool to detect threats and risks in a company’s network. They focus on detecting risks for companies using the ELK framework as a base. For example, detecting unauthorized access attempts, brute force logins, network parameters for safety, and more.
With SIEMonster, an open source alternative to Shield or Marvel, organizations have global, real-time security monitoring without the development headaches, documentation integration and price tag of other SIEM solutions.
Pros & Cons
When deciding which systems to use to build SIEMonster, Rock looked for a solution that was mature. He considered Cisco Open Framework (now Apache Metron). But “companies like Netflix and Ebay were already using it, so ELK as a backend was perfect for us to build an SIEM around it,” he explained.
Many security vendors have general solutions. SIEMonster customers need something to show them what is happening on their specific network.
Rock also likes ELK for its scalability. Elastic Stack is the perfect Database and Log parsing suite because it will scale indefinitely and grow with their SIEM tools.
Elastic Stack’s versatility allowed Rock to develop a solution for monitoring different metrics for each customer. Using ELK, Rock built out plugins, dashboards and parses to collect, process, and visualize data, then automatically detect risks, create tickets, and send alerts and reports. There are almost no limits to what you can build.
“The metrics we are after depend on what the customer wants to see. For example, the top 10 offending IP addresses, countries performing the most amount of attacks, etc.”
And each customer can get exactly what they need.
The biggest drawback for Elastic Stack from Rock’s perspective are the compression rates. Compared to commercial solutions, like Splunk, which compresses data to 20% of its original size, ELK has room for improvement. ELK compresses data closer to about 80%, so that is better than nothing, but worth considering.
Finally, Rock rounds out the SIEMonster solution with Skedler. To send automated Kibana reports like those mentioned earlier, as a PDF, Excel file or PNG, “We needed a reporting tool for SIEMonster, Skedler is perfect for this situation.”
If SIEMonster detects any issues for their clients, they can immediately send a report with all the pertinent information. Additionally, they can send summary reports regularly, such as weekly or monthly, to give customers a status update for peace of mind.
The Elastic Stack – Elasticsearch, Logstash, Kibana – is a powerful suite of tools for centralized logging, system monitoring, business intelligence, monitoring and security, and more. There is no shortage of examples of the applications of ELK.
Here we explored how one SIEM solution was built using ELK stack, including the benefits and some drawbacks. Rock and the SIEMonster team found ELK tools can help them deliver a better product for their customers. Hopefully, this will help to give you inspiration and ideas for implementing it yourself.
Get future tech posts and use cases delivered directly to you by subscribing to our newsletter.