top of page

SPLUNK SIEM LAB

Introduction:

The purpose of this Splunk lab is to take traffic from the public internet, try to access a web server and its website, and aggregate web traffic to Splunk. This project is a Splunk SIEM lab; it's three virtual machines created in Linode and Jump box using Kali. Splunk and WordPress using Ubuntu.

 

The jump box machine is used to SSH into the Webserver and Splunk Server. This is a more secure process than directly connecting SSH into Webserver and Splunk Server.

 

The webserver is running services such as  Apache2, MariaDB, and SSH. This allows for public access to a fresh install of a publicly accessible WordPress site and remote access to the server via Secure Socket Shell (SSH). The web server is also configured with a Splunk Forwarder to send logs to the Splunk Server to be indexed and monitored in real-time.

 

The Splunk Server has a control panel for Splunk accessible via a web browser using the server's IP and port 8000, as well as a listener on port 9997 to gather the logs being sent by the Forwarder/Webserver.

This is a paragraph where you can add any information you want to share with website visitors. Click here to edit the text, change the font and make it your own.

This is a paragraph where you can add any information you want to share with website visitors. Click here to edit the text, change the font and make it your own.

image.png

Setup

 

The setup of this lab was done mostly through Linode prebuilt Ubuntu and kali boxes and then allowing ssh through the boxes. The most troubling factor for this was using the prebuilt Splunk box from Linode. While it may work for some, I had issues using it so I had to download a newer version using this command after reading documentation from Splunk and others on ho to remove all Splunk services.

Download Command: wget -O splunk-9.1.1-64e843ea36b1-linux-2.6-amd64.deb "https://download.splunk.com/products/splunk/releases/9.1.1/linux/splunk-9.1.1-64e843ea36b1-linux-2.6-amd64.deb"

 

From that point, I also logged into the WordPress site and downloaded the new version of Splunk Forwarder to match the Splunk server.

Splunk Forwarder Download Command: wget -O splunkforwarder-9.1.1-64e843ea36b1-linux-2.6-amd64.deb "https://download.splunk.com/products/universalforwarder/releases/9.1.1/linux/splunkforwarder-9.1.1-64e843ea36b1-linux-2.6-amd64.deb"

 

Even though It was a new download, I had run into some issues where I found it was best to keep the machine on in Linode, which does not charge for how long the machine is on if replicating in aws or Azure, that may not work for you. Additionally, I restarted the services and edit the various .conf so that the logs would output to the Splunk server from the web server.

  1. I SSH into my Jump Box to then later SSH into my webserver

image.png

2. Here I initially added the directories I wanted to monitor, and I then later upgraded the forwarder and added them again. The directory to complete this is cd /opt/splunkforwarder/bin then to add the directories /var/log , /var/log/apache2/

image.png

3. Then I added the port 8000 TCP on the Splunk server and reloaded the firewall in order to access the Splunk concole. I also made sure that port 9997 was open.

image.png

4. Create Splunkadmin user for the Splunk console.

image.png

5.

Here I added the Splunk server for the forwarding server to push to the deployment server. With the command ./splunk set deploy-poll <splunk server IP:8000>

Then I went to /opt/splunkforwarder/etc/system/local then cat deployment.conf. Also later modified outputs.conf.

image.png
image.png

6. I then added the forward sever again and the  forward port 9997

image.png
image.png

7. I then added  the directories that I wanted to monitor for a second time.

image.png

8. I then restarted the forwarder service 

image.png

9. Check the status of the Splunk services ./splunk status

image.png
image.png

10. Looking at the forwarder management screen in the Splunk application, I was able to see that the web server was in.

image.png

Brute Force

  1. To simulate a brute force (a password cracking method) attack on the WordPress login hosted on the Webserver, a local machine was used along with a tool called Hydra. The Hydra syntax for WordPress that was used is as follows: hydra -l <wordlist> -P <wordlist> <ip> http-post-form '/wp-login.php:log=^USER^&pwd=^PASS^&wp-submit=Log+In:incorrect' -vV -t 30

image.png
image.png
image.png

2. In splunk I was able to pull the brute force attempt we see that the fake usernames/passwords were posted to the login site /wp-login.php.

image.png
image.png

Ping

In order for me to simulate a DDOS attack, I created a ping script and ran from my local Kali machine to attack the webserver. However this only one machine a true DDOS would be multiple machine aka a botnet running this attack at once to bring down a webserver services. Monitoring this kind of traffic can be beneficial to detect a possible DDOS attack with hopes of catching and stopping the malicious traffic before server takedown occurs.

image.png
image.png
image.png

To be able to see when the Webserver was pinged via a log file, the following command was ran from the Webserver:

iptables -I INPUT -p icmp --icmp-type 8 -m state --state NEW,ESTABLISHED,RELATED -j LOG --log-lev el=1 --log-prefix "Ping Request"

The pings are now stamped in the kern log ,which allowed me to pull out the event within Splunk.

image.png

Dashboard

1. In the dash board I was able to create important statistics and virtualizations
Using this command I was able to see the difference between host , uri_path , and status . source="/var/log/apache2/access.log" | top limit=20 uri_path by clientip,req_time | eval percent=round(count / total * 100, 2) | fields - percent

image.png
image.png

2.Using this splunk command source="/var/log/auth.log" ssh failure | iplocation rhost | stats count by rhost, Country ,City | sort - count it allowed me to see the different countries, cities and the host.

image.png
image.png

3.I was then able to include the URI path source="/var/log/apache2/access.log" | iplocation clientip | stats count by Country, City, clientip, uri_path | table Country, clientip, City, uri_path , count it also showed the different countries and host trying to connect to different directories.

image.png

4.For the Dashboard there a multiple splunk commands used:
 
Word Press Login : source="/var/log/apache2/access.log" uri_path="/wp-login.php"| iplocation clientip| stats count by Country, City, clientip| table Country, clientip, City, count
 
SSH FAILURE DASHBOARD source="/var/log/auth.log" ssh failure | iplocation rhost | table rhost, Country, City | chart count by Country
 
URI_PATH DASHBOARD source="/var/log/apache2/access.log" | iplocation clientip | stats count by Country, City, clientip, uri_path | table Country, clientip, City, uri_path , count

image.png

Firewall

1. From Looking at Splunk and the multiple attempts to attack our webserver. The script created block_ip.sh was used to block the 103.139.44.55 for Vitenam from our WordPress server aswell as the 146.190.128.77 from the United states.

image.png
image.png
image.png
image.png

2.

In this screenshot we see the script processed the firewall rules to block both IP's .

To run the script it has to be made executable: chmod +x block_ip.sh

Then run command sudo ./block_ip.sh 123.456.789.123

image.png
image.png

3.A more simple way is to add the webserver/wordpress to the Lab_firewall VIA the Linode platform and utilize the rules assigned to block outside traffic.

image.png
image.png

 Lessons Learned

This Splunk Lab SIEM has demonstrated the reality of cyber attacks as various attackers from different IP addresses attack the Webserver. The web server was never shared outside my project, and this demonstrated how important proper security architecture is to protect servers and websites from potential dangers.

 

Had the Webserver indeed been compromised, lateral movement from an adversary could not take place, and the one IP logged to access the Webserver's backend would be the private IP of the Jumpbox. This keeps those accessing the servers through the Jumpbox safe, as their true IPs are not disclosed, and a new Jumpbox can always be established.

 

This project allowed me to configure cybersecurity architecture and figure out the issues and solutions to create a successful lab. The lab did give me a few issues as far as SPLUNK. I learned how to troubleshoot, restart the service, and keep the server on. Overall, The essential skills I learned were:

  • SIEM Management/ DATA Collection

  • Incident Detection and Response

  • Cybersecurity Architecture Configuration

  • SPLUNK Troubleshooting

  • Lab Configuration and Management

bottom of page