Welcome to the SecurityisFutile blog

I welcome comments and suggestions, I take criticism very lightly (at least most of the time). My goal for this blog is to document various experiments and research projects I feel are both relevant and prominent in the field of computer security (or lack there of) and share my results and experiences with other fellow computer security enthusiests. Most of my topics are based soley on open source technology and methodology, mostly due to availability and cost. I believe that effective security measures help keep people honest with their technology, for the most part. Security is futile (usless) or at least it feels that way when an inspired opportunist comes around and exploits your weaknesses. With that being said I leave you with a quote of inspiration; There is no security on this earth, there is only opportunity.-- General Douglas MacArthur

Thursday, December 23, 2010

iWatch my logs in Splunk....do you?

iWatch is a program written in perl that performs real-time filesystem monitoring and requires inotify support for Linux-based kernels (Linux Kernel >= 2.6.13).  Its purpose is to monitor changes in a specific directory, file or even recursively monitor a directory and perform event notification immediately after a change.  This program is somewhat similar to the open source versions of Tripwire, OSSEC and AIDE however, it is more simplistic in nature and can easily be tied into your central syslog monitoring solution.

Configure iWatch to monitor some critical files on a local Ubuntu Linux server and report changes to syslog.  Then configure Splunk (the standard Search App) to monitor the local syslog file and modify syslog-event transformations to display iWatch specific fields.
  • It will be assumed that Ubuntu is already installed and operating 
  • Local Ubuntu box is configured to have syslog messages forwarded to /var/log/syslog 
  • Splunk 4.1.x is already configured and the default Search app is available
Step 1:
  1. Download inotify support and iwatch for Ubuntu (I used archive.offensive-security.com as my source repository)
  • # apt-get install inotify-tools
  • # apt-get install iwatch
Step 2: Configure the local syslog file index in Splunk
  • Log into the Splunk Manager web interface Manager --> Data inputs --> Files & Directories --> Add New
  • Click the Index a file on the Splunk server 
  • Use /var/log/syslog as the path
  • Select From list as the sourcetype, then select syslog as the source type
  • Select main as the index, unless you know what you are doing
  • Select Follow tail (so Splunk performs a tail -f on the file and reads in new events after you create the index)
  • The click Save 
Step 3: Modify the syslog-event transformations file:
In the splunk web interface, now go to Manager --> Fields --> Field transformations --> syslog-extractions
Modify the fields as follows, then click the Save button when completed.

Regular Expression:


Event Format:
process::$1 pid::$2 iwatch_event::$3 iwatch_file::$4

Step 4: Start local iWatch process
-s (log events to syslog)
-v (verbose)
-r (recursive monitor directories/files)
-e (monitor all file modifications/changes)

(run in foreground)
# iwatch -s -v -r -e all_events /etc /bin
(run in background)
# iwatch -s -v -r -e all_events /etc/bin &

Step 5: create, delete some files  (in bash shell as root)
# cd /etc; i=0; while [ $i -le 10 ]; do touch file.$i; i=`expr $i + 1`; done
# cd /etc; j=0; while [ $j -le 10 ]; do rm file.$j; j=`expr $j + 1`; done

(view contents of sylog and make sure we have some events)
# cat /var/log/syslog

Step 6: Search for the new iWatch events in Splunk
Using the Splunk Search App, search for your new events using the following regex:

sourcetype="syslog" process="iWatch"

You should see "iwatch_event" and "iwatch_file" under the interesting field column on the left.  Click on those fields and search for your specific events.  Now you have the ability to search and build reports on file modifications/changes to your environment!!!  I would recommend reading more on iWatch to see how it can best be configured to work in your environment.  The example above should just get you started.

Happy Spelunking!!!


No comments:

Post a Comment