Tuesday, April 21, 2009

Building A Syslog Server

Well I'm going to skip continuing my postings on Border Router Security because my Syslog server is acting up and I need to build a new one. Since my build procedure is out of date I figured this a good time to update and share it. I don't claim to be a Linux guru, but I've worked this out the best I can from various online resources. If you have any suggestions on how to make this more secure, or improve it, etc please let me know. I'm always open to alternatives.

So this is my procedure for build a Syslog server using:
1. Fedora Core 10
2. rsyslog
3. MySQL
4. Apache
5. php-syslog-ng

1. Install Fedora with MySQL and Apache both with PHP support also add in the dev tools so you can compile anything else required. For mine I do not install a GUI only CLI so no Xwindows required unless you want it. Some other tidbits:
    a. Set the time zone to UTC - I do this on all networking equipment and synchronize with NTP
    b. Enable SELinux
    c. When the installation is complete perform a yum update
    d. Static IP address
    e. WebServer
        a. Crypto-Utils
        b. Mod_auth_mysql
        c. Mod_perl
        d. Mod_ssl
        e. Php
        f. Php_mysql
    f. MySQL Database
        a. MyODBC
        b. Mod_auth_mysql
        c. Mysql-devel
        d. Mysql-server
        e. Perl-DBD-MySQL
        f. Php-mysql
2. Now lets secure Linux a bit and setup some basic services
    1. Create a user
        a. Add a user for to login with
        useradd -c "John Doe" jdoe
        passwd jdoe
    2. SSH
        a. SSH into the box and logon as this user.
        b. su - and use the root password
        c. nano -w /etc/ssh/sshd_config
        d. Find '#PermitRootLogin yes' and change it to 'PermitRootLogin no'
        e. Find '#PermitEmptyPasswords no' and change it to 'PermitEmptyPasswords no'
        f. service sshd restart
    3. Turn off unneeded services: netfs, nfslock, acpid, bluetooth, cpuspeed, cups, gpm, haldaemon
        chkconfig --levels 0123456 netfs off
        chkconfig --levels 0123456 nfslock off
        chkconfig --levels 0123456 acpid off
        chkconfig --levels 0123456 bluetooth off
        chkconfig --levels 0123456 cpuspeed off
        chkconfig --levels 0123456 cups off
        chkconfig --levels 0123456 gpm off
        chkconfig --levels 0123456 haldaemon off
    4. Install and configure NTP - if you happened to be running this in a Virtual machine this may not work.
        a. yum install ntp.i386
        b. Optionally edit the ntp.conf file and update with your own NTP server (nano -w /etc/ntp.conf)
        c. ntpdate -u time.nist.gov
        d. service ntpd start
        e. chkconfig --levels 234 ntpd on
        d. service ntpd start
        f. NOTE: if you run into issues because this is a VM. Put this ntpdate -u time.nist.gov into crontab.
            EDITOR=nano
            export EDITOR
            crontab -e
            * 00 * * * ntpdate -u time.nist.gov
        g. You can use ntpstat, ntpdc -p, and ntpq -p to check on the status of ntpd
    5. Update iptables -
        a. nano -w /etc/sysconfig/iptables
            # Firewall configuration written by system-config-securitylevel
            # Manual customization of this file is not recommended.
            *filter
            :INPUT ACCEPT [0:0]
            :FORWARD ACCEPT [0:0]
            :OUTPUT ACCEPT [0:0]
            :RH-Firewall-1-INPUT - [0:0]
            -A INPUT -m state --state ESTABLISHED,RELATED -j ACCEPT
            -A INPUT -p icmp -j ACCEPT
            -A INPUT -i lo -j ACCEPT
            -A INPUT -m state --state NEW -m tcp -p tcp --dport 22 -j ACCEPT
            -A INPUT -m state --state NEW -m udp -p udp --dport 514 -j ACCEPT
            -A INPUT -m state --state NEW -m tcp -p tcp --dport 443 -j ACCEPT
            -A INPUT -j REJECT --reject-with icmp-host-prohibited
            -A FORWARD -j REJECT --reject-with icmp-host-prohibited
            COMMIT
        b. service iptables restart
    6. Configure crontab editor and add updatedb to schedule
        EDITOR=nano
        export EDITOR
        crontab -e
        Add Line: 00 00 * * * /usr/bin/updatedb
        crontab -l
3. Now let's get MySQL going
    1. chkconfig --levels 234 mysqld on
    2. service mysqld start
    3. Change the root password for MySQL
       mysqladmin -u root password "mypassword"
4. Let's download and setup php-syslog-ng
    1. Download php-syslog-ng (as of this post the latest version is 2.9.8m)
        wget http://php-syslog-ng.googlecode.com/files/php-syslog-ng-2.9.8m.tar.gz
        tar -xvzf php-syslog-ng-2.9.8m.tar.gz
    2. Edit the php.ini file
        nano -w /etc/php.ini
        find 'display_errors = Off' and change to 'display_errors = on'
        find 'magic_quotes_gpc = Off' and change to 'magic_quotes_gpc = On'
        find 'max_execution_time = 30' and change to 'max_execution_time = 60'
        find 'memory_limit = 32M' and change to at least 'memory_limit = 128M'
    3. yum install php-gd.i386
    4. Let setup the code in the site now
        mkdir /var/www/html/syslog
        cp -r /root/php-syslog-ng-2.9.8m/html/* /var/www/html/syslog/
        chmod 776 /var/www/html/syslog/config/
        chmod -R 777 /var/www/html/syslog/jpcache/
    5. Start up the web server
        service httpd start
        chkconfig --levels 234 httpd on
    6. Browse to https://<syslog-IP>/syslog
        a. everything should be green except the config.php file should be un-writeable.
        b. Click Next then except the license and click next again.
        c. MySQL database configuration - I won't go into much info hear expect change the password. Once you click next it will create the DB.
        d. Enter a name for you site for example Geek37 PHP-Syslog-NG and click next
        e. In Step 3 the only items I need to change is the Site URL, email address, and password. The site URL needs to be /syslog/. The others are up to you.
        f. Step 4 provides a copy of the configuration which needs to be pasted into config.php.
            nano -w /var/www/html/syslog/config/config.php
        g. Click Install CEMDB
        h. At this point you should get a screen that asks you to "Start Import". If you do great, however if you're like me you received an unexpected error "Request-URI Too Large". If so you need to complete this part manually.
            cd /var/www/html/syslog/install/sql
            gzip -d cemdb.sql.gz
            mysql -uroot -p syslog < cemdb.sql
        i. chmod -R 777 /var/www/html/syslog/config/
        j. chmod 777 /var/www/html/syslog/graph.jpeg
        k. Last bit is to go through the config and do your own tweaking.
            nano -w /var/www/html/syslog/config/config.php
        l. Set seLinux to allow for write access to graph files
            chcon -t httpd_sys_content_rw_t /var/www/html/syslog/graph.jpeg
            chcon -R -t httpd_sys_content_rw_t /var/www/html/syslog/jpcache
    7. At this point if you browse to https://<syslog-IP>/syslog you will be prompted for logon credentials. User is admin and the password is whatever you set it to earlier. Once in you can go to the config page and add users.
5. Now it's time work on rsyslog
    1. Install rsyslog-mysql.i386
        yum install rsyslog-mysql.i386
    2. Install MS True Type Fonts (http://corefonts.sourceforge.net/)
        yum install cabextract.i386
        yum install ttmkfdir.i386
        wget http://dl.atrpms.net/all/chkfontpath-1.10.1-2.fc9.i386.rpm
        nano -w /etc/yum.conf
            gpgcheck=no
        yum localinstall chkfontpath-1.10.1-2.fc9.i386.rpm
        nano -w /etc/yum.conf
            gpgcheck=yes
        wget http://corefonts.sourceforge.net/msttcorefonts-2.0-1.spec
        rpmbuild -bb msttcorefonts-2.0-1.spec
        rpm -ivh $HOME/rpmbuild/RPMS/noarch/msttcorefonts-2.0-1.noarch.rpm
        nano -w /var/www/html/syslog/includes/jpgraph/jpg-config.inc
        Change DEFINE("TTF_DIR","/usr/share/fonts/truetype/msttcorefonts/"); to DEFINE("TTF_DIR","/usr/share/fonts/msttcorefonts/");
    3. Configure rsyslog
        nano -w /etc/rsyslog.conf
        a. Add MySQL support
            Insert a line for the MySQL module $ModLoad MySQL
        b. Add UDP Support
            Change #$ModLoad imudp.so to $ModLoad imudp.so
            Change #$UDPServerRun 514 to $UDPServerRun 514
        c. Add TCP Support if required
            Change #$ModLoad imtcp.so to $ModLoad imtcp.so
            Change #$InputTCPServerRun 514 to $InputTCPServerRun 514
        d. Add a template with a destination of MySQL and then remove all remote logs from the list. These need to be placed before the line "*.info;mail.none;authpriv.none;cron.none" or all your remote logs will go into /var/log/messages as well.
            $template syslog-ng,"insert into logs(host,facility,priority,level,tag,datetime,program,msg) values ('%fromhost-ip%', '%syslogfacility-text%',$
            *.* >127.0.0.1,syslog,root,password;syslog-ng
            :fromhost-ip,!isequal,"127.0.0.1" ~
6. Maintenance Scripts
    1. Copy the script files logrotate.log and reloadcache.php to a location of your choice I use /etc/php-syslog-ng
        mkdir /etc/php-syslog-ng
        cp ~/php-syslog-ng-2.9.8m/scripts/logrotate.php /etc/php-syslog-ng
        cp ~/php-syslog-ng-2.9.8m/scripts/reloadcache.php /etc/php-syslog-ng
        cp ~/php-syslog-ng-2.9.8m/scripts/lpdcache.php /etc/php-syslog-ng/
    2. Set the log retentione period and edit the logrotate.php
        nano -w /var/www/html/syslog/config/config.php
        Change the 30 in define('LOGROTATERETENTION', 30); to the number of days your like to save your logs. Please remember to keep an eye on disk usage.
        nano -w /etc/php-syslog-ng/logrotate.php
        Change include_once "/www/php-syslog-ng/html/includes/common_funcs.php"; to include_once "/var/www/html/syslog/includes/common_funcs.php";
        Change include_once "/www/php-syslog-ng/html/config/config.php"; to include_once "/var/www/html/syslog/config/config.php";
    3. Edit reloadcache.php
        nano -w /etc/php-syslog-ng/reloadcache.php
        Change require_once "/www/php-syslog-ng/html/includes/common_funcs.php"; to require_once "/var/www/html/syslog/includes/common_funcs.php";
        Change require_once "/www/php-syslog-ng/html/config/config.php"; to require_once "/var/www/html/syslog/config/config.php";
    4. Edit lpdcache.php
        nano -w /etc/php-syslog-ng/lpdcache.php
        Change require_once "/www/php-syslog-ng/html/includes/common_funcs.php"; to require_once "/var/www/html/syslog/includes/common_funcs.php";
        Change require_once "/www/php-syslog-ng/html/config/config.php"; to require_once "/var/www/html/syslog/config/config.php";
    5. Edit crontab to execute the scripts automatically
        mkdir /var/log/php-syslog-ng/
        EDITOR=nano
        export EDITOR
        crontab -e
        # PHP-Syslog-NG
        @daily php /etc/php-syslog-ng/logrotate.php >> /var/log/php-syslog-ng/logrotate.log
        @daily find /var/www/html/syslog/jpcache/ -atime 1 -exec rm -f '{}' ';'
        0,5,10,15,20,25,30,35,40,45,50,55 * * * * php /etc/php-syslog-ng/reloadcache.php >> /var/log/php-syslog-ng/reloadcache.log
    5-NOTE: for the log rotate script I prefer to confirm daily that this runs and cleans up the database. Also this report is a good indication of DB health. So I have it emailed to me automatically.
        @daily php /etc/php-syslog-ng/logrotate.php | mail -s "Daily Syslog Log Rotate Report" security@company.com
7. Daily Report Scripts
    1. Create mysql_report report script
        a. nano -w /etc/php-syslog-ng/mysql_report
            #!/bin/bash
            ############################################################################
            ## Script to run MySQL Syslog report                                      ##
            ##                                                                        ##
            ## Created 06.30.2006 FJS                                                 ##
            ##                                                                        ##
            ##                                                                        ##
            ##                                                                        ##
            ############################################################################

            TODAYSDATE=`date '+ %m/%d/%y'`
            echo "********** NCC Syslog Report for $TODAYSDATE **********"
            echo ""
            echo "*************** Syslog Messages per day ***************"
            mysql -uroot --password=password syslog < /etc/php-syslog-ng/mysql_syslog_report1.sql
            echo ""
            echo "************ Syslog Messages per host/day ************"
            mysql -uroot --password=password syslog < /etc/php-syslog-ng/mysql_syslog_report2.sql
            echo ""
            echo "********** Syslog Messages per host/level/day **********"
            mysql -uroot --password=password syslog < /etc/php-syslog-ng/mysql_syslog_report3.sql
            echo "********************** Available Drive Resources **********************"
            df
            echo "*******************************************************"
        b. chmod 775 /etc/php-syslog-ng/mysql_report
    2. Create the mysql_syslog_report.sql file
        a. nano -w /etc/php-syslog-ng/mysql_syslog_report1.sql
            SELECT DATE(datetime) as Date, COUNT(*) as Count
            FROM all_logs
            GROUP BY DATE(datetime)
            ORDER BY DATE(datetime) DESC;
        b. nano -w /etc/php-syslog-ng/mysql_syslog_report2.sql
            SELECT host as Host, DATE(datetime) as Date, COUNT(*) as Count
            FROM all_logs
            GROUP BY host, DATE(datetime)
            ORDER BY DATE(datetime) DESC, Host;
        c. nano -w /etc/php-syslog-ng/mysql_syslog_report3.sql
            select DATE(datetime) as Date, host, level, COUNT(*) as Count
            from all_logs
            GROUP BY host, level, DATE(datetime)
            ORDER BY DATE(datetime) DESC;
    3. Setup crontab to execute and email the report
        EDITOR=nano
        export EDITOR
        crontab -e
        #Execute Daily Email Report
        @daily /etc/php-syslog-ng/mysql_report | mail -s "Daily Syslog Log Report" security@company.com
8. Last but not least start logging. Point all your devices to your new syslog server.

I think that's everything. If you have any recommendations or if you have any issues with this procedure please let me know. One of these days I may make a more formal document with screen shots but for now this is it.

4 comments:

  1. Fred - I just wanted to say "thank you" for taking the time to post this detailed install. I run the site syslog.org and had someone struggling mightily with getting php-syslog-ng to work: http://www.syslog.org/forum/syslog-ng/how-to-customize-php-syslog-ng/msg2788/

    Thanks again.

    Jerry

    ReplyDelete
  2. thank you Fred for posting in detail.

    Saju

    ReplyDelete
  3. This comment has been removed by the author.

    ReplyDelete
  4. Hi Fred Sir, Thank U so much.....

    Tshering
    Bhutan

    ReplyDelete