Pfsense Host listing to bash auto-complete

Setup a daily cron tab as root (sudo crontab -e) to login to pfsense and grab your listing of hosts.

@daily /usr/bin/scp root@firewall.com:/etc/hosts /dev/stdout | grep ^10. | awk ‘{print $2}’ | tr A-Z a-z > /data/network_devices.txt

Next lets edit our personal bashrc file to perform host name auto-completing using the list of host names in the file:

vi .bashrc

Add the following at the bottom of the file:

#Perform auto-complete for the following commands:
complete -W “$(cat /data/network_devices.txt;)” telnet
complete -W “$(cat /data/network_devices.txt;)” ssh
complete -W “$(cat /data/network_devices.txt;)” ping
complete -W “$(cat /data/network_devices.txt;)” traceroute
complete -W “$(cat /data/network_devices.txt;)” dig
complete -W “$(cat /data/network_devices.txt;)” scp

#Optional – create an alias to update all your screen sessions:
alias update-screen=”screen -X at # stuff \”source ~/.bashrc\n\””

Lastly, change to root, generate ssh keys if you don’t already have them created, copy the keys up to your pfsense firewall

sudo -s -H -u root
ssh-keygen
ssh-copy-id root@firewall.com

Reload your personal bashrc file and you are all set:
source ~/.bashrc

ASA syslog parsing…

The ASA syslog format can be a bit of a pain to parse, especially when you want a concise list of the access a host has been granted or denied via the firewall. The original log format looks like this:

00:00:00 local4.info %ASA-6-106100: access-list inside permitted tcp Inside/10.1.1.1(1024) -> Outside/10.2.2.42(80) hit-cnt 1 first hit [0x62c4905, 0x0]

Lets clean this up a bit with the following command, for this example we only care about the permitted access across the inside interface

$grep access-list\ inside\ permitted <SYSLOG FILE>.log | sed -e 's/\// /g' | sed -e 's/(.*>//' | awk '{print $7,$8,$9,$10,$11}'

This will result in the following:

tcp Inside 10.1.1.1 Outside 10.2.2.42(80)

Obviously you would be left with a few thousand entries like the one above, now you can add a unique sort to the mix and clean things up, the whole thing will look like this:

$ grep access-list\ inside\ permitted <SYSLOG FILE>.log | sed -e 's/\// /g' | sed -e 's/(.*>//' | awk '{print $7,$8,$9,$10,$11}' | sort | uniq -c | sort -nr

Find all log files in various directories and grep them….

My central syslog server creates a folder hierarchy as follows:

/data/logs/[system name]/month/day.log

Sometime I need to grep multiple logs across several specific devices.  Using find, grep and xargs we can accomplish this as follows:

find *server* -type f | grep 08/03.log | xargs grep interesting_traffic

The above will return a list of all the files within any folder containing the phrase `server`, then it filters out the logs for the date 08/03 and lastly it greps them for the phrase `interesting_traffic`.  Easy!