The server access log records all requests processed by the server. The location and content of the access log are controlled by the CustomLog directive. I got bored today and create a simple script with menu so you can easily pull the interesting info from your logs on webs server :-).
[easyazon_link identifier=”1565922255″ locale=”US” tag=”wn0d5-20″] AWK [/easyazon_link] does the job very well as it pull the requested info and you can slice and dice the info to your desired format. Here is basic break down of the log file with awk:
awk ‘{print $1}’ access.log # ip address (%h)
awk ‘{print $2}’ access.log # RFC 1413 identity (%l)
awk ‘{print $3}’ access.log # userid (%u)
awk ‘{print $4,5}’ access.log # date/time (%t)
awk ‘{print $9}’ access.log # status code (%>s)
awk ‘{print $10}’ access.log # size (%b)
awk -F\” ‘{print $2}’ access.log # request line (%r)
awk -F\” ‘{print $4}’ access.log # referer
awk -F\” ‘{print $6}’ access.log # user agent
I do not going to boring you with more deep details around the log files – here is my simple script – self explanatory what you can pull from the menu 🙂
#!/bin/bash
######################################################
# SCRIPT : Analyzing Apache Log file #
# DESCRIPTION: Select option for desired request #
# CREATED by : TechNotesDesk blog 2015 #
######################################################
cd /var/log/apache2
clear
echo Please select a menu item
echo
echo "1) Blank User Agents"
echo "2) Top 10 IPs"
echo "3) Top 10 referrers"
echo "4) Top user-agent"
echo "5) Requests per day"
echo "6) Total unique visitors"
echo "7) Real time Requests"
echo "8) Most popular URLs"
echo "9) Sorted number of visit per IP"
echo "10) Unique visitors this month"
echo
read CHOICE
case $CHOICE in
1) awk -F\" '($6 ~ /^-?$/)' $log| awk '{print $1}' | sort | uniq;;
2) cat access.log | awk '{ print $1 ; }' | sort | uniq -c | sort -n -r | head -n 10;;
3) cat access.log | awk -F\" ' { print $4 } ' | grep -v '-' | grep -v 'http://www.YOURDOMAIN.com' | sort | uniq -c | sort -rn | head -n 10;;
4) cat access.log | awk -F\" ' { print $6 } ' | sort | uniq -c | sort -rn | head -n 10;;
5) awk '{print $4}' access.log | cut -d: -f1 | uniq -c;;
6) cat access.log | awk '{print $1}' | sort | uniq -c | wc -l;;
7) tailf access.log | awk '{ printf("%-15s\t%s\t%s\t%s\n", $1, $6, $9, $7) }';;
8) cat access.log | awk '{ print $7 }' | sort | uniq -c | sort -rn | head -n 25;;
9) cat access.log | awk '{print "requests from " $1}' | sort | uniq -c | sort;;
10) cat access.* | grep `date '+%b/%G'` | awk '{print $1}' | sort | uniq -c | wc -l;;
*) echo You made an invalid selection;;
esac
echo Have a great day!
Simple select option and you will get the output, customized as you wish. Instead of type the line every time you can now select option and get the info you were looking for.
[easyazon_image align=”none” height=”160″ identifier=”1430211598″ locale=”US” src=”http://blog.technotesdesk.com/wp-content/uploads/2016/01/41JKb7O3elL._SL160_.jpg” tag=”wn0d5-20″ width=”130″]