Exclude certain files when creating a tarball archive

I have noted that my tarball archives are growing really big on my Linux server. They used to be around 50GB and now I am reaching 220GB. It forced me to look in to the problem and I have discovered that some files grew over time especially some logs and temp files that are generated during testing and simulations on my Red Hat Linux ES server.

In order to bring my archive in size down I had to just add ” –exclude” in to my script. You can do this two ways. One is to add “–exlude=’*.wlf”” which will skip all files with the file extension or you create a text file exclude.txt that is more easier to update every time and the script will run without additional touching. The “exclude.txt” file should be at the same directory as is the bash script.

So I have created a the text file with this inside:

*.wlf
/sim

This way every file with file extension .wlf will be skipped as well as all files in directory /sim during my tarball creation. here is the full script I am using to generated my backup archive

#!/bin/bash
##############################################################################################
# This script creates a tarball archive with date when the tarball was created and send email
# once the tarball is created
# if you want skip only one file extension use this
# tar cvzf $dest/$archive_file --exclude='*.wlf' $backup_files
##############################################################################################

backup_files="/data/development"
dest="/backups/develop"

#day=$(date +%A)
day=$(date +"%m-%d-%Y")
hostname=$(hostname -s)
archive_file="$hostname-$day.tar.gz"

echo " Running backup of selected files $backup_files do $dest/$archive_file"
date
echo

tar cvzf $dest/$archive_file -X exclude.txt $backup_files

echo "Backup has been created"
date

ls -lh $dest | mail -s 'DEVELOP-BACKUP has been Compressed' roman@domain.com

This simple create the tarball archive with the host name and date when the archive has been created and send me email upon completion.