Exclude Files and Directories from rsync

rsync is my preferred tool for creating backups from remote machines. It's fast, and convenient and the incremental backups save time, disk space and bandwidth.

You may encounter a situation where you have to copy most of the content but also exclude some files and directories while taking backup with rsync.

Thankfully, rsync allows you to exclude files and directories of your choice by adding the exclude flag to your rsync command:

--exclude 'dir_name'

Let me go into detail with a practical example where I encountered this need to exclude a particular directory from rsync backup.

Exclude directory from rsync

I use a self-hosted Ghost CMS instance on Digital Ocean for the Linux Handbook website.

So, I have automated backup setups at the server level. Apart from that, I make weekly backups of my Ghost setup and save it to my local machine.

Here's the content of the Ghost CMS directory. I want to use rsync to make backups for all the folders except the logs directory. I guess it makes sense to exclude logs, right?

Now, here's what I use for copying all the directories except logs:

rsync -av --partial --inplace --append --exclude 'logs' ghostlinuxhandbook:/var/www/ghost/content/ .

I provided rsync the entire /var/www/ghost/content directory but I asked it to exclude the logs.

As you can see in the screenshot below, the logs directory has not been copied.

If you are wondering about other flags and options, their combination is used for delta updates. Here's their explanation:

That's what I use for making the backup of my Ghost instance with rsync while excluding the logs directory.

There can be more scenarios for excluding files and directories. Let me quickly share them.

Exclude multiple directories from rsync

If you want to exclude multiple directories from rsync, you can specify them one by one with the exclude flag:

rsync -av --exclude --exclude 'dir1' --exclude 'dir2' source_dir destination_dir

The above is a very generic example of using rsync to copy the contents of the source_dir to destination_dir while excluding dir1 and dir2 subdirectories of the source_dir. This happens on the local machine but you can use the same multiple exclude flags for the directories you want to exclude.

-v is for the verbose mode that gives you additional information on file transfers.

Exclude files from rsync

Similarly, you can specify if there are files to exclude from rsync:

rsync -av --exclude --exclude 'file1' --exclude 'file2' source_dir destination_dir

You can even mix and match files and directories the same way:

rsync -av --exclude --exclude 'file1' --exclude 'dir2' source_dir destination_dir

Exclude files or directories based on naming patterns

It's okay to provide directory and file names if there are a few. But if you have a huge list of files or directories, then it could be tiresome.

The good thing is that if their names match a pattern, you can use it in the exclusion.

Let's say you want to exclude all the files and directories that start with log; you do this:

rsync -av --exclude 'log*' source_dir destination_dir

Exclude files of specific types

The wildcard pattern is quite useful for excluding multiple files of certain patterns.

Let's say, you want to exclude all the files that end with .gz and .zip. Here's what you can do:

rsync -av --exclude '*.gz' --exclude '*.zip' source_dir destination_dir

Exclude multiple files and directories from a list

If you need to exclude a number of files and directories based on their name, pattern, type, etc, typing them all could be very difficult.

Instead, you can play smart and create a new text file and enter the files and directories to exclude on individual lines. You can surely mention types and patterns.

Let's say you have this ex-list.txt file containing this:

data.txt
*.gz
dir3
dir4
log*
*.zip

The above will exclude the data.txt, files ending in .gz extension, directories dir3 and dir4, files and directories starting with log and files ending with .zip extension.

Now you can provide it to rsync using the --exclude-from flag:

rsync -av --exclude-from={'ex-list.txt'} source_dir destination_dir

And thus, you can exclude multiple files and directories from rsync easily.

More on rsync

If you are interested in learning about other usage of rsync command, this tutorial will help you out.

I hope you find this tip useful and include it in your Linux command arsenal.