Home > File & Directory Management > rsync-exclude

rsync-exclude: Excluding Specific Files/Directories During File Synchronization

`rsync` is a powerful tool used for efficient file synchronization between local and remote systems. Specifically, the `--exclude` and `--exclude-from` options allow you to exclude specific files, directories, or items matching patterns from the synchronization target. This guide focuses on `rsync`'s exclusion capabilities, which are highly useful for preventing unnecessary file transfers and for backing up or copying only specific data.

Overview

The exclusion feature of `rsync` is crucial for complex file synchronization scenarios. By excluding specific log files, temporary files, version control system metadata (.git, .svn), etc., from the synchronization target, you can reduce transfer times, save destination space, and avoid copying unnecessary data.

Key Features

The core features provided by `rsync`'s exclusion functionality.

  • **Pattern-based Exclusion**: Exclude files or directories based on their names or paths using wildcard (glob) patterns.
  • **Exclusion from File List**: Manage exclusion patterns by writing them in a separate file.
  • **Flexible Rule Application**: Define complex include/exclude rules in combination with the `--include` option.
  • **Performance Optimization**: Improve synchronization speed by reducing unnecessary file transfers.

Key Options

The essential options used in the `rsync` command to exclude files and directories.

Specifying Exclusion Rules

Other Useful Options

Generated command:

Try combining the commands.

Description:

`rsync-exclude` Executes the command.

Combine the above options to virtually execute commands with AI.

Usage Examples

Practical examples of using `rsync`'s exclusion options.

Exclude Specific File Extensions

rsync -av --exclude='*.log' --exclude='*.tmp' /path/to/source/ /path/to/destination/

Synchronizes files from the source directory to the destination directory, excluding `.log` and `.tmp` files.

Exclude Specific Directories

rsync -av --exclude='node_modules/' --exclude='.git/' /path/to/source/ /path/to/destination/

Synchronizes files from the source directory, excluding the `node_modules` and `.git` directories.

Using an Exclusion List File

rsync -av --exclude-from=exclude_list.txt /path/to/source/ /path/to/destination/

Excludes items from the synchronization target using patterns defined in the `exclude_list.txt` file. **Example `exclude_list.txt` content:** ``` *.bak /temp_files/ logs/ ```

Include Only Specific Files, Exclude Others

rsync -av --include='*.conf' --exclude='*' /path/to/source/ /path/to/destination/

Excludes all files by default and includes only files with the `.conf` extension for synchronization. The order of `--include` and `--exclude` is important.

Exclude Specific Files When Synchronizing to a Remote Server

rsync -avz --exclude='.DS_Store' --exclude='cache/' /path/to/local/user@remote_host:/path/to/remote/

Synchronizes the contents of a local directory to a remote server, excluding `.DS_Store` files and the `cache/` directory.

Tips & Precautions

Tips and precautions for effectively using `rsync`'s exclusion options and avoiding potential issues.

Understanding Pattern Matching Rules

Exclusion patterns operate according to `rsync`'s filter rules.

  • **Wildcards**: `*` matches any sequence of characters, and `?` matches a single character. `**` matches across directory boundaries.
  • **Paths**: A pattern starting with `/` is relative to the root of the source directory. A pattern ending with `/` matches only directories.
  • **Relative Paths**: If a pattern does not contain `/`, it matches at all directory levels. For example, `--exclude='temp'` excludes both `/path/to/source/temp` and `/path/to/source/subdir/temp`.

Importance of Option Order

The `--include` and `--exclude` options are processed in the order they appear on the command line. Therefore, to include specific files while excluding the rest, you should use `--include` first, followed by `--exclude='*'`.

  • `--include='*.txt' --exclude='*'` (Includes all `.txt` files, excludes the rest)
  • `--exclude='*' --include='*.txt'` (Excludes all files, including `.txt` files – unintended result)

Utilizing Test Runs (`--dry-run`)

Always use the `-n` or `--dry-run` option before synchronizing important data to preview the expected behavior. This is highly effective in preventing unintended file exclusions or inclusions.

  • `rsync -avn --exclude='*.log' /path/to/source/ /path/to/destination/`

Using `-vv` Option for Debugging

When exclusion rules are not working as expected, adding the `-vv` (verbose) option provides detailed information about which files `rsync` skips and by which rules they were excluded, aiding in debugging.

  • `rsync -avv --exclude='*.bak' /path/to/source/ /path/to/destination/`

Same category commands