Folder Sync Tool for Amazon S3

Folder Sync Tool Overview

CS Browser includes a Folder Sync Tool that allows you to upload or download only new and changed files, significantly optimizing bandwidth usage and saving time when performing backups to Amazon S3.

The tool currently supports one-way synchronization (also known as mirroring). You can define the Source and Destination, which can be either a local folder or a bucket/folder on Amazon S3.

The smallest unit of synchronization is a file. If any part of a file changes in the source folder, the entire file will be transferred to the destination folder.

CS Browser compares files based on size and modification date. Alternatively, you can use file hash comparison, which is slower but detects file changes even if the modification date is incorrect.

Saved Sync Jobs can also be listed and started from the command line. See the run-sync-job command for unattended synchronization scenarios.

Working with the Folder Sync Tool

To open the Folder Sync Tool

Click Tools → Folder Sync Tool..

The Folder Sync Tool window will open.

To configure a new Sync Job

1. Click 1 to choose the source folder and 2 to choose the destination folder.

   The Select Folder Type dialog will open:

   

2. Select the folder type by clicking the corresponding button.

   You will see the Browse for Folder dialog for selecting an Amazon S3 folder:

   

   Or the system Select Folder dialog for a local folder:

   

3. Once you have defined the source and destination folders, click the Analyze button to compare the folders.

   

   CS Browser will compare the selected folders. During analysis, Sync Exclusion Rules are applied before synchronization actions are prepared. When analysis is complete, the New, Changed, Deleted, and All switches become available. Excluded appears when files or folders have been excluded.

   

To work with comparison results

Use the result switches to review New, Changed, Deleted, Excluded, or All items. For excluded files and folders, the Reason column shows which rule excluded the item.

You can right-click a result item to open it, open the containing folder, create an exclusion rule from the selected item, or delete selected items from the source, the destination, or both sides.

After deleting files from the source or destination, CS Browser refreshes the comparison and recalculates the pending actions.

The Exclude from Sync submenu offers common presets, such as excluding the selected file, files with the same extension, files in the current folder, or files matching size and date conditions. Choose Custom Rule... to define the rule manually. You can also right-click the File column header to sort results by file name or by file extension.

4. Click Synchronize to add the required transfers and deletions to the Tasks queue.

   

Sync Job Options

Each Sync Job has a set of options that define its behavior. To configure them, click Options on the toolbar.

The Sync Job Options dialog will open:

You can choose which types of modifications to include in synchronization:

• Include only new files - Only new files will be synchronized; changed files will be ignored.
• Include only changed files - Only changed files will be synchronized; new files will be ignored.
• Include both new and changed files - Both new and changed files will be synchronized.

Propagate deletions - If this option is enabled, files deleted from the source folder will also be deleted from the destination folder.

Use hashes to compare files - CS Browser will analyze file contents by calculating hashes for local files and comparing them to hashes stored in remote file metadata (remote files are not downloaded). This method is slower than comparing modification dates but is useful when those dates are unreliable.

Use metadata cache file - If this option is enabled, CS Browser saves file metadata to a metadata cache file to reduce HEAD requests and speed up subsequent synchronizations. If it is disabled, no metadata cache file is created or updated.

Keep metadata cache locally - When Use metadata cache file is enabled, this option stores the cache file in the configured temporary folder instead of uploading it to the Amazon S3 bucket.

Sync Exclusion Rules

Sync Exclusion Rules let you skip files and folders that should not be included in synchronization. Excluded items are still visible in the comparison results, so you can review what was skipped and why.

To open Sync Exclusion Rules

Click the Exclusion Rules... link on the Folder Sync Tool toolbar.

The Sync Exclusion Rules dialog will open:

Rules and conditions

Each rule can contain one or more conditions. Use All conditions when every condition must match, or Any condition when matching a single condition is enough. Rules are evaluated independently: if any enabled rule matches an item, that item is excluded.

Click Add to create a new rule, or Edit to change the selected rule.

A condition can check one of the following file properties:

• File/folder name - matches only the item name, for example *.png.
• Relative path - matches the path relative to the sync root, for example Photos/**.
• File size - compares the file size.
• Last modified - compares the modification date and time.
• Item type - matches files or folders.

For File/folder name and Relative path, wildcards are supported. Use * inside one folder level and ** across folder levels. The Matches any operator accepts several wildcard patterns separated with semicolons, for example *.png;*.jpg;*.gif.

Example: exclude small image files

To exclude PNG, JPG, and GIF files smaller than 10 KB in any Images folder, create a rule with All conditions and add these conditions:

• Item type - Equals - File
• File size - Less than - 10 KB
• Relative path - Matches any - Images/**;**/Images/**
• File/folder name - Matches any - *.png;*.jpg;*.gif

Use Images/** to match a folder named Images directly under the sync root, and **/Images/** to match Images folders deeper in the tree.

Additional rule operations

Right-click the rules list to duplicate rules, move selected rules up or down, and import or export rules. The same commands are also available from the More menu.

Managing Sync Jobs

To add a new Sync Job

Click Add new job next to the last Sync Job tab:

A new Sync Job tab will appear:

To rename a Sync Job

Right-click the tab header of the Sync Job you want to rename and choose Rename job.. from the context menu.

The Rename Sync Job dialog will open:

Enter a new name for the Sync Job and click OK.

To delete a Sync Job

Right-click the tab header of the Sync Job you want to delete and choose Delete job.. from the context menu.

A confirmation dialog will appear:

Click Yes to delete the Sync Job, or No to cancel.