Skip to end of metadata
Go to start of metadata

Overview

Site Manager can automatically synchronize all backups stored in a Repository to a remote server or disk on a daily basis. This can be used to efficiently ensure that offsite replication of backup data is performed. The Site Manager server will manage the stopping of backups on the server and ensuring that the synchronization process has a consistent view of the repository. The synchronization process will also manage failed and partially uploaded files and ensure the remote copy of the repository is always in a consistent state.

This feature can be accessed through the 'Active Backups' page under the 'Backups' heading and is configured on a per-repository basis:

Configuring Remote Synchronization

The 'Active Backups' page has an entry for each repository which will either display information on the remote synchronization configuration or offer the option of creating a new remote synchronization (this excludes local repositories, which do not support remote synchronization). Clicking 'New Activity' or 'Configure Synchronization' will display the synchronization setup wizard. The steps of this wizard are described below:

Stage 1 - Activity Type

In the first stage, the type of activity and repository are chosen. For remote synchronization, this should always be 'Schedule Synchronization'. This stage may be skipped depending on how the wizard is launched.

Stage 2 - Synchronization Type

In the second stage of configuration, the type of remote sync is chosen. The types available are:

Synchronization TypeDescription
Windows Network Share

Synchronization to any remote server which can communicate as a SMB/CIFS share of the form \\servername\share

Local PathSynchronization with any file path local to the Site Manager server. This might be a folder, a local USB disk or any other path which can be read and written to as a folder
File TransferSynchronization to a server running SFTP, FTP or FTPS
Amazon S3Synchronization to a Amazon S3 server or other compatible store

The next step will change depending on which synchronization type is chosen.

Stage 3 - Connection Details

The third stage of configuration includes all the options specific to configuration type. This includes the path to the remote server or folder and any authentication details. The different remote synchronization types take the following details:

Synchronization TypeAvailable Options
Windows Network ShareNetwork share path in UNC format (\\server\share\folder, backslash separated), username, password, and domain. If a domain is entered as part of the username (in DOMAIN\User or [email protected] form), the domain field is disabled.
Local PathLocal path in Windows path format (backslash separated) with optional username, password, and domain. If these are omitted, the path is accessed using the SYSTEM account of the Site Manager server.
File Transfer

Type of protocol (FTP, FTPS or SFTP in the URI dropdown), Path to the server in URI format (sftp://example.com/home/backups/macrium, omitting the leading sftp:// and using forward slash separators), username and password for the remote server.

A custom port may also be supplied. 

SFTP Paths

When configuring an SFTP server, include the full path to the folder to sync to in the server path, NOT the path relative to the user's home folder. For example, when connecting to a server as user, to sync to the user's home folder the full path must be specified e.g. example.com/home/user/macrium not example.com/macrium

This may not apply to all SFTP servers or to FTP servers.

Amazon S3
  • S3 Service - Amazon, Wasabi, Backblaze, or custom for full configuration control.
  • Endpoint - URL per the service provider's specification, required only for custom configurations, protocol type (HTTPS or HTTP) with URL omitting the leading protocol type.
  • Bucket - Active bucket as configured within the service provider's console.
  • Region - The selected buckets' region identifier per the service provider's specification.
  • Concurrent Buffers - Number of buffers to use for simultaneous upload of a file. More buffers can improve upload speeds on faster internet connections. Warning, over saturation of the internet connection may lead to extensive timeouts.
  • Root Path - Location within the bucket to upload remote sync files to, should be forward slash separated.
  • Authentication - Access key and secret key as configured within the service provider's console.

Further information can be found in this article

Stage 4 - Configuration

The final stage configures options that apply to all synchronization types. The options are as follows:

OptionDescription
Custom NameThe custom name is shown in the UI to identify the sync. If left blank, the server/local path of the sync will be used instead.
Folder Name

The folder name set here will be created on the server/remote path to store the synchronized backups.

The path configured as part of the network or server path must exist on the remote server or the sync process will generate an error, however, Site Manager will create the folder set as the folder name if it does not already exist.

For example, if remote sync is configured with a server of sftp://192.168.0.1/home/backups and a folder name of SiteManager1, the sync process will store image files in /home/backups/SiteManager1, creating the SiteManager1 folder if necessary.

Multiple repositories can sync to the same server/network path so long as the folder name is different for each one.

Synchronization Window - TimesThe synchronization window controls what time of day the sync will run. When the start time is reached, Site Manager will stop scheduling backups on the repository, wait for any existing backups to finish then run the sync. After the sync has finished, backups resume. If the sync can't be started before the sync end time is reached, the sync attempt is abandoned until the following day. This option allows some control over when high internet bandwidth options like sync get run.
Synchronization Window - FrequencyThe sync process can be limited to only happening on some days of the week by selecting the desired days here.
Options - Allow Backups During Remote Sync

If this option is selected, backups to the repository will be allowed to continue during the synchronization process, with some limitations. 

  1. The remote sync can only start when no backups are running - this is so that there is a consistent, point in time view of which backups exist
  2. The sync process will find changes and data to upload and sort them by computer
  3. Changes are uploaded for the computer with the smallest amount of data to upload
  4. The computer from step 3 is 'unlocked' and backups for this computer may now start
  5. Steps 3 and 4 are repeated until all data is uploaded
  6. The change is committed in the remote location. Before this step, if the sync fails, the changes can be rolled back to the previous repository state
Enable per folder upload

Uploaded backups are committed to the remote location immediately after new data has been uploaded for each folder. A folder in the repository corresponds to the backups for an individual computer and backup definition. This option reduces the impact of upload failures by more frequently checkpointing so that less work is required to recover a consistent checkpoint and continue the sync on reupload. It can also reduce the amount of temporary storage space on the remote sync target as each checkpoint will flush old files that are no longer needed.

When using this option, the remote store will always remain in a state where it can be rolled back to the last checkpoint should a connection be interrupted. This is generally not an issue - if the sync is retried after interruption, the sync will continue from the checkpoint until completion, but if the sync cannot be retried (due to repository disk failure for example), some folders in the remote sync will have the pre-sync contents and some post-sync contents. If there is a requirement that the backups for two computers remain in close sync (e.g. a replication set of servers), this option may be inappropriate. 

Finishing the Wizard

When finishing the wizard, a connection test is performed by the server and any errors are highlighted to be fixed:

Viewing Remote Synchronization Status

Current Server Status

Once  Remote Synchronization has been configured, the Remote Synchronization activity in the Scheduled Activity page will display the following status:

The 'Run Now' option allows a sync to be started manually. This manually started sync will wait for any active backups in the repository to finish before starting and will prevent any backups from starting until the sync is completed.

Last Synchronization Results

This section shows the results of the last sync that ran, including whether it succeeded, any error messages, the amount of data uploaded, and which files were affected.

Viewing a Synchronization in Progress

Upcoming syncs are shown in the forecast of the repository and active syncs are shown in the 'Activity' widget on the 'Dashboard' page just as for backups

Common Issues and Fixes

Synchronization Failures

If a sync fails part way through the upload process, the next sync will roll the remote server back to its previous state before starting - there is no need to do anything to repair the remote sync repository state, this will happen automatically on the next sync. Communications issues are handled by internally retrying the upload for up to 10 minutes if a disconnection from the server happens.

Full Backup Files are Synchronized When Creating Incremental Backups

The remote synchronization process uploads any new or changed backups. As this works at a file level, a change to a full backup caused by the 'Create Synthetic Fulls' option in the backup schedule will cause the full image file to be re-uploaded. It is recommended to avoid this option if the amount of data uploaded is a problem.

Remote Server ID mismatch

Sometimes a remote sync connection or sync attempt will fail with the error 'Remote server ID does not match this repository'. This error means that the local repository's unique ID does not match the one recorded on the server for this sync. This can have many causes:

    • An error on the Repository - If the ID file is deleted from the repository, a new ID will be generated. If this happens the remote sync is stopped to prevent local repository data with errors from overwriting the remote copy.
    • Multiple Repositories syncing to the same server - if multiple repositories sync to the same server and path, the one that connects first and writes their unique ID file to the server will own it and sync to it. The other repository will generate an error instead. This prevents multiple repositories from overwriting each other's data.
    • Server space is being reused for a different Repository - If a Site Manager server is moved or the remote sync destination has old Site Manager sync data, there may be an ID error generated because the remote sync server still has the ID from the old repository.

If the ID mismatch has been caused by old data or is otherwise not relevant, the repository can write a new ID file to the remote server and take it over by using the 'Reinitialize' option. This will update the ID file and allow the remote sync to proceed. 

  • No labels