Backup & Disaster Recovery

rclone and S3-compatible storage: copy vs sync, dry runs, bandwidth limits and encryption

Configure rclone for an S3-compatible bucket, understand why sync can delete files, and add dry runs, delete limits, bandwidth caps and a crypt remote.

Published
Reading time
7 min read

rclone is a command-line program for moving files between a server and almost any cloud storage, including S3-compatible object storage. It is excellent for pushing website files, exports and archives to a bucket, but one of its commands, sync, deletes files on the destination by design. This guide sets up an S3 remote, explains when to use copy and when to use sync, and adds the safety nets that keep a typo from emptying a bucket.

Install a current version

Check what you are running before you rely on any flag. The latest upstream release at the time of writing is v1.75.2 (9 October 2026), while Debian 12 (bookworm) and Debian 13 (trixie) both package rclone 1.60.1. The rclone install page itself warns that distribution packages are often out of date and offers an install script plus a selfupdate command for installs that are not managed by a package manager:

rclone version
# official install script, as published on rclone.org/install
sudo -v ; curl https://rclone.org/install.sh | sudo bash

If you prefer to stay with the Debian package, keep in mind that the documentation on rclone.org describes the newest release, so test each flag with --help on your version.

Configure an S3-compatible remote

Run rclone config, create a new remote of type s3 and choose the provider that matches yours, or "Any other S3 compatible provider" (provider = Other). The result in ~/.config/rclone/rclone.conf looks like this:

[objstore]
type = s3
provider = Other
env_auth = false
access_key_id = YOUR_ACCESS_KEY
secret_access_key = YOUR_SECRET_KEY
endpoint = https://s3.example.com

For the generic provider rclone uses path-style bucket addresses by default (the force_path_style option defaults to true). If your key is not allowed to create buckets, add no_check_bucket = true (or the flag --s3-no-check-bucket) so rclone does not try. Then test:

chmod 600 ~/.config/rclone/rclone.conf
rclone lsd objstore:
rclone ls objstore:my-bucket --max-depth 1

The config file holds working credentials. Keep it readable only by the account that runs rclone, or add configuration encryption with rclone config and option s (Set configuration password); rclone then needs that password at every start, for example through the RCLONE_CONFIG_PASS variable or --password-command.

copy or sync: the decision that matters

rclone copy transfers new and changed files and never deletes anything from the destination. rclone sync makes the destination identical to the source, which includes deleting files on the destination that are no longer in the source. The rclone documentation puts it plainly: because sync can cause data loss, test it first with --dry-run or --interactive.

  • Use copy when the bucket should accumulate files: exports, archives, database dumps with dated names.
  • Use sync only when you truly want a mirror, and remember that a mirror is not a backup. If a file is deleted or encrypted by ransomware on the server, the next sync faithfully carries that change to the bucket.

Two details often surprise people. rclone always syncs the contents of the source directory, not the directory itself. And if any error occurs during a sync, rclone does not delete files on the destination for that run.

# see what would happen, change nothing
rclone sync /var/www/site objstore:my-bucket/site --dry-run
# confirm each change interactively
rclone sync /var/www/site objstore:my-bucket/site --interactive
# additive copy with live progress
rclone copy /var/backups/db objstore:my-bucket/db -P

Safety nets for sync

--max-delete N stops the run with a fatal error if more than N files would be deleted, which catches the classic mistake of syncing an empty or wrong source directory. --backup-dir moves every file that sync would overwrite or delete into another path on the same remote instead of removing it. Combined with a date, it gives you a simple history of changed files:

rclone sync /var/www/site objstore:my-bucket/current \
  --backup-dir objstore:my-bucket/old/$(date +%F) \
  --max-delete 200 \
  --log-file /var/log/rclone-site.log --log-level INFO

The backup directory must be on the same remote as the destination and must not overlap it, which is why the example uses sibling paths current and old. Old folders accumulate, so clean them up with a lifecycle rule on the bucket or a periodic rclone delete --min-age job tested with --dry-run.

Limit bandwidth

Large uploads can saturate a server's link. --bwlimit takes bytes per second, not bits: --bwlimit 10M means 10 MiB/s. You can set upload and download separately (--bwlimit 10M:off limits only uploads) or give a timetable so transfers slow down during working hours:

rclone copy /srv/archive objstore:my-bucket/archive \
  --bwlimit "08:00,2M 18:00,20M 23:00,off"

Verify what arrived

rclone check compares sizes and hashes between source and destination and reports differences without changing either side. With --one-way it only checks that everything in the source exists and matches in the destination, which suits a copy job whose destination holds older files too.

rclone check /var/backups/db objstore:my-bucket/db --one-way

Encrypt on the client with a crypt remote

A crypt remote wraps another remote and encrypts file contents, and optionally file names, before upload. Create it with rclone config, type crypt, pointing at a path such as objstore:my-bucket/encrypted, and use the new remote name in your commands. Points to understand before you commit data to it:

  • The crypt password is stored in rclone.conf in a lightly obscured form that only resists a casual look. It is not secure unless you also enable configuration encryption.
  • Keep the password (and the optional salt password) somewhere safe off the server. With them you can recreate the remote elsewhere; without them the data cannot be decrypted.
  • You cannot change the password of data that is already encrypted. Changing it in the config simply makes old files unreadable; the only way is to upload everything again through a new crypt remote.
  • rclone check cannot compare hashes through crypt; use rclone cryptcheck instead.

Schedule it

Run the job from a systemd timer or cron as the account that owns the config, with --log-file so you have a record, and alert when the command exits non-zero. Start every new job with a --dry-run pass and read the log before removing the flag.

What you should do

  1. Check rclone version and use a current release if you depend on newer flags.
  2. Use a bucket-scoped key, protect rclone.conf, and consider configuration encryption.
  3. Default to copy; use sync only for real mirrors, always with --dry-run first, --max-delete and preferably --backup-dir.
  4. Add --bwlimit on servers that also serve users.
  5. Verify with rclone check or cryptcheck, and keep crypt passwords in a safe place off the server.
  6. Remember that rclone moves files; for versioned, deduplicated backups with retention, use a backup tool such as restic on top of the same bucket.

Sources

SKYLINE Engineering

@skyline

The engineering team at SKYLINE Industrial Solutions. We publish field-tested guides drawn from real KSA and GCC deployments.

See author profile
SKYLINE engineering services

Need this implemented for you?

Reading is free — building it right takes a team. SKYLINE engineers ship Backup & Disaster Recovery for Aramco vendors, banks, hospitals and government agencies across Saudi Arabia. Talk to us before you start.

Aramco Approved Contractor ISO 9001 · ISO 27001 SAMA CSF aligned NCA ECC ready 247+ KSA clients

Comments

0 total · 0 threads
Be the first to leave a comment.