Object storage versioning, lifecycle rules and Object Lock explained for backup buckets

A bucket that holds backups needs more than a place to put files. Three S3 features decide whether you can recover from an accidental delete, a bad sync or a ransomware attack: versioning, lifecycle rules and Object Lock. This guide explains each one using Amazon's S3 documentation, which defines the API, shows the AWS CLI commands to configure them, and points out what to check when your bucket lives on an S3-compatible service instead of AWS.

First, a word on "S3-compatible"

Many object storage services speak the S3 API, but each implements its own subset. Ceph's Object Gateway, which a number of providers build on, publishes a feature table that lists bucket versioning and lifecycle as supported and marks some other features as partial; other platforms publish similar tables. Before you design around any feature below, read your provider's documentation and test it on a throwaway bucket. The AWS CLI works against most of these services when you pass the provider's endpoint:

aws s3api get-bucket-versioning --bucket my-backups \
  --endpoint-url https://s3.example.com

The remaining examples leave out --endpoint-url for readability; add it on every command if you are not using AWS.

Versioning: keep every version, recover from deletes

With versioning enabled, a bucket keeps multiple variants of the same object. Overwriting an object creates a new version, and a simple delete (one that does not name a version ID) does not remove data at all: S3 inserts a delete marker, which becomes the current version and makes the object look deleted. A bucket is in one of three states: unversioned (the default), versioning-enabled or versioning-suspended. Once you enable versioning you can suspend it, but the bucket can never return to unversioned.

aws s3api put-bucket-versioning --bucket my-backups \
  --versioning-configuration Status=Enabled
aws s3api list-object-versions --bucket my-backups --prefix reports/

To undelete an object, delete its delete marker by naming the marker's version ID; the previous version becomes current again:

aws s3api delete-object --bucket my-backups --key reports/q3.pdf \
  --version-id MARKER_VERSION_ID

Be aware that each version is a complete object, not a difference from the previous one, so three versions of a file use three times the storage. Without a cleanup rule, a versioned bucket only ever grows.

Lifecycle rules: decide what happens to old data

A lifecycle configuration is a set of rules the storage service applies to objects for you. There are two kinds of action: transitions, which move objects to another storage class, and expirations, which delete them. Rules apply to existing objects as well as new ones, and they run asynchronously, so expect some delay between an object becoming eligible and the action happening. For a backup bucket three actions are especially useful:

• NoncurrentVersionExpiration permanently removes old versions a set number of days after they stop being current. With NewerNoncurrentVersions you also keep a minimum number of recent versions; both conditions must be exceeded before a version is removed.
• AbortIncompleteMultipartUpload cleans up the parts of uploads that never finished, which otherwise keep using space.
• ExpiredObjectDeleteMarker removes delete markers that no longer have any versions behind them.

{
  "Rules": [
    {
      "ID": "old-versions-and-stale-uploads",
      "Filter": { "Prefix": "" },
      "Status": "Enabled",
      "NoncurrentVersionExpiration": { "NoncurrentDays": 30, "NewerNoncurrentVersions": 5 },
      "AbortIncompleteMultipartUpload": { "DaysAfterInitiation": 7 }
    },
    {
      "ID": "expired-delete-markers",
      "Filter": { "Prefix": "" },
      "Status": "Enabled",
      "Expiration": { "ExpiredObjectDeleteMarker": true }
    }
  ]
}

aws s3api put-bucket-lifecycle-configuration --bucket my-backups \
  --lifecycle-configuration file://lifecycle.json
aws s3api get-bucket-lifecycle-configuration --bucket my-backups

One point from the AWS documentation is easy to miss: on general purpose buckets a bucket policy cannot stop deletions made by a lifecycle rule. Treat lifecycle rules as carefully as you would a delete script, and choose the noncurrent-days value so it is longer than the time it might take you to notice a problem.

Object Lock: write once, read many

Object Lock protects object versions from being overwritten or deleted for a period, using a write-once-read-many model. It works only on buckets with versioning enabled, and once you enable Object Lock on a bucket you cannot disable it or suspend versioning. Protection comes in two forms:

• A retention period with a retain-until date. You can set it per object version or as a default on the bucket.
• A legal hold, which has no expiry and stays until someone with permission removes it.

Retention has two modes:

On AWS the only way to remove a compliance-mode object before its date is to delete the whole account, so start in governance mode, which AWS suggests for testing retention settings, and move to compliance only when you are sure of the period. A default retention rule looks like this:

aws s3api put-object-lock-configuration --bucket my-backups \
  --object-lock-configuration '{ "ObjectLockEnabled": "Enabled", "Rule": { "DefaultRetention": { "Mode": "GOVERNANCE", "Days": 30 }}}'

Locking does not stop new versions or delete markers from being created. A simple delete still returns success and adds a delete marker; a delete that names a locked version is refused with an access-denied error. The locked data therefore stays recoverable even if an attacker with your keys deletes everything they can see.

Combining the three for backups

• Enable versioning so deletes and overwrites are reversible.
• Add a lifecycle rule that expires noncurrent versions after a period longer than your restore window, and cleans up stale multipart uploads.
• Add Object Lock retention if you need protection against someone who has your access keys. Check your backup tool's documentation first: tools that delete or rewrite data during pruning will run into locked versions, and storage use will follow the retention period rather than your tool's policy.

What you should do

1. Confirm in your provider's documentation which of versioning, lifecycle and Object Lock it supports, and test each on a scratch bucket.
2. Enable versioning on every bucket that holds backups or important files, and practise one undelete.
3. Add lifecycle rules for noncurrent versions, delete markers and incomplete multipart uploads, and review them like code.
4. If you adopt Object Lock, start in governance mode with a short default retention, and decide on compliance mode only after a test period.
5. Keep access keys for backup buckets separate from keys used by applications.

Sources

• docs.aws.amazon.com/AmazonS3/latest/userguide/Versioning.html
• docs.aws.amazon.com/AmazonS3/latest/userguide/DeleteMarker.html
• docs.aws.amazon.com/AmazonS3/latest/userguide/ManagingDelMarkers.html
• docs.aws.amazon.com/AmazonS3/latest/userguide/object-lifecycle-mgmt.html
• docs.aws.amazon.com/AmazonS3/latest/userguide/lifecycle-configuration-examples.html
• docs.aws.amazon.com/AmazonS3/latest/userguide/object-lock.html
• docs.aws.amazon.com/AmazonS3/latest/userguide/object-lock-configure.html
• docs.aws.amazon.com/cli/latest/reference/s3api/put-bucket-versioning.html
• docs.aws.amazon.com/cli/latest/reference/s3api/put-bucket-lifecycle-configuration.html
• docs.aws.amazon.com/cli/latest/reference/s3api/put-object-lock-configuration.html
• docs.aws.amazon.com/cli/latest/userguide/cli-configure-endpoints.html
• docs.ceph.com/en/latest/radosgw/s3