Can a Stolen Backup Key Delete Your Backups? Keys and Bucket Locks

On MinIO, a stolen backup key deleted old backups unless it lacked delete rights. A key without them still overwrote objects, and under a lock a delete by version id was refused and the old version stayed. R2 documents no token without delete rights, so I'd assume an R2 key can delete, and I didn't run that.

I ran the key and lock tests in a lab under WSL2, with MinIO in a container as the storage. Nothing ran on real R2, B2, S3, or Hetzner. Claims about those come from their docs and price pages, and I mark them. The keys on this page protect the objects that pg-backup uploads, the script in my off-site backups guide.

What Can a Stolen Server Key Delete?

On MinIO, removing the delete right stops deletion and leaves overwriting allowed. The keys were admin, bucket-rw (list, read, write, and delete on one bucket), put+list, and put-only (trimmed):

operation                          admin          bucket-rw      put+list       put-only
read from another bucket           200            403 AccessDenied 403 AccessDenied 403 AccessDenied
overwrite old backup               200            200            200            200
delete old backup                  204            204            403 AccessDenied 403 AccessDenied

This server key policy ran on MinIO. It keeps read because a put-only key broke rclone copyto:

{"Version":"2012-10-17","Statement":[
 {"Effect":"Allow","Action":["s3:ListBucket"],"Resource":["arn:aws:s3:::shop-backups"]},
 {"Effect":"Allow","Action":["s3:GetObject","s3:PutObject"],"Resource":["arn:aws:s3:::shop-backups/*"]}]}

That key can read every backup, so a stolen server key is also a copy of your data.

Encrypting with age on the server would address that, though a lost key means lost backups, and I didn't wire it into pg-backup or test a lost key. The same key can also add a junk object that looks newest, and my freshness check accepts it. The restore commands in my restore drill guide pick the newest object, so they would pick that junk (trimmed):

OK zz-not-a-backup, 60000000 bytes, 0 s old, 2 objects
junk newest, freshness exit 0
restore step would pick: zz-not-a-backup

A killed multipart rclone upload left parts that the no-delete key couldn't list or remove. An admin key cleared them with rclone backend cleanup offsite:shop-backups -o max-age=1s, while plain rclone cleanup left that young upload in place. offsite is the rclone remote from my off-site backups guide. R2's docs say it expires incomplete uploads after seven days, so on other providers add a rule that aborts them.

What Does a Bucket Lock Change?

The commands below created my MinIO bucket under COMPLIANCE, and they are for MinIO only. Two warnings first.

Compliance warning: on AWS or B2, compliance mode can't be shortened or removed (AWS, B2). I tested neither. The data stays until it expires, so I'd expect to keep paying for it. I'd start with governance mode and a one-day period. On a separate MinIO bucket with that setting, a delete by version id was refused, and only a key with s3:* on the bucket or the admin key could delete, using the bypass header.

Lifecycle warning: the mc ilm rule deletes by age and doesn't check newer objects. rclone delete --min-age has the same weakness. If backups fail for 30 days, the last good one expires, so the failure alerts and the freshness check have to reach you before then.

The rule also keeps running after a server dies, so turn it off at the provider when you start a recovery. My rule never reached its date, so it never fired.

I'd prune at the provider, because a server key that can prune can also delete. The tradeoff: the provider rule is age-only. The prune in pg-backup runs after a verified upload, so failed runs deleted nothing.

Here are the MinIO commands, with my lab's values, not the ones I'd start with. mc is the MinIO client, adm is my mc alias for the MinIO admin user, and shop-backups is my lab's bucket name:

mc mb --ignore-existing --with-lock adm/shop-backups
mc retention set --default COMPLIANCE 7d adm/shop-backups
mc ilm rule add --expire-days 30 --noncurrent-expire-days 1 adm/shop-backups

This table puts the MinIO results under COMPLIANCE next to R2. The R2 column comes from its S3 API page and bucket lock page, and none of this ran on R2. The delete row ran on a separate MinIO bucket with a 1-day default, not the 7-day bucket above:

On R2 I'd try a 7-day lock rule, a 30-day lifecycle rule, and an Object Read & Write token scoped to the one bucket, set up from its bucket lock page, and I didn't run any of that. Cloudflare says locks take precedence, so keep the lifecycle rule longer than the lock.

If that token can delete, the lock protects only the last 7 days of backups from a stolen key. That is my reading of the docs, and I didn't test it.

Which Storage Should I Use, and What Does It Cost?

I have no tested pick among these. I'd choose on key scope and lock behavior, because those decide what a stolen server key can do. These are list prices read on the dates shown (R2, B2, AWS S3, Hetzner), so check them before you commit. The key scope comes from the R2 token page, the B2 key page, and the Hetzner credentials page.

Thirty nightly copies of my 92 MB dump come to about 2.8 GB. That is under the free tiers listed for R2 and B2. Hetzner's base price applies at any size.

Summary

On MinIO, a key without delete rights could not delete, but it could overwrite, add junk that looks newest, and read every backup. Under COMPLIANCE, the lock refused a delete by version id and kept the old version after an overwrite.

The R2 column comes from its docs, the lifecycle rule never fired, and I have no tested pick among the providers. Try your own token and lock rule on your real provider before you count on them, and keep the lifecycle rule longer than the lock.

The full backup script is in my off-site backups guide, and my self-hosting course sets up database backups to storage outside the server.

Thanks for reading.

And stay awesome!