Docs / Keep it running
Backups & restore tests
Planned design. TractorStack is in development. This page describes how it is designed to work; nothing here is available to install yet. See the roadmap for the status of each part.
In design for stage 1 backups and test restores. Planned for off-site copies from stage 2.
A backup you have never restored is a guess. TractorStack’s design treats the restore as the real product: every night a backup is taken, and every night a copy of it is restored and checked automatically.
What gets backed up
- PostgreSQL, through pgBackRest. pgBackRest takes full and incremental backups, archives the write-ahead log for point-in-time recovery, and can write to S3-compatible repositories. See PostgreSQL.
- RustFS buckets that hold application files, copied to a second location. Backups of PostgreSQL also live in RustFS, so RustFS itself must not be the only copy.
- The configuration repository: the Git repository with OpenTofu, Ansible, Compose and runbooks. With it, a lost host can be rebuilt; without it, you are rebuilding from memory. OpenTofu state is stored in PostgreSQL, so it is covered by the database backups.
Where the copies live
| Copy | Purpose | Stage |
|---|---|---|
| On-site, in RustFS | Fast restores for everyday mistakes | 1 |
| Off-site, for example a Tier III colocation facility | Survives the loss of the building or server room | 2 |
| Offline, on media disconnected from the network | Survives ransomware or an attacker with admin access | Customer choice |
Stage 1 has a single server, so its on-site copy protects against mistakes and corruption, not hardware loss. If the server matters, plan an off-site copy early, even before stage 2.
The nightly test restore
After the backup finishes, the design restores it into a scratch PostgreSQL container that is thrown away afterwards. The checks planned for it are:
- the restore completes and PostgreSQL starts cleanly;
- WAL replay reaches a recent point, so the archive has no gaps;
- a short list of queries returns plausible results, such as row counts on key tables and the newest record timestamp;
- the time taken, so you know how long a real restore would be.
The results go into the one-page status report. A failed restore is treated as an incident, not a warning to read later.
Retention
How long to keep backups is a decision for each customer, not a default we pick for you. It depends on contracts, regulation, how far back mistakes are usually found, and storage cost. A common starting point is a few weeks of daily backups plus monthly copies kept longer, but we agree the numbers with you and write them into the repository so they are visible and reviewed.
RPO and RTO in plain terms
- RPO (recovery point objective): how much recent data you can afford to lose. With continuous WAL archiving, this can be minutes rather than a full day.
- RTO (recovery time objective): how long you can be down while restoring. The nightly test restore measures this for you.
Lower numbers cost more: a streaming replica in stage 2 shortens RTO, and more frequent archiving shortens RPO.
Recovery runbooks
The runbooks/ folder is planned to hold step-by-step guides for the cases that actually happen: restoring one table or database to a point in time, restoring a whole server onto new hardware, recovering a deleted bucket object, and failing over to the replica in stage 2. Each is written to be followed under pressure by someone who didn’t write it, and rehearsed.
Backups to S3 from day one are also one of the Kubernetes-ready rules: later stages are designed to start from these same backups.
Last updated 25 October 2026 · Questions or corrections: sales@infratractor.com