Control after deployment

How to Recover From a Failed Deployment

A deploy that fails should not take the working version with it. Here is what the platform recovers on its own, what you trigger, and the line where recovery stops being possible.

Deployments fail. A dependency changes, a migration is missing, an environment variable is wrong. What matters is whether the failure takes your running app down with it, and how much of the damage you can undo.

This guide covers the four controls that exist for that on Impreza, and is deliberate about their limits. None of them is a substitute for a backup.

What you get

  • A health requirement that makes an unhealthy release count as a failure instead of a success
  • Progress you can read: the named step the deployment is on, and a recorded history
  • Cancellation while the work is still safe to stop
  • Release history and manual rollback to a retained previous release
  • Runtime health reported separately from the result of the last operation

Stop a bad release from being called good

By default, a deployment that starts its containers is treated as done. require_healthy_start changes that: the release only succeeds once your healthcheck_path answers 2xx, with no redirect, on the target port. It applies to the first install too, which is where the silent failures usually hide.

When a first install never becomes healthy, its containers are removed and its volumes are preserved. When a replacement fails, an eligible healthy previous release can be recovered. Requiring health needs agent 0.6.3 or newer, and it needs an explicit health path: there is no useful default for a route that means “ready”.

Read what is actually happening

Two different questions have two different answers, and mixing them up is the most common mistake.

The last operation tells you how the most recent deploy ended. Runtime health tells you what the containers are doing right now, which needs agent 0.6.4 or newer. A deployment can report success while the app is unhealthy, and it can report failure while the previous release keeps serving traffic perfectly.

Progress reporting, from agent 0.6.6 onward, gives you the named step the deployment is on, along with a bounded history of the steps it passed. It is a step, not a live percentage: nothing here estimates a finish time. If the agent restarts mid deploy, the saved result can still be read afterwards, and a guarded recovery path exists for a host reboot.

Cancel, while cancelling still means something

Read the current operation first

Check the command identifier and the cancellation state of the operation you intend to stop. Cancelling blind is how people cancel the wrong deploy.

Cancel queued work

Work that has not started yet is cancelled immediately. This is the clean case.

Request a stop during preparation

Preparation returns requested, not cancelled. The agent stops at a safe point and restores the previous configuration, and only then does the state become terminal. Running cancellation needs agent 0.6.5 or newer.

Poll until it is terminal

Keep reading the operation until it reports cancelled or another final result. Never treat requested as confirmation that the work stopped.

Cancellation is not a fixed time stop, and it has a point of no return

A request stops the deployment at the next safe point, which may not be immediate. Once container replacement or recovery has started, it cannot be cancelled at all. Interrupting a build that is already running is only possible from agent 0.6.12, on supported Ubuntu 24.04 hosts, and only after the server administrator has enabled controlled builds.

Roll back to a previous release

Retained releases are listed with the deployment, each marked with whether rollback is supported. Pick one, confirm, and the platform restores it.

What comes back is the image and the configuration. What does not come back is your database, or any other mutable data. A rollback after a migration that rewrote a table restores old code on top of new data, which is usually worse than the failure you were fixing.

A rollback can also interrupt traffic while it happens, and it is reported as failed when the release has expired or when ports, storage or routing have changed since it was created. Read the deployment history for the real outcome rather than assuming the request succeeded.

Plan the data path before you need it

Take a backup before shipping anything that changes the database, and know which volumes it covers. See app backups and restores. A copy taken while a database is writing is not proof of a consistent database.

Catch it before production

For eligible branches, a temporary preview address lets the change be exercised before it reaches the live deployment. See previewing Git branches over Tor. Build credentials are never inherited by a preview, so a preview that needs them has to be given them deliberately.

The honest summary

The platform protects availability of the previous release. It does not protect your data, it does not guarantee an instant stop, and it does not switch traffic without interruption. Treat health requirements and release history as the seatbelt, and backups as the thing that actually restores what you lost.

Start now

Get an offshore VPS with the agent preselected, read deployment safety and rollback in the docs, or set the health policy while deploying with the Node, Python and PHP recipes.

Ready to build privacy-first?

No KYC, no email required, crypto payment. Deploy an offshore server in minutes, or do it all by chat with the Impreza agent.