Auto-Restart a Railway Service When It Goes Down (Self-Healing, With Guardrails)

Your Railway service died at 2:47 a.m. The fix, applied at 9:15 a.m. after a customer emailed, took eleven seconds: you clicked Restart. Everything between 2:47 and 9:15 was pure, avoidable downtime, because the fix was known, mechanical, and waiting for a human.

This article automates that click. External detection catches the failure Railway's own restart policy can't see, a workflow restarts the service through Railway's API, and guardrails make sure the automation never makes things worse. Demand for this is not hypothetical: Railway's community template marketplace has a popular "RAM Restart" template that exists purely because people keep needing exactly this.

Railway's restart policy catches exits, not hangs

Railway lets you configure a restart policy (ON_FAILURE, ALWAYS), and it works for the failure it's designed for: the process exits. Crash with a non-zero code and Railway brings you back.

But the failures that cause the 9:15 a.m. email are the other kind:

• OOM thrash: the process is alive but spends its life in GC or swap, timing out every request.
• Deadlock: the event loop or thread pool is stuck; the process holds the port and never answers.
• Poisoned state: a wedged connection pool, a corrupt in-memory cache. The app runs, and every response is a 500.

In all three, the process never exits, so the restart policy never fires. From Railway's point of view the service is running. Only an observer outside the process, probing it the way a user would, can tell it's dead.

The pattern: detect outside, act on the platform, verify

The loop has four parts, and you should refuse to ship fewer:

1. Detect: an external HTTP check fails from multiple regions, and an incident opens.
2. Act: a workflow triggers a restart via Railway's API.
3. Verify: after a delay, check whether the incident is still active; escalate to a human if it is.
4. Audit: every action is recorded where the team can see it: executed, failed, or skipped, with the reason.

upti.my ships this as a native Railway integration, so the "act" step is a drag-and-drop node rather than a hand-rolled API call. (If you'd rather hand-roll it, the raw GraphQL is at the end of this article.)

Step 1: connect Railway with a project token

In Railway: Project → Settings → Tokens → create a project token. Project tokens are scoped to a single project and environment, are revocable in Railway's UI, and grant no account-wide access. Paste it into upti.my (Settings → Integrations, or during onboarding). The token is validated against your project, encrypted at rest with AES-256-GCM, never returned by any API, and shown redacted in the UI.

The integration immediately discovers your services (names, public URLs, current deployment status) and offers to import them as HTTP checks. If you haven't set up basic monitoring yet, start with the Railway monitoring guide and come back; the rest of this article assumes a check exists.

Step 2: a check that detects the hang (without false restarts)

An automated restart demands a higher standard of evidence than a Slack ping. Configure the check so a restart can only be triggered by a real failure:

• Tight timeout. A hung service usually manifests as timeouts, not errors. If your p99 is 800ms, a 5s timeout is a failure signal, not noise.
• Multi-node confirmation. Require 2+ regions to agree before the incident opens. One slow probe from one continent must never restart production.
• Body assertion. Assert on a health-endpoint response, not a bare 200, so "up but serving garbage" counts as down. (Why: HTTP 200 is not a health check.)

Step 3: the Railway Self-Heal node

In the workflow builder, the flow is: trigger on incident.created → filter to the monitor(s) for this service → Railway Self-Heal node → notification. The node takes your Railway integration, a service picked from a live dropdown of your project's services, and one of two actions:

• Restart latest deployment. The equivalent of the dashboard's Restart button; right for hangs, OOM thrash, and poisoned state.
• Full redeploy. Rebuilds and redeploys the service instance. Heavier, for when a restart isn't enough.

If you imported your monitors through the integration, upti.my offers to generate this workflow for you, with one filter + heal branch per service and a notification destination of your choice, as an ordinary, editable workflow. Nothing about it is magic; you can open it and see every node.

The guardrails you don't have to build

An uptime tool with restart access to production must be visibly paranoid. These are on by default, not homework:

• Resolved incidents never act. If the incident resolved while the workflow was in flight, the heal is skipped.
• Cooldown window (default 10 minutes): after a heal action, further heals for the same target are suppressed. No restart storms.
• Per-incident action cap (default 3): if three restarts didn't fix it, a fourth won't either; that incident needs a human.
• Full audit trail: every action lands in a remediation log visible in the workflow run details, whether it was executed, failed, or skipped, with the reason. Nothing is silent, in either direction.

Catch the crashes polling can't see: deploy events

An HTTP check polls every minute or so. A service that crash-loops (dies, restarts, dies again) can look "mostly up" to a poller while being unusable. And workers or cron services with no public URL can't be polled at all.

The integration closes this with Railway webhooks: copy the Webhook URL from the integration card and paste it into Railway → Project → Settings → Webhooks. Deployment state changes become first-class incident signals: railway.deployment.crashed opens a high-severity incident, railway.deployment.failed a medium one, and a successful deploy resolves it. Three crashes in ten minutes escalate the incident to critical with a crash-loop note on the timeline.

The elegant part: your self-heal workflow filters on the service, not the signal type. The same heal branch that catches an HTTP-check incident also catches a deploy-crash incident, with zero changes. Between webhook events and external checks, both halves of the failure space are covered, and the cooldown and action cap keep a crash loop from ever becoming a restart loop.

Verify recovery, escalate if it loops

After the Self-Heal node, don't just notify and hope. The verification pattern in the workflow builder is: Delay node (give the service time to come back) → Filter node with the "incident still active" condition → escalation destination (PagerDuty, on-call, or a louder channel). If the restart worked, the check recovers, the incident resolves, and the escalation branch filters out. If it didn't, a human gets paged with full context: what failed, what was tried, and that it didn't work. Add a Deduplicate node so a flapping service doesn't multiply notifications.

Don't fight your own deploys

A deploy briefly takes the service down; your monitor sees downtime; your workflow restarts the deployment you're in the middle of shipping. Prevent this with maintenance windows in CI:

- name: Open maintenance window
  id: maint
  run: |
    UUID=$(uptimyctl maintenances create \
      --start-at "$(date -u +%Y-%m-%dT%H:%M:%SZ)" \
      --finish-at "$(date -u -d '+15 minutes' +%Y-%m-%dT%H:%M:%SZ)" \
      --description "deploy ${{ github.sha }}" | jq -r '.uuid')
    echo "uuid=$UUID" >> "$GITHUB_OUTPUT"

- name: Deploy to Railway
  run: railway up --service api-server

- name: Close maintenance window
  if: always()
  run: uptimyctl maintenances resolve ${{ steps.maint.outputs.uuid }}

During the window, incidents are suppressed and no heal fires. See maintenance docs and uptimyctl docs.

Appendix: the DIY way (raw Railway GraphQL)

Prefer to own the plumbing? Railway's public GraphQL API at backboard.railway.com/graphql/v2 accepts a Project-Access-Token header, and any workflow's Custom Webhook node (or your own script) can call it. Find the latest deployment, then restart it:

# 1. Get the latest deployment for the service
query LatestDeployment($projectId: String!, $environmentId: String!, $serviceId: String!) {
  deployments(
    first: 1
    input: { projectId: $projectId, environmentId: $environmentId, serviceId: $serviceId }
  ) {
    edges { node { id status } }
  }
}

# 2. Restart it
mutation Restart($id: String!) {
  deploymentRestart(id: $id)
}

It works. It's the same API surface the Self-Heal node uses. What you're signing up to build yourself is everything around the call: the cooldown, the action cap, the resolved-incident refusal, the audit trail, and keeping up with an unversioned API. That surrounding machinery, not the mutation, is the actual product of this article.

If you control the box: webhook recovery actions

On a VPS or your own hardware, the same philosophy still applies: the workflow's recovery step calls a small authenticated remediation endpoint you expose (one that runs docker restart or systemctl restart behind auth), with the same cooldowns and attempt caps guarding it. Keep the healthcheck verification step after the action so a restart that didn't fix anything escalates to a human. See the workflow docs and the practical self-healing guide.

The 2:47 a.m. failure will happen again. The only question is whether the fix takes eleven seconds or six and a half hours. Build the workflow on the 14-day Growth trial. Detection and the status page stay free either way.