Blog
200 OK Is Not a Health Check
- software-engineering
- laravel
- typescript
- reliability
- devops
The dashboard is green. Every server says healthy. Users are getting errors on every page.
That happens when the health check is a route that returns 200 and nothing else. It proves the web server is running. It doesn't prove the app can do anything.
A health check that can't fail isn't a health check. It's a light that's always on.
Two different questions
Most setups ask one question when they actually need two.
Is it alive? If not, restart it. This is liveness.
Can it serve traffic right now? If not, stop sending it requests until it can. This is readiness.
Kubernetes has a separate probe for each. Load balancers usually just have one health check per target group, and an unhealthy target gets pulled out of rotation. Either way, the thing calling your endpoint is going to act on the answer. So the answer had better mean something.
Liveness should be boring
A liveness check should only fail when restarting the process would actually fix it.
Database down? Restarting your app won't bring it back. If liveness checks the database, a database blip turns into every container restarting at once, and then everything comes back cold at the same moment. That's a worse outage than the one you started with.
So liveness stays shallow. Does the process respond? Fine.
app.get('/live', (_req, res) => {
res.sendStatus(200);
});Laravel 11 and up ships one for you. In bootstrap/app.php:
->withRouting(
web: __DIR__.'/../routes/web.php',
commands: __DIR__.'/../routes/console.php',
health: '/up',
)/up returns 200 if the app boots, and 500 if it doesn't. Laravel also fires a DiagnosingHealth event on that route, and if a listener throws, it returns 500. That's handy. It's also the trap: hang a database check on /up and point liveness at it, and you're back to restarting containers because MySQL is having a moment.
Readiness checks the stuff you actually need
Readiness is where the real checks go. Can this instance reach the database? The cache? Whatever it needs to handle a normal request?
use Illuminate\Support\Facades\Cache;
use Illuminate\Support\Facades\DB;
Route::get('/ready', function () {
$checks = [
'database' => rescue(fn () => (bool) DB::select('select 1'), false, report: false),
'cache' => rescue(fn () => Cache::put('ready-check', 1, 10), false, report: false),
];
$ok = ! in_array(false, $checks, true);
return response()->json(['ok' => $ok, 'checks' => $checks], $ok ? 200 : 503);
});rescue() runs the closure and gives you the fallback if it throws. So every check comes back as a plain true or false. A 503 when anything fails tells the load balancer to stop sending traffic here for now.
Same idea in Node. The one thing to add is a timeout on each check, because a health check that hangs is as useless as one that lies:
async function check(name: string, fn: () => Promise<unknown>, ms = 1000) {
try {
await Promise.race([
fn(),
new Promise((_, reject) => setTimeout(() => reject(new Error('timeout')), ms)),
]);
return [name, true] as const;
} catch {
return [name, false] as const;
}
}
app.get('/ready', async (_req, res) => {
const results = await Promise.all([
check('database', () => pool.query('SELECT 1')),
check('redis', () => redis.ping()),
]);
const ok = results.every(([, passed]) => passed);
res.status(ok ? 200 : 503).json({ ok, checks: Object.fromEntries(results) });
});Run them in parallel, cap each one, and report which one failed. Future you, staring at a 503 at night, will want the name.
Don't let Stripe take you offline
This is the one people learn the hard way.
If readiness checks a third-party API and that API goes down, every instance fails readiness at the same time. The load balancer pulls all of them. Now you're completely down because a dependency was partly down.
Readiness should check what's yours and required: your database, your cache, your queue connection. For outside services, degrade. Show a "payments are delayed" message, queue the work, alert someone. Don't fail the whole box over something restarting or rerouting can't fix.
Workers need a pulse too
Your web servers can be perfectly healthy while the queue worker died three hours ago. Nothing in the request path notices. The jobs just pile up.
A cheap fix is a heartbeat. Schedule a tiny job every minute, and have it write a timestamp when a worker picks it up:
// routes/console.php
Schedule::job(new Heartbeat)->everyMinute();
// app/Jobs/Heartbeat.php
class Heartbeat implements ShouldQueue
{
use Queueable;
public function handle(): void
{
Cache::put('heartbeat:queue', now()->getTimestamp());
}
}If that timestamp is fresh, the scheduler ran and a worker processed the job. If it's three minutes old, something stopped.
$last = Cache::get('heartbeat:queue');
$workersOk = $last !== null && now()->getTimestamp() - $last < 180;Put that behind its own endpoint for your uptime monitor, not in web readiness. A stuck worker is a reason to alert someone, not a reason to pull healthy web servers out of rotation.
Laravel also has php artisan queue:monitor, which fires a QueueBusy event when a queue gets past a size you set. Pair it with the heartbeat and you can tell a dead worker apart from one that's just behind.
Keep it quiet in public
Health endpoints are usually public, and people forget that. Don't return stack traces, hostnames, versions, or connection strings. Names and true/false are enough for a load balancer. If you want more detail, put it behind auth or on an internal network.
The habit
When you add a health check, ask what's going to read it and what that reader does with a failure.
- A failed liveness check gets the process restarted, so only check things a restart can fix.
- A failed readiness check pulls the instance out of rotation, so only check things this instance needs and controls.
- Workers and schedulers don't serve requests, so give them a heartbeat and alert on it.
If your health check can't fail, it's not checking anything. It's just saying hi.