PagerDuty vs actually fixing the problem automatically

PagerDuty did its job perfectly. At 3:07am it rang three people because PHP-FPM hit pm.max_children. By 3:22am someone SSHed in, ran sudo systemctl restart php8.3-fpm, and went back to bed. At 3:45am it rang again — same root cause, a slow report endpoint nobody prioritized. The on-call tool routed pain to humans; it did not remove the pain.

PagerDuty, Opsgenie, and similar platforms excel at escalation policies, schedules, and incident timelines. They answer who gets notified and when. They do not answer whether anyone needed to wake up at all for a failure with a known, safe fix.

When paging is the right tool

Human judgment is mandatory for:

• Data corruption or ambiguous database state
• Security incidents and suspected breaches
• Deploy rollbacks that might lose in-flight transactions
• Novel failures with no documented playbook
• Client communication during visible outages

PagerDuty shines here. Keep it for incidents that require brains, not for restarting a stuck supervisor worker for the fourth time this month.

When paging is expensive theater

Many Laravel production incidents are mechanical:

• PHP-FPM pool exhausted — workers busy or leaking memory
• Queue workers crash-looping on a poison job
• Disk full on /var/log — logs not rotated
• Opcache serving stale code after deploy until FPM restart
• Redis connection blip recoverable with worker bounce

These have safe automatable first responses. Paging first burns on-call capacity and trains engineers to ignore alerts — alert fatigue follows.

Diagnosis: classify your last twenty pages

For each incident, ask:

1. Could a documented script have fixed it in under sixty seconds with no data risk?
2. Did the same alert fire more than once in seven days?
3. Did customers notice before PagerDuty resolved the ticket?

If yes, yes, and sometimes — you need auto-fix before page, not better ringtones.

Build a simple spreadsheet of incident type, frequency, manual fix duration, and data risk. Sort by frequency times duration — top rows are automation candidates. PagerDuty should page what's left after that filter, not everything.

Inspect production before you automate. For FPM saturation:

grep 'max_children' /var/log/php8.3-fpm.log

For queue workers:

sudo supervisorctl status
php artisan queue:failed

For disk:

df -h && du -sh /var/log/*

Document findings. Automation without diagnosis automates the wrong thing.

Manual fixes (what on-call does at 3am)

Restart PHP-FPM when pool is stuck:

sudo systemctl restart php8.3-fpm

Restart queue workers:

sudo supervisorctl restart laravel-worker:*
php artisan queue:restart

Clear disk pressure safely — rotate logs, truncate known fat files, never blind rm:

sudo logrotate -f /etc/logrotate.conf

Tune FPM so the incident is less likely tomorrow — use the PHP-FPM calculator before raising limits.

These steps are repeatable. They should be playbooks, not heroics.

Define policy tiers: Tier 1 auto-fix without approval (worker restart), Tier 2 auto-fix with notification (FPM restart), Tier 3 page human immediately (database corruption suspected). PagerDuty handles Tier 3 routing; Reflex handles Tier 1 and 2 when configured.

Measure mean time to repair separately from mean time to acknowledge. PagerDuty improves the latter. Auto-fix improves the former — customers care about the former.

Integrating both tools

Many teams keep PagerDuty for schedule and compliance while Reflex executes first-response playbooks. Failed repair escalates to the same PagerDuty service with context attached — symptom, actions attempted, logs — so the human waking up starts mid-debug, not at zero.

Document which failure classes are Tier 1 versus Tier 3 in your internal wiki and in Reflex policy — ambiguity causes either over-automation or alert spam. Review quarterly after PHP or queue driver upgrades.

The auto-fix model Reflex uses

Reflex separates detection, repair, and escalation:

1. reflexd reports PHP-FPM pressure, queue depth, disk, deploy markers
2. The Brain evaluates policy-bound playbooks — restart FPM, bounce workers, clear safe temp paths
3. Humans get paged only when repair fails, policy requires approval, or the failure class is novel

Every action writes an audit entry: symptom, action, outcome — suitable for postmortems and agency client reports. Read how the Brain repair cycle works.

Pair with health-gated deployments so bad releases never trigger on-call in the first place.

Comparison summary

Full comparison: Reflex vs PagerDuty. You may keep PagerDuty for escalation while Reflex handles repair — they solve different layers.

Post-incident, compare time-to-acknowledge from PagerDuty with time-to-resolve from Reflex audit logs. If acknowledgment is fast but resolution slow, you have a repair gap — not an escalation gap.

The goal is not zero pages — it is zero unnecessary pages for failures your runbook already solves in under a minute without risk.

Laravel teams on Forge should read PHP-FPM OOM in production — the incident PagerDuty pages for most often has a safe first response before anyone opens a laptop.

View pricing. Laravel monitoring guide. Reflex for agencies when you manage client on-call rotations you cannot sustain.