OpsKeeper

Final Demo · PostgreSQL pool exhaustion

Business Impact & Incident Console

A PostgreSQL application pool-exhaustion case connects real business queries, alerts, multi-agent diagnosis, controlled preview, human approval, repair, and independent verification. The static shell stays available while database cards degrade truthfully.

Environment

Not initialized

incident pending

One-click fault injection

The request first asks Manager to create an incident; Manager then invokes the controlled fixture. This page holds no database or fixture credentials.

Duration
90 seconds (TTL safety net)
Allowed target
pg:pool-fixture
Target fingerprint
0123456789abcdef0123456789abcdef

Real business queries

Three cards poll independently through the same stressed application pool.

3-second polling · 2-second client timeout

Paid orders

Order query

Querying

No successful query yet; this page never shows simulated success data.

Latency pending

Recent warehouse

Inventory query

Querying

No successful query yet; this page never shows simulated success data.

Latency pending

Audit events

Audit query

Querying

No successful query yet; this page never shows simulated success data.

Latency pending

Business impact

Healthy cards

0/3

Degraded cards

3/3

Avg last-success latency

Pending

Closed-loop stages

Manager owns authoritative state. Preview approval eligibility never replaces human approval.

Not started

  1. Scenario start

    starting

  2. Awaiting alert

    awaiting_alert

  3. Alert correlated

    alert_correlated

  4. Diagnosis

    diagnosis_dispatched

  5. Preview ready

    preview_ready

  6. Awaiting approval

    awaiting_approval

  7. Repair

    repair_dispatched

  8. Verification

    verifying

  9. Recovered

    recovered

  10. Archived

    closed

  11. Start failed

    start_failed

Repair preview decision

Compact evidence before HITL; the full comparison is available in the archive after closure.

Controlled workload boundary: preview-pg replays a fixed workload and never copies active sessions from the source instance.

Candidate A
No usable candidate

Not eligible for human approval

Candidate B
Not returned

Preview rejected · No human approval

Demo switch entries

Monitoring

Primary board: verify pool 4/4, wait queue, and query errors

https://teams.yueming.xin/grafana/d/opskeeper-pgpool-live/?orgId=1&from=now-15m&to=now&refresh=5s

Manager status

Secondary board: Manager health, HTTP latency, and internals

https://teams.yueming.xin/grafana/d/opskeeper-manager-internals/opskeeper-manager-internals?orgId=1&from=now-1h&to=now&refresh=30s

Element room

Watch Manager coordinate diagnosis, preview, and repair

https://rooms.yueming.xin/#/room/#benyue-lumos-ops:matrix-local.agentteams.io:18080

AgentTeams Dashboard

Inspect the task board and OpsKeeper runtime

https://teams.yueming.xin/#plugin-route:opskeeper-teamharness/home

OpsKeeper console

Switch to the standard demo user to review incidents, approvals, audits, and read-only settings

https://opskeeper.yueming.xin/login?account=switch

Incident archive

Review the full evidence chain and A/B comparison after closure

https://teams.yueming.xin/#plugin-route:opskeeper-teamharness/archive

Preview report

Open the preview-pg deep link to verify all metrics

https://opskeeper.yueming.xin/preview/

Human action prompts

Only two nodes require human intervention. Injection, diagnosis, preview, and verification are coordinated by Manager and Workers.

Use only at awaiting_approval
Approve Candidate A for incident <incident_id>. Verify candidate_id, execution_id, target fingerprint, scope, parameters, and expiry. Preview PASS only grants approval eligibility; it is not human approval.
Use only after recovered
Close incident <incident_id>: confirm business queries, pool metrics, and independent verification, then archive the evidence chain and A/B preview comparison.