Skip to main content

Container memory exhaustion

Use this runbook when a container approaches its memory limit, is repeatedly OOM-killed, or causes host memory pressure.

Diagnose​

  1. Confirm the alert time window and affected container.
  2. Compare working-set memory with the configured container limit.
  3. Check restart count and kernel OOM events.
  4. Review recent deployments, workload changes, and request volume.
  5. Capture logs and metrics before restarting the workload.

Mitigate​

  • Reduce traffic or concurrency when the workload can shed load safely.
  • Restart only after evidence is captured and an operator approves the action.
  • Adjust memory limits only when host capacity and workload behavior justify it.
  • Roll back a recent release when memory growth correlates with that deployment.

Verify​

Confirm memory stabilizes, restart count stops increasing, and the service's latency and error rate return to baseline. Record the evidence and chosen action in the incident timeline.

Store for retrieval​

Store the runbook text with an embedding generated by the same model configured as EMBEDDING_MODEL. The application expects a table with title, service, content, and embedding columns. The vector dimension must match the selected embedding model.