A telemetry change took down ChatGPT for four hours

The conversation

On 11 December 2024, OpenAI's ChatGPT, API and Sora services went down for several hours. OpenAI's own incident postmortem, titled “API, ChatGPT & Sora Facing Issues,” states that “between 3:16 PM PST and 7:38 PM PST on December 11, 2024, all OpenAI services experienced significant degradation or complete unavailability,” and specifies that ChatGPT and Sora reached “full recovery at 7:01 PM PST” and the API “across all models at 7:38 PM PST.” The report states plainly that the event “was not caused by a security incident or a recent launch.”

What the documents show

OpenAI's postmortem attributes the outage to an internal change: at 3:12 PM PST, engineers “deployed a new telemetry service to collect detailed Kubernetes control plane metrics.” The document states the new service's configuration “unintentionally caused every node in each cluster to execute resource-intensive Kubernetes API operations whose cost scaled with the size of the cluster,” overwhelming the Kubernetes API servers and breaking the DNS-based service discovery that lets internal services find one another. The change had been “tested in a staging cluster, where no issues were observed,” and the problem was “most pronounced in our largest clusters, so our testing didn't catch it,” while DNS caching “made the issue far less visible until the rollouts had begun fleet-wide.”

The system boundary

The postmortem describes a boundary between infrastructure that operates independently and infrastructure that cannot: the “Kubernetes data plane can operate largely independently of the control plane,” so running workloads did not immediately fail, but “DNS relies on the control plane,” so once cached records expired over roughly twenty minutes, dependent services began failing outright. The document also describes a boundary in OpenAI's remediation ability: engineers “identified the issue within minutes” but could not fix it because doing so required access to “the Kubernetes control plane — which we could not do due to the increased load,” forcing three parallel workarounds before regaining enough control to remove the offending service.

Where it fails

OpenAI names the failure as a monitoring gap: pre-deployment checks assessed “resource utilization metrics... (CPU/memory)” but, the report states, “no precautions were taken to assess Kubernetes API server load,” and the rollout “monitored service health but lacked sufficient cluster health monitoring protocols.” A cascading dependency on a control plane for service discovery, combined with a rollout that outran the visibility needed to catch it, is a pattern any large distributed system should guard against.

• Does a service's ability to discover other services depend on infrastructure that a routine change could overload?
• Is a new deployment tested at the scale of the largest production cluster, not just a representative staging environment?
• Can engineers reach the control layer needed to roll back a change once that same layer is under load?

OpenAI's status page, as retrieved on 16 September 2026, continues to report aggregate uptime for ChatGPT and its APIs on a rolling basis, distinct from any single incident's postmortem; a builder should treat a published root-cause account like this one as OpenAI's own explanation of a specific date, not a general reliability guarantee.