monit.rs · Zenofex LLC

The field that was removed eleven times: 62% of the breaking changes we count come from flapping specs

PostHog's spec "removed" the field id in eleven separate weekly snapshots; Vercel added a oneOf variant in 17 of 18. Seven vendors account for 62% of every breaking change we counted in 90 days, and the bottom of our own leaderboard is measuring spec regeneration rather than API change. The…

The field that was removed eleven times: 62% of the breaking changes we count come from flapping specs

TL;DR: We shipped a leaderboard ranking all 47 public APIs we track by breaking changes per 100 operations, then went and read the bottom of it by hand. PostHog's spec "removed" the field id in eleven separate weekly snapshots. Vercel's added a oneOf variant in 17 of 18. A field cannot be removed eleven times. Across the whole set, seven vendors show 20% or more of their breaking messages recurring in a later snapshot, and those seven account for 3,839 of the 6,205 breaking actions we counted in 90 days — 62%. The top of the leaderboard is sound. The bottom is measuring spec regeneration as if it were API change, and we are saying so before anyone quotes it at us.

Update, 7 November 2026 — this is fixed. Everything below stood when published, and the counting rule has since been changed. Methodology v3 counts a breaking message once per vendor, at the snapshot where it first appeared. 90-day totals across the 44 scored vendors fell 39%, from 6,205 to 3,795. PostHog went from 3,472 to 1,438 and is no longer the least stable API we track. The before-and-after table is at the end of this post, and the numbers on every page are already v3. One correction worth recording: the fix proposed in the original "What we are changing" section below — suppressing a message already reported for the same vendor and operation — does not work. Measured, it cut totals by 4% and moved not one vendor's score. When findings differ by operation but share a message, the counting rule already counts the message, so a key including the operation leaves the counted unit untouched. The key has to be the unit that is actually counted: (kind, message).

See it live: the full leaderboard, all 47 ranked. No signup. Atom feed, or one per vendor.

What we built, and what we found when we read it

Last week we added a page that answers a question no vendor can answer about itself: of every public API we track, which one breaks its consumers most often? It ranks on breaking changes per 100 operations over 90 days, because ranking on raw counts just tells you which API is biggest.

Then we did what the September methodology rewrite taught us to do, which is to read the findings behind the worst number before publishing anyone's name next to it. That rewrite happened because Deepgram was about to be labelled the worst offender on the strength of 49 "breaking changes" that turned out to be a single auth cleanup repeated across 49 operations.

This time the number at the bottom was PostHog: 3,472 counted breaking actions in 90 days, 137.18 per 100 operations, a stability score of 2 out of 100. That is 56% of every breaking action we counted across all 44 scored vendors, from one API.

PostHog's tracked spec produced 1,523 distinct breaking messages over 15 weekly snapshots, from 12 August to 1 November. 438 of them — 28% — appear in more than one snapshot. Restricting to removals makes it starker: of 586 distinct Field removed: messages, 398 recur in a later snapshot. 68%.

id was removed on 16 August, then again on 23 August, 29 August, 6 September, 13 September, 20 September, 27 September, 11 October, 18 October, 25 October and 1 November. Eleven removals of one field across eleven weeks is not eleven breaking changes. It is one spec document being regenerated with unstable output, and our engine reading each regeneration as a fresh wave of removals.

The single largest snapshot makes the mechanism obvious: 2,836 changes, of which 1,126 breaking findings, spread across 1,120 distinct operations but only 399 distinct messages. "Field removed: hasMore" 29 times, "limit" 29 times, "offset" 28 times, "columns" 23 times. That is a pagination wrapper appearing and disappearing from hundreds of response schemas at once — the signature of a generated document, not of a team shipping 1,126 decisions in a week.

Vercel shows the same thing in a different shape. 664 distinct breaking messages over 18 snapshots, 247 of them recurring, and Returned variant added to oneOf present in 17 of the 18. A returned variant cannot be newly added seventeen weeks running.

Two vendors dominate that 62%: PostHog at 3,472 and Vercel at 319. The others in the group have high recurrence rates on tiny samples — Figma has two distinct breaking messages, one of which recurs, which is 50% and means nothing. We are not calling Figma, Box, PlanetScale, Perplexity or Discord flappy on two to five messages. The finding is about PostHog and Vercel; the rest is noise we are reporting so the threshold isn't hiding behind a round number.

The flapping is concentrated at the bottom. It does not touch the top of the board, and the median is not moved by it.

Eleven of the 44 scored APIs shipped nothing breaking at all in 90 days: Algolia Search, CircleCI, DocuSign eSignature, Loops, Replicate, Slack Web API, Spotify Web API, and four Twilio surfaces — Twilio core, Messaging, Verify and Video.

The median vendor sits at 2.49 breaking changes per 100 operations over 90 days. The mean is 13.72, and the gap between those two numbers is the whole story of this dataset: a long quiet middle and a short, loud tail.

By category, median rate per 100 operations:

Every category with one vendor in it is that vendor, not a category — Analytics is PostHog, Fintech is Plaid, Observability is Datadog, DevOps is CircleCI. They are listed because dropping the unflattering single-vendor rows while keeping the flattering one would be the easiest way to lie with this table.

Of the categories with enough vendors to mean anything, infrastructure and AI churn roughly five to six times as fast as the median vendor. That holds with PostHog and Vercel set aside, and it is the result we would stand behind from this window.

A flapping spec is still a real problem — just a different one

It would be convenient to conclude that PostHog's API is fine and only our metric was wrong. That is not what we think.

If you generate a client from a vendor's published OpenAPI document — and that is the entire point of publishing one — then a document that regenerates with id, name, description and the pagination wrapper moving in and out of schemas breaks your build every week. Your diff is noisy, your generated types churn, and you cannot tell a real removal from regeneration artefact without reading every line. That is a genuine failure with a genuine cost. It is simply not the same failure as "this vendor removed a field your code depends on", and we were reporting both as the second one.

Two different problems deserve two different numbers. Right now we publish one.

What we are changing, and what we are not

Today: every page that prints these counts for PostHog or Vercel carries the caveat inline, naming both vendors and linking here — the leaderboard, the analytics and infra category rankings, and the PostHog and Vercel pages themselves. Putting it only on the leaderboard would have left the same overstated number uncaveated on four other pages.

Shipped the same day. The counting rule collapses findings within a single snapshot — that was the September fix, and it works. It did not look across snapshots, which is why a field removed eleven times counted eleven times. Methodology v3 adds that: a breaking message counts once per vendor, at the snapshot where it first appeared.

What we got wrong on the first attempt is instructive. The obvious key was (vendor, operation, message) — suppress a message already reported for the same operation. Measured against the real data, that reduced 90-day totals by 4% and changed no vendor's score at all. The reason is in the September rule itself: when a vendor's findings differ by operation but share a message, least(distinct operations, distinct messages) already counts the message, so a dedupe key containing the operation never touches the counted unit. Keying on (vendor, kind, message) instead cut totals by 39%.

The cost is stated plainly: a breaking message that legitimately recurs on a genuinely new operation months later is now undercounted. We take that trade because overstating a named company's breakage by 59% is the worse error, and the engine cannot yet tell a regenerated document from a repeated decision.

Not changing: the per-vendor pages and the score keep using the same counted-action figure, so nothing on the site disagrees with anything else on the site. When the cross-snapshot rule lands, every number moves together, and we will publish the before and after exactly as we did in September.

Every figure above comes from the same place the public pages do: weekly snapshots of each vendor's published OpenAPI document, diffed against the previous snapshot, following $ref into shared components.schemas. A breaking change is one that can break an existing integration — a removed endpoint or field, a tightened constraint, a changed response type, a newly required request property.

Within one snapshot, a change is counted once per kind as the smaller of distinct operations and distinct messages, so one edit to a shared schema that surfaces across fifty endpoints counts once rather than fifty times. The methodology post explains why that rule exists and where it falls short.

"Recurring" above means: the identical breaking message text appeared in two or more separate weekly snapshots for the same vendor. That is a deliberately conservative test. It will miss flapping that changes its wording between regenerations, so 62% is a floor, not a ceiling.

Window: the 90 days ending 7 November 2026. Vendors without a known operation count or with fewer than two snapshots are excluded from scoring and named as unranked on the leaderboard rather than silently dropped.

Before and after: every vendor whose count moved

Nineteen of the 44 scored vendors changed. The other 25 never repeated a breaking message, so v3 leaves them exactly where they were.

The least stable API we track is no longer PostHog. It is Supabase (Data), at 69.41 breaking changes per 100 operations against a 170-operation surface — a number v3 barely touched, because Supabase was not repeating itself. PostHog now sits fourth from the bottom.

That is the point of fixing a metric rather than defending it: the vendor our old number shouted about was the one the number was wrong about, and the vendor it was right about was further up the page.

Monitor your own API this way

The figures on this page come from the same diff engine monit.rs runs against customer endpoints. Point it at your own API and it works the same way: it learns each response schema by example, diffs every subsequent response against that baseline, and alerts you when a breaking change ships. The free tier is 3 endpoints with the full feature set and no card required. Start monitoring free.

What monit.rs does

monit.rs compares observed response fields and types with a learned baseline. Run OpenAPI contract checks against the latest stored response when you want to compare the implementation with a declared contract.

See the difference when a field disappears or changes type. Optional AI summaries suggest possible causes and next steps; review the evidence before accepting a new baseline.

What you get on the free tier

  • Monitor 3 endpoints, full feature set, no credit card
  • Schema-aware regression detection — catches missing fields, type drift, new properties, status shifts
  • OpenAPI contract testing — validate the latest stored response on demand
  • Optional AI summaries — suggested causes and next steps alongside the observed diff
  • Public status pages at <slug>.status.monit.rs — share uptime history with customers
  • Multi-channel alerts — email, Slack, Discord, PagerDuty, webhook
  • Security fuzz testing — AI-generated adversarial edge-case tests to surface input validation gaps
  • MCP server for Claude Desktop / Cursor / Cline (no auth, three tools)

How it works

  1. Add your endpoint. Paste a URL. Set method + auth. Done.
  2. Baseline learned. monit.rs probes your endpoint and records the expected schema, types, and structure automatically.
  3. Regressions caught. Review detected changes and send a test alert to confirm your notification destination.

Public API changelog surface

monit.rs also runs a public changelog of 47+ well-known API vendors (Stripe, GitHub, OpenAI, Cloudflare, Datadog, PostHog, Vercel, Supabase, and 39 more). Every detected OpenAPI change is published at /changelog with per-vendor RSS feeds, stability badges, and side-by-side comparison pages.

See how we compare with existing monitors: vs Pingdom · vs Checkly · vs Uptime Robot · vs Statuspage.io · vs Bump.sh · vs Optic · vs Postman Monitors

Get started

Start monitoring free →

3 endpoints. Full features. No credit card required.