You paste an LLM’s output into your pipeline and JSON.parse throws. The standard response is to re-prompt: “fix the JSON.” Sometimes that works; sometimes it returns the same mistake with different words around it. This loop is more common than it should be, and it persists because the failure looks random. It is not random—it follows a small, stable catalog, and it can be repaired without asking the model to try again.
The failure catalog
The mistakes are not infinite. Every JSON failure we have seen in generated output falls into one of the patterns below. Each was reproduced and the exact error recorded from Node 20’s JSON.parse:
| Failure | Example | JSON.parse error |
|---|---|---|
| Trailing comma | {"a": 1,} | Expected double-quoted property name in JSON at position 37 |
| Single quotes | {'a': 1} | Expected property name or '}' in JSON at position 1 |
| Unquoted keys | {a: 1} | Expected property name or '}' in JSON at position 1 |
| Line comment | // note before a field | Expected property name or '}' in JSON at position 4 |
| Block comment | /* note */ between fields | Expected double-quoted property name in JSON at position 18 |
| Missing comma | {"a": 1 "b": 2} | Expected ',' or '}' after property value in JSON at position 17 |
| Literal newline in a string | Multi-line string value | Bad control character in string literal in JSON at position 17 |
| NaN / Infinity | {"score": NaN} | Unexpected token 'N', ... is not valid JSON |
| Markdown fences | ```json … ``` wrapping | Unexpected token '`', "```json … |
| Truncation | "tags": ["dev", "json" (unclosed) | Expected ',' or ']' after array element in JSON at position 60 |
| Ellipsis | ["dev", "json", ...] | Unexpected token '.', ... is not valid JSON |
One more thing about the error messages themselves: the position they report is where the parser first lost track, not necessarily where the deviation began. A trailing comma at the end of an object is reported as “Expected double-quoted property name … at position 37”—one character past the actual mistake. The parser can only tell you the first place it got confused; tracing the root cause is still manual.
Why JSON leaves no room for error
JSON is the strictest popular data format: double quotes only, no comments, no trailing commas, no bare identifiers, no multi-line strings. Every other format developers write—Python dicts, JavaScript objects, YAML, TOML—forgives at least one of these rules.
Language models are trained on text that mixes all of those formats. When sampling the next token, a model assigns a high but not absolute probability to the syntax-correct choice. Most of the time it lands correctly. Occasionally it samples the plausible-but-wrong token: a single quote where JSON requires double, a trailing comma after the last field, a comment the model is “helpfully” adding.
Two properties make this structural rather than incidental:
- The model cannot parse its own output. There is no feedback loop between emitting a token and checking the accumulated result against JSON’s grammar.
- The failure is all-or-nothing. JSON has no error tolerance: one misplaced comma invalidates the entire payload, no matter how correct the other ten thousand characters are.
So the real question is not whether an AI will emit broken JSON, but how your pipeline absorbs the failure when it happens.
The re-prompt trap
“Ask the model to fix it” is the most common recovery, and it has three costs:
- Tokens and latency. Every retry is a full generation pass—seconds of wall-clock time and additional spend per request.
- Non-determinism. The same prompt may fix the JSON this time and break it differently the next. You cannot build reliable retry logic around a distribution.
- Content drift. A regeneration can quietly change values—a number, a name, a classification—with no diff highlighting it.
Re-prompting is the right tool when the content is wrong: the model missed a field or misread the input. It is the wrong tool for syntax. Syntax failures have a finite catalog, and a finite algorithm can handle them.
What deterministic repair actually does
A deterministic repairer applies a fixed set of token-level rules: strip comments, normalize quotes and bare keys, remove trailing commas, insert clearly missing commas, escape control characters, close unclosed structures, drop markdown fences. Same input, same output, every time, in microseconds, with no API call.
Failure What the repairer does Trailing comma Removed Single quotes, unquoted keys Converted to double-quoted strings Line and block comments Stripped Missing commas Inserted Literal newlines in strings Escaped to \n NaN / Infinity Kept as "NaN" / "Infinity" strings—the token is preserved, not guessed Markdown fences Removed Truncated structure Brackets closed with best-effort guesses Ellipsis Dropped
The practical differenceNo tokens, no latency, no non-determinism. The same input always produces the same output, which means repair results are testable and cacheable. Where repair honestly stops
A repairer fixes syntax; it cannot recover content the model never emitted. Three cases are worth knowing before you trust repaired output:
- A string cut off mid-word.
{"bio": "she works at closes to {"bio": "she works at"}. The rest of the sentence is gone—no algorithm can recover text that is not in the token stream. - Missing separators are guesses.
[1 2] becomes [1, 2] because a comma is the most likely intent—but the repairer decides that for you. - Empty values are invented.
{"a": } becomes {"a": null}. null is not in the source; review it before treating the repaired document as ground truth.
Rule of thumb: treat repair as a recovery net, not a validator. Validate the repaired output before it enters anything stateful.
Prevention comes first
- Request structured output where the API supports it. OpenAI’s
response_format / JSON schema mode, Anthropic structured outputs, and Gemini JSON mode force valid JSON at generation time. - Lower the temperature. Setting it to 0 reduces—though does not eliminate—variation.
- Keep payloads small. Truncation is the one failure that scales with output length. Ask for one record at a time instead of fifty.
- Validate at the boundary, retry once. If the second attempt is also broken, repair deterministically instead of entering a re-prompt loop.
- In production, repair with review. Never silently write repaired output into a database without a validation step.
How JSONDock repairs AI output
JSONDock’s Repair tab applies the open-source jsonrepair engine and then re-formats the result with a lossless parser, so the repair never rounds numbers while fixing syntax. Take an AI payload with a trailing comma and a 64-bit identifier:
{"orderId":900719925474099312345, "status": "paid",} It comes out of the Repair tab with the comma fixed and the identifier still exactly 900719925474099312345:
{
"orderId": 900719925474099312345,
"status": "paid"
} That lossless behavior is what our guide on JSON.parse precision loss explains in detail. The principle is the same: repair, like formatting, should never be the step that corrupts your data.