toolhq.io

All posts
7 min readby Jameel Haider

Why large IDs break in JSON: numbers, precision and the 53 bit limit

A payment API returns an order ID of 9007199254740993. Your frontend reads it back as 9007199254740992. Nothing threw, nothing logged, and the support ticket says the customer's receipt links to somebody else's order.

This is the most common silent data corruption bug in web APIs, and it comes from a gap between what the JSON specification allows and what JSON parsers actually do.

JSON does not have an integer type

The JSON grammar describes a number as a sequence of characters: an optional minus, digits, an optional fraction, an optional exponent. That is all. The specification places no limit on how many digits you may write, and it says nothing about how a parser should represent the result in memory.

RFC 8259 is explicit about the consequence. Interoperable implementations are expected to stay within the range of an IEEE 754 double precision number, because that is what most parsers use. Anything outside that range is legal JSON that different parsers will read differently.

A double has 53 bits of significand. Every integer up to 2^53 is representable exactly. Above that, the gaps between representable values grow, and any integer landing in a gap is rounded to the nearest one that exists.

Number.MAX_SAFE_INTEGER        // 9007199254740991
JSON.parse('{"id": 9007199254740993}').id
// 9007199254740992

The value changed during parsing. No error, no warning. By the time your code sees the object, the original digits are gone.

Where the 64 bit values come from

Almost every ID scheme in wide use produces values that exceed 53 bits.

SourceWidthExceeds 2^53
Postgres bigserial, MySQL BIGINT64 bitOnce the table passes ~9 quadrillion, rarely in practice
Snowflake IDs (Twitter, Discord, Instagram)64 bit, timestamp seededAlways, from the first ID
Stripe style random numeric IDsVariesOften
Unix time in nanoseconds64 bitAlways since 1970
Blockchain amounts, token balances (wei)256 bitAlways

Snowflake IDs are the usual first encounter. They embed a millisecond timestamp in the high bits, so even the very first ID a Snowflake generator ever emits is around 2^63. A Discord message ID has never fit in a double.

The database is fine. The server language is often fine: Python integers are arbitrary precision, Java has long, Go has int64. The loss happens at the JSON boundary, and only when one side of that boundary uses doubles for every number. That includes JavaScript, and it also includes any language whose JSON library defaults to a float type.

Decimals have the same problem, differently

Integers above 2^53 lose precision because they fall between representable values. Decimal fractions lose precision because most of them have no exact binary representation at all.

0.1 + 0.2              // 0.30000000000000004
19.99 * 100            // 1998.9999999999998
Math.round(1.005 * 100) / 100   // 1 , not 1.01

This is not a JSON bug, it is binary floating point working as designed, but JSON transports the damage. A price serialised as 19.99, read into a double, multiplied by a quantity and rounded, will eventually be off by a cent. Financial reconciliation jobs exist largely because of this.

Four fixes, in order of preference

1. Send IDs as strings.

{ "id": "9007199254740993", "parent_id": "9007199254740992" }

This is what Twitter did when Snowflake IDs broke every JavaScript client, and what Stripe, Discord and most modern APIs do now. An ID is an opaque handle, not a quantity. You never add two IDs together, so the number type buys you nothing and costs you correctness.

If you are changing an existing API, add the string field alongside the numeric one (id_str was Twitter's name for it), deprecate the numeric field, and remove it a release later. Breaking every client at once is worse than the bug.

2. Send money as minor units, or as a decimal string.

Either {"amount": 1999, "currency": "USD"} with the amount in cents, or {"amount": "19.99"} parsed into a decimal type on arrival. Both avoid doubles entirely. Minor units are simpler and are what Stripe and most payment processors use. Decimal strings are better when you need more than two places, such as unit prices or exchange rates.

3. Parse with BigInt on the consumer side.

If you cannot change the producer, you can still avoid the loss, but only by intercepting the raw text before it becomes a Number. Modern JavaScript engines support a reviver that receives the original source string:

JSON.parse(text, function (key, value, context) {
  if (typeof value === 'number' && !Number.isSafeInteger(value)) {
    return BigInt(context.source)
  }
  return value
})

Where that is unavailable, libraries such as json-bigint do the same job by replacing the parser. Be aware of what you inherit: BigInt does not mix with Number in arithmetic, and JSON.stringify throws on it unless you add a replacer. For values you only ever display or pass through, that is a fair trade.

4. Use a decimal aware parser in the server language.

Go has json.Number, which keeps the original text until you ask for a specific type. Jackson has USE_BIG_DECIMAL_FOR_FLOATS and USE_BIG_INTEGER_FOR_INTS. Python's json.loads accepts parse_float=decimal.Decimal. None of these help if the JavaScript client has already rounded the value, but they stop your own services from being the lossy hop.

How to tell whether it is already happening

The failure is invisible in logs, because by the time anything logs the value it has already been rounded. Two checks find it.

Compare the raw response text with what your code holds. Capture the body before parsing, parse it, serialise it again, and diff the two. Any number that changed digits is a number you are corrupting. The diff checker is a quick way to eyeball a captured pair, and the JSON formatter will normalise whitespace first so the diff shows only real changes.

Check your schema. If you generate types from a sample payload, a numeric ID becomes number and the assumption gets baked into the codebase. The JSON to TypeScript converter makes that visible: seeing id: number on a Snowflake field is the signal to change the contract, not the type.

For a one off check on a specific payload, paste it into the formatter and look for any integer longer than 16 digits. That is the practical threshold, since 2^53 is a 16 digit number.

What to write in the schema

If you publish an OpenAPI or JSON Schema document, say what you mean:

id:
  type: string
  pattern: '^[0-9]+$'
  description: 64 bit identifier, serialised as a string

type: integer with format: int64 is legal and common, and it is exactly the case that breaks. Validators accept it, code generators emit a 64 bit type in Java and a number in TypeScript, and the mismatch ships. Writing the field as a string with a numeric pattern removes the ambiguity for every consumer. JSON Schema validation covers the rest of the vocabulary.

The rule underneath all of this is short. If a value is a quantity you will do arithmetic on, keep it in a numeric type and keep it small, or move to minor units. If it is an identifier, make it a string and stop thinking about precision entirely.

Related reading: can you use comments in JSON covers another gap between the specification and what people expect, JSON vs TOML for config looks at where JSON is the wrong format, and XML vs JSON covers the typing differences between the two.