How Can a Dataset Represent Unknown Values?
From the earlier reference collection. Read the current software blog.
Define what missing fields and explicit nulls mean, and keep unknown data distinct from valid zeroes and empty text.
Separate different states
A dataset needs to distinguish at least three cases: a value is known, the value is unknown, or the property does not apply. Those states should not be compressed into a single placeholder unless the format and documentation make the distinction clear.
For a numeric reading, zero can be a real observation. For a text field, an empty string can be a supplied value, such as an intentionally blank label. Treating either as a generic synonym for “unknown” makes it impossible for clients to interpret the original data reliably.
Choose omission and null deliberately
In JSON, an omitted property and a property whose value is null are different representations. A schema can define omission to mean that the provider does not include that field in this response, perhaps because it is optional or unavailable. It can define null to mean that the field is recognized but no value is known for this record.
For example, a station record might omit maintenance_note when the provider does not collect notes for that station type, while returning maintenance_note: null when notes are supported but none is currently known. That distinction is useful only if it is stable and documented.
Use explicit codes in tabular data
CSV has no built-in null value, so a dataset must define how it encodes one. An empty cell is a common choice, but it can be confused with an empty text value. A reserved token such as \N can be clearer for machine processing, provided the documentation states that the exact token is reserved and how literal occurrences are escaped.
Avoid assigning a normal-looking number like -1 or 9999 as an unknown marker unless the domain guarantees it cannot be valid. Such sentinels can leak into calculations when a consumer misses one rule.
Make the rules testable
Write a small table in the schema documentation that names each state and shows its representation in the chosen format. Include examples for a real zero, blank text, unknown value, and inapplicable field. Then validate sample records against those rules before publishing.
Clients should not need to infer meaning from patterns in the data. A precise missing-value contract allows them to preserve uncertainty, calculate only with valid values, and display a useful explanation when information is unavailable.