How Can a Data Format Distinguish Observation from Estimate?
From the earlier reference collection. Read the current software blog.
A value alone may not reveal whether it was measured, calculated, interpolated, or forecast. Record the method or status alongside it.
Numbers need context
A value such as 18.4 can look precise while concealing how it was obtained. It might come directly from a sensor, be calculated from several readings, or be filled in for a missing period. Users may reasonably treat these cases differently, so a data format should make the distinction visible.
The distinction is not a judgment that one kind of value is always better. An estimate can be useful where a measurement is absent, and an observation can still have uncertainty or quality problems. The format's job is to say what process produced the value, not to imply more confidence than the evidence supports.
Choose explicit metadata
Add a method or status field with defined values such as observed, estimated, interpolated, or forecast, as appropriate to the dataset. Define each term and indicate whether the status applies to one value, an entire record, or a time span. If some records have unknown provenance, represent that state explicitly rather than silently treating it as observed.
For example, a river-level record could include value, unit, timestamp, and method. A reading taken by an instrument might be marked observed; a gap filled from nearby readings could be marked interpolated. A user can then filter or display them differently without guessing from decimal precision.
Record the method when it matters
A status label may be enough for simple cases. Where reproducibility or interpretation depends on the process, include a method identifier, version, or a link to documentation. A derived value might also identify its input records or the period used. Avoid embedding a long explanation in every row if a stable method reference can explain it once.
Uncertainty can be represented separately where it is meaningful and supported. Do not invent confidence intervals or imply that an estimate has a measured uncertainty when none was calculated. If uncertainty is unavailable, document that limitation and retain the method label.
Make status part of validation
Check that every value has a valid status, that status values belong to the documented set, and that combinations make sense. A forecast status might require a forecast horizon; an observed status might require an instrument or collection source if the dataset captures that detail. The rules should match actual practice.
When publishing derived data, keep the status attached through exports, aggregation, and display. If multiple kinds of inputs contribute to one summary, describe how the output status is assigned. This lets readers understand not just what number appears, but what kind of evidence it represents.