Why Should Raw Sensor Values Be Preserved?
From the earlier reference collection. Read the current software blog.
Keeping original sensor records makes corrections traceable and lets later users reproduce transformations.
Keep the source record intact
A raw sensor value is the value received from the instrument or its source before later cleaning, conversion, rounding, or aggregation. Preserving it creates a reference point for every subsequent interpretation. If a dashboard replaces a suspicious value without retaining the original, readers cannot tell what the device reported or how the published series changed.
A record should preserve the value together with its original unit, station identifier, and available timestamps. If the source supplies quality flags or other metadata, keep those too. The exact fields depend on the data, but the goal is to retain enough information to understand the record in its original context.
Make transformations reproducible
Later processing may convert Celsius to Fahrenheit, remove a known invalid code, calculate an hourly mean, or round values for display. Each transformation should be documented and, where practical, applied as a separate derived field or dataset. Then a reader can follow the path from source value to displayed result.
For example, suppose a source value is 20.0 °C and a dashboard shows 68.0 °F. If both the original value and conversion method are recorded, another user can reproduce the display and check whether the conversion was applied correctly. If only the converted number remains, it may be unclear what unit or source value produced it.
Handle corrections as history
Sources sometimes correct past data, and a publisher may discover a parsing or mapping error. Preserve the earlier record and note the correction, its time, and the reason when known. A change log can distinguish a revised source value from a local correction. This is particularly useful when different reports were generated before and after a correction.
Suppose an import process accidentally assigns one station’s data to another station for a day. A clear correction record can identify the affected dates and the corrected mapping. Keeping the mistaken import in an auditable history does not mean continuing to present it as valid; it means making the repair understandable.
Separate storage from display
A dashboard can show a rounded or cleaned value while a data store retains the source record and the processing notes. Make clear which value is displayed and what operations were applied. If values are excluded or flagged, expose that status rather than silently dropping them. Readers can then decide whether a filtered series suits their analysis.
Raw preservation also supports future reinterpretation. A later analyst may use a better method or discover that an earlier assumption was mistaken. Keeping source values, transformation steps, and correction history makes that work possible without claiming that any particular dataset is error-free.