Why Should a Data Format Preserve Original Values?
From the earlier reference collection. Read the current software blog.
Normalization improves consistency, but retaining source values can preserve distinctions needed for auditing and future reinterpretation.
Cleaning can erase meaning
Data pipelines often standardize spelling, units, capitalization, or date formats so values are easier to compare. That work is useful, but a transformation can collapse distinctions. If an input contains N/A, an empty string, and unknown, converting all three to null may erase clues about whether the source marked a value unavailable, left it blank, or used a special code.
Once a transformation discards those differences, later users may not be able to recover them. This matters when auditing a correction or when a field’s original convention was not understood at the time of import.
Keep raw and interpreted forms distinct
When provenance matters, store the received value separately from the normalized value. A record might retain source_temperature_text as 68 F and provide temperature_celsius as a numeric conversion. Document which field is authoritative for calculations and how the conversion was made.
Do not duplicate raw data without a reason. The goal is to preserve information that normalization could remove, not to make every output field carry several competing versions. A source identifier, retrieval time, or source release can also help explain where the raw value came from.
Make conversions reproducible
Write down conversion rules, including units, rounding, and treatment of malformed input. For example, converting Fahrenheit to Celsius yields a decimal that may be rounded for display. Keeping the source value alongside the normalized result lets a later user recompute at a different precision if needed.
A raw field should not be treated as automatically correct. It records what was received, while validation and interpretation determine how the value can be used. Mark rejected or suspect values clearly rather than silently replacing them with a plausible-looking substitute.
Preserve just enough history
Versioned snapshots or an audit log can preserve meaningful changes without carrying every intermediate representation in the main response. A small example dataset can show the original value, normalized value, and transformation note so consumers understand the relationship.
Preserving original values gives future maintainers evidence about the input and lets them revisit earlier assumptions. With clear provenance, users can distinguish what the source supplied from what the pipeline inferred or reformatted.