What Is a Reliable Pattern for Temporary Output Files?
From the earlier reference collection. Read the current software blog.
Build and validate a candidate output beside its destination, then replace the destination only after the candidate is complete.
Keep unfinished bytes away from the final path
Writing directly to a final output file exposes readers to partial content if the program crashes, runs out of disk space, or is interrupted. A safer pattern is to write a temporary candidate, finish all writes, close it, and validate it before making it visible under the final name. Until then, the old completed output remains available.
For example, generate report.csv.tmp while report.csv remains unchanged. The temporary name should be clearly associated with the intended destination and should not be mistaken for a finished report by downstream jobs.
Place the temporary file carefully
Create the temporary file in the destination directory when possible. A rename or replacement within one filesystem is commonly the operation used to publish the candidate, whereas moving across filesystems may require a copy and lose that property. Use a secure temporary-file creation method rather than inventing a name that another process could pre-create.
If multiple runs might target the same destination, decide how they coordinate. Unique temporary names prevent collisions during generation, but they do not decide which completed run should win. A lock, versioned output path, or explicit last-writer policy may be needed.
Validate before replacing
Close the file and check the conditions that matter: parse it again, verify expected columns, confirm record counts where meaningful, or compare a checksum to the generated bytes. Only after validation succeeds should the program replace the final path. If validation fails, report the problem and retain or remove the candidate according to a documented diagnostic policy.
A replacement operation should be chosen for the platform and filesystem behavior the tool supports. If durable storage after sudden power loss matters, additional syncing may be needed; a rename alone should not be described as a complete durability guarantee.
Handle failure states explicitly
On failure before publication, leave the previous destination untouched and return an error. On success, report the final path only after the replacement completes. A crash can still leave temporary files behind, so a later run should recognize its own temporary naming scheme and avoid treating those files as inputs.
This pattern creates a clear lifecycle: candidate, validated candidate, published output. That lifecycle makes retries safer and gives users a dependable answer to the question, “Is this file complete?”