Evidence asset

Survey data cleaning checklist: 20 tests before analysis

Turn a raw survey export into an auditable analysis file with 20 duplicate, range, routing, missingness, timestamp and transformation tests.

Published: 25 September 2026
Reading time: 13 min
Author and reviewer: The Survey Review

Survey data cleaning is the documented process of finding and resolving records that violate the questionnaire, fieldwork plan or analysis specification. Keep the raw export immutable, run checks in code or a versioned query, write every decision to an issue log, and produce a new analysis file. Cleaning should make errors visible; it should not silently force answers to look plausible.

What is survey data cleaning?

Survey data cleaning converts a raw response export into a documented analysis file by checking it against the questionnaire, routing rules, sample records, fieldwork dispositions and variable definitions. It is separate from a data dictionary, which defines fields and codes, and from pre-launch QA, which tests the instrument before live collection.

Start by preserving the exact raw export and its file hash. Work only on a copy, keep transformation code under version control, and save a row-level issue log. That design makes the process repeatable and lets a reviewer distinguish the source value from the action taken.

20-test survey data cleaning checklist

ID	Test	Pass condition	Evidence to retain
I1	Stable row key	Every response has one non-changing unique key	Key rule and duplicate count
I2	Exact duplicate records	Repeated platform records are identified without assuming similar answers are duplicates	Compared fields and retained-row rule
I3	Sample-to-response join	Invitation or panel IDs join at the declared cardinality	Matched, unmatched and multiply matched counts
I4	Direct identifiers	Analysis files contain only identifiers explicitly required by the plan	Identifier inventory and removal rule
S1	Expected columns	Export fields match the approved dictionary and questionnaire version	Added, missing and renamed fields
S2	Variable types	Dates, numbers, booleans and text parse without silent coercion	Parsing errors by field
S3	Allowed codes	Every stored value is documented or logged as an exception	Out-of-domain values
R1	Branch eligibility	Downstream answers occur only for respondents in the item's universe	Questionnaire rule and failing keys
R2	Structural missingness	Items not shown use the declared structural-missing state	Displayed-state or route marker
R3	Impossible paths	No record reaches mutually incompatible branches or terminal outcomes	Route trace and failing condition
R4	Stale hidden values	A changed upstream answer cannot leave an ineligible downstream value active	Answer history or replay test
C1	Displayed unanswered	Item nonresponse is distinct from not shown	Rendered-state evidence
C2	Partial-response rule	Completion status follows the predeclared threshold	Threshold, denominator and status
C3	Technical missingness	Failures, timeouts and interrupted writes are not recoded as respondent choices	System state and incident note
V1	Numeric ranges	Values fall inside documented bounds or appear in the issue log	Minimum, maximum and exception rows
V2	Date and timezone	Timestamps parse in the documented timezone and valid sequence	Original value, timezone and normalized value
V3	Multi-select encoding	Selected, unselected, not shown and missing are distinguishable	Export shape and code rule
V4	Other-text linkage	Free text appears only with the relevant “Other” state, unless the instrument permits otherwise	Parent answer and text field
O1	Declared exclusions	Test, preview and known operational records are removed only by a written rule	Exclusion reason and count
O2	Reproducible output	Rerunning the same code on the same raw export produces the same analysis file hash	Input, code and output hashes

Reusable issue log

Keep one row per flagged field or record. The minimum fields are issue_id, check_id, row_key, variable, observed_value, expected_rule, evidence, action, code_version, reviewer and resolution_time. Add a status such as open, corrected, excluded, retained or accepted limitation. Never overwrite the raw value to hide what happened.

Worked example: routing and missingness

Question Q2 asks whether a respondent used the service. Only users see Q12 satisfaction. A non-user with Q12=5 fails the branch-eligibility test because the path should never expose Q12. A non-user with the documented structural-missing code passes. A user with Q12 blank is different again: it may be item nonresponse or a technical failure, depending on whether Q12 was rendered and whether the response write completed.

The cleaning pipeline should not replace all three states with zero. It should preserve the source values, derive an issue classification from route evidence and apply only the declared action. The accompanying data dictionary must define Q12's allowed range, missing codes and universe so the test is machine-checkable.

Duplicate decision tree

Are record IDs identical? Inspect whether the platform or transfer process duplicated one submission.
Do invitation tokens match but answers differ? Apply the declared repeat-submission rule; do not automatically keep the first or last response.
Do only IP or device attributes match? Do not assume identity. Shared networks and devices are common.
Are answers merely similar? Retain both unless independent evidence establishes duplication.
Was one record excluded? Log both keys, the retained record, the excluded record and the governing rule.

Validation queries before analysis

Every analysis row has exactly one stable key.
Every stored code is allowed by the dictionary or appears in the issue log.
Every branch condition agrees with downstream values and structural-missing codes.
Every derived score can be recomputed from retained source variables.
Every exclusion maps to a predeclared rule and final disposition.
Counts reconcile from raw export to retained, corrected, excluded and unresolved records.
Rerunning the pipeline on the same raw export produces the same file hash.

Transparent method

We converted the private draft into a field-level release gate and checked its metadata requirements against current institutional documentation. Harvard and OSF identify variable names, definitions, allowed values and units as core dictionary fields. ICPSR explicitly documents missing codes and universe or skip patterns, while DDI describes structured metadata as a basis for accuracy checking and replication. The duplicate, routing and output-hash tests are operational controls derived from those documented fields; they are not claimed as universal exclusion standards.

Limitations

Cleaning cannot repair a biased sample, recover data that were never collected, validate a construct or justify deleting inconvenient responses. Speeding, straight-lining and open-text quality flags are context-dependent; they should not become automatic exclusions without a study-specific rationale established before seeing the result. A technically valid value can still be substantively wrong, and a statistical outlier can be a genuine answer. Privacy, access and retention rules apply to raw, intermediate and cleaned files.

Sources and limitations

Harvard Biomedical Data Management: Data Dictionary, for variable definitions, allowed values, units and reproducible documentation.
ICPSR: What is a codebook?, for response codes, missing-data labels, summary checks and survey universe or skip patterns.
Open Science Framework: How to Make a Data Dictionary, for exact file-variable names, accepted values and range checks.
DDI Alliance: Create a Codebook, for structured metadata, replication and accuracy checking.

Verification date: 25 September 2026. This is operational survey-design guidance, not legal advice. Requirements can differ by jurisdiction, audience and research purpose. Send corrections with a primary source through our corrections process.