Matrix survey questions: design, mobile layout and coding

A matrix survey question places several related row items under one shared set of response columns. Use one only when every row has the same construct, scale, reference period and instructions; keep it compact; provide an item-by-item mobile fallback; assign stable row and column codes; and treat straightlining as a review signal rather than automatic proof of inattention.

What is a matrix survey question?

A matrix, grid or matrix-rating question combines several row items with one response set. The shared choices must mean the same thing for every row. If one row asks frequency, another asks agreement and a third asks importance, the screen contains three different measures disguised as one grid.

Matrix-or-separate decision table

Worked redesign

Suppose a grid says “Rate our website” with rows for speed, prices, checkout, customer support and delivery, and columns from Very poor to Excellent. Those rows do not share one experience: someone who has not contacted support or received a delivery needs a legitimate not-applicable route. Keep a small website-experience matrix for speed, navigation and checkout. Ask support and delivery separately only when routing confirms exposure. Give the retained rows stable variables such as web_speed, web_navigation and web_checkout.

Six-row design template

1. Write one stem that states the construct and reference period.
2. Keep only directly comparable row items.
3. Use a fully labeled ordered scale when the measure is ordered.
4. Add a genuine not-applicable option when exposure varies.
5. Repeat the item and all response labels in the mobile fallback.
6. Put necessary explanation beside the relevant item rather than in a distant note.

Mobile layout and accessibility checks

The U.S. Census Bureau's mobile-optimized National Survey of College Graduates testing found that respondents could encounter grids that required horizontal scrolling to reach response options. Current Qualtrics documentation says its mobile-friendly matrix setting converts the grid to an accordion on reduced screens. That vendor behavior is an example, not a guarantee for every platform or questionnaire.

Test the published instrument at narrow widths, with zoom, large text, keyboard navigation and a screen reader. Every control needs an accessible name that combines the row item with its response label. Focus must follow the visible order, and an error must identify the incomplete item. Do not rely on color, hover, sticky headings or visual alignment alone. If the grid becomes item-by-item cards or an accordion, confirm that labels, codes, required-state behavior and any randomized order are preserved.

Sixteen-field coding specification

Store matrix_id, row_id, row_label, column_code, column_label, scale_order, not_applicable_code, missing_code, display_position, layout_mode, randomization_group, required_rule, questionnaire_version, language_version, export_variable and export_column. Qualtrics currently documents that each statement in a single-answer matrix is exported to its own column; verify the same one-row-to-one-variable relationship in the chosen platform's actual export.

Example: service_reliability uses 1 Very poor, 2 Poor, 3 Fair, 4 Good, 5 Excellent, 97 Not applicable, 98 Prefer not to answer and blank for routed-not-shown. Do not code not applicable as a midpoint or include it in a mean.

Straightlining review

Straightlining is a review signal, not proof of inattention. Selecting the same response down a matrix can reflect satisficing, but it may also be the honest description of a consistent experience. Review the pattern with elapsed time, contradictory items, item missingness, attention checks and open-text evidence. Declare any exclusion rule before analysis, retain the original value and report how many records the rule changes.

Twelve boundary tests

1. Longest row label at the narrowest supported width.
2. Longest translated response label.
3. Keyboard-only selection and correction.
4. Screen-reader announcement of row item plus response.
5. One optional row skipped.
6. One required row left blank.
7. Not applicable selected legitimately.
8. Randomized row order retained in the export.
9. Mobile item order matches the recorded display position.
10. A response-label change creates a new questionnaire version.
11. One response column selected for every row.
12. A routed respondent never sees the matrix.

Transparent method

We rechecked all three primary sources on 14 October 2026. The definition, current mobile-friendly behavior and export boundary use Qualtrics' first-party documentation. Mobile risk and redesign context use the Census Bureau's grid usability evaluation. The peer-reviewed web-survey study provides evidence about grid versus item-by-item presentation, breakoff, missing data and non-differentiation. The decision table, coding specification, worked redesign and boundary tests are editorial synthesis.

Limitations

There is no universal safe number of rows or columns. Results depend on label length, device, construct, population and platform rendering. Splitting a grid can itself change context and response behavior, so trend measures need a controlled comparison. A lower straightlining rate does not prove better validity, and a higher rate does not prove fraud. Product behavior can also change by plan, setting and release, so verify the live instrument and its export.