AI basics & verificationPractical AI workflows

AI summaries of multi-select surveys: count people separately from selections

An AI summary may normalize choices to fit a pie chart and quietly replace how many people selected something with its share of all selections. Both calculations can be valid while answering different questions. The reporting unit must stay visible.

A person writing a checklist in a notebook
Illustrative photo: Glenn Carstens-Peters · Unsplash License

The workflow at a glance

Supply the exact question, multi-select rule, valid-response definition and original choices. Check option counts person by person first. Report people and selections separately; percentages above100% do not automatically establish an error.

Freeze the question before summarizing it

Our original question asks which file functions someone needs, allowing PDF,OCR andPNG together, none for explicitly no need, and blank for unanswered. It asks about the presence of needs, not a single highest priority. The Pew Research Center questionnaire guide discusses wording and response format, including forced-choice versus select-all lists. We use no survey results from that source; every record below is an editorial exercise.

Retain the question and option dictionary with the data. Decide what blank and none mean before involving AI. If no dictionary is supplied, request clarification instead of merging both states simply because neither lists a specific tool.

Count six responses by hand

R01 selects PDF andOCR, R02 selects OCR, R03 selects PDF andPNG, R04 selects PNG, R05 explicitly selects none, and R06 is blank. This example defines the first five as valid answers, including explicit no need, and keeps R06 separately as unanswered. The sample does not represent actual KitNelo users.

PDF,OCR andPNG each have two people among five valid respondents:40% each. Their sum is120% because R01 andR03 belong to two options each. Only four people selected at least one tool. Six tool selections must not become six people with a need.

People and selections answer different questions
ItemPeople /5 valid answersTool-selection basis
PDF2/5 =40%2/6 ≈33.3%
OCR2/5 =40%2/6 ≈33.3%
PNG2/5 =40%2/6 ≈33.3%
none1/5 =20%, separateExcluded from the six tool selections
Blank R06One unanswered, separateNot evidence of zero need

Request counting evidence before prose

Use two passes. First ask for the respondent IDs behind each option and its count. Check those yourself before requesting the written summary. If the model supplies only percentages, require numerators, denominators and IDs. Do not start with a fluent customer-persona paragraph and work backward to justify it.

A reusable multi-select verification request

This is a fictional multi-select question, not a single-choice ranking.
Question: Which file functions do you need? Options: PDF,OCR,PNG; none means explicitly no need; blank means unanswered.
R01: PDF|OCR
R02: OCR
R03: PDF|PNG
R04: PNG
R05: none
R06: [blank]
Count each person once per selected option. Valid respondents include none but exclude blank.
Show respondent IDs, counts, denominator and percentages for each tool. Report none and unanswered separately.
Also show total tool selections and unique people selecting at least one tool. Do not force respondent percentages to sum to100%. Ask before changing the base.
This defines an acceptance task and expected basis, without claiming a model already produced a correct result.

A percentage needs its reporting unit

PDF at33.3% can describe its share of six tool selections. It cannot be silently rewritten as33.3% of valid respondents selecting PDF.40% of valid respondents is correct under this example’s five-person base. Excluding none changes the denominator to four and the result to50%; that change must be stated.

A pie chart of respondent percentages suggests mutually exclusive groups, even though one person can occupy several options. Our editorial recommendation is a bar chart labeled multi-select, with valid respondent count and raw option counts. If a second chart describes selection share, name that basis separately rather than reusing the people-based title.

Keep the question intact

Changed question

PDF,OCR andPNG each account for33.3%, totaling100%, so the three groups do not overlap.

Evidence-based description

Among five valid answers, two people select each option. Multi-select percentages total120%, and four people select at least one tool.

Selection share does not establish mutually exclusive respondent groups. Check overlap in the original records.

Handle missingness, duplicates and weights explicitly

This exercise has no sampling weights or repeat submissions by the same person. Real duplicates, branching questions and subgroup filters require agreed rules before calculation. A repeated PDF within one answer counts once for that person in this task. Counting every mention instead would define a different analysis.

Do not replace blank with none or ask AI to guess the missing answer. Use the missing-versus-zero checks, then the denominator and percentage checks when comparing reports. This six-person exercise establishes no population preference, statistically significant difference or measured ranking.

Evidence before accepting the summary

  • The original question and multi-select rule are retained.
  • Every option traces to respondent IDs.
  • Numerator, denominator and unanswered count are visible.
  • Explicit none and blank are not interchanged.
  • People and selection counts have separate labels.
  • The chart does not turn overlapping choices into exclusive groups.

Common questions

Must percentages above100% be wrong?

No. A respondent can select several options. Check the base and person-level counts before normalizing anything.

Should none be excluded from the denominator?

That depends on the predeclared question. This example includes explicit no need as a valid answer. Explain and recalculate any different base.

Do precise decimals remove the need to count manually?

No. Precision does not prove the correct reporting unit. Check IDs, counts and denominator before choosing display precision.

Further reading & sources

Related articles