The workflow at a glance
Calculate successful outcomes divided by opportunities within every channel and overall, then inspect the channel mix. Different weights can reverse the pooled ranking. Common weights answer a defined standardized descriptive question; they do not automatically establish that the campaign caused an effect.
Define the numerator, denominator and row unit
These are editorially created data, not real business results or a test of a model. Each row gives successes and eligible opportunities for one campaign/channel pair. The exercise assumes the listed opportunities can be added across those groups. First ask which rate is higher within channels and which is higher in the current pooled sample; do not silently replace that with a causal-effect question.
| Campaign and channel | Successes | Opportunities | Within-channel rate |
|---|---|---|---|
| A / mobile | 90 | 100 | 90% |
| A / desktop | 1 | 10 | 10% |
| B / mobile | 19 | 20 | 95% |
| B / desktop | 16 | 80 | 20% |
Two rankings can both be arithmetically correct
A’s pooled rate is(90+1)/(100+10)=91/110, approximately 82.73%. B’s is(19+16)/(20+80)=35/100, or 35%. If the question concerns the observed success fraction across these exact opportunities, A is higher.
Within mobile, B’s95% exceeds A’s90%; within desktop, B’s20% exceeds A’s10%. This is not an arithmetic contradiction. Approximately 90.91% of A’s opportunities come from the higher-rate mobile channel, compared with 20% of B’s opportunities.
Repair an editorially constructed flawed summary
A is 82.73% overall and B is 35%, so A is more effective in every channel.
A has the higher current pooled observed rate; B has the higher observed rate in both channels. Report those scopes separately.
Do not average percentages without their denominators
The pooled rate uses each campaign’s actual opportunity weights. Averaging A’s90% and 10% to get 50% ignores the denominators 100 and 10 and does not reproduce its original sample. The weighted-average reconstruction guide explains that distinction.
Now deliberately change the question: suppose both campaigns are described using half mobile and half desktop. Under those common weights, A=0.5×90%+0.5×10%=50%, while B=0.5×95%+0.5×20%=57.5%. This is a standardized description under an explicit hypothetical mix, not the original pooled sample or a revenue forecast.
| Question | A | B | Supported description |
|---|---|---|---|
| Original pooled sample | 82.73% | 35% | A has the higher current pooled observation |
| Within mobile | 90% | 95% | B has the higher mobile observation |
| Each channel weighted 50% | 50% | 57.5% | B is higher under the stated common weights |
Ask AI to preserve the reversal
The request below makes the comparison explicit. Asking for one winner encourages a fluent sentence that may omit the constraints. Retain original counts and unrounded intermediate calculations so you can locate an error before accepting the final percentages.
You can also open the AI productivity studio, choose Data insights & hypotheses, and supply the four exercise rows with the review request above. This uses server-side Gemini generation and sends input only when you actively generate; availability and usage limits follow the page status. This article makes no model call, and its reference answer comes from independent calculation.
Use the four original rows in this article. Calculate successes/opportunities within every channel and overall for each campaign. Explain how channel mix reverses the ranking. Then apply common 50% mobile/50% desktop weights, labeling those results as standardized descriptions rather than the original pooled sample. Do not invent records, significance tests or causal lift. End with three sentences covering overall results, channel results and unknown conditions.Common weights still do not establish a treatment effect
The OpenStax experimental-design chapter discusses lurking variables and random assignment when interpreting differences. This exercise provides no random allocation, matched period, equivalent audience composition or comparable exposure conditions. The standardized gap alone cannot establish that switching to B causes an increase of 7.5 percentage points.
Channel is only one observed grouping variable. Audiences, timing, attribution and success definitions could also differ. Check those definitions and missing records rather than inventing confidence intervals, significance or a larger sample. Review denominators and percentage-point wording before presenting the conclusion.
A complete three-sentence reference conclusion
In this original sample, A’s overall success rate is about 82.73%, above B’s35%, while B is higher within mobile at 95% versus 90% and within desktop at 20% versus 10%. Different channel weights reverse the pooled ranking; with an explicitly hypothetical 50%/50% mix, A is 50% and B is 57.5%. These are descriptive comparisons, and the supplied evidence does not establish causal impact or statistical significance.
Inspect charts as well as prose. A chart showing only one pooled bar per campaign conceals the reversal. Put channel counts alongside the overall result, with denominators visible. A polished visualization cannot restore a comparison dimension omitted from the analysis.
Minimum evidence for the AI summary
- All four original rows and their channels are retained.
- Pooled rates divide summed successes by summed opportunities.
- Common weights are labeled hypothetical and distinct from the original sample.
- The conclusion does not generalize an observation to every person, channel or causal effect.
Common questions
Does a reversal establish a data error?
No. Different group weights can create it naturally. Still check denominators, duplicates, missing records and consistent outcome definitions.
Should I discard the overall rate?
No. The current sample’s total rate and comparisons within comparable channels answer different useful questions. Report both rather than deleting one to force a single ranking.
Are 50%/50% the recommended real-world weights?
No. They are an illustrative choice. A real standardized analysis needs a stated target population and a reason for its weights.
Further reading & sources
Put the guide into practice
Open the relevant tool, preview the result and check your exported file.
