What a poll can and cannot show
A poll can be numerically precise and still support a weak conclusion. Sample size is only one part of quality. Who could be selected, who responded, how the question was worded, how responses were weighted, when the survey was fielded, and what kind of uncertainty is reported all affect what the number can reasonably tell you. Reading a poll well means checking each of these steps before accepting the number at the end of them.
Start with the population and the sample
A poll's target population is the full group a pollster wants to describe, such as all registered voters in a state. Because surveying everyone is rarely practical, pollsters draw a sample, a smaller group whose responses estimate the population's views. A probability sample gives every member of the population a known, nonzero chance of selection, which is what allows statistical estimates to generalize back to the full population. A nonprobability sample does not guarantee that known chance of selection, which limits how confidently its results can be generalized, even when the sample is large.
Four poll types to identify on sight
| Poll type | Best identification clue |
|---|---|
| Opinion poll | Measures views on issues or public questions |
| Benchmark poll | Creates an early baseline for a candidate or campaign |
| Tracking poll | Repeats comparable measures to show change during a campaign |
| Exit poll | Asks voters after voting how or why they voted |
Public opinion can affect campaign strategy, electoral messaging, and policy debates, but the credibility of any conclusion depends on methodology: sampling, margin of error, neutral wording, and accurate reporting.
What random selection protects against
Random selection protects against systematic bias in who ends up in the sample, the risk that certain kinds of people are consistently more or less likely to be included than the population they are meant to represent. A large sample cannot correct a biased selection method. A well-selected sample of one thousand can represent a population better than a self-selected sample of one hundred thousand.
Question wording can change the result
The wording, order, and response options of a poll question can shift results independent of any change in underlying opinion. A question that states a specific policy consequence can produce a different result than a neutrally worded version of the same question. Comparing polls that worded a question differently risks mistaking a wording effect for a change in genuine opinion.
Describe the data before explaining it
Before drawing a conclusion, first describe the values, direction, magnitude, and consistency of the pattern; then evaluate explanations. A single data point is not a trend. Two variables moving together on a graph establish an association, not a demonstrated causal explanation, unless the study design supports the stronger conclusion.
What the poll cannot prove: the margin of error
A reported margin of sampling error describes only the uncertainty that comes from sampling a subset of the population. It does not capture error from nonresponse, coverage gaps in who could be reached, question wording effects, or the accuracy of methods used to model who will vote. A poll reporting a margin of three points can still be wrong by considerably more once these other sources are taken into account.
| Candidate | Support |
|---|---|
| Candidate A | 48% |
| Candidate B | 45% |
| Undecided | 7% |
What this poll can and cannot support. It supports a description: Candidate A leads Candidate B by three points among the voters sampled. It does not support a confident claim that Candidate A is ahead in the full population, since a three-point lead sits within the reported 3.5-point margin, meaning the true gap could run in either direction. It also does not establish why either candidate holds that level of support, since the table reports only a snapshot of stated preference.
A polling example with a specific flaw
A news outlet reports a poll finding that 62% of respondents support a proposed policy, based on responses collected through a link posted on the outlet's own website over one weekend. The flaw: this is a nonprobability, self-selected sample drawn only from people who both visit the site and choose to click the link. The 62% describes those respondents; it does not reliably describe public opinion generally, regardless of how many people responded.
A poll-checking routine
- Population: who is the poll supposed to describe?
- Selection: did every member have a known chance of being chosen, or did people select themselves?
- Wording: is the question neutral, and is it the same as in the poll being compared?
- Margin: is the difference larger than the sampling error, and what other errors are not covered?
- Claim: does the conclusion stay with description, or does it jump to causation or to a trend from a single point?