AP US Government study guide · Chapter 33
What a poll can and cannot show
UNIT 4 · BELIEFS
CHAPTER 29
A poll can be numerically precise and still support a weak conclusion. Sample size is only one part of quality. Who could be selected, who responded, how the question was worded, how responses were weighted, when the survey was fielded, and what kind of uncertainty is reported all affect what the number can reasonably tell you.
Reading a poll well means checking each of these steps before accepting the number at the end of them.
START WITH THE POPULATION AND THE SAMPLE
A poll’s target population is the full group a pollster wants to describe, such as all registered voters in a state. Because surveying every member of that population is rarely practical, pollsters draw a sample, a smaller group whose responses are used to estimate the population’s views. A probability sample gives every member of the population a known, nonzero chance of selection, which is what allows statistical estimates to generalize back to the full population. A nonprobability sample does not guarantee that same known chance of selection, which limits how confidently its results can be generalized, even when the sample is large.
FOUR POLL TYPES TO IDENTIFY ON SIGHT
An opinion poll measures public views on an issue or political question. A benchmark poll establishes a candidate’s baseline level of support or the electorate’s starting attitudes, usually early in a campaign. A tracking poll repeats comparable measurements over time to follow changes in views of a candidate during a campaign. An exit poll collects information from people after they vote to estimate how and why different groups voted.
Public opinion can affect campaign strategy, electoral messaging, and policy debates, but the credibility of any conclusion depends on methodology. Sampling, margin of error, neutral wording, and accurate reporting determine how confidently the result can be generalized.
| Poll type | Best identification clue |
|---|---|
| Opinion poll | Measures views on issues or public questions |
| Benchmark poll | Creates an early baseline for a candidate or campaign |
| Tracking poll | Repeats comparable measures to show change during a campaign |
| Exit poll | Asks voters after voting how or why they voted |
Table 29.1 Four required poll types.
WHAT RANDOM SELECTION PROTECTS AGAINST
Random selection protects against systematic bias in who ends up in the sample, the risk that certain kinds of people are consistently more or less likely to be included than the population they are meant to represent. A large sample cannot correct a biased selection method. A well-selected sample of one thousand can represent a population better than a self-selected sample of one hundred thousand.
QUESTION WORDING CAN CHANGE THE RESULT
The specific wording, order, and response options of a poll question can shift results independent of any underlying change in opinion. A question that states a specific policy consequence can produce a different result than a neutrally worded version of the same underlying question. Comparing results across polls that worded a question differently, or comparing a single poll’s results across different questions, risks mistaking a wording effect for a change in genuine opinion.
DESCRIBE THE DATA BEFORE EXPLAINING IT
Before drawing a conclusion, first describe the values, direction, magnitude, and consistency of the pattern; then evaluate explanations. A single data point is not a trend, and two variables moving together on a graph establish an association, not a demonstrated causal explanation, unless the study’s design supports drawing that stronger conclusion.
WHAT THE POLL CANNOT PROVE
A poll’s reported margin of sampling error describes only the uncertainty that comes from sampling a subset of the population; it does not capture error from nonresponse, coverage gaps in who could be reached, question wording effects, or the accuracy of methods used to model who will vote. A poll reporting a margin of sampling error of three points can still be wrong by considerably more than three points once those other sources of error are taken into account.
| Candidate | Support |
|---|---|
| Candidate A | 48% |
| Candidate B | 45% |
| Undecided | 7% |
Table 29.2 A hypothetical statewide poll of 800 likely voters, margin of sampling error ±3.5 points.
WHAT THIS POLL CAN AND CANNOT SUPPORT
Table 29.2 supports a description: Candidate A leads Candidate B by three points among the voters sampled. It does not support a confident claim that Candidate A is ahead in the full population, since a three-point lead sits within the reported ±3.5-point margin of sampling error, meaning the true gap could run in either direction. It also does not, by itself, establish why either candidate holds that level of support, since the table reports only a snapshot of stated preference, not a causal explanation for it.
A POLLING EXAMPLE WITH A SPECIFIC FLAW
Example. A news outlet reports a poll finding that 62% of respondents support a proposed policy, based on responses collected through a link posted on the outlet’s own website over one weekend.
The specific flaw: this is a nonprobability, self-selected sample drawn only from people who both visit the outlet’s website and choose to click the link, not a sample giving the broader population a known chance of selection. The 62% figure describes those respondents; it does not reliably describe public opinion generally, regardless of how many people responded.
QUESTIONS TO ANSWER WITHOUT NOTES
1. What is the difference between a population and a sample in polling?
The population is the full group a pollster wants to describe. The sample is the smaller group surveyed to estimate the population’s views.
2. What does a probability sample provide that a nonprobability sample does not?
A known, nonzero chance of selection for every member of the population, which allows results to generalize back to the population with a stated level of confidence.
3. Does a larger sample fix a biased selection method?
No. A biased selection process produces a biased result regardless of sample size; a large biased sample simply repeats the bias with more apparent precision.
4. Can question wording alone change a poll’s result?
Yes. Wording, order, and response options can shift results independent of any actual change in underlying opinion.
5. Does a poll’s margin of sampling error capture every source of survey error?
No. It captures only sampling uncertainty, not nonresponse, coverage gaps, wording effects, or likely-voter modeling error.
6. Does two variables moving together on a graph establish causation?
No. It establishes association; causation requires additional evidence or a study design that supports that stronger conclusion.
7. Is a single data point a trend?
No. A trend requires a pattern across multiple points, not one observation.
8. A campaign surveys 1,200 likely voters in February to establish its starting support before advertising begins. What type of poll is this?
A benchmark poll.
9. A campaign surveys comparable samples every three days to see whether support changes after a debate. What type of poll is this?
A tracking poll.
10. Researchers interview voters after they leave polling places about candidate choice and issue priorities. What type of poll is this?
An exit poll.