Inference, Margin of Error, and Statistical Claims: SAT Math Study Guide

A study measures a few and reasons to the many. What it may honestly claim (about cause, and about whom) is decided not by the data but by how the data was gathered.

The last section of the domain is the one that asks not what the numbers are but what they are allowed to say. A study never measures everyone; it measures a sample and reasons outward to a whole population. Whether that reasoning is sound (whether a result generalizes beyond the people studied, and whether it shows that one thing causes another) depends almost entirely on how the study was designed. This is where the caution from the scatterplot section gets its full answer: association becomes evidence of cause only under a specific condition, and a result describes a population only under another. Knowing those two conditions, and reading a study against them, is what this section trains.

▸ Use representative samples to make population inferences

A sample is the group actually measured; the population is the larger group you want to know about. A number computed from the sample (a sample mean, a sample percentage) is an estimate of the corresponding true value in the population. The estimate is trustworthy as a stand-in for the population when the sample is chosen at random from the population you want to describe; random selection is the design feature that justifies generalizing, because it keeps the sample from systematically over- or under-representing any part of the population in expectation. Without random selection, the sample may describe only itself.

▸ Interpret margin of error around a sample estimate

A sample estimate need not equal the population value, so a study may report the estimate together with a margin of error to describe its uncertainty. A poll finding 52 percent support with a margin of error of 3 points is really saying the true population figure plausibly lies between 49 and 55 percent. All else equal, increasing the sample size generally decreases the margin of error and makes the estimate more precise.

$Figure from the book$

FIGURE 1 A sample estimate is a band, not a point. A larger sample narrows the band, pinning the true value down more tightly.

A confidence interval is this same band given a confidence level: a 95 percent confidence interval of, say, 48 to 56 means the procedure produces intervals that capture the true value most of the time, so those bounds are the plausible range for it. A value inside the interval is consistent with the data; a value outside is not well supported.

▸ Distinguish random sampling from random assignment

Two separate uses of randomness sit at the heart of study design, and keeping them distinct is the single most valuable idea in the section. They answer different questions and license different conclusions.

Random sampling vs. random assignment

Random sampling: randomly choosing who is in the study from the population. This is what lets results generalize to that population.

Random assignment: randomly choosing who receives which treatment among the participants. This is what lets a difference be attributed to cause.

They are independent choices: a study can use either, both, or neither, and each conclusion it earns comes from a different one.

▸ Distinguish observational studies from experiments

The distinction in randomness produces the distinction in study type. In an observational study, the researcher only watches, recording who already does what, and so can detect that two things go together but never rule out that some third factor drives both. In an experiment, the researcher intervenes, using random assignment to impose the treatment, which tends to balance potential confounders across the groups and so supports a causal conclusion. The grid below collapses everything into one picture: the columns ask whether the sample was random, the rows whether the assignment was, and each cell names exactly what may be concluded.

$Figure from the book$

FIGURE 2 What a study earns: random assignment buys cause; random sampling buys generalization. Each is needed for its own conclusion.

Read the grid as two independent switches. Random assignment moves you from “association” up to “cause.” Random sampling moves you from “this group only” out to “generalizes to the population.” A study earns a conclusion only on the axis whose randomness it actually used.

▸ Match generalization and causation claims to study design

A common task is matching the study design to the conclusions it can support. An experiment with random assignment but volunteer participants can claim cause, the treatment worked, but only for people like those volunteers, since no random sample was drawn. A random survey with no assigned treatment can generalize its finding to the whole population but can show only association, never cause. The strongest claim, both cause and generalization, requires both randomizations; the weakest, an observational study on a self-selected group, supports only an association among those particular people. In every case, you match the study's two design choices to the two axes and read off precisely what is earned, no more.

WORKED Read a study against the grid

A study randomly assigns volunteer patients to a drug or a placebo; the drug group improves more. The patients were not randomly sampled.

Random assignment was used, so a causal claim is justified: the drug likely caused the improvement.

Random sampling was not, so the result holds only for patients like these volunteers; it may not generalize to the broader population.

Conclusion: cause, but not generalization. That is the upper-right cell of the grid.

▸ Identify sampling bias and limits on generalization

Finally, the most common way a study goes wrong: a sample that is not random and so does not represent its population, a source of bias. A voluntary-response sample, made of only those who chose to reply, over-represents people with strong opinions. A convenience sample, made of whoever was easiest to reach, over-represents the nearby and the available. In both, the sample tilts away from the population in some systematic direction, so the estimate it gives is off in that direction no matter how large the sample grows. Increasing the size of a systematically biased sample does not remove the bias; it can produce a more precise estimate of the wrong target. Random selection is what reduces this systematic tilt, which is why every claim to generalize rests on it; it reduces sampling error in expectation, though it does not eliminate every source of nonresponse or measurement error.

▸ Practice: inference and study design

Question 1

A random sample of 800 adults finds that 64 percent favor a measure, with a margin of error of 3 percentage points. Which is the most appropriate conclusion?

A. exactly 64 percent of all adults favor it
B. it is plausible that between 61 and 67 percent of all adults favor it
C. at least 64 percent of all adults favor it
D. fewer than 64 percent of all adults favor it

Question 2

A pollster wants to reduce the margin of error of an estimate. Which change accomplishes this?

A. survey fewer people
B. increase the sample size
C. ask more questions
D. report a higher percentage

Question 3

Volunteers are randomly assigned to take either a new supplement or a placebo, and the supplement group sleeps longer on average. Because the participants volunteered rather than being randomly selected, the most defensible conclusion is:

A. the supplement causes longer sleep for everyone
B. the supplement likely caused longer sleep, but the result may not generalize beyond people like these volunteers
C. there is only an association, with no cause
D. nothing at all can be concluded

Question 4

An observational study finds that people who eat breakfast tend to weigh less. Which conclusion is justified?

A. eating breakfast causes lower weight
B. there is an association, but the study cannot show breakfast causes lower weight
C. skipping breakfast causes higher weight
D. breakfast and weight are unrelated

Question 5

A researcher randomly selects 1,000 residents from a city of 200,000 and surveys them. The results can be generalized to:

A. only the 1,000 surveyed
B. all residents of that city
C. all cities of similar size
D. no one

Question 6

A website poll draws 50,000 voluntary responses. A separate study surveys a random sample of 1,000 people drawn from the target population. Which better supports generalizing to that population, and why?

A. the website poll, because a larger sample always generalizes better
B. the random sample, because its selection method avoids the systematic bias of self-selection
C. both are equally trustworthy, since 50,000 exceeds 1,000
D. neither, since only a census of the full population would be trustworthy

Question 7

Which type of study can establish that one variable causes a change in another?

A. a randomized experiment
B. any large observational study
C. a voluntary-response survey
D. a convenience sample

Question 8

A website invites visitors to rate a product, and only those who choose to respond do so. The resulting average rating:

A. accurately represents all customers
B. may be biased and cannot be reliably generalized to all customers
C. is always too low
D. is more reliable with more responses

Question 9

In a study, participants are randomly selected from a large population and then randomly assigned to a treatment or a control group, and the treatment group improves more. The strongest valid conclusion is:

A. the groups merely differ by chance
B. the treatment is associated with improvement only
C. the treatment causes the improvement, and the finding generalizes to the population
D. nothing can be concluded without more data

Question 10

A study reports a 95 percent confidence interval of 48 to 56 for a population mean. Which interpretation is best?

A. exactly 95 percent of the data lies between 48 and 56
B. it is plausible that the true population mean lies between 48 and 56
C. the sample mean must be 52
D. every future sample mean will fall between 48 and 56

Question 11

A sample yields an estimate of 50 with a margin of error of 4. A claim that the true value equals 53 is:

A. contradicted by the data
B. consistent with the data, since 53 lies within 46 to 54
C. impossible to evaluate
D. proven correct by the data

12. A poll reports that 64 percent favor a measure, with a margin of error of 3 percentage points. What is the greatest percentage the true population value could plausibly take? (grid-in)

Question 13

Two polls each estimate support at 55 percent. Poll A surveyed 2,500 people; Poll B surveyed 400. Which estimate is more precise, and why?

A. Poll B, because smaller samples are more focused
B. Poll A, because a larger sample gives a smaller margin of error
C. they are equally precise
D. cannot be determined

Question 14

A study finds that students who attended an optional tutoring program scored higher than those who did not. Because students chose whether to attend, the most defensible conclusion is:

A. tutoring causes higher scores
B. attending tutoring is associated with higher scores, but causation cannot be established
C. tutoring has no effect
D. the higher scores caused students to attend

▸ Solutions: inference and study design

Show the answers and explanations

1: B. The estimate is a range: 64 percent give or take 3 points, so the plausible interval for the population is 61 to 67 percent. The other choices treat a single sample figure as if it were the exact or a bounding population value.

2: B. A larger sample produces a smaller margin of error, tightening the estimate. The number of questions or the size of the reported percentage has no bearing on the margin.

3: B. Random assignment supports a causal claim, so the supplement likely caused the effect. But without random sampling, the result holds only for people like these volunteers and may not generalize, the upper-right cell of the grid.

4: B. An observational study can detect that breakfast and lower weight go together, but it cannot rule out a third factor driving both, so it shows association, not cause. Causation would require random assignment.

5: B. Because the sample was drawn at random from the city's residents, the results generalize to all residents of that city, but no farther, since other cities were not sampled.

6: B. Sample size alone does not make a sample trustworthy; how the sample was selected does. The website poll is a voluntary-response sample, so it systematically over-represents people with strong opinions no matter how many respond. The random sample of 1,000, though far smaller, avoids that systematic tilt and is the one that supports generalizing to the population. A bigger biased sample is not a better one; it is a more confidently wrong one.

7: A. Only a randomized experiment, by assigning treatments at random, balances out other factors and can establish cause. Observational and self-selected designs can show association at most.

8: B. A voluntary-response sample over-represents those with strong opinions, biasing the result in a systematic direction that more responses cannot fix, so it cannot be generalized to all customers.

9: C. Random assignment earns the causal claim, and random sampling earns generalization to the population. Both randomizations are present, so the strongest conclusion is justified: the treatment causes the improvement and the finding generalizes to the population.

10: B. A confidence interval gives the plausible range for the true population mean, here 48 to 56. It describes the parameter's plausible values, not the spread of the data or of future samples.

11: B. The interval of plausible values is 50 give or take 4, namely 46 to 54. Since 53 falls inside that range, the claim is consistent with the data.

12: 67. The plausible range is 64 give or take 3 points, so the greatest plausible value is 64 + 3 = 67 percent.

Show the answers and explanations

13: B. A larger sample yields a smaller margin of error, so Poll A, with 2,500 respondents, gives the more precise estimate. Sample size, not focus, drives precision.

14: B. Because students self-selected into tutoring, no random assignment was made, so the study shows an association between tutoring and higher scores but cannot establish that tutoring caused them.

▸ Inference and study design: chapter summary

A study reasons from a sample to a population, and what it may claim is fixed by its design, not its data. An estimate comes with a margin of error (a band that generally narrows as the sample grows) and a confidence interval is that band of plausible values for the true parameter. Two randomizations govern the conclusions: random sampling lets a result generalize to a population, and random assignment lets a difference be called a cause. Match a study to those two axes and you can read off exactly what it earns. And never trust a non-random sample, however large, to speak for a population it was never drawn to represent. With that, the four domains of the test's mathematics are complete; what remains is putting them to work.