Judge news, polls and political ads: bias, wording, sample size and context

Grade 12 Social Studies Worksheet - Challenge Me

Circle the best answer.

1. Read this first

   Use this

   Cards A to D on three phone calls in Fenwick (invented, for practice). Card A, "Three kinds of call" (summary in our words, not a quotation): A genuine poll tries to measure opinion. A push poll is a call dressed as a poll whose real aim is to spread a claim; it asks leading questions, often calls many people and reports no results. A message test asks a sample how different messages change opinions, to decide what to say. For a poll of n people chosen at random, the margin of error is about 100 divided by the square root of n, in percentage points. For a change between two separate polls of the same size, the margin is about 1.4 times that. A poll in which people choose themselves to answer is not a random sample, so the rule does not apply to it. Card B, "Call 1": Over one weekend 12,000 people are asked, "Would you vote for Rae if you knew Rae voted to close the fire station?" No other question is asked and no results are reported. Card C, "Call 2": A firm phones 800 randomly chosen adults, asks 12 questions including age and party, and reports results with a margin of error and the name of who paid. Forty-six percent of those asked say they would vote against Rae. Card D, "Call 3": A campaign phones 300 voters. Half hear "Rae cut taxes" before the vote question and half hear "Rae cut library hours." Among the first half 58 percent would vote for Rae; among the second half 52 percent would.

2. 1.

   A new call in Fenwick asks 20,000 people one question, 'Do you agree the unsafe bridge should be fixed now?', then reports no results. Bo uses Card A. What does it resemble?

   • A fraud, since any call that reports no results is a trick
   • A push poll: a leading question, many calls, no results
   • A genuine poll, since it asks about a public safety issue
   • A message test, since it tests one message on many people
3. 2.

   Esme reads this claim: 'Call 3 shows that the tax message is the more popular one.' What is the strongest objection that Cards A and D support?

   • Each half has about 150 people, a margin near 8, so 6 points could be chance
   • A message test is not a poll, so Card A says it cannot be used to compare messages
   • The tax half had 58 percent, so it cannot be compared with the 52 percent half
   • The two halves total 300 people, a margin of about 6 points, which equals the gap
4. 3.

   Jun reads this claim: 'Call 2 shows 46 percent of Fenwick voters will vote against Rae.' What does the claim assume?

   • That Call 1's 12,000 people would answer the same way
   • That the 800 adults match the people who will actually vote
   • That the margin of error is zero, since the firm reported it
   • That the 800 adults were not chosen at random

5. Use this

   Cards A to D on three surveys in Oakley (invented, for practice). Card A, "Rule of thumb" (summary in our words): For a poll of n people chosen at random, the margin of error is about 100 divided by the square root of n, in percentage points. For a change between two separate polls of the same size, the margin is about 1.4 times that. A poll in which people choose themselves to answer is not a random sample, so the rule does not apply to it. Card B, "Poll W": a news site's reader poll on banning phones in class; 9,000 readers click, and 71 percent say yes. Card C, "Poll Z": a firm tries 20,000 phone numbers and 1,600 randomly chosen adults answer, a response rate of 8 percent. The firm adjusts its results so the ages of the answers match the town's census ages. Card D, "Survey S": a school hands a form to its 300 students in class, with a teacher in the room: "Do you like school lunch?" All 300 answer.

6. 4.

   Kira reads Cards B and C. Which poll better supports a claim about Oakley adults, and what is its main caution?

   • Neither, since a response rate of 8 percent makes any poll useless
   • Poll Z, chosen at random, cautioned by its 8 percent response rate
   • Poll W, since 9,000 is more than five times 1,600, which makes it more accurate
   • Poll Z, since 1,600 gives a margin of 2.5 points, which covers each source of error
7. 5.

   Luz reads this claim: 'Poll Z is accurate within 2.5 points.' What is the best-supported reply from Cards A and C?

   • The 2.5 covers random sampling error only, not the 92 percent who did not answer
   • The claim is false, since a response rate of 8 percent makes the margin meaningless
   • The claim is true, since the firm adjusted the ages to match the census
   • The 2.5 is wrong, since the margin for 1,600 people is 5 points
8. 6.

   Noah reads Card C: Poll Z adjusts its results so ages match the census ages. What does that adjustment assume?

   • That age is the sole way answerers and non-answerers differ
   • That older and younger people answer polls at the same rate
   • That answerers match non-answerers of the same age
   • That the town's census ages are exactly right

9. Use this

   Cards A to D on a budget and two ads in Ridgemont (invented, for practice). Card A, "Library budget" (the city's published figures): Books and materials: $500,000 last year, $300,000 this year. Staff: $1,500,000 last year, $1,600,000 this year. Building: $1,000,000 both years. Total: $3,000,000 last year, $2,900,000 this year. Card B, "Ad 1" (with the line "Paid for by Citizens for Full Shelves"): "The mayor cut the library budget by 40 percent!" Card C, "Ad 2" (with the line "Paid for by Friends of the Mayor"): "Library spending fell just 3 percent, and staff pay rose." Card D, "Ad 3" (with the line "Paid for by Friends of the Mayor"): "Library staff pay rose 7 percent."

10. 7.

    Pia reads Cards A to C. How should the two ads be read?

    • Ad 1 is false, since the total fell about 3 percent
    • Each is accurate about a different part of the budget
    • Ad 2 is false, since books fell 40 percent
    • Both are false, since staff pay rose and the budget rose
11. 8.

    Ren wants a fair one-line summary of Card A for a neutral reader. Which is best?

    • Books fell 40%, staff rose 7%, total fell 3%
    • Each line was cut, so the library budget fell 40 percent
    • Books fell 40 percent while the total fell 10 percent
    • Staff rose 7 percent, so the books line rose too
12. 9.

    Tala reads this claim: 'Ad 1 is a lie.' What is the strongest objection that Cards A and B support?

    • The objection is that Ad 1 says 'library budget,' which means the books line alone
    • The 40 percent matches the books line; the fault is what the ad leaves out
    • Ad 1 comes from a group that wants book funding, so its claim can be ignored
    • Ad 1 is true in each respect, since the payer line is shown
13. 10.

    Zee reads Ad 1: 'the library budget' fell 40 percent. What does it assume about readers?

    • That readers will know the books line is just a small part of the budget
    • That readers believe everything a paid ad says
    • That readers will take 'the library budget' to mean total spending
    • That readers will check the figures in the city's published budget