Judge news, polls and political ads: bias, wording, sample size and context

Grade 12 Social Studies Worksheet - Challenge Me

Circle the best answer.

1. Read this first

   Use this

   Cards A to D on three phone calls in Fenwick (invented, for practice). Card A, "Three kinds of call" (summary in our words, not a quotation): A genuine poll tries to measure opinion. A push poll is a call dressed as a poll whose real aim is to spread a claim; it asks leading questions, often calls many people and reports no results. A message test asks a sample how different messages change opinions, to decide what to say. For a poll of n people chosen at random, the margin of error is about 100 divided by the square root of n, in percentage points. For a change between two separate polls of the same size, the margin is about 1.4 times that. A poll in which people choose themselves to answer is not a random sample, so the rule does not apply to it. Card B, "Call 1": Over one weekend 12,000 people are asked, "Would you vote for Rae if you knew Rae voted to close the fire station?" No other question is asked and no results are reported. Card C, "Call 2": A firm phones 800 randomly chosen adults, asks 12 questions including age and party, and reports results with a margin of error and the name of who paid. Forty-six percent of those asked say they would vote against Rae. Card D, "Call 3": A campaign phones 300 voters. Half hear "Rae cut taxes" before the vote question and half hear "Rae cut library hours." Among the first half 58 percent would vote for Rae; among the second half 52 percent would.

2. 1.

   Ari reads Card A and the three calls. Which pairing is right?

   • Call 1 is a push poll, Call 2 a genuine poll, and Call 3 a message test
   • Call 1 is a genuine poll, Call 2 a push poll, and Call 3 a message test
   • Call 1 is a push poll, Call 2 a message test, and Call 3 a genuine poll
   • Call 1 is a message test, Call 2 a genuine poll, and Call 3 a push poll
3. 2.

   A new call in Fenwick asks 20,000 people one question, 'Do you agree the unsafe bridge should be fixed now?', then reports no results. Bo uses Card A. What does it resemble?

   • A genuine poll, since it asks about a public safety issue
   • A push poll: a leading question, many calls, no results
   • A message test, since it tests one message on many people
   • A fraud, since any call that reports no results is a trick
4. 3.

   Esme reads this claim: 'Call 3 shows that the tax message is the more popular one.' What is the strongest objection that Cards A and D support?

   • Each half has about 150 people, a margin near 8, so 6 points could be chance
   • A message test is not a poll, so Card A says it cannot be used to compare messages
   • The tax half had 58 percent, so it cannot be compared with the 52 percent half
   • The two halves total 300 people, a margin of about 6 points, which equals the gap
5. 4.

   Jun reads this claim: 'Call 2 shows 46 percent of Fenwick voters will vote against Rae.' What does the claim assume?

   • That the 800 adults match the people who will actually vote
   • That Call 1's 12,000 people would answer the same way
   • That the 800 adults were not chosen at random
   • That the margin of error is zero, since the firm reported it

6. Use this

   Cards A to D on three surveys in Oakley (invented, for practice). Card A, "Rule of thumb" (summary in our words): For a poll of n people chosen at random, the margin of error is about 100 divided by the square root of n, in percentage points. For a change between two separate polls of the same size, the margin is about 1.4 times that. A poll in which people choose themselves to answer is not a random sample, so the rule does not apply to it. Card B, "Poll W": a news site's reader poll on banning phones in class; 9,000 readers click, and 71 percent say yes. Card C, "Poll Z": a firm tries 20,000 phone numbers and 1,600 randomly chosen adults answer, a response rate of 8 percent. The firm adjusts its results so the ages of the answers match the town's census ages. Card D, "Survey S": a school hands a form to its 300 students in class, with a teacher in the room: "Do you like school lunch?" All 300 answer.

7. 5.

   Luz reads this claim: 'Poll Z is accurate within 2.5 points.' What is the best-supported reply from Cards A and C?

   • The 2.5 covers random sampling error only, not the 92 percent who did not answer
   • The claim is true, since the firm adjusted the ages to match the census
   • The 2.5 is wrong, since the margin for 1,600 people is 5 points
   • The claim is false, since a response rate of 8 percent makes the margin meaningless
8. 6.

   Milo reads this claim: 'Survey S is the most reliable because all 300 students answered.' What is the strongest objection?

   • The claim is right, since 300 of 300 answering means no student was left out
   • Full response helps, but a teacher in the room may change answers
   • The claim is false, since 300 students give a margin of 5.8 points
   • The claim is false, since a school survey cannot measure students' views
9. 7.

   Noah reads Card C: Poll Z adjusts its results so ages match the census ages. What does that adjustment assume?

   • That answerers match non-answerers of the same age
   • That the town's census ages are exactly right
   • That age is the sole way answerers and non-answerers differ
   • That older and younger people answer polls at the same rate

10. Use this

    Cards A to D on a budget and two ads in Ridgemont (invented, for practice). Card A, "Library budget" (the city's published figures): Books and materials: $500,000 last year, $300,000 this year. Staff: $1,500,000 last year, $1,600,000 this year. Building: $1,000,000 both years. Total: $3,000,000 last year, $2,900,000 this year. Card B, "Ad 1" (with the line "Paid for by Citizens for Full Shelves"): "The mayor cut the library budget by 40 percent!" Card C, "Ad 2" (with the line "Paid for by Friends of the Mayor"): "Library spending fell just 3 percent, and staff pay rose." Card D, "Ad 3" (with the line "Paid for by Friends of the Mayor"): "Library staff pay rose 7 percent."

11. 8.

    Ren wants a fair one-line summary of Card A for a neutral reader. Which is best?

    • Each line was cut, so the library budget fell 40 percent
    • Books fell 40 percent while the total fell 10 percent
    • Staff rose 7 percent, so the books line rose too
    • Books fell 40%, staff rose 7%, total fell 3%
12. 9.

    Tala reads this claim: 'Ad 1 is a lie.' What is the strongest objection that Cards A and B support?

    • Ad 1 is true in each respect, since the payer line is shown
    • The 40 percent matches the books line; the fault is what the ad leaves out
    • The objection is that Ad 1 says 'library budget,' which means the books line alone
    • Ad 1 comes from a group that wants book funding, so its claim can be ignored
13. 10.

    Zee reads Ad 1: 'the library budget' fell 40 percent. What does it assume about readers?

    • That readers will check the figures in the city's published budget
    • That readers will take 'the library budget' to mean total spending
    • That readers will know the books line is just a small part of the budget
    • That readers believe everything a paid ad says