Grade by Grade
E. Culminating Data Management Investigation · Culminating Data Management Investigation

Critiquing Investigations Constructively

E2.4About 65 minutes
Learning goals
  • I can evaluate another student's investigation in five areas: question, data, analysis, conclusions and communication.
  • I can check calculations and spot common errors such as overclaiming, biased samples and misleading graphs.
  • I can give feedback that is specific, respectful and useful: a strength, a question and a suggestion with a reason.
  • I can critique the work, not the person.
  • I can use what I learn from critiquing others to improve my own investigation.

Words to know

constructive critique
Feedback that identifies strengths and weaknesses with reasons and suggests specific improvements, aiming to make the work better.
peer review
Evaluation of work by others with similar knowledge. In science, research is peer-reviewed before publication.
overclaiming
Stating a conclusion more strongly or more widely than the evidence supports, such as claiming cause from a survey.
truncated axis
An axis that starts at a value other than 0, which can exaggerate differences between bars.
voluntary response sample
A sample made of people who chose to respond (for example, to a social media poll); it is usually biased.

Seeing with fresh eyes

It is hard to spot weaknesses in your own work, because you know what you meant. Other people see what you actually wrote. That is why universities and journals use peer review, and why the curriculum asks you to provide constructive critiques of the work of others, critiquing their mathematical work in a constructive way.

Critiquing well is a skill. It needs your statistical knowledge (to see what is wrong) and care (so the person can use your feedback). In this lesson Amira reviews a classmate's project, and you practise the same process.

Questionsignificant?precise?Datarandom? enough?biased? ethical?Analysisright tools?correct numbers?Conclusionsmatch evidence?limits stated?Communicationclear graphs?sources cited?For each area: name one strength, ask one question, make one suggestion with a reason.
Five areas to check in any investigation, each with two key questions.

How to give feedback people can use

  • Be specific. "Your graph is confusing" does not help. "Figure 2's y-axis starts at 70%, which makes a 6-point difference look huge" does.
  • Give reasons. Say why something matters: what could go wrong, what a reader might wrongly conclude.
  • Start with a real strength. Not as sugar-coating, but because knowing what works is useful too.
  • Ask questions when you are unsure: "How were the 15 students chosen?" may reveal a good answer.
  • Suggest a fix that is realistic for a student project.
  • Critique the work, not the person. Say "the conclusion overstates the evidence", not "you exaggerated".
  • Prioritize. Two or three important points beat ten small ones.

A simple format for each point: strength, question, suggestion (with reason).

Worked example

The model investigation, Stage 25: a project to critique

Daniel's project (a fictional example): Question: "Does listening to music while studying improve marks?" Method: Daniel posted a poll on social media and messaged 15 friends who answered. Each reported the minutes of music they listen to while studying on a usual day and their latest math test mark. Results: the 9 who listen to music had a mean mark of 82%; the 6 who don't had a mean of 76%. He found r = 0.45 between minutes of music and mark, which he called "a strong correlation". His bar graph (below, left) starts its axis at 70%. Conclusion: "Listening to music causes higher marks, accurate to ±3%, 19 times out of 20. All students should study with music." Critique it in the five areas.

  1. Question: interesting and relevant to students (strength). But "improve" implies cause, which a survey cannot test; "music" is not defined (any genre? lyrics?).
  2. Data: a voluntary response sample of friends from social media is not random, so it cannot represent any population. n = 15 is very small (9 and 6 in the groups). Marks were self-reported, possibly rounded up. Did the friends consent to their marks being used?
  3. Analysis: the group means and r are reasonable summaries, and r was calculated correctly (strength). But r = 0.45 is moderate, not strong (|r| between 0.33 and 0.67), and r² ≈ 0.20, so about 80% of the variation in marks is unexplained. No standard deviations, so we cannot tell whether a 6-point difference is large compared with the spread.
  4. Conclusions: overclaiming twice: "causes" (observational data; lurking variables such as study habits, subject, or motivation could explain it) and "all students" (a friend group cannot be generalized). The ±3% is copied from nothing: a margin of error needs a random sample, and with n = 15 it would be far larger than 3%.
  5. Communication: the truncated axis makes 82% look more than twice as large as 76% (bars of 12 and 6 units above 70). n is not shown. The question and results are stated clearly (strength).

Strengths: an engaging question, clear statement of results, correct calculations. Main issues: non-random voluntary sample, causal and general overclaiming, misused margin of error, truncated axis.

Daniel's graph: axis starts at 70%70%75%80%85%90%95%100%76%No music82%MusicHonest version: axis starts at 00%20%40%60%80%100%76%No music82%MusicMusic bar looks twice as tallDifference looks as it is: 6 points
The same two means (made-up example) graphed two ways. Starting the axis at 70% makes a 6-point difference look like a doubling.
Worked example

The model investigation, Stage 26: Amira's written feedback

Turn the critique into three pieces of constructive feedback for Daniel, using strength, question, suggestion.

  1. 1. Question and sample. Strength: "Your question is one lots of students care about, and you stated it clearly." Question: "How did you choose the 15 people? Could people who like music have been more likely to answer your poll?" Suggestion: "Use a random sample from a class list (with permission) and report the response rate; then you can describe a real population. Because a survey can't show cause, you could rephrase the question as 'Is listening to music while studying associated with marks?'"
  2. 2. Correlation and conclusions. Strength: "Your r calculation is correct." Question: "What does r² tell you about how much of the variation in marks music explains?" Suggestion: "Describe r = 0.45 as moderate, report r² ≈ 0.20, and say 'associated with' instead of 'causes', since other factors like study habits could explain both. Limit the conclusion to the people you surveyed, and remove the ±3%: a margin of error needs a random sample, and with 15 people it would be much wider."
  3. 3. Graph. Strength: "The bars are labelled clearly." Question: "What happens to the picture if the axis starts at 0?" Suggestion: "Start the axis at 0 (or clearly mark a break), show n for each group, and add standard deviations so readers can judge whether 6 points is a big difference."
  4. Tone check: every sentence is about the work, each point has a reason, and the suggestions are doable for a student.

Three focused, respectful points, each with a strength, a question and a specific suggestion with a reason.

Receiving critiques of your own work

When you receive feedback, thank the reviewer, read it all before reacting, and sort each point: agree and fix, agree and add as a limitation, or disagree and explain why (using evidence, as in Answering Questions and Defending Your Reasoning). Keep a short "changes made" list. Many teachers value a revised report that shows what you learned from feedback.

A quick checklist for spotting common problems

  • Is the sample random? Who could be missing?
  • Are n, means and spreads reported?
  • Do words match evidence: "causes", "proves", "all"?
  • Is r described correctly (strength by |r|; r² for variation explained)?
  • Is a margin of error used only with a random sample, and calculated correctly?
  • Do graphs start at 0 (for bars), have labels, units and sources?
  • Are calculations correct? Check at least one yourself.
Just for fun

A good reviewer is like a good spell-checker: it finds the problems, suggests the fix, and never says "wow, you really can't spell."

Hands-on activity

Your investigation, step 13: peer review exchange

You need: a classmate's or friend's draft report (or, if you are learning alone, a news article that reports a survey or study), the five-area checklist, and your own draft.

  1. Read the work once for the big picture. Then read it again with the five-area checklist.
  2. Check at least two calculations yourself (a mean, a percentage, a margin of error or r).
  3. Write three pieces of feedback using strength, question, suggestion with a reason.
  4. Read your feedback as if you were receiving it. Remove anything that is about the person rather than the work.
  5. Exchange: give your feedback, receive feedback on your own draft, and make a changes made list.
  6. Revise your report and presentation.

If you are working alone: critique a news story that reports a statistic (look for sample size, who was surveyed, and the exact wording of the claim), then ask a family member to critique your draft using the checklist.

Your turn

1

Rewrite as constructive feedback: "Your sample is bad."

2

Daniel's r = 0.45. Find r² and interpret it.

3

Why is "accurate to ±3%, 19 times out of 20" wrong in Daniel's conclusion? Estimate a more realistic margin of error for a proportion with n = 15.

4

In Daniel's truncated graph, how many times taller does the music bar look than the no-music bar?

5

Give one lurking variable for Daniel's association.

6

State the three parts of the feedback format in this lesson.

More practice: warm-up

1

Name the five areas to critique.

2

Is a social media poll a random sample?

3

Is r = 0.45 weak, moderate or strong (by this course's classroom guide)?

4

Should feedback be about the work or the person?

5

What is peer review?

More practice: on your own

1

A project reports "Mean commute: 23 minutes" with no other statistics. What two questions would you ask?

2

A student's report concludes, "Taller students are better at basketball because r = 0.8." Write a constructive comment.

3

You check a classmate's mean: values 6, 8, 9, 10, 12 and their mean is 9.5. Is it right? Phrase your feedback.

4

How should Amira respond to a reviewer who says her conclusions should apply to "all Ontario students"?

5

Suggest a realistic way Daniel could test whether music causes changes in performance.

Challenge

1

Daniel's groups: music (n = 9) mean 82, s = 9; no music (n = 6) mean 76, s = 10. Express the difference in units of standard deviation and comment on the strength of evidence compared with Amira's.

2

Write a short critique of this news headline: "Survey: 70% of Canadians love winter!" (an online poll on a ski-resort website, 2000 responses).

3

Explain why critiquing other students' work can improve your own investigation, with two specific examples.

Check your understanding

Quiz

1.

Which feedback is most constructive?

2.

Daniel's sample of friends who answered a social media poll is a:

3.

Daniel called r = 0.45 "strong". The best correction is:

4.

Which conclusion is overclaiming for a survey of 15 friends?

5.

Critiquing "the work, not the person" means saying:

6.

When you receive a critique you disagree with, you should:

For parents and teachers

Big idea (E2.4): constructive critique is specific, reasoned and respectful: it names strengths, asks questions and suggests realistic fixes, focusing on the work rather than the person.

How to help: Pick a news story that quotes a survey and critique it together using the five areas (question, data, analysis, conclusions, communication). Model kind, specific feedback when you read your teen's draft, and invite them to critique something you wrote.

About the example: Daniel and his project are fictional and invented to show common mistakes.

Common difficulties:

  • Feedback that is vague ("good job", "confusing").
  • Pointing out only flaws, or only praise.
  • Not checking any calculations.
  • Taking critiques of one's own work personally: remind your teen that every researcher, at every level, gets critiqued.