Research II: Evaluating Sources for Reliability, Objectivity and Completeness

Forty per cent better marks?

A post goes around your school: New study: students who start at 10 a.m. get 40% higher marks! It has thousands of shares. It would be perfect for your report on start times.

But where did 40% come from? Who did the study? Was it one school or a thousand? Did the marks rise because of the later start, or did something else change too?

The Grade 11 expectation 1.3 includes evaluating sources to determine their reliability, objectivity and completeness. That is a higher bar than Is it on a real-looking website? In this lesson you will learn to judge sources the way professional fact-checkers do.

Reliability: can I trust it to be accurate?

• Author and expertise. Is the author named? What are their qualifications for this topic? A doctor writing about sleep has relevant expertise; the same doctor writing about tax law does not.
• Evidence. Does it give data, name its sources, link to studies? Claims with no evidence are opinions, however confidently stated.
• Process. Was it edited (a reputable newspaper or magazine), peer-reviewed (a scholarly journal) or published by an accountable body (a government department, a university)? A personal blog or social post usually has no checking step.
• Track record. Does the publisher correct its mistakes publicly? Reputable news outlets publish corrections.

Objectivity: is it fair?

• Purpose. Is it trying to inform, persuade or sell? Persuasive sources can be useful, but you must know they are arguing.
• Sponsorship and funding. A study of a sports drink paid for by the drink's maker is not automatically wrong, but it needs extra checking. Look for sponsored, partner content or paid post labels.
• Language. Loaded words (disaster, miracle, so-called experts) signal an agenda.
• Balance. Does it fairly describe views it disagrees with, or only mock them?

Completeness: does it give the whole picture?

• Sample. How many people or places were studied? A survey of 30 students at one school cannot speak for all of Ontario.
• Limits. Good studies state their own limitations. Missing limits is a warning.
• Context. Are numbers given with a comparison (40% higher than what?)?
• Perspectives. Whose voices are included? Who is affected but not heard?

Read laterally: do what fact-checkers do

Researchers at Stanford University found that professional fact-checkers judged unfamiliar websites faster and more accurately than many highly educated readers, because they left the site almost immediately to see what other sources said about it. This is lateral reading. Reading vertically (staying on the page and studying its design and its own About us page) is easy to fool, because any site can describe itself as trustworthy.

Use the four SIFT moves (developed by digital-literacy educator Mike Caulfield):

1. Stop. Before you share or cite, ask whether you know this source. Notice your own reaction: a claim that makes you angry or delighted needs more checking.
2. Investigate the source. Open a new tab and search the organization or author's name. What do reliable sources say about who they are and who funds them?
3. Find better coverage. Search the claim itself. Do reputable sources report it? Is there a better, more original source?
4. Trace claims, quotes and media to the original context. Follow links back to the study, the full speech or the original photo. Check whether the claim was stretched or cut out of context.

Trace claims upstream

Information changes each time it is retold. A study finds a small improvement at one school. The university's press release calls it promising. A news article writes linked to better grades. A social post says 40% higher marks. Each step is a little less careful than the last.

When you trace back to the original, check three things:

• What exactly was measured? Attendance is not marks; marks in one course is not all marks.
• Correlation or causation? If two things happen together, one did not necessarily cause the other. Did the school change anything else at the same time?
• Size and setting. How many students, where, for how long?

Bias: theirs and yours

• Sponsorship bias: results lean toward whoever paid.
• Selection bias: the sample is not typical (an online poll answered only by people who care a lot).
• Omission bias: important facts or perspectives are simply left out.
• Confirmation bias (yours!): you search for, notice and believe sources that agree with your thesis. The fix is built into your research plan: a sub-question that deliberately looks for the other side.

Whose perspective?

Completeness also means hearing from the people most affected. If your topic affects First Nations, Métis or Inuit communities (for example, northern food prices, water systems, land use or language revitalization), look for sources created by those communities and their organizations, not only outside commentary. National and regional organizations such as Inuit Tapiriit Kanatami, the Assembly of First Nations, the Métis Nation of Ontario and the Chiefs of Ontario, as well as Indigenous-led media and Indigenous researchers, publish reports and statements. Many First Nations follow the OCAP® principles (ownership, control, access and possession) for research about their communities, set out by the First Nations Information Governance Centre. The same rule applies to any topic: a report on accessibility should include people with disabilities speaking for themselves.

A note on AI tools

AI chatbots can produce fluent answers that contain invented facts and invented sources (references to articles that do not exist). Never cite anything you have not opened and read yourself, and follow your teacher's rules about whether and how AI tools may be used. An AI summary is not a source; the documents it points to might be, once you have checked them.