C. Organization of Data for Analysis · Organization of Data for AnalysisDesigning Experiments and Ethical Data Collection
C2.4 C2.1About 70 minutes
Learning goals- I can design a completely randomized, randomized block or matched-pairs experiment and explain when each is best.
- I can explain the placebo effect and the purpose of single-blind and double-blind designs.
- I can describe the ethical principles that guide research with people in Canada, including informed consent, privacy and confidentiality.
- I can identify ethical concerns in a proposed student survey or experiment and suggest how to address them.
- I can explain why research involving First Nations, Inuit and Métis communities follows additional principles such as OCAP®.
Words to know
completely randomized design
All experimental units are assigned to treatments completely at random.
randomized block design
Units are first sorted into blocks of similar units (by a variable that may affect the response); treatments are then randomly assigned within each block.
matched-pairs design
A design that compares two treatments using pairs of very similar units, or the same unit receiving both treatments in a random order.
placebo effect
An improvement in a subject's condition caused by believing they are receiving treatment, not by the treatment itself.
blinding
Keeping subjects (single-blind), or both subjects and the people measuring results (double-blind), from knowing who received which treatment.
informed consent
A participant's voluntary agreement to take part, given after they understand the purpose, procedures, risks and their right to withdraw.
research ethics board (REB)
A committee that reviews research involving people to make sure it is ethical before it begins.
OCAP®
First Nations principles of Ownership, Control, Access and Possession, which state that First Nations communities own and control data about them. OCAP® is a registered trademark of the First Nations Information Governance Centre.
Good science, done right
Last lesson, the principles of control, randomization and replication gave you a basic experiment: randomly split the units into groups and compare. Real researchers have more tools. If they know that some variable (age, location, soil) affects the response, they can block on it. If they can test both treatments on the same person, they can use matched pairs. And if people's expectations might affect the results, they blind the study.
But good design is not only about getting the right answer. When people (or animals) are involved, research must also be ethical. The curriculum asks you to design surveys and experiments with attention to ethics, privacy and honest responses, and to think about the concerns an ethical review of student surveys might raise.
Three experimental designs
1. Completely randomized design (CRD). Every unit is randomly assigned to a treatment. Simple and good when units are fairly similar.
2. Randomized block design. If some variable is likely to affect the response (grade level, a sunny versus a shady shelf, a field's soil type), first sort the units into blocks of similar units. Then randomly assign treatments within each block and compare treatments within blocks. Blocking removes the block variable's effect from the comparison, giving more precise results.
The statistician's motto: "Block what you can; randomize what you cannot."
3. Matched-pairs design. A special block design for two treatments, where each block is a pair of very similar units (twins, two plots side by side) or the same subject tested under both treatments in a random order. Differences between individuals are almost removed, because each comparison is made within a pair.
The placebo effect and blinding
People who believe they are being treated often feel better, try harder or report improvement even when the treatment does nothing. This is the placebo effect, and it is real and sometimes large. To separate it from the treatment's true effect:
- Give the control group a placebo (a sugar pill, a fake app, a sham procedure) that looks just like the real treatment.
- Single-blind: subjects do not know which treatment they received.
- Double-blind: neither the subjects nor the people who give the treatments and measure the results know who got what. This prevents researchers' expectations from influencing how they treat subjects or judge results.
Double-blind, placebo-controlled, randomized experiments are the standard for testing new medicines.
| Design feature | Protects against |
|---|
| Control group | Changes over time, practice effects, outside events |
| Random assignment | Lurking variables and researcher selection bias |
| Blocking | Known variables that affect the response |
| Placebo | The placebo effect in subjects |
| Double-blinding | Expectations of subjects and researchers |
| Replication (enough units) | Chance differences between individuals |
Ethics in research with people
In Canada, research with people at universities, hospitals and colleges must follow the Tri-Council Policy Statement: Ethical Conduct for Research Involving Humans (TCPS 2), and must be approved by a research ethics board (REB) before it starts. Many school boards also have their own process for approving research with their students. TCPS 2 is built on three core principles:
- Respect for persons: people take part only with free and informed consent. They must understand the purpose, what they will do, the risks and benefits, and that they can withdraw at any time without penalty. For young people, a parent's or guardian's consent is usually needed too, along with the young person's own agreement.
- Concern for welfare: minimize harm (physical, emotional, social). Protect privacy: collect only the information you need, keep it confidential and secure, and report results so that individuals cannot be identified.
- Justice: treat people fairly and equitably; do not place the burdens of research on some groups while others get the benefits.
Honesty is part of ethics too: report results accurately, including results you did not hope for, and do not use data for a purpose participants did not agree to.
Research with First Nations, Inuit and Métis communities
TCPS 2 includes a chapter on research involving First Nations, Inuit and Métis peoples. It calls for community engagement: communities help shape research that affects them and share in its benefits. First Nations have set out the OCAP® principles: Ownership, Control, Access and Possession. They state that First Nations communities own data about them and control how it is collected, used and shared. (You may remember OCAP® from Grade 9 Mathematics.) Inuit and Métis organizations have also set out their own research principles. These principles respond to a long history of research done on Indigenous communities without their consent or benefit, and they support research done with and by communities.
Worked exampleExample 1: Plant growth with blocking
You want to compare how bean plants grow under three conditions: (1) natural light, (2) natural light plus a desk lamp for 4 extra hours each evening, (3) a shaded spot. You have 12 plants, but they must sit on 4 different windowsills, and the windowsills get different amounts of sun. Design the experiment.
- Response: plant height (cm) after 21 days. Factor: light condition, 3 levels.
- Blocks: the 4 windowsills. Each windowsill gets 3 plants, one of each treatment, so each sill is a block of 3.
- Randomization within blocks: on each sill, roll a die for each plant: 1–2 = natural, 3–4 = lamp, 5–6 = shade (reroll if that treatment is already used on that sill). For the shade treatment, use a cardboard screen.
- Control: same pot, soil, seed variety, planting date and watering for all plants. The natural-light plants are the control group.
- Analysis: compare treatments within each windowsill, then average the differences over the 4 sills.
A randomized block design: 4 blocks (windowsills) × 3 treatments, randomized within each block. Differences between windowsills no longer get mixed up with the treatment effect.
Worked exampleExample 2: Choosing a design
Which design fits best? (a) Comparing two sunscreens: each volunteer has sunscreen A on one arm and B on the other. (b) Testing three study apps on 90 students from three different schools. (c) Testing whether a new paint dries faster than an old paint, using 40 identical boards.
- (a) Each person gets both treatments, one on each arm: matched pairs. Use a coin to decide which arm gets A, because one arm may get more sun (for example, the driver's-side arm).
- (b) Schools may differ in ways that affect learning: randomized block design, with schools as blocks; randomly assign the three apps to 10 students each within every school.
- (c) The boards are identical: a completely randomized design (randomly choose 20 boards for each paint) is enough.
(a) matched pairs; (b) randomized block (block by school); (c) completely randomized.
Worked exampleExample 3: An ethical review of a student survey
The curriculum asks for examples of concerns that could arise from an ethical review of surveys generated by students in a school. A student plans a survey asking classmates for their name, their family income, how often they feel anxious, and whether they have ever vaped, and plans to post the results with comments on the class website. List the ethical concerns and fixes.
- Consent: students must be told the purpose and that taking part is voluntary; students under 18 may need parent or guardian consent according to school or board rules. Fix: an information letter and consent process approved by the teacher or school.
- Privacy: asking for names is unnecessary and makes answers identifiable. Fix: make the survey anonymous.
- Sensitive questions: family income, anxiety and vaping are personal and could cause discomfort or harm if revealed. Fix: remove questions that are not essential; offer "prefer not to say"; include where to find support (such as a school counsellor) for the anxiety question; get the teacher's or administration's approval.
- Small groups: in a small class, results by group (for example "the two students who take the bus") could identify people. Fix: report only totals for groups of a reasonable size.
- Pressure: classmates may feel they cannot say no to a friend or when the teacher is watching. Fix: collect responses privately, with no record of who participated.
- Data security and use: posting comments publicly and keeping raw data indefinitely is risky. Fix: report summaries only, store raw data securely, delete them when the project ends, and use them only for the stated purpose.
An ethical review would ask for informed consent, anonymity, removal of unnecessary sensitive questions, protection of small groups, freedom from pressure, secure storage and summary-only reporting.
Just for funIn a double-blind taste test, neither the tasters nor the person serving the cups knows which cola is which. It's the only time in science that everyone being clueless is the whole point.
Hands-on activityBean sprout experiment with blocks
You need: 8 dried beans (such as kidney or lima beans from the grocery store), 8 clear plastic cups or small jars, paper towels, water, a ruler, a coin, and two different locations in your home (for example a kitchen windowsill and a bedroom windowsill).
Question: Do bean seeds sprout longer roots in the light or in the dark?
- Line each cup with a damp paper towel and place one bean between the towel and the cup wall so you can see it.
- Blocks: put 4 cups in location 1 and 4 cups in location 2.
- Randomization within blocks: in each location, flip a coin for each cup: heads = light, tails = dark (cover the cup with a box or foil); stop when a treatment has 2 cups, so each location has 2 light and 2 dark cups.
- Control: add the same amount of water to every cup each day (about 1 tablespoon) and keep the towels damp.
- After 7 days, measure each root's length to the nearest millimetre.
- Compare light and dark within each location, then overall. Write a conclusion and two limitations.
What to look for: a clear table with location, treatment and length, and comparisons within each block. Results vary by bean and home; the important thing is the design. Typical limitations: only 4 beans per treatment; temperature may differ between locations and between covered and uncovered cups.
Your turn
1Explain the difference between single-blind and double-blind experiments.
In a single-blind experiment the subjects do not know which treatment they get. In a double-blind experiment neither the subjects nor the researchers who treat and measure them know.
2A researcher tests a new fertilizer on fields that differ in soil type (clay, sandy, loam). Which design should she use, and what are the blocks?
A randomized block design with soil type as the blocks; within each soil type, fields are randomly assigned to the new fertilizer or the control.
3Why should the order of treatments be randomized in a matched-pairs design where each person tries both treatments?
Because the order can matter (people may improve with practice, get tired, or learn from the first trial). Randomizing the order prevents one treatment from always having the advantage of going second (or first).
4What is informed consent?
A person's voluntary agreement to take part in research, given after they understand its purpose, what they will do, the risks and benefits, and that they may withdraw at any time.
5Name the three core principles of TCPS 2.
Respect for persons, concern for welfare, and justice.
6What do the letters in OCAP® stand for?
Ownership, Control, Access and Possession.
More practice: warm-up
1In a block design, are treatments assigned randomly within blocks or between blocks?
2What is the placebo effect?
An improvement caused by believing you are being treated, not by the treatment itself.
3May a participant withdraw from a study after it starts?
Yes, at any time, without penalty.
4What committee reviews research with people before it starts?
A research ethics board (REB).
5Name one way to protect respondents' privacy.
Sample answers: anonymous surveys; collect only needed information; store data securely; report only group summaries.
More practice: on your own
1Twenty pairs of identical twins are used to test whether a memory game improves recall. Describe how to assign treatments.
Matched pairs: in each twin pair, flip a coin to decide which twin plays the memory game; the other twin is the control. Compare recall within each pair.
2A trial of a new headache medicine gives the medicine to one group and nothing to the other, and both groups know which they got. Explain the problem and fix it.
The placebo effect: the medicine group may report feeling better just because they took something. Give the control group an identical-looking placebo and make the trial double-blind, so neither patients nor the researchers judging improvement know who got what.
3In the randomized block figure in this lesson, why is it useful to block the 40 students by grade?
If Grade 12 students respond differently from Grade 11 students (for example, they have more study experience), mixing them could hide or exaggerate the treatment effect. Comparing A and B within each grade removes the grade difference from the comparison and gives more precise results.
4A student wants to measure classmates' heart rates before and after climbing three flights of stairs. List two ethical precautions.
Sample answers: get informed consent (and follow school rules on parent consent); let anyone with a health condition opt out without explanation; allow anyone to stop at any time; keep individual heart-rate data confidential; have a teacher supervise.
5Explain why research ethics require that results be reported honestly even when they are disappointing.
If only favourable results are reported, the published evidence becomes biased and people may make harmful decisions based on it (for example, using a drug that does not work). Participants also gave their time on the understanding that the research would contribute honest knowledge.
Challenge
1Design a double-blind experiment to test whether a new energy drink improves performance on a reaction-time test, using 60 volunteers who are either athletes (20) or non-athletes (40). Include blocking, randomization, a placebo and blinding.
Blocks: athletes and non-athletes (they may differ in reaction time). Within the athlete block, randomly assign 10 to the energy drink and 10 to a placebo drink that looks and tastes the same; within the non-athlete block, randomly assign 20 and 20. Drinks are prepared in identical coded cups by someone not involved in testing, so neither the volunteers nor the testers know who got which (double-blind). Everyone takes the same reaction-time test 30 minutes later. Compare the drink and placebo groups within each block, then combine.
2A matched-pairs study of 6 runners measures sole wear (mm) on brands A and B: runner 1: A 2.1, B 2.6; runner 2: A 1.8, B 2.0; runner 3: A 2.5, B 2.4; runner 4: A 1.6, B 2.1; runner 5: A 2.2, B 2.6; runner 6: A 1.9, B 2.3. Find the mean difference (B − A) and interpret it.
Differences B − A: 0.5, 0.2, −0.1, 0.5, 0.4, 0.4. Sum = 1.9. Mean = 1.9 ÷ 6 ≈ 0.32 mm. On average brand B wore about 0.3 mm more than brand A on the same runner; brand A wore less for 5 of 6 runners. With only 6 runners this is suggestive, not conclusive.
3A health researcher wants to study diabetes rates in several First Nations communities using data held by the communities' health centres. Using TCPS 2 and the OCAP® principles, describe how the research should be planned.
The research should be planned with the communities from the start: community leaders and members help decide the research questions, methods and how results will be used. The communities keep ownership of their data, control decisions about collection, use and publication, decide who has access, and keep possession (physical custody) of the data. Formal agreements set out these terms, community consent is obtained in addition to individual consent, results are shared with the communities first, and the work should benefit the communities. It also needs REB approval under TCPS 2.
Quiz
1. A researcher sorts 60 volunteers by age group, then randomly assigns treatments within each age group. This is a:
2. In a double-blind experiment:
3. Each of 25 students takes a memory test once with music and once without, in a random order. This is a:
4. Which is not part of informed consent?
5. The OCAP® principles are:
6. Why do drug trials give the control group a placebo?
7. An ethical review of a student survey would most likely object to:
Big idea (C2.4, C2.1): experiments can be made more precise by blocking (grouping similar units) or using matched pairs, and more trustworthy with placebos and blinding. Research with people must be ethical: informed consent, the right to withdraw, privacy, minimal harm, fairness and honest reporting. Research with First Nations, Inuit and Métis communities follows additional principles such as OCAP®.
How to help: If your teen plans a survey or experiment for the culminating investigation, act as an "ethics board": ask who will take part, how they will agree, whether any question could upset someone, and who will see the raw answers. The bean activity is an easy, safe experiment to run over a week.
Common difficulties:
- Confusing blocks (groups formed before randomizing, by a known variable) with treatment groups (formed by randomizing).
- Thinking a placebo is unethical. In a properly approved study with informed consent, participants know they might receive a placebo.
- Mixing up anonymity and confidentiality.
- Treating ethics as an afterthought. It belongs in the plan step, before any data are collected.