A. Programming Concepts and Skills · Arrays and Subprograms

Testing and Debugging Programs with Lists and Functions

A4.1 A4.3 A4.4 A4.5About 75 minutes

Learning goals

I can tell syntax, run-time and logic errors apart and fix each kind.
I can interpret Python's error messages at different stages: before running, while running and while testing.
I can locate a logic error by tracing, by printing variable values, and with a debugger.
I can write a test plan with normal, boundary and special cases, and compare expected with actual output.

Words to know

syntax error
A mistake in the grammar of the language (a missing colon, an unclosed bracket, bad indentation). The program does not start.

run-time error
An error that happens while the program is running, which stops it with a traceback (e.g. IndexError, ZeroDivisionError).

logic error
A mistake that lets the program run to the end but gives the wrong result. Python cannot detect it; testing does.

test case
A set of input values plus the expected output.

boundary case
A test at the edge of a range, e.g. a mark of exactly 50 or a target at index 0.

debugger
A tool in an IDE that runs a program one line at a time (stepping), stops at breakpoints, and shows the values of variables.

breakpoint
A marked line where the debugger pauses the program so you can inspect it.

Bugs are normal; finding them is a skill

Professional programmers spend a large part of their time testing and debugging. The difference between a beginner and an expert is not that experts make no mistakes; it is that experts find them fast, with a calm, systematic method. The curriculum asks you to identify and correct syntax, logic and run-time errors, to interpret error messages, to trace programs, and to validate programs with a full range of test cases. You have practised pieces of this in every lesson. Here it all comes together.

Three kinds of errors

Syntax errors break the grammar of Python. Python checks the whole file before running anything, so the program does not start at all. Examples: SyntaxError: expected ':', SyntaxError: '[' was never closed, IndentationError: expected an indented block. Your editor often underlines them as you type.
Run-time errors happen while the program is running, at the exact line that fails. Everything before that line has already run. Examples with lists and functions: IndexError, TypeError, ValueError, ZeroDivisionError, NameError, UnboundLocalError.
Logic errors produce no message. The program finishes but the answer is wrong. Only testing, with an expected answer to compare against, reveals them.

A quick way to classify a bug. If nothing runs, it is a syntax error; if the program stops with a traceback, it is a run-time error; if it finishes with a wrong answer, it is a logic error.

Error (last line of the message)	Typical cause with lists and functions
IndexError: list index out of range	index ≥ len(a), often an off-by-one in a loop bound
TypeError: unsupported operand type(s) for +: 'NoneType' and 'int'	using the result of a function that prints instead of returning
TypeError: f() missing 1 required positional argument	too few arguments in a call
ValueError: invalid literal for int() with base 10: 'abc'	converting text that is not a number
ZeroDivisionError: division by zero	averaging an empty list
UnboundLocalError: cannot access local variable ...	assigning to a global name inside a function

Run-time errors you have met in this unit and what usually causes them.

Finding a logic error

Suppose a function should count the values greater than a limit. The comments show what each call should print.

def count_above(values, limit):

"""Return how many values are greater than limit."""

count = 0

for i in range(1, len(values)):

if values[i] >= limit:

count += 1

return count

print(count_above([5, 10, 15], 9)) # expected 2

print(count_above([12, 3, 7], 9)) # expected 1

print(count_above([4, 9, 20], 9)) # expected 1

Output: 2, 0, 2. The first test passes, but the second and third are wrong. One passing test proved nothing.

Three ways to locate the problem:

Trace by hand in a trace table, as you did in Traversing a List.
Print debugging: add temporary print lines that show the values of variables on each pass.
A debugger: in IDLE, Thonny or VS Code you can set a breakpoint, step one line at a time and watch variables change. Thonny's "Debug current script" button is especially easy for beginners.

def count_above(values, limit):

count = 0

for i in range(1, len(values)):

print(" i =", i, "value =", values[i], "count =", count)

if values[i] >= limit:

count += 1

return count

print(count_above([12, 3, 7], 9))

Output: i = 1 value = 3 count = 0, i = 2 value = 7 count = 0, then 0. The printout shows that index 0 (value 12) is never checked. The loop starts at 1.

Worked example

Example 1: Fix both bugs and re-test

Use the evidence to fix count_above, then explain the third wrong answer.

Bug 1 (from the printout): range(1, len(values)) skips index 0. Fix: loop over every element, for v in values:.
Bug 2 (from the third test, [4, 9, 20] with limit 9, which gave 2): 9 was counted. The specification says greater than, but the code uses >=. Fix: if v > limit:.
Re-run all the tests after a fix, not only the one that failed. A fix can break something else.
Remove the debugging prints when you are done.

Both bugs fixed: the function now returns 2, 1 and 1.

The debugging cycle. Testing shows a difference between expected and actual output; tracing or printing locates the cause; after the fix, run every test again.

Planning a full range of test cases

A test plan lists each test before you run it. Include:

Normal cases: typical data.
Boundary cases: values at the edges, such as exactly the limit, the first and last index, a mark of exactly 50.
Special cases: an empty list, a list with one element, all elements equal, negative numbers.
Invalid input (for programs that read input): letters instead of numbers, numbers out of range.

For each case, write the input, the expected output (worked out by hand) and, after running, the actual output.

def count_above(values, limit):

"""Return how many values are greater than limit."""

count = 0

for v in values:

if v > limit:

count += 1

return count

tests = [([5, 10, 15], 9, 2), ([12, 3, 7], 9, 1), ([4, 9, 20], 9, 1),

([], 9, 0), ([9, 9], 9, 0), ([-1, -5], -3, 1)]

for values, limit, expected in tests:

actual = count_above(values, limit)

result = "pass" if actual == expected else "FAIL"

print(values, limit, "expected", expected, "actual", actual, result)

Output: six lines, each ending in pass. Putting the test plan in a list and checking it with a loop is an automated test: it can be re-run after every change in a fraction of a second.

Boundary values on a number line

For a function with ranges, such as achievement_level(mark) from Writing Your Own Functions, the bugs hide at the edges. Test just below, at, and just above each boundary.

Boundary test marks for achievement_level: 49 and 50, 59 and 60, 69 and 70, 79 and 80. A mistake such as mark > 80 instead of mark >= 80 only shows up for a mark of exactly 80.

Worked example

Example 2: A test plan for achievement_level

Write a test plan for the function that returns "Level 4" for 80 to 100, "Level 3" for 70 to 79, "Level 2" for 60 to 69, "Level 1" for 50 to 59 and "below Level 1" under 50.

Normal: 92 → Level 4; 74 → Level 3; 65 → Level 2; 55 → Level 1; 30 → below Level 1.
Boundaries: 80 → Level 4 and 79 → Level 3; 70 → Level 3 and 69 → Level 2; 60 → Level 2 and 59 → Level 1; 50 → Level 1 and 49 → below Level 1.
Extremes: 100 → Level 4; 0 → below Level 1.
Special: what should 105 or −3 do? The specification does not say. Ask! Clarifying the specification is part of testing.

About 17 test cases, most of them at the boundaries.

Preventing run-time errors from bad input

A program that reads input should validate it rather than crash. One simple way checks the text before converting it:

def read_mark():

"""Keep asking until the user enters a whole number from 0 to 100."""

while True:

text = input("Mark (0-100): ")

if text.isdigit() and int(text) <= 100:

return int(text)

print("Please enter a whole number from 0 to 100.")

m = read_mark()

print("Saved", m)

Sample run: the user types abc, then 120, then 87. The program prints the help message twice, then Saved 87. (isdigit() rejects negative numbers and decimals too, which is what we want here. The Software Development unit adds exception handling with try and except.)

Just for fun

The word "bug" for a computer problem is older than you might think, but a famous moment came in 1947: Grace Hopper's team found a real moth stuck in a relay of the Harvard Mark II computer and taped it into their logbook, labelled the "first actual case of bug being found".

Hands-on activity

Break your partner's code

You need: a computer with Python and a partner (a family member is fine, even with no coding experience).

Choose a function you wrote in an earlier lesson (for example, the linear search or count_below).
On paper, write a test plan of at least 8 cases: normal, boundary, special and (if it reads input) invalid. Write the expected output for each before running anything.
Give your partner the plan and let them invent 3 more nasty cases ("What if the list is empty? What if every number is the same?").
Run every case and fill in the actual output. For each FAIL, use print debugging or the debugger in Thonny or IDLE to locate the cause, fix it, and re-run every test.

What to look for: expected outputs are written first; the student re-runs all tests after each fix, not just the failed one.

Your turn

Classify each error: (a) print("hi" (b) a = [1, 2] then print(a[2]) (c) an average that is always too big.

A program stops with ZeroDivisionError: division by zero in average(values). What test case caused it, and how do you fix it?

Why can't Python detect a logic error?

Name three special cases to test for any function that takes a list.

What is print debugging?

List the boundary test values for a function that is True for ages 13 to 17 inclusive.

What should you do after fixing one bug?

More practice: warm-up

What kind of error is IndentationError?

In a traceback, which line do you read first?

What is a breakpoint?

What two outputs does a test plan compare?

What does int("12a") raise?

More practice: on your own

This should return the largest element but returns 0 for [-4, -2, -9]. Find and fix the bug.

def largest(values):

best = 0

for v in values:

if v > best:

best = v

return best

This program crashes. Read the error and fix it.

names = ["Liam", "Sara", "Omar"]

for i in range(len(names) + 1):

print(names[i])

IndexError: list index out of range

Write a test plan (input and expected output) of 5 cases for is_sorted(values), which returns True if values is in ascending order.

Predict the output and name the bug:

def total(values):

for v in values:

t = 0

t += v

return t

print(total([4, 5, 6]))

Explain how to use a debugger to find the bug in the last question without adding prints.

Challenge

A function remove_all(values, x) uses for i in range(len(values)): and calls values.pop(i) when values[i] == x. Trace remove_all([5, 5, 1], 5) and explain the two problems it shows.

Write an automated test loop (like the one in this lesson) for achievement_level that runs at least 8 boundary cases and prints only the failures.

A program reads 5 marks and prints the average. It works when tested with 80, 70, 90, 60, 100. List four more tests that a full range of test cases should include and what you would expect each to show.

Check your understanding

Quiz

A program prints nothing at all and shows SyntaxError: expected ':'. What kind of error is it?

A function returns the wrong total but never crashes. What kind of error is this?

Which is a boundary test case for a function that accepts marks from 0 to 100?

What should a test plan contain for each case?

Which tool lets you run a program one line at a time and watch its variables?

for i in range(len(a) + 1): print(a[i]) will most likely cause:

After fixing a bug that one test revealed, you should:

For parents and teachers

Big idea: This lesson pulls together the code-maintenance expectations: A4.1 (identify and correct syntax, logic and run-time errors), A4.3 (interpret error messages from programming tools at different stages), A4.4 (tracing to find logic and run-time errors) and A4.5 (validate with a full range of test cases).

How to help: You do not need to code to help with testing. Ask your teen, "What is the strangest input someone could give this program?" and "What did you expect to happen?" Insist that expected outputs are written down before running.

Common difficulties:

Testing only one easy case and declaring the program finished.
Changing code randomly until the error goes away instead of locating the cause.
Forgetting to remove debugging prints.
Fixing the symptom (e.g. adding global or a try that hides an error) rather than the real bug.