A Formal Test Plan, Part 1: Unit Testing with unittest

"It worked when I tried it"

In Grade 11 you wrote test plans with expected and actual results, and you used assert and small test loops. That works for one person and one program. A team project needs more: hundreds of tests, run automatically, by anyone, after every change, with results that are easy to read.

The curriculum asks you to develop and implement a formal testing plan (e.g., unit testing, integration testing, regression testing) for a software project to ensure program correctness (A4.2), and to produce the software according to specifications: code, test, deploy (B1.3). This lesson covers the plan and unit testing. The next lesson covers integration, regression and acceptance testing. (Testing the correctness of particular algorithms, such as searches and sorts, is part of the sibling unit Algorithm Design and Analysis.)

What a formal test plan contains

A formal test plan is a document, written during design and updated during development. Amara's plan for LendIt has these sections:

1. Scope: what will be tested (every Must and Should requirement, R1 to R7 and N1 to N4, plus any Could, R8 or N5, that gets built) and what will not (online booking, R9, which is out of scope).
2. Levels and approach: unit tests for every public function; integration tests for module pairs and the whole program; acceptance tests with the client; regression testing by re-running everything before every merge.
3. Test cases: each with an ID, the requirement it checks, the kind (normal, boundary, invalid), the input, the expected result, and later the actual result and pass/fail.
4. Environment: Python 3 on Windows (like the centre's laptop) and on the team's computers; sample data files.
5. Responsibilities and schedule: each module's author writes its unit tests; Amara reviews them and writes integration and acceptance tests; tests run at every merge.
6. Entry and exit criteria: testing of a module starts when it is code-complete; the release is ready when every Must requirement's tests pass and no known bug is rated "critical".
7. Bug reporting: how failures are recorded and tracked.

Choosing unit test cases

For each function, pick inputs from three groups (you used these in Grade 11):

• Normal: typical values (3 days late).
• Boundary: values on both sides of every edge in the rules. For the late fee: returned on the due date (0 days), one day late, exactly at the cap (20 days × $0.25 = $5.00), and beyond the cap.
• Invalid: values the function should reject. What should happen if returned is not a date? The interface should say, and a test should check it.

Also think about white-box cases: look at the code and make sure every branch and every loop path (zero times, once, many times) runs in at least one test.

Python's unittest module

Python includes a testing framework called unittest. You write tests as methods of a class that inherits from unittest.TestCase. (Classes and inheritance are taught in the sibling unit Modular and Object-Oriented Design; here you only need the pattern.) Rules:

• The test file is named test_<module>.py.
• Each test method's name starts with test_ and describes the behaviour being checked.
• Each test arranges inputs, acts (calls the function) and asserts the result.
• setUp (optional) runs before every test, giving each one fresh data.

Useful assertions: assertEqual(a, b), assertTrue(x), assertFalse(x), assertAlmostEqual(a, b) for floats (it rounds the difference to 7 decimal places), assertIn(item, collection), and assertRaises(ErrorType) for code that should raise an exception.

Here is LendIt's loans.py at the end of sprint 1. It builds on the design-stage version from the design documents lesson, adding days_late and can_borrow.

And here are its unit tests. Save both files in the same folder.

Run it with python3 test_loans.py (or press Run in your editor). unittest writes its report to the error stream, which most editors show in the same window. The time on the Ran line will be different on your computer.

You can also run every test file in a folder at once with python3 -m unittest. It finds every file named test_*.py, runs everything, and prints one dot per passing test: ........... for these 11 tests, then Ran 11 tests and OK.

When a test fails

Reading a failure report uses the same skills as reading any traceback (see Code Maintenance: Using Documentation to Fix Errors Independently in Data Types, Files and Expressions). Suppose a teammate "tidies" can_borrow and changes its last line to return out <= MAX_ITEMS. The tests now report:

Testing invalid input

What should late_fee do if someone passes the text "2027-01-21" instead of a date? Subtracting a string from a date raises a TypeError. The team decides that this is correct behaviour for a programming mistake (the menu, not loans.py, is responsible for turning user input into dates), and writes a test to pin that decision down:

• with self.assertRaises(TypeError): followed by an indented late_fee(self.due, "2027-01-21")

The test passes only if that call raises a TypeError. Input that users type is validated in main.py, and those checks get their own tests.

Tests first?

Many developers write the test before the code, an approach called test-driven development (TDD):

1. Red: write a test for the next small behaviour. Run it; it fails (the code does not exist yet).
2. Green: write just enough code to make it pass.
3. Refactor: tidy the code, re-running the tests to make sure it still works.

Whether you write tests first or right after, the rule for the team is the same: no function is "done" until it has tests (it is in LendIt's definition of done).