A. Programming Concepts and Skills · Data Types, Files and Expressions

Binary Files and JSON

A3.1 A1.2 A1.4 A1.5About 75 minutes

Learning goals

I can explain the difference between text and binary files and convert between str and bytes.
I can write and read fixed-size binary records with struct, and jump to a record with seek.
I can save and load structured data, including lists of records, as JSON.
I can choose a suitable file format for a task and justify the choice.

Words to know

binary file
A file read and written as raw bytes, without any text encoding or line-ending changes ("rb", "wb").

bytes
Python's type for a sequence of raw byte values 0 to 255, written like b"Hi".

encode / decode
Convert text to bytes (s.encode("utf-8")) and bytes back to text (b.decode("utf-8")).

`struct`
A standard module that packs values into a fixed binary layout and unpacks them again, described by a format string such as "<10sih".

endianness
The order of bytes in a multi-byte number: little-endian stores the least significant byte first.

random access
Jumping directly to any position in a file with seek(offset), instead of reading from the start.

JSON
JavaScript Object Notation: a text format for nested objects, arrays, strings, numbers, true, false and null.

serialization
Turning data in memory into a form that can be saved or sent (and deserialization turns it back).

Beyond lines of text

Text files are great for people, but not for everything. A photo, a song or a game's level map is not made of lines of characters. Millions of sensor readings would waste space as digit characters. A settings file with options inside options does not fit neatly into CSV columns.

This lesson covers two more kinds of external file from the curriculum: binary files, which store raw bytes in an exact layout, and JSON, a text format for nested data that is used by almost every web service.

The number 300 stored two ways. A text file stores the characters "3", "0", "0" (codes 0x33, 0x30, 0x30). A binary file stores the value in a fixed number of bytes; little-endian puts the low byte (0x2c) first.

Bytes and binary mode

In Python 3, text (str) and raw data (bytes) are different types:

"é".encode("utf-8") gives b'\xc3\xa9' (two bytes).
b'\xc3\xa9'.decode("utf-8") gives "é".
Indexing a bytes object gives an int from 0 to 255.

Open a file with "rb" or "wb" to read or write bytes. Python then does no encoding and no newline conversion: you get exactly what is on the disk. (Do not give an encoding in binary mode.)

data = "Montréal 300".encode("utf-8") # str -> bytes

print(data, len(data))

print(list(data[:3]), data[:7].hex(" "))

print(data.decode("utf-8")) # bytes -> str

with open("blob.bin", "wb") as f:

f.write(bytes([137, 80, 78, 71])) # any byte values 0-255

with open("blob.bin", "rb") as f:

print(f.read())

bytes are raw numbers 0 to 255. Encoding text gives bytes (é takes 2 bytes in UTF-8, so 12 characters become 13 bytes); decoding turns them back into text. Files opened with "b" read and write bytes, with no encoding and no newline changes. The four bytes written here happen to be the start of every PNG image file. Output, line by line: b'Montr\xc3\xa9al 300' 13 | [77, 111, 110] 4d 6f 6e 74 72 c3 a9 | Montréal 300 | b'\x89PNG'

Fixed-size records with struct

The struct module packs values into an exact byte layout described by a format string:

< little-endian (the order most PCs use), > big-endian.
i 4-byte signed int, h 2-byte signed int, q 8-byte signed int.
f 4-byte float, d 8-byte float.
10s a 10-byte string (bytes), padded with zero bytes.

These are exactly the integer bounds and float precisions from Lessons 5 and 6. Packing 40,000 with h (maximum 32,767) raises struct.error; a long name is cut to 10 bytes. Choosing field sizes is a design decision.

When every record has the same size, record i starts at byte i × size. f.seek(i * SIZE) jumps straight there. That is how databases and many file formats find data quickly.

A binary file of fixed-size records. Each record packs a 10-byte name, a 4-byte integer and a 2-byte integer (shown in hexadecimal: "Amara", 12 goals, number 7). Because every record is exactly 16 bytes, the program can jump to any record without reading the ones before it.

import struct

FMT = "<10sih" # little-endian: 10-byte str, 4-byte int, 2-byte int

SIZE = struct.calcsize(FMT) # 16

roster = [("Amara", 12, 7), ("Ben", 8, 19), ("Chloé", 15, 4)]

with open("roster.dat", "wb") as f:

for name, goals, number in roster:

f.write(struct.pack(FMT, name.encode("utf-8"), goals, number))

with open("roster.dat", "rb") as f:

f.seek(2 * SIZE) # jump to record 2

raw_name, goals, number = struct.unpack(FMT, f.read(SIZE))

name = raw_name.rstrip(b"\0").decode("utf-8")

print(SIZE, name, goals, number)

Fixed-size binary records with struct. pack turns values into exactly 16 bytes; unpack turns them back. seek moves to any byte position, so record 2 is read directly (random access). The name is padded with zero bytes, which rstrip(b"\0") removes. Output: 16 Chloé 15 4

Worked example

Example 1: How big is the binary file, and where is record 999?

A weather station stores readings as binary records with format "<iff": a 4-byte integer time stamp, then two 4-byte floats (temperature and wind). It has logged one record a minute for 30 days. How large is the file, where does record 999 start, and how does the float size affect precision?

struct.calcsize("<iff") is 4 + 4 + 4 = 12 bytes per record (with < there is no padding).
Records: 30 days × 24 h × 60 min = 43,200. Size: 43,200 × 12 = 518,400 bytes, about 0.5 MB.
Record 999 starts at byte 999 × 12 = 11,988, so f.seek(11988) then f.read(12) gets it directly.
f in struct is a 4-byte single-precision float: only about 7 significant digits, compared with about 15 to 17 for Python's own 8-byte floats (Lesson 6). For temperatures to 0.1 °C that is plenty; for a bank balance it would not be.
The i time stamp is a signed 32-bit int, so it shares the Year 2038 limit from Lesson 5 if it counts seconds since 1970. Using q (8 bytes) avoids that.

12 bytes per record, 518,400 bytes in total, record 999 at byte 11,988. Choose field sizes with Lessons 5 and 6 in mind.

JSON for structured data

JSON looks like Python dictionaries and lists. It is plain text, so people can read it, and every major language can load it.

json.dump(data, f) writes; json.load(f) reads. (dumps and loads work with strings.)
Types map like this: dict ↔ object, list ↔ array, str ↔ string, int and float ↔ number, True/False ↔ true/false, None ↔ null.
Tuples become lists, and dictionary keys always become strings. Dataclass objects must be converted with dataclasses.asdict first.
indent=2 makes the file readable; ensure_ascii=False keeps letters such as é as they are (use with encoding="utf-8").

import json

from dataclasses import dataclass, asdict

@dataclass

class Player:

name: str

number: int

goals: int

captain: bool = False

team = {"team": "Northern Lights", "season": 2026,

"players": [asdict(Player("Amara", 7, 12, True)), asdict(Player("Ben", 19, 8))]}

with open("team.json", "w", encoding="utf-8") as f:

json.dump(team, f, ensure_ascii=False, indent=2)

with open("team.json", encoding="utf-8") as f:

loaded = json.load(f)

players = [Player(**p) for p in loaded["players"]]

print(players[0])

print(json.dumps({"ok": True, "missing": None, "pair": (1, 2), 3: "x"}))

JSON saves nested dictionaries and lists as readable text. asdict turns a dataclass record into a dictionary, and the double asterisk in the list comprehension unpacks each dictionary back into Player arguments. The last line shows the conversions: True → true, None → null, a tuple → a list, and the integer key 3 → the string "3". Output, line by line: Player(name='Amara', number=7, goals=12, captain=True) | {"ok": true, "missing": null, "pair": [1, 2], "3": "x"}

Security note

Python's pickle module can save almost any object, but loading a pickle file can run code hidden inside it. Never load a pickle from a source you do not trust. JSON and CSV only contain data, so they are safe to read (though you still validate the values).

Worked example

Example 2: Text, binary or JSON?

Choose a file format for each, and justify: (a) a high-score table edited by hand, (b) 10 million sensor readings needing fast random access, (c) a game's settings with nested options, sent to a web service.

(a) A text or CSV file: people can open and edit it in any editor, and it is small.
(b) A binary file of fixed-size records: compact (no digit characters or commas), and any record can be reached with seek in one step. A database (Lesson 13) is another good choice.
(c) JSON: it handles nesting (dictionaries inside lists inside dictionaries), it is readable, and nearly every language and web service understands it.
Trade-off to remember: text formats are portable and readable but larger and slower to parse; binary formats are compact and fast but need a precise description of the layout (field sizes, byte order) to be read correctly.

(a) CSV/text, (b) binary records (or a database), (c) JSON.

Just for fun

Why do binary files make terrible comedians?

Their jokes only make sense to people who know the format string. Everyone else just sees \x00\x00\x00.

Hands-on activity

Look inside real files

You need: Python and a few files already on your computer: a photo (.jpg or .png), a short text file, and a .json or .csv file if you have one (many apps export them).

Step 1. Open each file in binary mode and print the first 16 bytes with f.read(16).hex(" "). PNG files start with 89 50 4e 47; JPEG files start with ff d8 ff. Compare with what you see.

Step 2. Write your own list of 5 records (for example, books with title, year and pages) to a binary file with struct, then read back only record 3 using seek.

Step 3. Save the same records as JSON with indent=2, open the JSON file in a text editor, and compare its size (os.path.getsize) with the binary file.

What to look for: correct magic numbers; a binary file whose size is exactly 5 × struct.calcsize(FMT); and an explanation of why the JSON file is larger but easier to read.

Your turn

What is len("café".encode("utf-8"))? Why is it not 4?

What is struct.calcsize("<hhi")?

A file holds 20-byte records. At which byte does record 7 start (counting from 0)?

What does json.dumps([1, None, True]) produce?

After json.loads('{"5": "x"}'), is the key the int 5 or the string "5"?

Why must you not pass encoding="utf-8" when opening a file with mode "rb"?

What goes wrong with struct.pack("<h", 40000)?

More practice: warm-up

Which mode writes a binary file?

What method turns a str into bytes?

What does json.load return for a JSON object?

What file method jumps to a byte position?

What JSON value represents Python's None?

More practice: on your own

Explain why a 16-byte binary record format cannot store the name "Alexandria-Rose" in its 10s field, and what happens.

Write code that reads every record from roster.dat (format <10sih) in a loop until the end of the file.

A teammate saves a list of Player dataclass objects with json.dump(team, f) and gets TypeError: Object of type Player is not JSON serializable. Explain and fix.

The 2-byte value 01 2c is read with format "<h" and with ">h". Give both results.

Why is a JSON file usually larger than the same data in a binary file?

Challenge

Write update_goals(path, index, new_goals) that changes only the goals of record index in roster.dat without rewriting the whole file.

Explain why the variable-length format of a CSV file makes it impossible to jump straight to line 1 000 000, and how a program could make that fast anyway.

JSON has no date type. Propose a way to store a due date in JSON so that it sorts correctly as text and loads back as a datetime.date.

Check your understanding

Quiz

What is "é".encode("utf-8")?

Which file mode reads raw bytes?

Records of 24 bytes each are stored one after another. Where does record 10 begin (first record is 0)?

Which Python value becomes JSON null?

Which format best suits nested settings that a web service will read?

Why should you never load a pickle file from an unknown source?

For parents and teachers

Big idea: A3.1 lists text files, binary files, databases and XML files. This lesson covers binary files and JSON (a widely used structured text format), and links back to integer bounds and float precision (A1.4), type conversion between text and bytes (A1.2) and records (A1.5).

How to help: the "look inside real files" activity is safe as long as files are only read. Remind your teen to work on copies if they experiment with writing into existing files.

Common difficulties:

Mixing str and bytes (for example, writing a string to a file opened with "wb").
Forgetting that struct fields have fixed sizes and silently truncate strings.
Expecting JSON to restore tuples, integer keys or dataclass objects automatically.
Using the wrong byte order when reading binary data from another system.