JSON in Python: Reading and Writing Files, CSV, pathlib and UTF-8 Encoding

Working with JSON in Python requires no third-party libraries: the json module is part of the standard library and turns dictionaries and lists into text and back. Almost always you also need three more things: the open function with the with construct, the pathlib module for paths and the csv module for tables. Below is everything in order, with working code and the pitfalls people most often stumble on.

The open function in Python: modes and encoding

Any text file is opened with the built-in function open(path, mode, encoding=...). It returns a file object that has the read, readline and write methods and that you can iterate over line by line in a loop.

Why you need with

A file must be closed: otherwise the data may not be written to disk, and the system keeps the handle busy. The with construct closes the file automatically — even if an error happens inside the block.

with open("notes.txt", "w", encoding="utf-8") as f:
    f.write("First line\n")
    f.write("Second line\n")

with open("notes.txt", encoding="utf-8") as f:
    for line in f:
        print(line.rstrip("\n"))

Iterating over a file in a loop reads it one line at a time without loading it entirely into memory, so you can process even gigabyte-sized logs this way. The f.read() method returns the whole content as one string, and f.readlines() returns a list of lines.

How to write a line to a file without erasing the old content

The most common beginner mistake is opening a file in "w" mode to "add" a line. The "w" mode empties the file at the moment it is opened, before the first write. To append a record to the end, use "a", and to make sure you never overwrite someone else's file, use "x".

with open("log.txt", "a", encoding="utf-8") as f:
    f.write("new record\n")

Note that write does not add a line break by itself — "\n" must be written explicitly. You can also use print(..., file=f), which adds the line break for you.

UTF-8 encoding: why "mojibake" appears

Before Python 3.15, the open() function without the encoding parameter used the system encoding. On Linux and macOS that is UTF-8, while on Windows with a Russian locale it is cp1251. A script that worked fine on a server would print "Еда" instead of "Еда" on a colleague's laptop, or crash with a UnicodeDecodeError.

In Python 3.15, under PEP 686, UTF-8 mode is enabled by default: open() without encoding reads and writes UTF-8 on all operating systems. You can restore the old behaviour with the environment variable PYTHONUTF8=0 or the flag -X utf8=0, and explicitly request the system encoding with encoding="locale".

import sys
print(sys.flags.utf8_mode)   # 1 in Python 3.15

open("legacy.txt", encoding="locale")   # system encoding
open("old_excel.csv", encoding="cp1251")  # a file saved in cp1251

The flip side: an old CSV from Excel or a Windows script saved in cp1251 will now be read as UTF-8 and cause a UnicodeDecodeError. The fix is to read it with encoding="cp1251" and save it again as UTF-8. The good practice stays the same — always specify encoding explicitly, and the code then behaves identically on any version.

pathlib in Python: paths as objects

Joining paths as strings ("data" + "/" + name) is awkward and breaks on different operating systems. The pathlib module provides the Path class, where paths are joined with the / operator and the object has clearly named properties.

from pathlib import Path

data = Path("data")
csv_path = data / "expenses.csv"

print(csv_path.name)    # expenses.csv
print(csv_path.stem)    # expenses
print(csv_path.suffix)  # .csv
print(csv_path.parent)  # data
print(csv_path.exists())

data.mkdir(parents=True, exist_ok=True)
(data / "hello.txt").write_text("Hello", encoding="utf-8")
print((data / "hello.txt").read_text(encoding="utf-8"))

for p in data.glob("*.csv"):
    print(p)

The read_text and write_text methods replace the open + with pair for small files. A Path object can be passed to the ordinary open, to json and to csv — all of them accept it. To get a path relative to the script itself rather than the current folder, use Path(__file__).parent / "data".

In Python 3.14, Path gained the copy() and move() methods, so shutil is no longer needed for simple copying. In the same version the standard library added the compression.zstd module for Zstandard compression (PEP 784).

JSON in Python: load, dump, loads, dumps

JSON is a text format for exchanging data: APIs return it, and configs and reports are stored in it. The json module can do two operations: serialization (a Python object → JSON text) and deserialization (text → object). There are four functions, and confusion between them is a frequent question.

An easy way to remember: the letter s at the end stands for "string" — working with a string; without it — working with a file.

Type mapping

When read back, a tuple becomes a list, while the module does not know sets, dates and Decimal and raises TypeError: Object of type ... is not JSON serializable.

How to save JSON to a file

import json
from pathlib import Path

report = {
    "month": "2026-10",
    "total": 15230.5,
    "categories": {"food": 8200, "transport": 2100},
    "tags": ["home", "work"],
}

path = Path("data/report.json")
path.parent.mkdir(parents=True, exist_ok=True)
with path.open("w", encoding="utf-8") as f:
    json.dump(report, f, ensure_ascii=False, indent=2)

The ensure_ascii=False parameter keeps non-ASCII characters as they are. Without it, "Еда" in the file would appear as "Еда" — valid JSON, but impossible for a human to read. indent=2 produces nicely indented formatting.

How to read JSON from a file

with open("data/report.json", encoding="utf-8") as f:
    data = json.load(f)

print(data["total"])               # 15230.5
print(data["categories"]["food"])  # 8200

The result is an ordinary dictionary or list, and you work with it using the usual tools. If the file is not valid JSON or is truncated, a json.JSONDecodeError exception is raised with the line and position — it is worth catching when the data comes from outside.

Parsing a JSON string and pretty printing

import json

raw = '{"name": "Anna", "skills": ["python", "sql"], "active": true}'
user = json.loads(raw)
print(user["skills"][0])   # python

print(json.dumps(user, ensure_ascii=False, indent=2, sort_keys=True))

sort_keys=True sorts the keys on output — handy for comparing files in git. To quickly format a file from the terminal there is a built-in utility: python -m json.tool data/report.json.

The response of an HTTP API is also a JSON string. The requests library parses it with the response.json() method, and underneath it is the same json.loads.

A catch: JSON keys are always strings

d = {1: "January", 2: "February"}
back = json.loads(json.dumps(d))
print(back == d)   # False
print(back)        # {'1': 'January', '2': 'February'}
# back[1] -> KeyError, you need back["1"]

fixed = {int(k): v for k, v in back.items()}

In JSON, object keys can only be strings, so numeric keys turn into strings after a serialization round trip. Convert them back when reading.

Custom types: dates and Decimal

import json
from datetime import date

def default(obj):
    if isinstance(obj, date):
        return obj.isoformat()
    raise TypeError(f"cannot serialize {type(obj).__name__}")

print(json.dumps({"day": date(2026, 10, 1)}, default=default))
# {"day": "2026-10-01"}

The function passed to the default parameter is called for each unknown object. For more complex cases you can subclass json.JSONEncoder and override its default method.

How to append an item to a JSON array

Unlike a text log, JSON cannot simply be appended to at the end: the file has to be read, changed and written back in full.

path = Path("data/items.json")
items = json.loads(path.read_text(encoding="utf-8")) if path.exists() else []
items.append({"id": len(items) + 1, "title": "new"})
path.write_text(json.dumps(items, ensure_ascii=False, indent=2), encoding="utf-8")

If there are many records and they are added constantly, the JSON Lines format is more convenient: one JSON object per line, and the file is opened in "a" mode.

CSV in Python: the csv module

CSV is a table in a text file: rows are records, and values are separated by commas. The csv module handles quotes, commas inside values and line breaks correctly, which a manual split(",") does not.

import csv

with open("data/expenses.csv", newline="", encoding="utf-8") as f:
    reader = csv.DictReader(f)
    total = 0.0
    for row in reader:
        total += float(row["amount"])
        print(row["date"], row["category"], row["amount"])
print("Total:", total)

csv.DictReader turns each row into a dictionary keyed by the file's header. All values are strings: convert numbers and dates yourself (float, int, date.fromisoformat). A file with a ";" delimiter (as Excel saves it in many European locales) is read with the parameter delimiter=";".

rows = [
    {"date": "2026-10-01", "category": "food", "amount": 450},
    {"date": "2026-10-02", "category": "transport", "amount": 60},
]
with open("data/out.csv", "w", newline="", encoding="utf-8") as f:
    writer = csv.DictWriter(f, fieldnames=["date", "category", "amount"])
    writer.writeheader()
    writer.writerows(rows)

When writing, newline="" is essential: without it, empty lines appear between rows on Windows. If the file will be opened in Excel, use encoding="utf-8-sig" — the BOM marker helps Excel recognise UTF-8.

From CSV to JSON: a complete example

import csv
import json
from collections import defaultdict
from pathlib import Path

src = Path("data/expenses.csv")
dst = Path("data/report.json")

by_category = defaultdict(float)
with src.open(newline="", encoding="utf-8") as f:
    for row in csv.DictReader(f):
        by_category[row["category"]] += float(row["amount"])

report = {"total": sum(by_category.values()), "categories": dict(by_category)}
dst.write_text(json.dumps(report, ensure_ascii=False, indent=2), encoding="utf-8")
print(dst.read_text(encoding="utf-8"))

This is the core of any report script: CSV input, aggregation in a dictionary, JSON output.

Common mistakes

• FileNotFoundError — the path is relative and is resolved from the folder the script is launched from, not from the script itself. Use Path(__file__).parent.
• The file was wiped — it was opened in "w" instead of "a".
• UnicodeDecodeError — the file is in another encoding; specify it explicitly (cp1251, utf-8-sig).
• TypeError: Object of type set is not JSON serializable — convert it to a list or pass a default function.
• KeyError after loading JSON — numeric keys have become strings.
• Your own file is named json.py or csv.py — it shadows the standard module, and import json imports it instead.

Summary

For JSON in Python the standard library is enough: json.load/json.dump for files, loads/dumps for strings, ensure_ascii=False and indent for readable output. Open files with with and an explicit encoding, build paths with pathlib, and read tables with csv.DictReader. In our Python course this topic is practised hands-on: the expense tracker reads expenses from a CSV file and saves a report as JSON.