# JSON in Python: Reading and Writing Files, CSV, pathlib and UTF-8 Encoding

URL: https://skillaria.top/en/guides/python/json-python-files-csv

Python has a built-in json module for working with JSON: json.load reads data from an open file, json.dump writes it to a file, and json.loads and json.dumps do the same with strings. Open files with with open(path, encoding="utf-8"), build paths conveniently with pathlib.Path, and read tables with the csv module.

Working with JSON in Python requires no third-party libraries: the `json` module is part of the standard library and turns dictionaries and lists into text and back. Almost always you also need three more things: the `open` function with the `with` construct, the `pathlib` module for paths and the `csv` module for tables. Below is everything in order, with working code and the pitfalls people most often stumble on.

## The open function in Python: modes and encoding

Any text file is opened with the built-in function `open(path, mode, encoding=...)`. It returns a file object that has the `read`, `readline` and `write` methods and that you can iterate over line by line in a loop.

| Mode | What it does |
| --- | --- |

| `"r"` | Reading (the default). If the file does not exist — `FileNotFoundError` |

| `"w"` | Writing. The file is created or **emptied immediately on opening** |

| `"a"` | Appending to the end; existing content is left untouched |

| `"x"` | Create a new file; if it already exists — `FileExistsError` |

| `"b"` | A modifier for the mode: working with bytes (`"rb"`, `"wb"`) |

### Why you need with

A file must be closed: otherwise the data may not be written to disk, and the system keeps the handle busy. The `with` construct closes the file automatically — even if an error happens inside the block.

```
with open("notes.txt", "w", encoding="utf-8") as f:
    f.write("First line\n")
    f.write("Second line\n")

with open("notes.txt", encoding="utf-8") as f:
    for line in f:
        print(line.rstrip("\n"))
```

Iterating over a file in a loop reads it one line at a time without loading it entirely into memory, so you can process even gigabyte-sized logs this way. The `f.read()` method returns the whole content as one string, and `f.readlines()` returns a list of lines.

### How to write a line to a file without erasing the old content

The most common beginner mistake is opening a file in `"w"` mode to "add" a line. The `"w"` mode empties the file at the moment it is opened, before the first `write`. To append a record to the end, use `"a"`, and to make sure you never overwrite someone else's file, use `"x"`.

```
with open("log.txt", "a", encoding="utf-8") as f:
    f.write("new record\n")
```

Note that `write` does not add a line break by itself — `"\n"` must be written explicitly. You can also use `print(..., file=f)`, which adds the line break for you.

## UTF-8 encoding: why "mojibake" appears

Before Python 3.15, the `open()` function without the `encoding` parameter used the system encoding. On Linux and macOS that is UTF-8, while on Windows with a Russian locale it is cp1251. A script that worked fine on a server would print "Еда" instead of "Еда" on a colleague's laptop, or crash with a `UnicodeDecodeError`.

In Python 3.15, under PEP 686, UTF-8 mode is enabled by default: `open()` without `encoding` reads and writes UTF-8 on all operating systems. You can restore the old behaviour with the environment variable `PYTHONUTF8=0` or the flag `-X utf8=0`, and explicitly request the system encoding with `encoding="locale"`.

```
import sys
print(sys.flags.utf8_mode)   # 1 in Python 3.15

open("legacy.txt", encoding="locale")   # system encoding
open("old_excel.csv", encoding="cp1251")  # a file saved in cp1251
```

The flip side: an old CSV from Excel or a Windows script saved in cp1251 will now be read as UTF-8 and cause a `UnicodeDecodeError`. The fix is to read it with `encoding="cp1251"` and save it again as UTF-8. The good practice stays the same — always specify `encoding` explicitly, and the code then behaves identically on any version.

## pathlib in Python: paths as objects

Joining paths as strings (`"data" + "/" + name`) is awkward and breaks on different operating systems. The `pathlib` module provides the `Path` class, where paths are joined with the `/` operator and the object has clearly named properties.

```
from pathlib import Path

data = Path("data")
csv_path = data / "expenses.csv"

print(csv_path.name)    # expenses.csv
print(csv_path.stem)    # expenses
print(csv_path.suffix)  # .csv
print(csv_path.parent)  # data
print(csv_path.exists())

data.mkdir(parents=True, exist_ok=True)
(data / "hello.txt").write_text("Hello", encoding="utf-8")
print((data / "hello.txt").read_text(encoding="utf-8"))

for p in data.glob("*.csv"):
    print(p)
```

The `read_text` and `write_text` methods replace the `open` + `with` pair for small files. A `Path` object can be passed to the ordinary `open`, to `json` and to `csv` — all of them accept it. To get a path relative to the script itself rather than the current folder, use `Path(__file__).parent / "data"`.

In Python 3.14, `Path` gained the `copy()` and `move()` methods, so `shutil` is no longer needed for simple copying. In the same version the standard library added the `compression.zstd` module for Zstandard compression (PEP 784).

## JSON in Python: load, dump, loads, dumps

JSON is a text format for exchanging data: APIs return it, and configs and reports are stored in it. The `json` module can do two operations: serialization (a Python object → JSON text) and deserialization (text → object). There are four functions, and confusion between them is a frequent question.

| Function | What it does |
| --- | --- |

| `json.dumps(obj)` | Object → JSON string |

| `json.loads(s)` | JSON string → object |

| `json.dump(obj, f)` | Object → write to an open file |

| `json.load(f)` | Read JSON from an open file |

An easy way to remember: the letter `s` at the end stands for "string" — working with a string; without it — working with a file.

### Type mapping

| Python | JSON |
| --- | --- |

| `dict` | object `{}` |

| `list`, `tuple` | array `[]` |

| `str` | string |

| `int`, `float` | number |

| `True` / `False` | `true` / `false` |

| `None` | `null` |

When read back, a tuple becomes a list, while the module does not know sets, dates and `Decimal` and raises `TypeError: Object of type ... is not JSON serializable`.

### How to save JSON to a file

```
import json
from pathlib import Path

report = {
    "month": "2026-10",
    "total": 15230.5,
    "categories": {"food": 8200, "transport": 2100},
    "tags": ["home", "work"],
}

path = Path("data/report.json")
path.parent.mkdir(parents=True, exist_ok=True)
with path.open("w", encoding="utf-8") as f:
    json.dump(report, f, ensure_ascii=False, indent=2)
```

The `ensure_ascii=False` parameter keeps non-ASCII characters as they are. Without it, "Еда" in the file would appear as `"Еда"` — valid JSON, but impossible for a human to read. `indent=2` produces nicely indented formatting.

### How to read JSON from a file

```
with open("data/report.json", encoding="utf-8") as f:
    data = json.load(f)

print(data["total"])               # 15230.5
print(data["categories"]["food"])  # 8200
```

The result is an ordinary dictionary or list, and you work with it using the usual tools. If the file is not valid JSON or is truncated, a `json.JSONDecodeError` exception is raised with the line and position — it is worth catching when the data comes from outside.

### Parsing a JSON string and pretty printing

```
import json

raw = '{"name": "Anna", "skills": ["python", "sql"], "active": true}'
user = json.loads(raw)
print(user["skills"][0])   # python

print(json.dumps(user, ensure_ascii=False, indent=2, sort_keys=True))
```

`sort_keys=True` sorts the keys on output — handy for comparing files in git. To quickly format a file from the terminal there is a built-in utility: `python -m json.tool data/report.json`.

The response of an HTTP API is also a JSON string. The `requests` library parses it with the `response.json()` method, and underneath it is the same `json.loads`.

### A catch: JSON keys are always strings

```
d = {1: "January", 2: "February"}
back = json.loads(json.dumps(d))
print(back == d)   # False
print(back)        # {'1': 'January', '2': 'February'}
# back[1] -> KeyError, you need back["1"]

fixed = {int(k): v for k, v in back.items()}
```

In JSON, object keys can only be strings, so numeric keys turn into strings after a serialization round trip. Convert them back when reading.

### Custom types: dates and Decimal

```
import json
from datetime import date

def default(obj):
    if isinstance(obj, date):
        return obj.isoformat()
    raise TypeError(f"cannot serialize {type(obj).__name__}")

print(json.dumps({"day": date(2026, 10, 1)}, default=default))
# {"day": "2026-10-01"}
```

The function passed to the `default` parameter is called for each unknown object. For more complex cases you can subclass `json.JSONEncoder` and override its `default` method.

### How to append an item to a JSON array

Unlike a text log, JSON cannot simply be appended to at the end: the file has to be read, changed and written back in full.

```
path = Path("data/items.json")
items = json.loads(path.read_text(encoding="utf-8")) if path.exists() else []
items.append({"id": len(items) + 1, "title": "new"})
path.write_text(json.dumps(items, ensure_ascii=False, indent=2), encoding="utf-8")
```

If there are many records and they are added constantly, the JSON Lines format is more convenient: one JSON object per line, and the file is opened in `"a"` mode.

## CSV in Python: the csv module

CSV is a table in a text file: rows are records, and values are separated by commas. The `csv` module handles quotes, commas inside values and line breaks correctly, which a manual `split(",")` does not.

```
import csv

with open("data/expenses.csv", newline="", encoding="utf-8") as f:
    reader = csv.DictReader(f)
    total = 0.0
    for row in reader:
        total += float(row["amount"])
        print(row["date"], row["category"], row["amount"])
print("Total:", total)
```

`csv.DictReader` turns each row into a dictionary keyed by the file's header. All values are strings: convert numbers and dates yourself (`float`, `int`, `date.fromisoformat`). A file with a ";" delimiter (as Excel saves it in many European locales) is read with the parameter `delimiter=";"`.

```
rows = [
    {"date": "2026-10-01", "category": "food", "amount": 450},
    {"date": "2026-10-02", "category": "transport", "amount": 60},
]
with open("data/out.csv", "w", newline="", encoding="utf-8") as f:
    writer = csv.DictWriter(f, fieldnames=["date", "category", "amount"])
    writer.writeheader()
    writer.writerows(rows)
```

When writing, `newline=""` is essential: without it, empty lines appear between rows on Windows. If the file will be opened in Excel, use `encoding="utf-8-sig"` — the BOM marker helps Excel recognise UTF-8.

## From CSV to JSON: a complete example

```
import csv
import json
from collections import defaultdict
from pathlib import Path

src = Path("data/expenses.csv")
dst = Path("data/report.json")

by_category = defaultdict(float)
with src.open(newline="", encoding="utf-8") as f:
    for row in csv.DictReader(f):
        by_category[row["category"]] += float(row["amount"])

report = {"total": sum(by_category.values()), "categories": dict(by_category)}
dst.write_text(json.dumps(report, ensure_ascii=False, indent=2), encoding="utf-8")
print(dst.read_text(encoding="utf-8"))
```

This is the core of any report script: CSV input, aggregation in a dictionary, JSON output.

## Common mistakes

- `FileNotFoundError` — the path is relative and is resolved from the folder the script is launched from, not from the script itself. Use `Path(__file__).parent`.
- The file was wiped — it was opened in `"w"` instead of `"a"`.
- `UnicodeDecodeError` — the file is in another encoding; specify it explicitly (`cp1251`, `utf-8-sig`).
- `TypeError: Object of type set is not JSON serializable` — convert it to a list or pass a `default` function.
- `KeyError` after loading JSON — numeric keys have become strings.
- Your own file is named `json.py` or `csv.py` — it shadows the standard module, and `import json` imports it instead.

## Summary

For JSON in Python the standard library is enough: `json.load`/`json.dump` for files, `loads`/`dumps` for strings, `ensure_ascii=False` and `indent` for readable output. Open files with `with` and an explicit encoding, build paths with `pathlib`, and read tables with `csv.DictReader`. In our Python course this topic is practised hands-on: the expense tracker reads expenses from a CSV file and saves a report as JSON.

## FAQ

**How is json.load different from json.loads?**

json.load reads JSON from an open file object, while json.loads parses a string. Similarly, json.dump writes to a file, and json.dumps returns a string.

**How do I keep non-ASCII characters in JSON without \u sequences?**

Pass ensure_ascii=False to json.dump or json.dumps and open the file with encoding="utf-8".

**Do I need to install a library to work with JSON in Python?**

No, the json module is built into the standard library; you just write import json.

**Can I store a date or a set in JSON?**

Not directly: you have to convert them — for example a date to an ISO string with isoformat, and a set to a list — or pass a function to the default parameter.

**How do I read a large JSON file that does not fit in memory?**

The standard json.load reads the whole file; for large volumes store the data in JSON Lines format and parse it line by line with json.loads.


---
Source: Skillaria.Top (https://skillaria.top)