Reading and Writing JSON, CSV and Parquet in Python

Three formats, three jobs

CSV: always state the types

import csv
from decimal import Decimal
 
with open("orders.csv", newline="", encoding="utf-8") as f:
    for row in csv.DictReader(f):
        amount = Decimal(row["amount"])   # never float for money
        qty = int(row["qty"])

CSV has no types, no standard for nulls and many quoting dialects. Always parse explicitly and decide what an empty string means.

JSON Lines: one record per line

import json
 
with open("events.jsonl") as f:
    for line in f:                       # streams: memory stays flat
        event = json.loads(line)

JSONL appends and streams well, so it suits raw landing zones. A single large JSON array must be loaded whole.

Parquet: columns, compression, types

import pyarrow.parquet as pq
 
table = pq.read_table("sales.parquet", columns=["store_id", "net_sales"])  # reads 2 columns only

Parquet stores each column together. A query that needs 2 of 40 columns reads about 5% of the bytes, and min/max statistics let engines skip whole row groups.

Common mistakes

• Using float for money. Use Decimal or integer cents.
• Opening CSVs without newline="", which mangles embedded line breaks.
• Writing thousands of tiny Parquet files. Aim for roughly 128 MB–1 GB per file.