Project ideas

Blueprints for builds I plan to make next: the problem, the architecture, the steps and how to tell when it works. When one is built, it moves to Projects with real results.

~/ideas

BlueprintAdvanced

Real-Time Clickstream Analytics with Kafka, Spark and Delta Lake

A planned streaming pipeline: website events flow through Kafka into Spark Structured Streaming, land as Delta tables with exactly-once guarantees, and feed a live sessions dashboard — with late data, replays and schema changes handled deliberately.

PythonKafkaApache SparkDelta LakeDockerPostgreSQLApache Airflow

BlueprintIntermediate

RAG Assistant That Answers From These Notes, With Citations

A planned retrieval-augmented assistant over this site's technical notes: structure-aware chunking, hybrid search, answers that cite the exact note section, and an evaluation set that gates every change.

PythonFastAPIHugging FaceQdrantLLMsRAGDocker

BlueprintAdvanced

Agentic Data-Quality Investigator with LangGraph and MCP

A planned AI agent that picks up a failed data-quality check, investigates with read-only SQL and lineage tools exposed over MCP, and drafts an incident note with evidence — while a human approves anything beyond reading.

PythonLangGraphMCPClaudeSQLPostgreSQLApache Airflow