About

What it is for

It asks how seven bands treat their own discography on stage: do they live off their past or bet on new material, do shows change from night to night, and how long does a song survive in the live repertoire. Two more questions, show structure and differences by country or venue type, are planned for later versions.

It is also a worked example of a small end-to-end data pipeline, with the method written down next to the charts.

How it is built

• Ingestion. Python clients for the setlist.fm and MusicBrainz APIs, orchestrated by Apache Airflow (LocalExecutor) in Docker Compose on an ARM64 virtual machine.
• Storage and transformation. PostgreSQL 16 and dbt Core. Raw setlist.fm data lives in an ephemeral schema truncated at the end of every run; staging and intermediate models are views; only aggregated marts persist.
• Analysis. pandas and lifelines (Kaplan-Meier) for rotation and song survival.
• This site. A static site generated from the marts with Python, Jinja2 and matplotlib (charts are inline SVG, with no JavaScript chart library), in Portuguese and English, on Cloudflare Pages. No API and no database at runtime.

Source

The code, the methodology notes and the specs are on GitHub. Built by alexarnoni.