01 / Data engineering · Analytics
CivicPulse Data Platform.
A pipeline for collecting and analyzing New York City 311 service requests, with data quality checks built into the workflow.
01 / THE PROBLEM
Context matters.
NYC 311 data is large and changes over time. Before it can support useful comparisons, records need to be collected consistently, loaded reliably, and checked for quality.
02 / APPROACH
From idea to workflow.
Loaded a 30-day backfill of 318,529 requests into PostgreSQL and began modeling the data with dbt. The next step is to finish Airflow orchestration for incremental extraction, loading, transformations, and tests. Early analysis compares open requests and resolution times across boroughs.
03 / TOOLS & METHODS
The stack.
04 / EVIDENCE
Show the work.
Add a real architecture diagram, dashboard screenshot, or model evaluation output here. Explain what the evidence shows and how it was produced.
Add a real diagram, chart, screenshot, or model output here.
05 / REFLECTION
What I learned.
Describe one technical decision, one challenge, and one limitation or next step. Specific details make the work easier to assess.
Airflow orchestration is still in progress. The current data load and dbt checks should be described separately from the DAG until the full workflow succeeds end to end. Validate any dashboard or timing claims before publishing.