← Back to selected work

01 / Data engineering · Analytics

CivicPulse Data Platform.

A pipeline for collecting and analyzing New York City 311 service requests, with data quality checks built into the workflow.

STATUSIn progress
KEY FIGURE318K+ records in 30-day backfill

01 / THE PROBLEM

Context matters.

NYC 311 data is large and changes over time. Before it can support useful comparisons, records need to be collected consistently, loaded reliably, and checked for quality.

02 / APPROACH

From idea to workflow.

Loaded a 30-day backfill of 318,529 requests into PostgreSQL and began modeling the data with dbt. The next step is to finish Airflow orchestration for incremental extraction, loading, transformations, and tests. Early analysis compares open requests and resolution times across boroughs.

03 / TOOLS & METHODS

The stack.

PythonNYC 311 APIPostgreSQLSQLdbtApache Airflow (in progress)

04 / EVIDENCE

Show the work.

Add a real architecture diagram, dashboard screenshot, or model evaluation output here. Explain what the evidence shows and how it was produced.

PROJECT VISUAL

Add a real diagram, chart, screenshot, or model output here.

05 / REFLECTION

What I learned.

Describe one technical decision, one challenge, and one limitation or next step. Specific details make the work easier to assess.

Airflow orchestration is still in progress. The current data load and dbt checks should be described separately from the DAG until the full workflow succeeds end to end. Validate any dashboard or timing claims before publishing.