The question
Can information available when a 311 request is created help distinguish requests that close within 72 hours from those that take longer?
Data and scope
The final workflow uses 49,999 valid, closed NYC 311 records. It checks timestamp order, removes unusable rows, derives the 72-hour target, and documents the final class distribution.
What I built
- Python preparation and validation for timestamps, missing values, and categorical inputs.
- A preprocessing pipeline with one-hot encoding and a logistic-regression baseline.
- A train/test evaluation workflow with metrics beyond headline accuracy.
- A documented AWS path using S3 and Athena SQL, plus reproducible local instructions.
Results
The corrected model materially outperformed the majority-class benchmark and retained strong discrimination across classification thresholds.
| Metric | Test-set result |
|---|---|
| Accuracy | 78.71% |
| Balanced accuracy | 79.88% |
| Precision | 88.60% |
| Recall | 73.68% |
| F1 score | 80.46% |
| ROC AUC | 88.30% |
| Majority-class benchmark | 59.47% |
Validation
Six automated checks cover the target definition, timestamp logic, train/test separation, row counts, feature leakage, and metric generation. The repository also records the target bug found in the earlier version: weekday was mistakenly used as the response variable.
One limitation and the next test
The baseline is trained on closed historical requests. Before operational use, I would run time-based validation and monitor calibration and drift by agency, complaint type, borough, and season.