7.2 Downstream Impact

🖥️ Lecture slides — Session 27 (Fri Dec 4)

Every final project includes a half-page downstream-impact statement. This page defines the inquiry, shows two worked examples, and gives the format and rubric.

The point is not to argue your project matters. It is to trace, honestly, the chain from your result to the world: who would act, on what decision, with what consequences when — not if — the model is wrong. A project with a modest, precisely traced impact chain scores higher here than a grand claim with no chain at all.

The five questions¶

Answer all five. Specific names beat categories: “the county flood manager” beats “stakeholders,” and “the USGS duty seismologist writing the aftershock statement” beats “the community.”

1. Who could act on this result? Actual roles or institutions, and the form they would need the result in (an API? a map? a threshold crossing?).
2. What decision would it change? A decision that would be made differently with your result than without it. If no decision changes, you have a contribution to knowledge — legitimate, but say so instead of inventing users.
3. What could go wrong if it’s wrong — and who bears that cost? Model errors have a direction: false alarms and misses land on different people. Name both failure directions and who pays for each. Note when the people bearing the cost are not the people who chose to deploy the model.
4. What is the environmental and computational footprint? Training and inference compute, data storage and transfer, and whether the footprint is justified by the improvement over the cheap baseline (Chapter 3 discipline: you know the baseline number). A model retrained nightly on a GPU cluster to beat persistence by 2% deserves that sentence.
5. What would responsible deployment need that this project doesn’t yet have? The gap list: evaluation on out-of-distribution data, calibrated uncertainties, a human-in-the-loop protocol, monitoring for drift, a retraining policy. Naming the gaps is what makes the rest of the statement credible.

Worked example 1: a flood-forecasting model¶

The project: an LSTM predicting river stage 24 h ahead for two gauged basins in western Washington, beating persistence by 30% RMSE in winter storms.

Who could act: the county emergency manager (road closures, evacuation warnings) and the city stormwater utility (retention-basin drawdown), both consuming a stage forecast with an uncertainty band, twice daily. What decision changes: drawdown currently starts on a National Weather Service watch, which over-triggers; a calibrated 24 h stage forecast could cut unnecessary drawdowns roughly in half. What goes wrong: a miss (predicted below flood stage; river floods) means unwarned road flooding — the cost falls on commuters and on the manager’s credibility, which future warnings spend. A false alarm wastes stored water and, repeated, erodes trust until warnings are ignored. Both basins are gauged and train-adjacent; the model has never seen an ungauged basin, an atmospheric river beyond training range, or the upstream dam’s revised release schedule — and the residents relying on the warning did not choose the model. Footprint: training ~2 GPU-hours per basin per season; inference negligible; footprint dominated by the ERA5 forcing archive (~200 GB). Justified relative to the 30% gain; would not be justified for the 4% gain we saw in summer low flows. Responsible deployment would need: calibrated predictive intervals (current intervals are 55% empirical coverage at nominal 90%), a fallback to persistence when inputs go stale, two more winters of out-of-sample verification, and an agreement on who reviews the forecast before it triggers anything.

Worked example 2: an aftershock-forecast model¶

The project: a neural model forecasting the spatial distribution of aftershocks in the week after an M6+ mainshock, evaluated by log-likelihood gain over an ETAS baseline (spatially uniform Omori decay).

Who could act: the regional network’s duty seismologist drafting the public aftershock statement; a utility deciding inspection order for substations. What decision changes: inspection routing — inspect the 20 highest-forecast-probability substations first rather than in radius order. The public statement should not change on this evidence: the model’s gain over ETAS is modest and untested on the region’s fault geometry. What goes wrong: the failure directions are asymmetric in visibility. Over-forecasting a quiet zone costs inspection hours. Under-forecasting the zone where a damaging aftershock then occurs costs an unreinspected substation and, if the map was public, the field’s credibility — aftershock forecasts are one of the few seismological products the public actually consumes. The cost of the miss lands on people who never saw the model card. Footprint: small — training is hours on one GPU; the honest comparison is that ETAS is nearly free, interpretable, and 30 years validated, so the burden of proof sits with us, not it. Responsible deployment would need: prospective testing in a CSEP-style framework against ETAS over multiple sequences, calibration of the probability maps, and a protocol for what the duty seismologist does when model and ETAS disagree. Until then this is a research comparison, and our statement says so.

The anti-pattern¶

“This work could help save lives and protect infrastructure from natural disasters, contributing to a safer and more resilient society.”

Every project in the room can generate this sentence; a language model will generate it unprompted. It names no actor, no decision, no failure cost, no gap — it is unfalsifiable, and it reads as what it is: a substitute for having thought about the question. Compare the specific chain: result → the stormwater utility’s drawdown decision → over-trigger cost vs. miss cost → what calibration is missing. Notice the specific chain is also more modest — that is not a coincidence. Boilerplate inflates precisely because it costs nothing.

Format and rubric¶

Format: half a page (250–400 words) in the final report, headed “Downstream impact,” answering the five questions in order — a sentence or three each. Write it yourselves; AI drafting is permitted with disclosure, but every named stakeholder and cost must survive the question “how do you know?” in the oral presentation.