how much does an environmental-justice map depend on the choices behind it?
Washington’s Environmental Health Disparities (EHD) Map gives every census tract a 1–10 rank of cumulative environmental health burden, and a top rank — “highly impacted” — steers real funding and policy under the state’s HEAL Act. But that rank is assembled from a stack of defensible-but-arbitrary choices: which indicators, how to normalize them, how to weight and combine them. This work runs a variance-based global sensitivity analysis over those choices — measuring how much a tract’s rank, and its impacted status, actually depends on them.
Context
The EHD Map is a Washington Department of Health product, built CalEnviroScreen-style: pollution-burden indicators and population-characteristic indicators are decile-ranked, averaged into themes, and multiplied into a single composite that is ranked again. A tract in the top ranks is designated “highly impacted,” which carries funding and siting consequences. Every step in that pipeline is a reasonable choice — and a different reasonable choice would move tracts around. The question is how much.
What this is
A Monte-Carlo, variance-based global (Sobol’) sensitivity analysis, built on a customized fork of the COINr composite-indicator framework. It rebuilds the entire index thousands of times while varying seven methodological levers:
- Indicator selection — dropping one indicator at a time.
- Measurement error — injecting ±25% noise into raw values.
- Normalization — min–max, z-score, percentile, binned decile.
- Weighting — original DOH weights, equal, PCA, inverse-correlation.
- Aggregation formula — arithmetic mean, geometric mean, product, PCA.
- Classification — equal-interval, deciles, z-score.
- Hierarchy — keeping vs. flattening the theme structure.
First- and total-order Sobol’ indices (bootstrapped) quantify how
much each lever drives the result. The headline metric is MARC
— mean absolute rank change: how many ranks the average tract moves,
and how many flip in or out of “highly impacted.”
What it found
On average, tracts moved more than a hundred ranks across the permutations. The swings were largest in the middle of the impact spectrum and smallest for tracts in the top 10% — so the map is far more robust at identifying the most-impacted hotspots than at placing everyone else on an ordinal scale. The aggregation formula and the normalization method were the most consequential choices by a wide margin. In short: the EHD ranking is better suited to classifying hotspots than to fine-grained ranking, and a handful of upstream assumptions substantially drive the result.
The environmental-justice extension
A second analysis tested adding candidate indicators the current map leaves out — asthma, wildfire smoke, pesticide load, and a mortgage-lending-discrimination (redlining) measure — and mapped how ranks shift when they’re included. The point is not that any one belongs in the index, but that omitting a discrimination indicator is itself a consequential methodological choice: historically redlined tracts rank as more impacted when it’s counted.
What I owned
- The full sensitivity/uncertainty analysis — the seven-lever design, the Sobol’ sampling, and the MARC / impact-flip metrics.
- A faithful reproduction and QA of the official EHD rank calculation from the source data.
- The COINr fork and custom normalizers/aggregators that made the whole index re-runnable under alternative methods.
- A multivariate validation (PCA / PLS against life expectancy) of what actually drives the index.
- An interactive mapping tool for DOH staff to explore ranks and rank-shifts spatially.
What it changes
Composite indices like this decide where money and protection go, and their rankings are usually presented as settled fact. Treating the methodology as an uncertainty to be measured — rather than a choice to be defended — tells a program which parts of its map to trust and which to caveat.