About the model

A projection shows where past trends lead if they continue. It does not account for known upcoming changes, such as federal budget or policy shifts, and may not reflect what actually happens. Nor can it see survey revisions or an institution's own plans. Treat the shaded range, not the middle line, as the result.

What is projected

Fiscal years 2025 to 2029, for each institution, for every dollar measure (51): total R&D and each source of funds, each federal agency, each field of research, each type of cost, and subawards. Personnel counts are not projected; NCSES releases them only from FY 2022. Any group on the site (a sector, a state, public or private, a set of institutions you pick, the nation) is projected by adding up its institutions' simulated futures.

Data

NCSES HERD Survey, FY 2012 to FY 2024, one value per institution and year. Nominal dollars, on the log scale.
An institution is modeled if it reported a positive amount in at least four years and reported in FY 2024. Institutions that do not qualify are held at their last reported value when a group is projected.
Institution type is the 2021 Carnegie Classification, held fixed. Types with few institutions (associate's and tribal colleges, system offices, independent research institutes, unclassified units) are pooled as "Other" for modeling.

The model in words

University R&D budgets behave much like a random walk with drift: a jump in one year tends to persist, and spending grows at some typical rate with year-to-year surprises. So the model describes growth, the change in log spending from one year to the next, and every projection starts from the institution's last reported value.

Typical growth by type (fixed effect): R1 universities, R2 universities, master's institutions and so on each get their own average growth rate.
State and institution (random effects): each state and each institution may grow faster or slower than its type. These are estimated with partial pooling, so an institution with a short or noisy history is pulled toward its type and state. In the fitted models these effects come out very small: once you know an institution's type and the shocks every institution shared in a year, its own past growth rate tells you little about its future growth.
Shared shocks (random effects by year, and by year within type): good and bad years hit many institutions at once, such as the 2013 sequestration or the post-2021 jump. The model estimates how large those shocks are and simulates new ones for each future year. Because every institution in a simulated future feels the same shock, a group's total has a realistic range instead of an overly narrow one.
Size-dependent noise: what is left over is far larger for small institutions, whose budgets can double or halve, than for large ones. The size of that noise is estimated as a smooth function of institution size.

The model in symbols

Let $V_{it}$ be institution $i$'s spending on a measure in fiscal year $t$, in nominal dollars, $y_{it} = \log V_{it}$, and $g_{it} = y_{it} - y_{i,t-1}$ its growth from one year to the next (computed only across consecutive reported years). Institution $i$ has type $s[i]$ (2021 Carnegie) and state $k[i]$.

Model

$$\begin{aligned} g_{it} &\sim \mathcal{N}\!\left(\eta_{it},\ \sigma(y_{i,t-1})\right) \\ \eta_{it} &= \mu_{s[i]} + a_{k[i]} + u_i + \gamma_t + \delta_{t,s[i]} \\ a_k &\sim \mathcal{N}(0, \sigma_a) && \text{state} \\ u_i &\sim \mathcal{N}(0, \sigma_u) && \text{institution} \\ \gamma_t &\sim \mathcal{N}(0, \sigma_\gamma) && \text{year shock} \\ \delta_{t,s} &\sim \mathcal{N}(0, \sigma_\delta) && \text{year-by-type shock} \\ \log \sigma(y) &= f(y) && \text{noise by size} \end{aligned}$$

Normal distributions are written with their standard deviation.

$\mu_s$: typical growth for institution type $s$ (fixed effect)
$a_k$: state effect
$u_i$: institution effect: its own persistent growth relative to its type and state
$\gamma_t$: shock shared by every institution in year $t$
$\delta_{t,s}$: shock shared by institutions of type $s$ in year $t$
$f$: smooth function of last year's log size (thin-plate spline, 6 basis functions)

Fit by REML in mgcv, giving coefficients $\hat\beta$ (all of $\mu, a, u$) with covariance $V_\beta$.

Noise by size

The noise function $f$ is estimated from the residuals:

Fit the model above with a single residual standard deviation and keep the residuals $\hat\varepsilon_{it}$.
Regress the log absolute residual on last year's log size with a thin-plate spline (6 basis functions, REML): $$\log\left|\hat\varepsilon_{it}\right| = f(y_{i,t-1}) + \nu_{it}$$
Convert to a standard deviation. For normal errors $\mathbb{E}\left[\log|\varepsilon|\right] = \log\sigma - 0.635$, so $$\hat\sigma(y) = \exp\!\left(\hat f(y) + 0.635\right)$$

Each institution's projected noise uses its size in the last observed year, $\hat\sigma(y_{iT})$, for all five years.

Projection

From the last year of data $T$ (FY 2024), 200 draws $m$, horizons $h = 1, \dots, H$ with $H$ = 5, with every random term widened by $c$ = 1.3 (chosen by backtest):

$$\begin{aligned} \beta^{(m)} &\sim \mathcal{N}\!\left(\hat\beta,\ V_\beta\right) \\ \gamma^{(m)}_{T+j} &\sim \mathcal{N}\!\left(0,\ c\,\hat\sigma_\gamma\right) \\ \delta^{(m)}_{T+j,s} &\sim \mathcal{N}\!\left(0,\ c\,\hat\sigma_\delta\right) \\ \varepsilon^{(m)}_{i,T+j} &\sim \mathcal{N}\!\left(0,\ c\,\hat\sigma(y_{iT})\right) \\[4pt] y^{(m)}_{i,T+h} &= y_{iT} + \sum_{j=1}^{h}\left(\mu^{(m)}_{s[i]} + a^{(m)}_{k[i]} + u^{(m)}_i + \gamma^{(m)}_{T+j} + \delta^{(m)}_{T+j,s[i]} + \varepsilon^{(m)}_{i,T+j}\right) \\ V^{(m)}_{i,T+h} &= \exp\!\left(y^{(m)}_{i,T+h}\right) \end{aligned}$$

Groups are summed within each draw, with unmodeled institutions held at their last value:

$$V^{(m)}_{G,T+h} = \sum_{i \in G,\ \text{modeled}} V^{(m)}_{i,T+h} + \sum_{i \in G,\ \text{not modeled}} V_{iT}$$

The shaded ranges are the 10th to 90th and 5th to 95th percentiles across draws.

Scoring

Backtests are scored with the continuous ranked probability score on the log scale:

$$\mathrm{CRPS}(F, y) = \mathbb{E}_F\left|X - y\right| - \tfrac{1}{2}\,\mathbb{E}_F\left|X - X'\right|$$

where $X, X'$ are draws from the projection $F$ and $y$ is the observed value. Lower is better.

The model as code

The symbols above map to the R code below: $\mu_s$ is sector, $a_k$ is s(state, bs = "re"), $u_i$ is s(inst, bs = "re"), $\gamma_t$ is s(year, bs = "re"), $\delta_{t,s}$ is s(year_x_sector, bs = "re"), and $f$ is the second-step gam.

Fit with the R package mgcv, one model per measure:

growth = log(value[t]) - log(value[t - 1])

bam(growth ~ sector                      # typical growth by institution type
           + s(state, bs = "re")         # state random effect
           + s(inst,  bs = "re")         # institution random effect
           + s(year,  bs = "re")         # shock shared by every institution in a year
           + s(year_x_sector, bs = "re"), # shock shared within a type in a year
    method = "fREML")

# Noise that depends on size (two-step location-scale model)
gam(log(abs(residual)) ~ s(log(last_value), k = 6))
sigma(size) = exp(fitted + 0.635)        # E[log|z|] = -0.635 for normal z

How the projections are simulated

Draw 200 sets of model coefficients from their estimated distribution (including uncertainty in the smoothing parameters). Each draw is one possible version of the typical growth rates.
For each future year, draw a new national shock and a new shock for each institution type, shared by every institution in that draw.
Add each institution's own noise, sized to the institution.
Accumulate growth from the institution's FY 2024 value to get 5 years of possible future values.
Widen every source of randomness by a factor chosen in testing (below), so the ranges are as wide as the past says they should be.

All 200 futures are kept for every institution. A group's range comes from adding its institutions' futures draw by draw and then taking the middle 80% and 90%. (Ranges for individual institutions cannot simply be added; the sum of each institution's high value is far less likely than any one of them.) Those group sums are computed once a year for every group on the Trends page, and every institution's futures are stored in a small file of their own, so the site downloads only what the chart on screen needs. Every summed file carries a fingerprint of the institution-level futures it was built from; the build stops if any group total disagrees with its institutions, and the site will not draw a range from files whose fingerprints do not match.

How it was tested

The model was fit on data through FY 2018, then 2019, 2020 and 2021 in turn, and asked to project the next three years, which had already happened. Projections were scored with the continuous ranked probability score (CRPS) on the log scale: it rewards projections that are close and honest about their uncertainty, and lower is better. Two alternatives were scored the same way: a naive approach (each institution continues its own average growth) and a model of spending levels with Gaussian process trends by type and correlated year-to-year errors, which was the first design considered. The growth model was more accurate than both at every horizon and every institution size.

Loading…

By institution size

Loading…

Choosing how wide the ranges are

Before widening, only about 73% of real outcomes fell inside the 80% range. Each source of randomness was multiplied by several candidate factors, and the one whose ranges captured closest to 80% and 90% of real outcomes was kept.

Loading…

Fitted models

Loading…

Limitations

Dollars are nominal, so the projections include inflation.
13 years is a short history. Shocks larger than any seen since 2012 are not ruled out, only unlikely under the model.
The model projects institutions that keep reporting. It does not project institutions entering or leaving the survey, mergers, or a measure dropping to zero.
Policy changes that have been announced but have not yet shown up in the data, such as changes to indirect cost rates or agency budgets, are not in the model.
Testing used total R&D; the same model and widening factor are applied to every other measure.

Download the projections

Every simulated future is published as a dataset on Hugging Face: n8layman/herd-explorer-projections (CC BY 4.0), one Parquet file per measure, with the model and backtest tables.

Reproduce it

Everything is in the GitHub repository. Run Rscript pipeline/build.R to rebuild the data and Rscript model/fit.R to refit, test and simulate, then model/publish.sh to rebuild the group and institution files, check them and publish. The model is refit once a year when NCSES releases a new survey year.