The Safety Core Study
How the FRED scoring engine was built and tested: a stand-alone study of forward crash risk on public FMCSA data
Published: October 2026 • Origins 2024-11 to 2025-08, outcomes through 2026-08 • Locked evaluation split read once, 2026-10-22
The Question
How many recordable crashes will a motor carrier have over the next twelve months, per truck-year — and how sure can we be? The study set out to build, from the ground up, the best defensible model of that quantity from everything FMCSA publishes, and to turn the prediction into a rating that behaves the way an underwriter needs it to.
Five properties were fixed in the charter before any model was fit: a thin or unreliable record sits near its class rate; the top tier requires proof; the bottom tier requires observed adverse events, never attributes alone; events decay smoothly to zero weight; and no grade jumps upward without a cause that can be named. The study was run as a stand-alone exercise — earlier engines and FMCSA's own measures appear only as neutral benchmarks — and every number traces to a manifest with the inputs, origin and code revision that produced it.
What the model predicts:
R̄ = V · (a + N) / (a + Λ) — the relativity to class; 1.00× = a typical motor carrier of this class
The class rate is a Poisson regression on size, class, tenure, filing recency, the share of the filed fleet seen on the road and road activity. The violation relativity is three credibility-shrunk ratios (driver behaviour; out-of-service and violation ratios standardised for where the motor carrier was inspected). Crash experience is Gamma–Poisson credibility on 36 months of recordable crashes with a linear ramp to zero, N decayed crashes against Λ decayed expected crashes, with a prior a set by the motor carrier's size cohort and the expected crashes its record carries. The full specification is in the white paper; this page is the evidence.
Headline Results on the Locked Evaluation Split
The evaluation split — every motor carrier whose DOT number leaves a remainder of two when divided by four, 4.03 million motor-carrier-origin rows with 268,454 forward crashes — was never read during model development. It was scored once, on 2026-10-22, on the model as frozen that day, with every parameter fixed from the fit and selection splits. The frozen model trails the gradient-boosted ceiling on ranking (Gini 0.647 against 0.674; 0.863 against 0.877 with every motor carrier weighted once), is better calibrated (1.003 against 1.014) and trails it by 3.3 points of deviance, the price of staying three named, publishable factors. Grades assigned on that split realize 0.36 / 0.71 / 0.86 / 1.25 / 1.67 / 2.65 of their class-expected crashes from Excellent to Critical — monotone, with no Poor or Critical motor carrier lacking an observed event.
Data, Folds and Leakage
What was loaded
Everything FMCSA publishes on data.transportation.gov, loaded into typed Parquet with header validation and per-file manifests: the MCMIS crash file (5.0 million rows from 1982, with the date FMCSA added each record), the inspection file (8.3 million, 36 months), inspection violations (13.4 million), the company census (4.4 million rows, 147 columns), SMS inputs and results, licensing and insurance history, out-of-service orders and UCR. Point-in-time archiving of every input began on 2026-10-21.
Three facts about the data changed the design. The public SMS carries no BASIC percentiles or alerts for property motor carriers, so FMCSA's targeting cannot be observed directly. Crash reporting completeness varies by state from 25% (Colorado) to over 90% known within 60 days, so the domicile state was kept out of the model until a completeness factor exists. And the census is current-only, so historical origins carry today's power-unit count and status — a limitation stated with every result.
| Cleaning rule (crash file, 5.01M rows) | Rows removed | Share |
|---|---|---|
| Not a motor carrier | 425,073 | 8.5% |
| No motor carrier identified | 200,334 | 4.0% |
| No USDOT number (intrastate state motor carrier numbers) | 729,380 | 14.6% |
| Not federally recordable | 17,938 | 0.4% |
| Duplicate report | 18,916 | 0.4% |
| Identified recordable crashes remaining | 3.62M |
Every cleaning rule was adopted only if it removed a measured miscalibration or a known data defect, never on taste.
Fold design
- 1. Origins: ten monthly cutoffs, 2024-11 through 2025-08, each with a mature 12-month forward outcome window. The ten overlap eleven-twelfths in outcomes and are reported as temporal stability, not independent samples; confidence intervals come from a motor-carrier-cluster bootstrap.
- 2. Features as of the origin: dated by when FMCSA knew a record (its add date), not when the event happened. Crash history over 36 months; inspections, violations and vehicle sightings over 24 months. Exits (out-of-service orders, revocations) censor forward exposure.
- 3. Motor carrier split by DOT number: odd → fit; divisible by four → select (model choice, credibility constants); remainder two → eval, locked and read once. A motor carrier's rows stay in one split, so every out-of-sample number is also out-of-motor-carrier.
- 4. Leakage audit: features rebuilt from a file truncated at the origin, and at the origin plus one day, agree with the full-file features to floating-point noise. A pending-exit flag that turned out to be dated inside the outcome window was found by the inspection-bias analysis and removed from every model.
- 5. Survivorship: 70% of motor carriers crashing after a 2016 origin are still active today, 93% for 2025; historical folds therefore draw from the full census including inactive motor carriers and censor exits.
How Fast Should a Crash Fade?
Decay and credibility, estimated jointly
Chosen by how well the next twelve months of crashes were predicted — 13.3 million motor-carrier-origin rows
Two quantities decide how far a motor carrier's own record moves it from its class rate: the weight w(age) on each past event, and the credibility prior a of the experience term. Both were chosen by the out-of-time negative-binomial predictive likelihood of the following year's crashes — the quantity that makes “thin record → class rate” true forward, not just in the fitting window. Every common shape was tried on 24- and 36-month windows on identical rows.
| Decay shape | Window | Step at the edge | Log-likelihood vs best |
|---|---|---|---|
| Exponential, half-life 18 months, no truncation | 36 m | 0.25 of an event | 0 |
| Ramp to a one-third floor | 36 m | 0.33 | −473 |
| Linear ramp to zero (chosen) | 36 m | none | −1,257 (0.19%) |
| Equal weights | 36 m | 1.00 | −3,597 |
| Exponential, half-life 18 months, no truncation | 24 m | 0.41 | −4,594 |
| Ramp to a one-third floor | 24 m | 0.33 | −4,821 |
| FMCSA 3-2-1 step | 24 m | 0.33 | −5,510 |
Reading: recency matters (the unconstrained optimum has a half-life near 18 months) and so does the tail — every 36-month shape beat every 24-month shape. Months 25–36 of crash history carry real forward information. The step-free linear ramp to zero at 36 months costs 0.19% of the best achievable fit and has no cliff anywhere: an event loses one thirty-sixth of its weight each month and is gone at 36. Decision (2026-10-22): crash signal on 36 months, linear ramp to zero; violation and out-of-service signals on 24 months, since the inspection file carries only 36 months of history. The credibility prior a, re-estimated after the class model by the same forward likelihood, is set per size cohort and per band of expected crashes Λ (below 0.1, 0.1–0.3, 0.3–1, 1–3, 3–10, 10+) and interpolated in log Λ between bands: a thin record gets a diffuse prior, so one crash counts (10–19 power units: 0.24 below 0.1 expected crashes), and a record with a few expected crashes a narrow one (3.75 at 1–3). The cohort-level values run from 0.79 for single trucks to 2.23 — up from 0.12–0.94 against a cohort-only rate, because the class model absorbs heterogeneity the motor carrier's own record no longer has to explain.
What Predicts Crashes
1. Whether the motor carrier actually operates
Exposure and activity carry two thirds of everything the data can explain
A Poisson gradient-boosting model with a log-exposure offset on 16.8 million motor-carrier-origin rows explained 31.4% of forward-crash deviance on the select split. Fitting each family of features alone, and refitting without each one, shows where that comes from:
| Feature family | Alone: deviance explained | Unique contribution |
|---|---|---|
| Exposure & activity (PU, observed fleet, sightings) | 20.4% | 6.94 pts |
| Census class (state, class, for-hire, intrastate, hazmat) | 15.9% | 1.96 pts |
| Inspections (counts by level, OOS, type) | 14.0% | 0.22 pts |
| Violations by BASIC | 14.0% | 0.03 pts |
| Tenure & MCS-150 recency | 10.5% | 0.86 pts |
| Crash history (36-month ramp) | 5.3% | 0.87 pts |
| Exits | 2.3% | 0.45 pts |
| All families | 31.4% |
Reading: Inspections and violations each explain 14% — the same 14%: two measurements of one thing (how much the motor carrier is on the road and how it looks when stopped), which is why each adds almost nothing once the other is present. A last MCS-150 filing more than five years old meant 0.18× the cohort-rate prediction (1.57 million rows); motor carriers seen with more power-unit vehicles than they filed realized 1.71×, motor carriers never seen in 24 months 0.50×. This is where the class rate comes from.
2. The motor carrier's own crashes
Small alone, almost entirely unique — the experience term
Crash history explains 5.3% of deviance on its own, but 0.87 of those points survive every other family: it is the one family that is not a proxy for activity. In the bake-off it was the single largest step — +3.4 points of deviance and +0.035 Gini on top of the class rate (0.610 → 0.645) — while keeping aggregate calibration at 1.00. That is the role Gamma–Poisson credibility plays in the rating: a modest, independent correction on top of the class rate, with the posterior saying how much to believe it.
3. Inspections: activity plus what was found
Inspection-selection bias, measured — and how it shaped the violation term
Is a motor carrier inspected often riskier, or just more active, or just domiciled where enforcement is heavy? On 8.39 million fit-split rows with 41 class controls and the domicile state's inspection intensity cut into tertiles:
Reading: the inspection count's predictive power is activity plus what the inspections find; it is not a risk signal in itself. A motor carrier inspected often and found clean is safer than its class. So the class rate carries activity by inspection type (capped at four inspections per truck, where the realized curve flattens), and the violation term is a shrunk ratio whose credibility grows with inspections and whose value falls toward class for a clean record. Standardising violation and out-of-service rates for the motor carrier's inspection state × level mix beat raw per-inspection rates every time (coefficient 0.151 vs 0.118 for violations; 0.128 vs 0.064 for OOS, and in a joint model the raw OOS rate turns negative). In a per-BASIC test the admitted BASICs are Unsafe Driving (RR 1.87 per log unit), Hours-of-Service (1.21) and Vehicle Maintenance (1.16); Driver Fitness (0.66) and Hazardous Materials (0.64) are negative. In the rating, Unsafe Driving and Hours-of-Service violations per driver-level inspection form the driver-behaviour term; the standardised out-of-service ratio counts inspections that ended in an out-of-service order, whatever caused it, and the standardised all-violation ratio counts every violation, whatever its BASIC — each against the number expected for where and how the motor carrier was inspected. The forward-predictive violation groups are all driver behaviour — speeding, seat belt, phone, texting, reckless driving, RR 1.15–1.24.
The Model Bake-off
Deviance explained and Gini by candidate — select split
4.11 million motor-carrier-origin rows, 273,755 forward crashes; identical rows for every candidate; fit on odd DOT numbers
The transparent model trails the gradient-boosted tree on ranking (0.645 against 0.674 pooled) and by 3.4 points of deviance — the price of linearity among busy motor carriers. The tree also ranks better inside each cohort, but cannot decompose into named factors or be governed.
| Candidate (select split) | Realized ÷ predicted | Deviance explained | Gini |
|---|---|---|---|
| Power-unit cohort rate | 0.99 | 1.1% | 0.13 |
| Class GLM: size, class, tenure, MCS-150 age, mileage band, observed fleet, country | 1.01 | 21.9% | — |
| + activity by inspection type and regime tertile | 1.01 | 23.5% | 0.61 |
| + violation relativities (A before experience) | 1.01 | 23.7% | — |
| A: + Gamma–Poisson experience, 36-month ramp | 1.005 | 27.2% | 0.645 |
| A with domicile-state fixed effects (parked: reporting completeness) | 1.005 | 27.9% | — |
| Gradient-boosting ceiling (same rows, leak removed) | 1.013 | 30.5% | 0.674 |
Candidate A was frozen as the safety core because it is the best calibrated, transparent model: within ±4% in every cohort, per-cohort Gini 0.59–0.78, every factor a named multiplicative term with a published constant. Two refinements were tried and rejected: an isotonic recalibration of the class rate (+0.3 points of deviance, mis-calibrated single trucks) and activity band dummies (+0.4 points, a twenty-term step function in place of three logs). The one documented weakness is a U-shaped calibration by predicted-rate decile (0.62 … 1.19 … 0.92 on the locked split): the log-linear class rate over-predicts quiet, rarely inspected motor carriers and under-predicts the busy upper middle.
| Locked evaluation split, read once (4.03M rows) | Realized ÷ predicted | Deviance explained | Gini | Top-5% lift |
|---|---|---|---|---|
| Power-unit cohort rate | 1.001 | 1.0% | 0.13 | 1.22 |
| Class GLM + typed activity | 1.008 | 23.7% | 0.611 | 3.93 |
| A (frozen spec) | 1.003 | 27.4% | 0.647 | 4.24 |
| A with state fixed effects | 1.002 | 28.1% | — | 4.25 |
| Gradient-boosting ceiling | 1.014 | 30.6% | 0.674 | 4.50 |
Per cohort on the locked split, A's Gini runs 0.63 / 0.62 / 0.59 / 0.60 / 0.64 / 0.71 / 0.75 / 0.83 from single trucks to 200–499 power units, with realized ÷ predicted 0.98–1.08 in each. Posterior coverage: 86% of rows with at least 0.5 expected crashes realized at or below the posterior 80th percentile.
Do the Grades Predict Future Crashes?
Realized ÷ class-expected crashes in the following 12 months, by grade
1.00 = the motor carrier's class rate. Three views: the locked evaluation split, read once on 2026-10-22 on the model as frozen that day; the selection split graded under the engine's rules (ranked 10+ power-unit book, 241,481 motor-carrier-origins over the ten origins, before the interim exposure-corroboration hold); and the November 7, 2026 run's out-of-time gate year (1,677,353 motor carriers graded as of the start of the latest fully reported twelve months, against the crashes of those months)
On the locked split Excellent realizes 0.36 of its class expectation and Critical 2.65 times it, on motor carriers and years the grading never saw; in the 2026-11-07 run's out-of-time gate year the two realize 0.50 and 3.08. The charter's tier criteria (Excellent ≤ 0.6, Critical ≥ 1.9) are met with room.
| Grade | Locked eval: share of rows | realized ÷ class (±2 SE) | Selection split, engine rules, 10+ PU: share | realized ÷ class | 2026-11-07 run, gate year: realized ÷ class |
|---|---|---|---|---|---|
| Excellent | 0.03% | 0.36 ± 0.05 | 1.7% | 0.50 | 0.50 |
| Strong | 3.3% | 0.71 ± 0.01 | 10.0% | 0.76 | 0.68 |
| Satisfactory | 88.0% | 0.86 ± 0.01 | 77.5% | 0.92 | 0.94 |
| Marginal | 6.6% | 1.25 ± 0.01 | 8.6% | 1.30 | 1.25 |
| Poor | 1.8% | 1.67 ± 0.02 | 1.6% | 1.72 | 1.68 |
| Critical | 0.3% | 2.65 ± 0.08 | 0.6% | 2.72 | 3.08 |
The locked-eval columns are the 2026-10-22 read of the model as frozen that day: all motor carriers, 4.03 million motor-carrier-origins, read once. The right-hand column is the production engine's own out-of-time check from the November 7, 2026 run: every motor carrier graded as of the start of the latest fully reported twelve months (1,677,353 motor carriers), against the crashes of those months. Across all motor carriers in the November 7, 2026 run, 96.7% are Satisfactory, because a one- or two-truck record proves little either way (1.75 million of them are shown as Not enough record rather than a grade); the ranked book is where the distribution opens up.
Bigger the better, all else equal: the posterior screens
Between two motor carriers at the same point estimate, the larger fleet's relativity is known and the smaller fleet's is a wide guess. The design leaves Satisfactory in either direction only when 60% of the posterior lies beyond the cut. A sweep on the selection split:
| Screen strength (both sides) | 10+ PU book: Exc / Strong / Sat / Marg / Poor / Crit | 10–19 PU Marg / Poor | 50+ PU Marg / Poor | Single trucks Marg / Poor |
|---|---|---|---|---|
| Upside 0.6, no downside screen | 0.5 / 15.2 / 58.0 / 21.2 / 4.4 / 0.8 | 20.7 / 2.6 | 21.1 / 8.2 | 3.5 / 1.3 |
| 0.6 both sides (chosen) | 0.5 / 15.2 / 72.9 / 9.2 / 1.9 / 0.4 | 6.7 / 0.7 | 14.9 / 5.3 | 0.9 / 0.2 |
| 0.7 both sides | 0.1 / 6.1 / 86.6 / 5.7 / 1.2 / 0.2 | 3.2 / 0.3 | 12.3 / 3.9 | 0.5 / 0.0 |
| 0.8 both sides | 0.1 / 2.2 / 93.4 / 3.4 / 0.7 / 0.1 | 1.4 / 0.1 | 9.5 / 2.8 | 0.0 / 0.0 |
Shares in percent, on the cuts as fitted; the engine's Excellent cut is set at 0.60 by decision, above the fitted value, so the engine's Excellent share is larger than this column. With 0.6 on both sides, Marginal realizes 1.39 of class instead of 1.26 and Poor 1.82 instead of 1.70; the motor carriers sent back to Satisfactory realize about 1.1–1.2. Going to 0.7 would cut Strong from 15% to 6% of the book for little added purity.
Who was Excellent — two leaks found by reading the list
Inflated filed fleets: 122 of the 416 motor carriers ever graded Excellent on the selection split had fewer than 0.1 inspections per filed power unit in 24 months — one filed 4,150 trucks and had three inspections. A big filed fleet with no road presence buys a large expected count and a tiny relativity. Fix: Excellent and Strong require at least 0.1 inspections per filed power unit.
Activity over-extrapolation: the most-inspected Excellent motor carrier (61 trucks, 1,219 inspections, 177 out-of-service) crashed at 1.6 per 100 truck-years the next year, above the book average, because the log-linear activity terms had extrapolated its class rate to about 15×. Fix: activity terms are capped at four inspections per power unit, where the realized curve flattens; the class model was refitted and the locked evaluation run on the corrected spec.
Reachability: in production a clean 36-month record earns Excellent from about 50 power units upward, provided the record carries at least 0.3 expected crashes over 36 months and the roadside record corroborates the filed fleet; below 50 it also takes a violation record better than class; smaller clean fleets reach Strong; single trucks cap at Strong (682 heavily inspected single trucks had reached Excellent in a dry run without that cap). From 10 filed power units, a fleet the road does not corroborate (under a quarter of the filed units seen in 24 months and under 0.25 inspections per filed unit-year) is shown one band lower — Excellent as Strong, Strong as Satisfactory (5,245 grades in the November 7, 2026 run). Excellent is 2.0% of the ranked book in that run.
Stability and Reporting Lag
Churn with every parameter frozen
Consecutive monthly origins on the selection split, 401,000–418,000 motor carriers graded in both, every parameter frozen:
Upward moves were decay roll-off (910–1,020 a month), exposure and class-rate changes (530–890) and violation relativities (60–145); downward moves were new crashes (890–1,010), exposure and class-rate changes (80–165), and smaller numbers of decay and violation changes. The governor — an improvement not caused by decay, a data correction or a model recalibration is ramped one band per run — changed the realized relativity of the displayed grades by less than a hundredth.
In production the engine runs weekly: between the 2026-11-07 and 2026-11-07 runs, 0.00% of the 1,835,687 motor carriers graded in both moved at least one band (0.00% per 30 days), 0.000% two or more, 0.000% were governed, and 0 improved because of a new event.
Timing: when to count the newest events
Thirteen past monthly runs replayed with only the records each run could have seen (crashes by FMCSA add date, inspections and violations by their MCMIS add date), every timing judged on the crashes motor carriers had afterwards. Gains are against holding everything back two months:
Fresh inspections helped on every run date; fresh crash counts helped while reporting was normal and stopped helping when FMCSA’s crash reporting stalled in mid-2025, so the live engine counts inspections and violations through the latest full month on file and crashes through one month earlier (the November 7, 2026 run: inspections through September 2026, crashes through August 2026). Events dated after those cutoffs are shown as not yet scored.
The Live Run
The production engine is candidate A: a class rate that reads the roadside record in seen-share bands (interacted with 50+ filings and inspections per unit), three shrunk violation relativities, and crash experience with a credibility prior that depends on the record's expected crashes. On its own out-of-time check — the November 7, 2026 run's gate year, every motor carrier graded as of the start of the latest fully reported twelve months against the crashes of those months — the pooled Gini is 0.626 (exposure-weighted Lorenz). The November 7, 2026 run (inspections, violations and census facts through September 30, 2026, crashes through August 31, 2026, data as of October 29, 2026) graded 1,835,687 property motor carriers, 109,449 of them in the ranked 10+ power-unit book. Verdict of the 2026-11-07 run: all nineteen checks ran, every one inside its bound, with no hard or soft failure. Every run repeats these nineteen checks before anything is written; a hard failure leaves the previous grades standing.
Calibration by evidence bucket (realized ÷ predicted by expected crashes, thinnest to thickest): 1.05 / 0.98 / 0.99 / 0.98 / 1.03 / 1.02 / 0.99 — every bucket within 10%. Ranked-book mix as displayed (109,449 motor carriers of 10+ PU): Excellent 2,182 (2.0%) · Strong 11,382 (10.4%) · Satisfactory 84,905 (77.6%) · Marginal 8,693 (7.9%) · Poor 1,624 (1.5%) · Critical 663 (0.6%). 5,245 Excellent or Strong model grades are shown one band lower because the roadside record does not corroborate the filed fleet; the relativity, posterior and forecasts are untouched. Of the Satisfactory motor carriers, 1.75 million (67,928 in the ranked book) are shown as Not enough record with no score: their 80% range runs from below the Excellent cut to the Poor cut or beyond, so the record cannot tell a best-in-class fleet from a poor one. That rule is display only; the engine's grade, gates, history and forecasts are unchanged.
Limitations and Open Items
- The census is current-only: every historical origin carries today's power-unit count and status until FMCSA's historical extracts (requested 2026-10-21) arrive and the weekly archive accumulates.
- No public vehicle-miles per motor carrier; MCS-150 mileage is self-reported and stale, and is used only as a coarse band in the class rate.
- State crash-reporting completeness confounds domicile-state effects (+0.75 points of deviance left on the table); a completeness factor from the archive's own lag curves is planned, and would let crashes count without the one-month hold.
- Ten monthly origins are one independent outcome year, and are stated as such.
- Within single-truck motor carriers, Marginal realizes more than Poor (1.57 vs 1.39 on select): single trucks follow their out-of-service and overall violation ratios more than the pooled coefficients say. The pooled grade is monotone overall.
- Nothing driver-level (CDLIS/PSP, Clearinghouse), no ELD, IRP/IFTA, DataQs or enforcement-case detail is public.
- Severity is displayed from the cohort's crash mix; a separate severity model is a follow-up. Passenger strata await their own class models.
Key Takeaways
1. Activity is the backbone; the class rate carries it
Whether and how much a motor carrier operates — vehicles seen versus filed, inspections by type, filing recency, tenure — explains two thirds of what FMCSA data can explain. A thin record's prediction comes from here, which is what makes “thin → class rate” honest.
2. The motor carrier's own crashes are the one independent signal
Small alone, almost entirely unique, and the single largest step in the bake-off. Gamma–Poisson credibility on a 36-month linear ramp uses it without a cliff and with a posterior that says how sure to be.
3. Inspections measure activity plus what was found
A traffic stop is an observation of driving, a weigh-station pull-in an observation of being on the road, and a clean motor carrier inspected often is safer than its class. Standardised ratios beat raw rates; the violation term reads driver behaviour (Unsafe Driving and Hours-of-Service per driver-level inspection) and the motor carrier's standardised out-of-service and all-violation ratios.
4. Transparent trails the black box on ranking by 0.03
On the locked split the three-factor model trails a gradient-boosted ceiling's Gini by 0.027 (0.647 against 0.674) with better calibration, three points of deviance behind. Every factor is a named multiplicative term with a published constant.
5. Proof for the top, events for the bottom, posteriors for size
Excellent needs 60% of the posterior below the cut, driver behaviour no worse than class, no fatal crash, enough inspections, at least 0.3 expected crashes on the record and a filed fleet the road corroborates; Poor and Critical need observed crashes or out-of-service excess; a wide posterior falls back toward class in either direction. Grades realize 0.36 to 2.65 of class on motor carriers the grading never saw, and 0.50 to 3.08 in the 2026-11-07 run's out-of-time gate year; no motor carrier improves because of a new event.