Fleetidy — ML Crash Model

The Big Idea

Can we look at a truck's safety record from last year and predict whether that same truck will be involved in a crash this year?

We trained a machine learning model on 1.67 million individual trucks using publicly available federal safety data. The model learned which patterns in a truck's inspection history, violation record, and motor carrier profile are early warning signs for future crashes.

Why this matters:

● Insurance underwriters can identify higher-risk trucks before writing a policy
● Fleet managers can spot problem areas and take corrective action
● Safety regulators can focus limited resources where they matter most

How It Works

1

Gather the Data

We collected every inspection, violation, and crash report from 2024 for 1.67 million individual trucks across the U.S., all from FMCSA public records.

2

Teach the Model

The model studied which trucks that had certain patterns in 2024 actually went on to crash in 2025. It learned from 270 different data points per truck.

3

Test on Unseen Data

We held back 15% of the trucks to test the model on data it had never seen. This ensures the results are genuine, not just memorization.

Key Results

How do we measure accuracy?

This study measures accuracy with a metric called AUC (Area Under the Curve). Think of it as the model's ability to correctly rank trucks from safest to riskiest. A score of 0.50 means the model is no better than flipping a coin. A score of 1.00 would mean perfect prediction. In the real world, scores above 0.60 for individual-vehicle crash prediction are considered strong, because crashes are rare and inherently unpredictable events. These figures describe this study's truck-level model only; the motor carrier grade is validated separately, at motor carrier level, and not by AUC on individual trucks.

Full Model (270 factors)

0.614

Uses every available data point. Maximum predictive power, but complex to manage.

Simplified Model (20 factors)

0.589

Uses only the 20 most important factors. Nearly as accurate, but far easier to understand, audit, and explain.

Model Accuracy Comparison

Bottom line: The simplified 20-factor model reaches 96% of the full model's AUC (0.589 against 0.614 — measured above the 0.50 coin-flip floor, it keeps 78% of the full model's ranking power) while being dramatically easier to explain and govern.

What the Model Looks At

The simplified model uses 20 factors drawn from public FMCSA data. Here are the most important ones, ranked by how much they influence the prediction. Red bars increase crash risk; green bars decrease it. This is the study's factor list. The motor carrier grade reads for-hire status inside its class rate, but it reads none of the violation or out-of-service rows as a separate factor with its own sign: it reads violations through three ratios — unsafe-driving and hours-of-service violations per driver inspection, and out-of-service and total violation findings against what the motor carrier's inspections would be expected to produce — and it reads mileage only as miles per truck within the class rate, where more miles per truck broadly raises, rather than lowers, the expected crash rate: the rate climbs band by band up to 100,000–300,000 miles per truck and eases slightly above that.

Factor	What It Means	Effect
Hours-of-Service Driving Measure	How severely a motor carrier's drivers violate rest and driving-time rules	Increases risk
Authorized-for-Hire Status	Whether the motor carrier operates as a for-hire business (vs. private)	Increases risk
Hazmat Violations	Number of hazardous materials violations found during inspections	Decreases risk
Driver Out-of-Service Rate	How often a motor carrier's drivers are pulled off the road during inspections	Increases risk
Controlled Substance Measure	Motor carrier's score on drug and alcohol testing violations	Decreases risk
Driver Fitness Violations	Inspections where driver qualifications (license, medical cert) were deficient	Decreases risk
Vehicle Maintenance Violations	Number of vehicle defects (brakes, tires, lights) found on this truck	Decreases risk
Violation Severity Score	A time-weighted measure of how serious and recent the violations are	Increases risk
Unsafe Driving Violations	Inspections where unsafe driving behavior (speeding, lane changes) was cited	Increases risk
Total Miles Driven	Self-reported total mileage of the motor carrier's fleet	Decreases risk

Factor Importance (Top 12)

A note on "Decreases risk": Some factors may seem counter-intuitive. For example, in this fit, more maintenance violations found on a truck go with lower future crash risk. That sign is this fit's own coefficient on a raw count: a raw violation count mixes how often a truck was inspected with what the inspections found, and although inspection counts sit among the fit's 20 factors, nothing in it standardises for where and how the truck was inspected or sets the count against what the truck's own inspections would be expected to produce. The motor carrier model behind Fleetidy's grade reads maintenance findings the other way: vehicle-maintenance violations raise expected crash risk, and the grade compares a motor carrier's out-of-service and violation findings with what its own inspections, given where and how they were done, would be expected to produce, so a defect found is never treated as protective.

Full Model vs. Simplified Model

We built two versions of the model. The full version uses 270 different data points and squeezes out every bit of predictive power. The simplified version uses just 20 carefully selected factors. Here's how we chose them:

The Selection Process

We ran the model 8 separate times on different random samples of the data
We tracked which factors were consistently chosen as important across all 8 runs
Factors that were important in every run made the shortlist
We tested models with 20, 25, and 30 factors to find the sweet spot

Why Simplify?

✓ Transparency: Regulators and readers can understand exactly how the model's predictions are calculated
✓ Auditability: Easier to verify the model isn't using biased or inappropriate inputs
✓ Reliability: Fewer factors means less chance of the model breaking when data changes
✓ Minimal cost: Only gives up ~2.5 points of accuracy for all these benefits

Accuracy by Number of Factors

Adding more factors beyond 20 actually made the model slightly less accurate on validation data, confirming that 20 is the optimal choice.

What This Means in Practice

This model doesn't predict crashes with certainty — no model can, because crashes involve unpredictable human behaviour and road conditions. What it does do is reliably rank trucks from lowest to highest risk.

For Underwriters

When pricing a fleet policy, a truck-level model of this kind can flag which specific trucks in the fleet carry the most risk, pricing at the individual vehicle level rather than averaging across an entire fleet. Fleetidy's motor carrier grade does not do this: it rates each motor carrier as a whole, and individual trucks enter it only as roadside sightings that help measure the size of the operating fleet.

For Fleet Operators

Understanding which factors drive crash risk gives fleet managers a clear checklist: ensure drivers comply with hours-of-service rules, address maintenance issues promptly, and monitor unsafe driving citations.

The Strongest Signal

In this fit, the single biggest predictor of future crashes is hours-of-service driving violations — how severely a motor carrier's drivers violate rest and driving-time rules. This factor alone has 3x more influence on the prediction than any other, reinforcing what safety experts have long known: tired drivers are dangerous drivers. That 3x is this fit's own coefficient on the raw measure: inspection counts sit among the fit's 20 factors, but nothing in it standardises for where and how a motor carrier's trucks are inspected or sets a violation measure against what those inspections would be expected to produce, so inspection-selection bias is left in. The motor carrier model behind Fleetidy's grade, with inspection activity and regime controlled, reads it differently: hours-of-service and unsafe-driving violations are counted together, each violation once, in a single driver-behaviour rate per driver inspection (the motor-carrier-level study that chose that rate found unsafe driving the stronger of the two when it tested them separately), and that rate is one of three violation factors in the grade, none of which carries three times the weight of another.

Data Source and Integrity

Where the data comes from

All of this study's data is sourced from the FMCSA Safety Measurement System (SMS), the U.S. federal government's primary system for monitoring commercial vehicle safety. This is the same data that FMCSA uses for its own safety ratings.

We used inspection reports, violation records, crash reports, and motor carrier profile information — all publicly available.

How we prevent cheating

● No future data: The model only sees 2024 data when predicting 2025 crashes
● Held-out test set: 15% of trucks were never seen during training
● Stratified sampling: Both crash and non-crash trucks are proportionally represented

By the Numbers

1.67M

Individual trucks studied

38,136

Had at least one crash in Year 2

270

Data points per truck (full model)

20

Factors in the simplified model