SENSI

method · data

What the method actually does

Most evaluation work treats the human reference as fixed and measures models against it. SENSI measures the reference first, against the sensor returns it was drawn from, and only then measures models against the audited version.

The argument

The four sources of error are not independent. An annotator draws a box by reading the same returns a detector reads, so the reference is least certain exactly where the model is weakest. A residual measured against an unaudited reference understates the true error, and understates it most in the conditions that matter. That claim is checkable, and checking it is what stage 01 is for.

How a reference gets audited

Producers emit measurements and two decision steps emit every verdict. No single signal settles anything on its own: three lidar points inside a box at 40 m means empty or occluded depending on evidence that a stage deciding alone cannot see. The decision step is a deterministic rule table rather than a fitted classifier, because a fitted reference is a model output, and then there is nothing left to measure models against.

Canonical files are immutable. Every stage writes new files, and each corrected reference is a complete version with a recorded parent, so any result can be traced to the exact reference it was computed on.

The corpus

K-Radar
58
recordings · 4D radar · lidar · stereo cameras
nuScenes
850
recordings · 6 cameras · lidar · 5 radars
Waymo
100
recordings · 5 cameras · 5 lidars
NVIDIA
120
recordings · 7 cameras · lidar

1,128 recordings, all ingested to one canonical schema: int64 nanosecond timestamps, a right-handed ego frame with +x forward and +z up, and a single class vocabulary across all four sources. Comparison always runs on the class set the two streams share.

What is not claimed

No dataset ships model predictions, so every candidate stream here was produced by running a detector and is only as good as that detector on that domain. Two of the five models tried on K-Radar do not transfer to it at all and are excluded, with the measurement that says so kept alongside the ones that do. Findings that were published internally and later refuted are recorded as refuted rather than deleted.