Ground truth and tolerances
What "truth" means here, how error is computed, where tolerances come from — and why a small residual is not an accurate calibration.
What truth is
On the virtual machine, truth is the set of parameters used to generate the scene, not an approximation recovered from it. The starred fields in the machine description are the truth:
| Chain | Truth field | Value on demo-xy-galvo |
|---|---|---|
| (1) Pixel size | cameras[].pixelSizeUm |
1.7236 µm/px |
| (1) Mounting angle | cameras[].rotationDeg |
−0.80° |
| (2) Camera-actuator offset | scanHeads[].offsetXMm/offsetYMm |
0.412 / 55.180 mm |
| (3) Lens distortion | cameras[].k1/k2 |
0 / 0 (use the test-distortion machine to practise) |
| (4) Hand-eye | the 2x2 Jacobian formed by pixel size and mounting angle | — |
| (5) Galvo field | scanHeads[].scaleX/scaleY/k1/skew |
1.0012 / 0.9993 / 1.8e−5 / 3.0e−4 |
| (6) Laser focus Z | scanHeads[].focusZMm |
112.500 mm |
| (7) Camera focus Z | workingDistanceMm + zMountOffsetMm |
110 + 2.347 = 112.347 mm |
So truth scoring is not "solving it again with another algorithm." It reconciles your result against the generator's input. That is the one decisive advantage a virtual machine has over real hardware.
On real hardware there is no truth,
TruthComparesis empty, and the criterion falls back to residuals. In the code these are the same path, not two implementations — so the way of reading results you practise on the virtual machine transfers directly.
Residual is not error
The single most useful sentence in this documentation.
| Definition | What it tells you | |
|---|---|---|
| Residual | the model's fit error over the data used in the solve | how self-consistent model and data are |
| Error | the difference between your result and the truth | whether the calibration is accurate |
A small residual only says "your model explains the data you collected". If the data itself is systematically flawed — distortion uncalibrated, points crowded in the centre of the field, span too small — the residual will be small and the error large.
The easiest way to produce this: shrink the sampling grid and run chain (1). The residual RMS barely moves while the error multiplies.
Three residual statistics, all of them
One residual number cannot be interpreted, so every chain reports three (ResidualStats):
| Statistic | What it tells you | Compensatable? |
|---|---|---|
| Raw maximum | how bad the worst point is | — |
| De-translated | what remains after removing the overall translation | a translation can be absorbed by a station offset |
| Leave-one-out | recomputed with each point held out; predictive power | detects overfitting |
A constant translation and a random scatter are indistinguishable inside one RMS, yet the remedies are opposite: the first is one offset, the second means changing the process or the hardware.
Where tolerances come from
All of them live in CalibTolerance in core/src/PreciSim.Core/Constants.cs — never as magic
numbers in the chains:
| Chain | Tolerance |
|---|---|
| (1) Pixel size | 0.002 µm/px |
| (1) Mounting angle | 0.02° |
| (2) Camera-actuator offset | 0.01 mm |
| (3) Lens distortion | 0.3 px RMS residual |
| (4) Hand-eye | 0.005 mm residual |
| (5) Galvo field | 0.010 mm de-translated residual |
| (6) Laser focus | 0.02 mm |
| (7) Camera focus Z | 0.05 mm |
Those correspond to demo-xy-galvo. On real hardware they should be derived from an error
budget, not copied:
- Write down the figure you must guarantee, e.g. "positioning accuracy ±0.05 mm (3 sigma)";
- List every contributor: calibration, motion, vision repeatability, part tolerance, thermal drift;
- Combine them in RSS — they are independent;
- Allocate a budget to each so that
3*sigma_total <= 0.05 mm.
sigma_total = sqrt(sigma_calib^2 + sigma_motion^2 + sigma_vision^2 + sigma_part^2 + sigma_thermal^2)
Read the shares, not the absolute values. If vision repeatability is half the variance, further work on the calibration algorithm buys little; better illumination and larger dots buy a lot. The main value of an error budget is telling you what not to optimise.
Taking this to real hardware
There is no truth there, but the method survives; only the verification changes:
- Use a more accurate instrument as reference (dial gauge, gauge blocks, laser interferometer, a better target);
- Use consistency instead of accuracy: repeat the same position 30 times and look at the spread;
- Use a closed-loop check: after calibrating, perform a move of known size and measure the result.
The point of the virtual machine is to let you practise the method while the answer is known, so that on real hardware you have the judgement to ask whether a number is plausible.