Machine vision metrology is the use of calibrated cameras, optics and software to measure parts to a stated accuracy, in line and without contact. Done well, a vision gauge measures every part to a few microns. Done badly, it reports precise-looking numbers that are wrong, because the lens, the calibration or the part’s position, not the software, set the real limit.
This article covers the metrology side of machine vision: the difference between resolution, repeatability and accuracy, how sub-pixel measurement works, why lenses and calibration matter, how to prove a vision gauge with a gauge R&R study, and when vision should replace a CMM. For the measurement methods themselves, see automated dimensional inspection.
Resolution, repeatability and accuracy are different things
| Term | What it means | What sets it |
|---|---|---|
| Resolution | The smallest change the system can register | Pixel size on the part, sub-pixel algorithms |
| Repeatability | How closely repeated measurements of the same part agree | Image noise, lighting stability, part presentation, vibration |
| Accuracy | How close the measurement is to the true value | Calibration, lens distortion, perspective error, temperature |
| Measurement uncertainty | The range in which the true value lies, with a stated confidence | All of the above, combined |
A system can be very repeatable and still inaccurate: it gives the same wrong answer every time. Specifications that quote only resolution say little about what the gauge will do on the line.
How sub-pixel measurement works
An edge in an image is not a hard step between two pixels. It is a gradient across several pixels. Edge-detection algorithms fit a curve to that gradient and locate the edge to a fraction of a pixel. Under good conditions (sharp focus, high contrast, stable lighting, a clean backlit edge) repeatability of about one tenth of a pixel is a reasonable working assumption, and better is possible. Under poor lighting or with rough edges it can be worse than a whole pixel.
Sub-pixel results improve resolution and repeatability. They do not fix accuracy, which still depends on calibration and optics. Our article on edge detection techniques covers the algorithms.
Worked example: sizing a vision gauge for a ±0.025 mm tolerance
A feature must be held to ±0.025 mm, a total tolerance band of 0.05 mm. The common target is a measurement system whose variation uses no more than 10% of the tolerance (gauge R&R, with six standard deviations of measurement spread).
| Step | Option A: 40 mm field of view | Option B: 20 mm field of view |
|---|---|---|
| Camera width | 2,448 pixels | 2,448 pixels |
| Pixel size on the part | 40 ÷ 2,448 = 0.0163 mm | 20 ÷ 2,448 = 0.0082 mm |
| Repeatability at 0.1 pixel (1σ) | 0.0016 mm | 0.0008 mm |
| Measurement spread (6σ) | 0.0098 mm | 0.0049 mm |
| Share of 0.05 mm tolerance | 19.6%: marginal | 9.8%: acceptable |
Halving the field of view doubles the pixels across the feature and brings the gauge under 10%. If the part does not fit in 20 mm, the options are a higher-resolution camera, two cameras each viewing one edge, or moving the part under the camera. This is why the field of view is a metrology decision, not just a framing one.
Lenses: why telecentric optics matter for measurement
With a conventional lens, magnification changes with distance. Parts that sit slightly higher look slightly larger. At a 100 mm working distance, a part that is 0.5 mm closer is magnified about 0.5% more. On a 20 mm feature that is a 0.1 mm error: twice the entire tolerance band in the example above.
A telecentric lens keeps magnification nearly constant over its depth of field, so small height changes do not change the measured size, and it views edges straight on, which removes perspective error on thick parts. Combined with a collimated telecentric backlight it gives the sharpest, most stable silhouette edges. The trade-off is size and cost: a telecentric lens must be at least as wide as the field of view.
| Lens choice | Best for | Limits |
|---|---|---|
| Conventional (entocentric) | Presence, inspection, measurements with loose tolerances | Magnification changes with distance; perspective error |
| Object-space telecentric | Precision 2D gauging of parts that vary in height or position | Large and costly for wide fields of view |
| Bi-telecentric | The tightest 2D metrology | Largest and most expensive; limited depth of field |
Calibration and traceability
- Pixel-to-millimetre calibration with a certified target, such as a glass grid or dot plate, converts pixels into real units.
- Distortion correction maps and removes lens distortion across the field, which is largest near the edges.
- Traceability comes from calibration artifacts certified against national standards, recorded with dates and results.
- Verification with a master part at the start of each shift catches drift, contamination or a bumped camera.
Other error sources to budget for
- Temperature. Aluminum expands about 23 µm per metre per °C, so a 100 mm aluminum part 5 °C warmer than reference grows by 100 × 0.000023 × 5 = 0.0115 mm, almost a quarter of a 0.05 mm tolerance.
- Part presentation. Tilt, burrs and debris move the edge the camera sees.
- Vibration and motion blur soften edges and hurt repeatability; strobed lighting and short exposures help.
- Lighting drift as LEDs age or warm up shifts edge positions on front-lit parts.
Proving the gauge: measurement system analysis
Before a vision gauge replaces an existing one, prove it with a gauge R&R study: several parts spanning the tolerance, measured repeatedly by the system (and by different operators where loading is manual). The study separates repeatability from reproducibility and reports the measurement spread as a percentage of tolerance. Common acceptance targets are under 10% as good and 10 to 30% as conditionally acceptable, depending on the application. Correlate the results against a reference such as a CMM or calibrated comparator.
Vision metrology vs a CMM
| Factor | Inline vision gauging | Coordinate measuring machine (CMM) |
|---|---|---|
| Speed | Milliseconds to seconds per part; 100% inspection | Minutes per part; sampling |
| Contact | None | Usually touch probe; some optical heads |
| Best features | 2D outlines, holes, positions; heights with 3D sensors | Full 3D geometry, deep features, GD&T datums |
| Environment | On the production line | Usually a controlled lab |
| Typical role | Every part, in process | Reference, first article and audits |
The two work together. Vision gauges every part at line speed; the CMM verifies the vision system and handles features a camera cannot see. For height, flatness and volume, 3D methods such as laser profiling extend vision into the third dimension, and high-tolerance part inspection shows the approach on precision parts.
Need a vision gauge you can trust?
Sciotex designs calibrated vision measurement systems and proves them with real parts. Free testing of your sample parts is available to confirm the achievable accuracy before you commit.
Frequently asked questions
Machine vision metrology is precise, calibrated measurement of parts using cameras, optics and image analysis, usually in line and without contact, with stated accuracy and repeatability.
It depends on the field of view, the camera resolution, the lens and the calibration. With sub-pixel edge detection, a telecentric lens and good lighting, repeatability of a few microns is achievable on small fields of view.
Sub-pixel measurement locates an edge to a fraction of a pixel by fitting the brightness gradient across several pixels. It improves resolution and repeatability, but accuracy still depends on calibration and optics.
A telecentric lens keeps magnification nearly constant as the part moves closer or farther, so small height changes do not change the measured size, and it removes perspective error on thick parts.
For 2D features and in-line 100% inspection, often yes. A CMM is still the better reference for full 3D geometry and GD&T datums, and is commonly used to verify the vision system.