National comparison and outliers

This page is for quality control managers and Authorised Examiners who want to know exactly how MOTLogs decides that a tester's TQI figure is a Watch or an Outlier.

How does MOTLogs compare a tester with the national average?

For every coloured row, MOTLogs takes the tester's figure and the National figure from DVSA for the same row, month and vehicle group. It works out the gap as a proportion of the national figure:

gap = |tester figure - national figure| / national figure

A tester at 24% fail rate against national 15% has a gap of 60%.

What are the Watch and Outlier thresholds?

The thresholds are fixed on the TQI log page. They are MOTLogs' own guide, not DVSA limits. DVSA publishes no figure for an unusual fail rate or test time.

What stops small numbers from shouting?

Two guards apply before a figure is called Watch or Outlier:

1. The absolute gap must be at least 5. That means 5 percentage points on a rate, or 5 minutes on a test time. A tester on 0% against a national 1% is not flagged, however large the proportion.
2. A small sample is never more than Watch. If the tester did fewer than 30 tests, the cell can be Watch but never Outlier.

If the national figure is zero or missing, the cell is Within range because there is nothing to compare with.

Which tester does "What stands out" pick?

The What stands out note looks at two rows only: Average test time and Tests failed. It lists testers whose figure on either row is an Outlier, and picks the one with the largest combined gap from national. It never picks the site average column, a tester with a small sample, or a column without a tester ID.

The note says which figures are far above or below national, with the national figure beside them. If no tester qualifies it says "Nothing stands out against national this month." Other rows can be Watch or Outlier in the table without appearing in the note.

What should I do about an outlier?

An outlier is a prompt to look, not a verdict. A sensible order is:

1. Check the vehicle mix and class group.
2. Check the number of tests.
3. Look at the individual tests in the test log, especially their durations.
4. Talk to the tester and record what you found by commenting on the figure. See Commenting on and attesting a TQI figure.

For wider background on TQI, see What is MOTLogs and the glossary.

Reference