Breaking Boundaries in License Plate Recognition

What would an ideal LPR system look like?

Automated reading of license plates has been around for decades. Unfortunately, most of the technology has not changed much in that time. When deep learning revolutionized computer vision a few years ago, it was clear that there was soon going to be a better way of reading plates. But if LPR systems are already fairly accurate (Transport for London estimates 97% accuracy for its congestion charge systems), how can deep learning help?

It’s likely that the main impact deep learning will have is to make the systems more robust to the position of the plate in the field of view, more forgiving of national variations in plate styles and more flexible to the angles at which the camera captures the plate. After all, if a deep learning machine can teach itself how to read plates that are square-on, why can’t it teach itself more complicated use-cases too?

Sighthound’s customers have varying use cases. This has necessarily led to robustness in the Sentry LPR software. Take the example of the Jaguar XF below. Sighthound reads the plate correctly even at an acute angle.

What about plates that are harder to read than western European ones, such as US states? Sentry still does a good job, although if the contrast between lettering and background makes it harder for a human observer to read the plate, the angle at which it can be read will fall for the software as well as the human.

One complaint often levied at legacy systems is the calibration required. The positioning or focus of the camera must often be precise or accuracy rates fall. This also leads to a need for more cameras as a single camera fails to capture multiple lanes or in and out vehicular traffic. Sighthound’s Sentry doesn’t care how near or far the plate is, within the bounds of legibility, caters to multiple plates in the same frame, and has no requirement that the plate be in a certain position on a vehicle. Hold a plate out of the window and Sighthound will read it.

Compare the image of a distant Mercury Sable, with close up of the plate provided for verification, with the close up of a Philippines Toyota

An ideal LPR system would certainly display some robustness to multiple issuing authorities. The reading of US plates is made more difficult by the 50 states, and by the tendency of dealers to use frames which obscure some of the issuing state information (solution: let the deep networks train on frame-occluded plates). But the European Union also has a lot of vehicles crossing national borders that makes country-of-issuance an important issue. Not only is Sentry able to read plates from almost every country in the world with Roman character letters, but it has country identification for most of Europe, Turkey, China, the Philippines and others. See image selection below.

So where are the upper limits to ALPR accuracy? Is it realistic that accuracies can be pushed to 99%, or 99.9%? Given that computer vision aims to mimic the human eye and brain, the upper bound could be defined, in the near-term at least, by whether a human operator would have been able to read the plate. In the end, that will define the ideal LPR system – one that despite the angle, the distance, the frame around the plate - got the answer right.