Want a facial recognition system? Buy Chinese – says US government

Want a facial recognition system? Buy Chinese – says US government

Chinese and Russian facial recognition systems are leading the world in accuracy, according to a US report.

In an annual performance competition, the top 20 list of the most accurate facial recognition algorithms was dominated by Chinese and Russian companies, which took all top five positions, and most of the top 10.

Chinese AI startup YITU Technology grabbed first place for the second year running. Competitors from the Shenzhen Institutes of Advanced Technology at the Chinese Academy of Sciences, and facial recognition specialist megvii – owner of Face++ – came second and fifth, respectively.

Moscow-based NTechLab and Russian biometrics firm Vocord ranked third and fourth.

Embarrassment for the US

The report could be embarrassing for US innovators and policymakers, because it has been produced by the National Institute of Standards and Technology (NIST), the government agency responsible for establishing US security standards. As such, it provides the guidelines for official technology purchases in government.

By ranking algorithms from Chinese and Russian providers above others, NIST is saying that these are the most reliable facial recognition systems, out of those submitted for test.

NIST’s testing regime – probably the most rigorous in the world – finds that these algorithms currently produce the least numbers of false matches, which the tests are designed to measure.

NIST’s Facial Recognition Vendor Test (FRVT) is an ongoing process, and vendors are allowed to submit algorithms for (re)assessment every three months.

ID photos, mugshots, photos of child exploitation victims, and ‘wild’ (unsorted citizen) images are among those included in the test, and NIST changes the set of wild images periodically so that algorithms are constantly presented with new data.

Algorithms are tasked with correctly matching multiple images of the same person in each category, with the fewest ‘imposters’ (false matches).

The granular detail from NIST’s ongoing assessment is presented in this in-depth, 282-page report, which examines the performance of submitted algorithms against each image category, including factors such as age, ethnicity, and skin tone.

Source: NIST

Internet of Business says

As the trade war escalates between the US and China, the increasingly politicised technology landscape has seen some US technology companies, such as Google, face questions from politicians about their research and trade partnerships with Chinese companies.

At the same time, Chinese tech giants such as ZTE, Huawei, and China Mobile have also incurred the wrath of US officials.

Meanwhile, controversy has arisen over government and law enforcement adoption of facial recognition technologies in the US, such as two police forces’ experimental deployment of Amazon’s real-time Rekognition system.

Civil liberties campaigners have criticised facial recognition systems for producing inaccurate and biased results in these contexts, specifically against black Americans and other ethnic minorities.

Similar concerns have been aired in the UK. In a report into the government’s biometric strategy and forensic services by Parliament’s Science and Technology Committee, MPs quoted findings from privacy advocacy organisation Big Brother Watch, which revealed that the Metropolitan Police had achieved less than two percent accuracy with its live facial recognition technology, and made no arrests.

In this context – and in the heightened political climate – news that algorithms designed by Chinese and Russian providers are performing better than others submitted to NIST for testing are more than a little ironic.

One reason for the better results may be that China’s vast population and lax data protection regime mean that researchers have unfettered access to the one thing that AI, machine learning, and computer vision systems need: massive amounts of training data.

And in a world of GDPR in Europe, and US moves in a similar direction, that advantage can only grow. Indeed, it may be the underlying reason why companies such as Facebook and Google opposed California’s new consumer privacy act, which was approved yesterday and may see the introduction of de facto regulation across the US, albeit on a voluntary basis.

So: Will government buy Chinese or Russian and get the most accurate results? Or buy American and – according to NIST – face a higher risk of errors (at least among the algorithms that have been submitted for open testing)?

Refreshingly in a climate of fake news and alleged false reporting, the NIST report comes with nearly 300 pages of scientific evidence, provided by the US’ own national researchers and standards setters.

• Editor’s note: We have approached NIST about which companies’ algorithms have not been included in its tests and asked the organisation for a comment about whether its findings can be interpreted as a fair assessment of the facial recognition market overall. We will update this story with any response we receive.