Facial recognition technologies can quickly identify who you are by automatically analyzing your facial features. The characteristics of your face (your biometric information) may be collected when you apply for an identity document like a passport, when you get your photo taken for an employee badge, or when you upload photos online to social media websites.
Given how many opportunities there are to record our faces, it is time to ask: can we remain anonymous in a crowd?
How Face Recognition Works
Face recognition systems usually analyze the features of a face to create a “template” that records specific information that is likely to be unique. This might include the position of your eyes and mouth or the shape of your nose. Other templates might make a holistic representation of all the unique features of the face taken together. A face template represents a mathematical summary of the facial features, and it’s a piece of data that is distinctly separate from the photo itself.
Many face recognition systems only make comparisons between templates so original photos don’t have to be stored, while other systems keep photos just in case manual comparisons are required. Some of these face recognition systems store the templates locally so that the comparison can be made quickly without requiring remote communications. Other systems use remote databases, perhaps connected to the Internet, which might store information about thousands or millions of faces.
In order to make feature analysis easier, some face recognition systems require you to look straight at a camera with a neutral expression. And when it comes time to recognize your face, you may need to strike the same pose again to ensure that a match is made. Other face recognition technologies are, however, able to analyze your face in 3D, so having a straight-on view is not required. More sophisticated face recognition systems are now using more advanced algorithms so the system can recognize faces from a variety of angles over time.
Face recognition systems can be used for recognition or identification. In recognition systems your face is compared to a single image or template, which was captured previously, in an attempt to verify that you are who you claim to be. In identification systems, your face is compared to many other faces, perhaps millions, in order to determine who you are. Recognition systems tend to be faster and easier to build than identification systems since there are fewer comparisons to make, and less chances of an error.
Accuracy
The accuracy of face recognition systems varies considerably and depends on a number of factors, including the quality of cameras, lighting, the type of templates used, the matching algorithms, and the size of the face databases. Early face recognition systems did not perform well with low quality photos or large image databases, but recent projects such as MegaFace at the University of Washington show that newer, more advanced systems can perform very well.
Some face recognition systems show systematic errors or biases, with poorer results when analyzing the faces of women, different ethnic groups, or younger people. These inaccuracies often arise when the development of algorithms is based on a narrowly defined population that is not sufficiently representative of the eventual population the system will be applied to. The face recognition algorithms need to be trained and tested to ensure they accurately detect differences between age, gender, and ethnicity.
Some systems have falsely verified a person when a low-quality photo of them was actually being presented to the camera. Such erroneous verifications, called ‘spoofing attacks’, occur when the system cannot differentiate between a ‘real’ face and a facsimile of one. More sophisticated face recognition systems can, however, include a liveness detection feature to look for dynamic characteristics, such as eye movements, before searching for a match.
Private Uses
A variety of private companies use face recognition technologies to provide services to consumers. Google and Microsoft, along with other mobile device companies, are experimenting with using your face to log into your device. Microsoft’s ‘Hello’ service uses the cameras on your laptop or tablet to automatically sign you in. Other companies, such as MasterCard, are also testing face recognition to authenticate a purchase if your device doesn’t include a fingerprint reader.
Facebook uses face recognition to automate photo grouping and tagging so people can find and identify photos of themselves or their friends. Facebook has explained that, in their analysis, automatic tagging that links a face to an identity may not be compatible with privacy laws that require that the people being tagged provide prior consent (e.g., in the EU and Canada). In these jurisdictions, Facebook has recently enabled face recognition features that group photos together based on matching features without providing identity tags. Apple also offers face recognition, but their technology creates user-specific (and user-controlled) local databases (stored on the user’s devices) instead of a huge, system-wide database (controlled and operated by an organization).
Public Uses
Government agencies have adopted face recognition technologies for verification and identification purposes. The photo that you provide to Passport Canada, for example, is compared with a database of previous photos to check for duplicates and fraud. Governments can also use photos to verify that the person who presents the identity document is the same person who initially applied for the passport.
Government agencies also use face recognition to identify persons trying to enter Canada despite being banned from the country. This may involve analyzing live camera feeds at airports and border crossings to automatically identify banned persons. New York City is considering the use of face recognition technologies in public spaces to automate the detection of terrorists and suspects.
Law enforcement agencies have also shown interest in face recognition systems. Calgary police were amongst the first to use the technology to compare mugshots with crime scene photos and video. The RCMP has also expressed an interest in adopting face recognition systems to identify suspects, often from low-quality cameras used in CCTV or surveillance systems. The RCMP’s interest is, in part, linked to a desire to be interoperable with the FBI’s face recognition technologies—the American agency has access to hundreds of millions of photos, many of which were initially collected by other federal departments.
Facepalm—Mixed Public/Private Uses
After riots in Vancouver following a hockey game, the Insurance Corporation of British Columbia (ICBC) offered to give police access to its facial recognition system for local driver’s licenses, but the British Columbia Information and Privacy Commissioner ultimately decided that the face recognition system cannot be used to identify rioters without a court order. While this use of face recognition technology was not allowed, the increasing number of information sharing arrangements being struck between government agencies could lead to face images being used for a variety of purposes.
Using face recognition technology to unlock our personal devices is also causing concerns. While the legal principle of self-incrimination prevents law enforcement agencies from compelling someone to provide a password, it is not clear that the same protections apply when asking someone to present their face to unlock the device.
Millions of photos are being collected and analyzed to assemble very large databases of face images or templates, and face recognition systems are becoming increasingly more accurate. Besides the risk of lost or stolen data, the organization that collected your photo today could also share or sell it to another organization tomorrow for different purposes than you originally had in mind.
Suggested Reading
- Data at Your Fingertips: Biometrics and the Challenges to Privacy
- Border security turns to facial recognition to watch for terrorists
- Automated Facial Recognition in the Public and Private Sectors
- Facial recognition systems stumble when confronted with million-face database
- Are Face Recognition Systems Accurate? Depends on Your Race
- Investigation Into The Use Of Facial Recognition Technology By The Insurance Corporation of British Columbia