Businesses should not rely on “cheap” technology that claims to analyze employee sentiment, the Information Commissioner’s Office (ICO) has said, warning that such technology could discriminate against some people.
The data protection group’s hack refers to technology that uses AI that claims to analyze things like facial movements and expressions, movement, and even eye tracking, as a way to monitor the health and well-being of employees.
The ICO has said that a data collection system that may focus on behavior or emotional responses to try to understand emotions is more dangerous than traditional biometric techniques that are used to verify a person’s identity.
He said the algorithms used in these systems, which are not specifically designed to detect emotional signals, can show bias or even discriminate against certain people.
The only thing that will sustain the work of biometrics is those that have full functionality, accounting and scientific support
Stephen Bonner, ICO
The regulator urged organizations to assess the public risk before using such technology, and warned that all companies that do not act responsibly, pose a risk to vulnerable people or fail to meet the ICO’s goals .
“Developments in biometrics and the AI market are not great. They may not work yet, or ever,” said ICO deputy commissioner Stephen Bonner.
“While there are opportunities, the risks are currently greater.
“At the ICO, we are concerned that incorrect data analysis can lead to assumptions and judgments about the wrong person and lead to discrimination.
“The only projects that can be used will be those that are fully operational, accountable and scientifically supported.
“As it stands, we still haven’t seen any emotion in the development of AI technology in a way that is compatible with data protection, and it has more questions about consistency, accuracy and fairness in this area.
“The ICO will continue to explore the market, identify stakeholders who are looking to create or deploy these technologies, and highlight the importance of improving data privacy and compliance, while encouraging trust and confidence in how these systems work.”
The ICO has also confirmed that it will publish new guidance on biotechnology in the spring of next year to help businesses understand how and when to use the technology.