Employment law, AI and keeping the ‘human’ in human resources 
By
Preferred Choice

By Greg Clark

Artificial intelligence is a game changer in the workplace, but as Greg Clark warns, it shouldn’t be allowed to make the final decision on employment matters.

AI promises to revolutionise the world of work in a variety of ways, including helping HR professionals make all sorts of difficult decisions – from who to hire as the next CEO to who should be made redundant at the conclusion of a redundancy process.

But AI is not necessarily the rational and perfect decision maker it is often thought of as being. In 2018, Amazon realised that a recruitment AI they were developing was biased against women. It had been trained to rate candidates by looking at CVs submitted to Amazon over the previous 10 years. Most of those CVs came from men. As a result, Amazon’s AI concluded that it was better to hire men. It penalised CVs that it believed came from women – for example, because they mentioned being a member of a women’s sports team.

The moral of the story? If you put biased data into an AI, the AI will give you biased outputs. And given the datasets used to train AI are often historical data generated by humans, unbiased data is really hard to come by.

The Amazon recruitment AI was retired five years ago, but, despite the rapid advances in technology, the problem of AI bias is still here. In 2022, researchers predicted that 85 per cent of AI projects would be hampered by bias. So, while the direction of travel for AI may be that it’s getting more powerful, more affordable and more available, AI bias is proving a tough nut to crack.

Implications for employment law

AI bias is undoubtedly a new and growing frontier of discrimination law. As it is used more and more, there is likely to be an uptick in employment tribunal claims about whether decisions taken by AI are fairly applied to workers.

One prominent case currently working its way through the employment tribunal system is Manjang v Uber. The case is about an automated facial-recognition system introduced by Uber, which requires its drivers to take a selfie when they check in for work. The selfie is then checked against the picture on the driver’s Uber account profile. If the selfie matches, the driver can check in and start work.

Mr Manjang claims the automated software wrongly decided that his selfies were of someone else on several occasions, which led to him not being able to work via Uber’s app and, ultimately, to his account being deactivated. Manjang says this was, in part, due to his race (he describes himself as black and of African descent) because, Manjang claims, the AI used by Uber produces more false negatives for people of colour.

While Manjang’s case is yet to be decided, it highlights the care that needs to be taken when making AI-assisted decisions about staff.

GDPR rights for employees

It’s for that reason that the UK GDPR gives employees the right not to be subject to solely automated decisions if the decision would have a legal or ‘similarly significant’ effect on them. For decisions commonly taken by HR, that would cover (for example) decisions about hiring, pay, bonuses, disciplinary sanctions and termination of employment. In those cases, a human must review any AI decision before it is applied and have the ability to change it.

That means that even as AI becomes more prominent, there will need to be humans in human resources.

What actions can HR teams take now?

  • If you are thinking about implementing an AI system, ask the developers about the data used to train the AI and whether the AI has been tested for bias
  • Think carefully about how the human review of AI-decision making will work, and in particular, the criteria you will use to review AI-assisted decisions and when you might ‘overrule’ the AI
  • Make sure you are transparent with staff about how AI-assisted decisions about them are being made, and that when and how humans have been involved has been clearly explained.