Teaching AI how people work is fraught with problems

Artificial intelligence works best within companies when it knows the context in which it is operating. Some of that context is easy to codify upfront, in explicit rules and guidelines. Some of it can be captured by analysing raw data: Celonis, a German software firm, ingests information from enterprise systems to see how processes like invoices or procurement actually unfold within organisations

But some information is harder to pass on, and none more so than tacit knowledge, the know-how born of experience and intuition. Michael Polanyi, a philosopher, famously distilled tacit knowledge thus: “We can know more than we can tell.” How can AI learn about work when even humans cannot articulate why they do things a certain way?

One answer is that they don’t need to. Tacit knowledge is already embodied in the data that machines are trained on: a corpus of marketing copy for a specific brand, say, encapsulates organisational judgment and expertise that models can soak up. And one of AI’s defining characteristics, says Enrique Ide of IESE, a Spanish business school, is its ability to identify patterns in data that humans cannot describe. From facial recognition to chess, AI has proved very handy at mastering activities that it was never explicitly taught.

Sometimes, however, the details of a process matter. A brick wall encapsulates bricklayers’ tacit knowledge, for example, but staring at loads of them won’t tell you how best to build one. Monumental AI, a Dutch startup that uses AI-powered robots to lay bricks, interviewed masons as part of its early research into how to design its machines. But their answers tended to be maddenly vague things like: “That’s the way I’ve always done it.” The bricklayers knew more than they could tell.

What they could not articulate for themselves, hours of video footage could reveal. Monumental saw, for instance, that masons were vibrating their hands a little bit when they pushed a brick into the mortar. This small movement helps push mortar into the pores of the bricks, creating a stronger adhesive bond, which is why Monumental’s robots were then designed to replicate it.

An obvious answer, therefore, is to track how people work in ever finer detail. A few functions are already closely monitored. A customer-service call centre naturally throws off lots of relevant information: calls are usually recorded as a matter of course, and it is normal for agents’ screens to be monitored as well. AI could be let loose on much more data. A survey of American employees by Danielle Li of the Massachusetts Institute of Technology and her co-authors found that workers believe they are sitting on large amounts of uncodified knowledge about their organisations.

But more intensive monitoring of the specific actions of individual workers is also sensitive terrain. Meta’s employees have reacted noisily to the firm’s Model Capability Initiative, a programme to track keystrokes and mouse clicks in order to train AI. Ms Li’s survey found that workers think they have the ability to withhold a lot of valuable information from nosy employers. They may well choose to do so. In an experiment, the researchers offered to buy survey data from participants; those who had been shown a video on how data could be used to train AI were less willing to sell.

Even the most intrusive monitoring regime would struggle to capture what is going on in people’s heads: what they know about clients’ preferences or the thought processes behind certain judgments. So another option is to try and tease out experts’ knowledge through evaluation processes. On some tasks, it is easy for an AI model to know how it is getting on: a piece of code either runs or it does not. Other tasks are much fuzzier to verify—judging whether a design is aesthetically appealing, say, or how well a research brief has been carried out. By getting human experts to rate how AI performs on these more subjective criteria, the models can progressively be refined to match their standards.

There’s nothing wrong with trying to capture tacit knowledge. Organisations have long worried about losing expertise when old hands depart. But using AI to solve the problem raises a number of tricky questions. Who owns uncodified know-how? How much surveillance is acceptable? And as machines learn more and more, how will that affect the way that people acquire, practise and pass on the expertise that was previously gained through experience? ■

This Economist article was legally licensed by AdvisorStream.