The paradox of automation: How it increases incompetence
I previously blogged this Guardian article by Tim Harford, but I was rereading some of it and came across this passage:
This problem has a name: the paradox of automation. It applies in a wide variety of contexts, from the operators of nuclear power stations to the crew of cruise ships, from the simple fact that we can no longer remember phone numbers because we have them all stored in our mobile phones, to the way we now struggle with mental arithmetic because we are surrounded by electronic calculators. The better the automatic systems, the more out-of-practice human operators will be, and the more extreme the situations they will have to face. The psychologist James Reason, author of Human Error, wrote: “Manual control is a highly skilled activity, and skills need to be practised continuously in order to maintain them. Yet an automatic control system that fails only rarely denies operators the opportunity for practising these basic control skills … when manual takeover is necessary something has usually gone wrong; this means that operators need to be more rather than less skilled in order to cope with these atypical conditions.”
The paradox of automation, then, has three strands to it. First, automatic systems accommodate incompetence by being easy to operate and by automatically correcting mistakes. Because of this, an inexpert operator can function for a long time before his lack of skill becomes apparent – his incompetence is a hidden weakness that can persist almost indefinitely. Second, even if operators are expert, automatic systems erode their skills by removing the need for practice. Third, automatic systems tend to fail either in unusual situations or in ways that produce unusual situations, requiring a particularly skilful response. A more capable and reliable automatic system makes the situation worse.
The article concludes with an interesting bit:
. . . In the mid-1980s, a Dutch traffic engineer named Hans Monderman was sent to the village of Oudehaske. Two children had been killed by cars, and Monderman’s radar gun showed right away that drivers were going too fast through the village. He pondered the traditional solutions – traffic lights, speed bumps, additional signs pestering drivers to slow down. They were expensive and often ineffective. Control measures such as traffic lights and speed bumps frustrated drivers, who would often speed dangerously between one measure and another.
And so Monderman tried something revolutionary. He suggested that the road through Oudehaske be made to look more like what it was: a road through a village. First, the existing traffic signs were removed. (Signs always irritated Monderman: driving through his home country of the Netherlands with the writer Tom Vanderbilt, he once railed against their patronising redundancy. “Do you really think that no one would perceive there is a bridge over there?” he would ask, waving at a sign that stood next to a bridge, notifying people of the bridge.) The signs might ostensibly be asking drivers to slow down. However, argued Monderman, because signs are the universal language of roads everywhere, on a deeper level the effect of their presence is simply to reassure drivers that they were on a road – a road like any other road, where cars rule. Monderman wanted to remind them that they were also in a village, where children might play.
So, next, he replaced the asphalt with red brick paving, and the raised kerb with a flush pavement and gently curved guttering. Where once drivers had, figuratively speaking, sped through the village on autopilot – not really attending to what they were doing – now they were faced with a messy situation and had to engage their brains. It was hard to know quite what to do or where to drive – or which space belonged to the cars and which to the village children. As Tom Vanderbilt describes Monderman’s strategy in his book Traffic, “Rather than clarity and segregation, he had created confusion and ambiguity.”
Perplexed, drivers took the cautious way forward: they drove so slowly through Oudehaske that Monderman could no longer capture their speed on his radar gun. By forcing drivers to confront the possibility of small errors, the chance of them making larger ones was greatly reduced.
Monderman, who died in 2008, was the most famous of a small group of traffic planners around the world who have been pushing against the trend towards an ever-tidier strategy for making traffic flow smoothly and safely. The usual approach is to give drivers the clearest possible guidance as to what they should do and where they should go: traffic lights, bus lanes, cycle lanes, left- and right-filtering traffic signals, railings to confine pedestrians, and of course signs attached to every available surface, forbidding or permitting different manoeuvres.
Laweiplein in the Dutch town of Drachten was a typical such junction, and accidents were common. Frustrated by waiting in jams, drivers would sometimes try to beat the traffic lights by blasting across the junction at speed – or they would be impatiently watching the lights, rather than watching for other road users. (In urban environments, about half of all accidents happen at traffic lights.) With a shopping centre on one side of the junction and a theatre on the other, pedestrians often got in the way, too.
Monderman wove his messy magic and created the “squareabout”. He threw away all the explicit efforts at control. In their place, he built a square with fountains, a small grassy roundabout in one corner, pinch points where cyclists and pedestrians might try to cross the flow of traffic, and very little signposting of any kind. It looks much like a pedestrianisation scheme – except that the square has as many cars crossing it as ever, approaching from all four directions. Pedestrians and cyclists must cross the traffic as before, but now they have no traffic lights to protect them. It sounds dangerous – and surveys show that locals think it is dangerous. It is certainly unnerving to watch the squareabout in operation – drivers, cyclists and pedestrians weave in and out of one another in an apparently chaotic fashion.
Yet the squareabout works. Traffic glides through slowly but rarely stops moving for long. The number of cars passing through the junction has risen, yet congestion has fallen. And the squareabout is safer than the traffic-light crossroads that preceded it, with half as many accidents as before. It is precisely because the squareabout feels so hazardous that it is safer. Drivers never quite know what is going on or where the next cyclist is coming from, and as a result they drive slowly and with the constant expectation of trouble. And while the squareabout feels risky, it does not feel threatening; at the gentle speeds that have become the custom, drivers, cyclists and pedestrians have time to make eye contact and to read one another as human beings, rather than as threats or obstacles. When showing visiting journalists the squareabout, Monderman’s party trick was to close his eyes and walk backwards into the traffic. The cars would just flow around him without so much as a honk on the horn.
In Monderman’s artfully ambiguous squareabout, drivers are never given the opportunity to glaze over and switch to the automatic driving mode that can be so familiar. The chaos of the square forces them to pay attention, work things out for themselves and look out for each other. The square is a mess of confusion. That is why it works.