Noise: A Flaw in Human Judgment

Noise is a random deviation from the norm. This differs from bias, which is a uniform differentiation. They both lead to bad results. Noise in different directions does not necessarily cancel out, but is additive. (If you end up 10 feet north of the target once, and 10 feet south the other time, you are equally inaccurate both times.) Eliminating noise and eliminating bias are key to making good decisions.

A challenge with many decisions is that we don't know what the "correct" answer is. Job interviews and prison sentencing are particular cases where there are bad amounts of "noise". Some small nuances in behavior can help. A number of distinct observations is likely to more accurately lead to the correct outcome. However, that only works if they are truly independent. When people meet together to talk, there is a desire to have consensus. Thus people are likely to alter their opinions to meet the first stated view. This can often lead to settling on a more extreme view. Independently collected feedback can provide a better outcome. In interviews, it also helps if interviewers have a distinct area to focus on rather than a general feel.

Computer algorithms can help with noise. They tend to apply rules much more accurately and reduce noise. Many times they are better than humans. However, this can sometimes be seen as negative, especially if there is not an equal distribution throughout society. There can also be challenges of handling rare outcomes. (A rare outcome that is missed by computers will get a lot of publicity, even if there were a great many more successes.)

People want perfection. However, they don't want the sacrifices to get it. People will complain about wildly different sentences for similar crimes. But then they will complain about fixed sentences guidelines that take out judicial discretion. Do we mandate rules or simply have guidelines? Noise can be nice in leading to different outcomes. However, most often it just leads to things being worse than expected.