Bias is a much used and misused word. Variance too has more than one meaning. Words rule. Words trigger our thoughts, our feelings, and our opinions. Bias itself can mean different things. It can mean different things to different people. And it can mean different things to the same person, depending on context.
Bias can mean an inclination toward a particular view, and become a prejudice. But here we do not talk about this. Here we speak of analytic bias. Analytic bias is the result of a bad model or algorithm, (It may or may not have any connection to personal bias).
While analytic bias is very different from personal bias, it can become a cover for prejudice-bias. And analytic bias can be used as a way to conceal personal bias, and even try to sell a particular point of view to others.
Analytic methods can be used or misused. Personal bias (or opinion) can be concealed by (or be misleading ) using analytic bias in several ways. First, it is good to look at the four component parts of the analytic (and valuation) process and how it can be biased:
- The question (hypothesis) – wrong question, wrong work scope;
- Data selection – wrong data, irrelevant result;
- Predictives (adjustments) applied – bad logic, wrong result;
- Presentation (report), wrong words (explanation) – misleading report.
Variance vs bias – a tradeoff?
For now, lets just look at item 2 above: Data selection.
We have two goals: precision and accuracy (sureness and trueness). Sureness is nearness (on average), to the correct, true result. Trueness means we approach the right result.
Variance (variation) measures uncertainty. Accuracy measures any bias (analytic bias). Variance means you cannot depend on the answer much. Bias means you can depend on the answer to be in a particular (wrong) direction!
Aha! Now we see how analytic bias can be used to conceal or perpetrate personal bias!
The bias-variance tradeoff.
For valuation, with a limited number of ‘comps,’ There is a tradeoff. Too much data, and you get more variation. Uncertainty. Too little competitive data, and you get selection bias.
The bias weakness of the legacy ‘appraisal process’ is twofold: 1) the data selection ‘picking comps’ is subjective – subject to personal bias of all forms, and 2) the bias will appear directly from intentional or unintentional picks.
The variance weakness of the legacy process is seldom an issue. Appraisers are taught to pick a handful of comps. “Too much data” is a concept that simply does not come into the thoughts, feelings, or opinions of appraisers or evaluators. You only need a handful.
With data science principles (Evidence Based Valuation), the quantity of data as well as the quality of the data is critical. We eliminate bias (and the potential for prejudice) through assisted critical thinking. We assure best reliability by eliminating unneeded variance.
There is a trade-off: too much vs. too little data.
In the Valuemetrics course: Stats, Graphs, and Data Science — we call this the “Goldilocks point” as explained by Baby Bear. “Just Right!”