What is an “accurate” appraisal?
First, we note that in Standard 1 of USPAP (Uniform Standards of Professional Appraisal Practice), accuracy is not required. What is required is that an appraisal be credible, or “worthy of belief.”
Perhaps reviewers know how to judge “accuracy.” Good news. Standard 3 does say specifically that the reviewer must develop an opinion as to the accuracy of the appraisal! We can just ask some reviewers how they score accuracy — and we got it!
Nope. The reviewer also has the real requirement of being credible “worthy of belief.”
No help.
Yet our users of appraisals really need to know if they can rely on the opinion. They need reliability, not believability! The stand-off between the believability of the appraiser and the believability of the reviewer is no help. Judgment. Opinion. Trust me!
So how do we measure reliability? Is it possible?
Scientific literature says reliability has two parts: accuracy and precision (trueness and sureness).
There are only two ways to measure reliability.
One way is if you possess the actual true number, and can compare your opinion against it. Appraisals and AVMs both can be tested against sale prices. Wonderful!
Problem. The individual sale price itself has ‘variation’ – random variation. Variation which comes from iffy human preferences, which are patchy and changing and fluid. Darn.
Nevertheless, we know that that variation itself has some consistency. We just need to account for that variation itself.
The second way is to reduce the analysis into parts, then measure the reliability of each of those parts, and use the (difficult math) parts to measure the whole “probability distribution.”
So, what are the measurable parts of the valuation model? Easy. It’s the same, whether for an appraisal, evaluation, desktop, or AVM. Three parts: 1) problem/assumptions; 2) data; 3) predictive algorithm.
Science requires dealing with uncertainty. To measure uncertainty, we need that probability distribution. Appraisers can do that. Appraisers can enhance the “reconciliation” part of the process to include an estimate of reliability. That estimate of reliability can be subjective, like an enhanced reconciliation, or it can be objective, using the reduction method just mentioned.
The Community of Asset Analysts (CAA)© applies Evidence Based Valuation (EBV)©. EBV is designed from the ground up to enable measurable reliability and accuracy, providing transparency and reproducibility to the appraisal product.
Would your clients like to have measurable risk/reliability, transparency, and reproducibility?
Join us now in the introductory Stats, Graphs, and Data Science 1 class.
Join us in the future of data science applied to valuation.