If your work is reliable, it’s credible.  If it’s credible it may not be reliable.

USPAP (Uniform Standards of Professional Appraisal Practice) defines credible as “worthy of belief.”  So how does one strive to be worthy?  Well the automatic answers are . . . automatic:  1) have education/training; 2) have experience; 3) have integrity.  Let’s see how we can measure these for “level of believability.”

First, USPAP Standards Rule 3 makes it easy.  The reviewer develops opinions on the opinion of value.  The scoring is yes/no, pass/fail in believability (credibility). SR 3-3(a) the official reviewer decides if the analyses are (1) appropriate (yes/no); (2) credible (yes/no).  In SR 3-3(b) the reviewer decides if the report is (3) appropriate (yes/no), and ‘not misleading’ (yes/no).  These believe/disbelieve decisions are based on a gradient, or “judgment of the level of believability.”

Each of these gradient scores of believability is based on five elements[1]: completeness, accuracy, adequacy, relevance, and reasonableness.[2]   Although the language here is that the reviewer is to develop an opinion on each of these, it is not clear how these opinions are to be developed other than the reviewer’s subjective judgment.  The only help we get in the appraisal development standards, such as SR-1—is that the acceptable ‘scope of work’ must:  a) meet the expectations of users; and, b) do what appraiser peers do.  Of course, there are a few ‘checklist’ requirements, such as identifying the client/users, use, value definition, date, property characteristics, and any assumptions.

To sum up, believability is in the eye and checklist of the beholder:  the appraisal reviewer, or  commonly—the quality control person (formerly called the ‘administrative reviewer).  The reviewer’s belief then should result in a yes/no decision on the credibility (believability) of the appraisal.

The user then decides whether or not to rely on the appraisal by the yardstick: “is it worthy of belief?”

How would we measure reliability?

It is important to recall that reliability and risk are opposite sides of the same chip.  Clients always seek to forecast their potential for loss or possibility for gain.  A recent historical point value opinion of market price (value in exchange) is just on part of the risk puzzle.

Reliability comprises:  validity, and accuracy (precision and unbiasedness).

• Validity is simple— How well does your assay measure what it sets out to measure? My experience in litigation practice as an expert witness or consultant—is that more often opposing experts fail to measure the right thing.  (Big errors come from picking the wrong, or biased data).
• Accuracy itself comprises two parts: unbiasedness and precision.

Unbiasedness, in the appraisal context is interesting.  There is no mention of analytical bias in USPAP!  It is only about personal bias.  It is defined as a “preference or inclination that precludes an appraiser’s impartiality, independence, or objectivity.”

Precision is low variance in the result, given repeated analyses.  There are only two solutions to this question:  a) order several appraisals each time, and calculate the standard deviation; or, b) require that the valuation be reproducible.  If the work can be reproduced, any modeling decisions can be easily isolated.  Reproducibility is at the core of the scientific method.  A reproducible valuation can stand up in litigation, in collateral assessment, portfolio management, and investment potential.  Best of all, a reliable valuation is believable on its face.  On the other hand . . .

A ‘believable’ appraisal may not be reliable.  This is a critical weakness of vintage appraisal practice today.

Trust me.

[1] In future postings of this blog, we will consider how each of these five words individually might be judged for a level of believability.  The question is:  What do you compare the level of believability against?

[2] Appraisal Foundation, Appraisal Standards Board, Uniform Standards of Professional Appraisal Standards, 2018-2019, STANDARDS RULE 3-3(a)(iii), [line 873] Comment.