To regulate or improve something, you first measure it.
The word credible appears in the most recent version of USPAP (©1999) some 276 times. The Uniform Standards of Professional Appraisal Practice as published from time to time by the Appraisal Foundation. The word credibility is used an additional 39 times, specifically as a measure of compliance. (Colloquially called a “violation”.)
Let’s look at these words.
‘Violation’ is a binary result. You are either guilty or not guilty of a USPAP ‘violation’. On what basis is guilt or non-guilt measured?
Credibility! Is defined in USPAP as “worthy of belief”. The key words are worthy, and belief. So credibility is believability.
And how do we measure believability? This is the problem. We must make a decision – guilty/not guilty on the basis of whether I believe or not. And whose belief is it we seek? Your belief, or my belief. Or is there some true absolute believer out there that magically makes the referee call?
Believability itself is subjective, and person-dependent. On the other hand, USPAP is quite clear: “Credible assignment results require support by relevant evidence and logic. …always measured in the context of the intended use.”
Uh oh! More clear, precise evidence, and logical, words . . . : relevant; measured; context; intended.
It must be in the standards somewhere. It can’t be just ‘believe me’! It can’t.
It must be in the review standards. Let’s look. Here are the words we found. The measure of believability: “the reviewer is required to develop an opinion as to the completeness, accuracy, adequacy, relevance and reasonableness of the analysis.” Let’s look at each, and the any possible objective basis.
- Completeness: Assumes the reviewer is a better judge of all that is necessary.
- Accuracy: Assumes the reviewer magically knows the true accurate number.
- Adequacy: Requires an opinion of sufficiency.
- Relevance: Requires a judgment of what is important to the client, whether disclosed, or not.
- Reasonableness: Quite a weasel word – don’t be silly or unreasonable.
All this somehow presumes that the reviewer knows better and more than the original appraiser does.
Here’s the rub. Appraisal is all about numbers – the data collection, selection, adjustment, and delivery. We have numbers, and ways of establishing risk/reliability of valuations (given complete, or substantially complete data).
Why not measures of reliability? Why believability?
Why not reproducibility and auditability? Why subjective review opinion? Why?
The data has improved. The analytics are instantaneous. The brain/machine visualization is there. The goal is to measure markets, not compare comps. It is time for reliability, not believability.
It’s time.
February 22, 2023 @ 5:17 am
The days of solely conforming to the form or other standardized reporting format are numbered. The tools are there, the skills are there, it is time that appraisers demand more from their clients, by allowing the appraiser to show reliability in their analysis and conclusions in consistent reproducible and reportable formats.
February 22, 2023 @ 7:31 am
Being an appraiser, a reviewer, and a client, I wish the skills were there, but they are not. I ‘believe’ in what George is teaching, but that teaching is not universal. When appraisers won’t proofread their reports they don’t catch the fundamental errors, and I am returning 40% of the reports to be redone. Granted these are complex reports mostly for litigation, but these are also the most educated/designated/experienced appraisers.
February 22, 2023 @ 9:50 am
It is my firm belief that appraisers should first be trained in how to complete a Self Contained Report; berfore they are allowed to see or use a Form.
When I had a large office and had 4-trainees at a time, I took the Forms and cut each section out and [asted it onto a legal size page.
For each Check Box, the appraiser had to provide Support/Documentation. Where the report had lines for comments, they had to write a page, then summarize it.
This worked well, to help them think beyond the form.
Too many wwere never Trained to appraise but Told what to do, given a List of Bad Words not to put in a report, a List of Comments to pick from, a List of Adjustments, or even a Template that would auto adjust data. If the value soes not come out high enough, they were told to pick another sale that was higher, for Bracketing purposes.
My first day on the job, I questioned the Bracketing and told my boss we would be biasing the report toward the Sales Price as opposed to Meauring Market Value as defined.
When I question the List of Adjustments and asked if they were used in every City, Price Range, Quality Ranges and Size of homes? He looked astonished, as a supervisor for 5- years he had never been questions about their List of Adjustments.
Thousands of appraisers can complete a form and make it look good in terms of guidelines, but may not Know how to Appraise.
February 22, 2023 @ 9:54 am
This is a great topic. It would be cool to have measures of reliability attached to the appraisal results.
February 24, 2023 @ 10:56 am
Thanks for this George.
USPAP definitely needs to be updated. The appraisal review process should be objective, but as you describe, all of the criteria (completeness, accuracy, adequacy, relevance and reasonableness of the analysis) are subjective and undefined.
The one that sticks out to me most is the criteria for accuracy. If USPAP were to define this as accuracy of the data, HC, EA, descriptions, etc. then I would feel better about this – as these are facts that can ‘reasonably’ be objectively measured. But for a reviewer to opine on the accuracy of the opinion of value ‘Assumes the reviewer magically knows the true accurate number.’ as you state.
Seems like a better review would be to judge whether the original appraiser followed USPAP or did not. Still subjective, but perhaps less so -and at least it’s the same criteria that is used by state boards.
Not to say that a reviewer couldn’t be requested to provide their opinion of value for the same property and describe why their opinion differs from the original appraisal.