Can I Reproduce? Please?

June 28, 2017

George Dell Appraiser, Dying Profession, EBV©, Risk Dying Profession, EBV, Future of Appraisal, Reproducible Work, Risk 1

Why won’t they teach me how to reproduce?

We hear of a “reproducibility crisis.” What is it? Who cares? How does it affect me? And finally – Do we reproduce properly in the appraisal process?

Depending on the author, the scientific method can be broken down into four parts, or five parts, six parts, or eight parts. What you will not usually find in any of these enumerations is an explanation of how reproducibility is important in science or in business decisions. Hmmm?

It is not part of the scientific method. It is not part of the “appraisal process.” It is not required in our valuation standards. So what is this “crisis” and who is causing this terrible thing? It appears that reproducibility is not a part of the scientific process – but is a measure of the reliability of the process itself! Interesting.

Well, I have been told that the appraisal process is both an art and a science. This makes sense. Readings on scientific research themselves assure us that art is a part of science. So what is the core logic of science? It’s simple: testing ideas with evidence. As simple as this sounds – – there has to be more. One of these deeper levels is reproducibility. For research to be tested, it has to be replicated.

Replication

This means a different researcher (appraiser) is given the same assignment, and gets the exact or nearly the exact the same result. So who is the second person in the case of replication? Well, we know that this replicator must also be competent in the subject matter – both in picking comps, units of measure, predictors, and in making adjustments.

Reproducibility

This means a person, given the same data, will get exactly the same result. We can call this person the “auditor.” The “reproducer” person needs not have the subject matter competence! The reproducer takes the data given, performs the computation described, and gets the same answer. In the accountant’s world, the reproducer function is called the ‘auditor.’ The auditor does not form a different opinion of gain/loss, or asset value. The auditor simply approves the financial books or not. A subjective start can only end in a subjective result. From: Can I Reproduce? Please? by George Dell, MAI, SRA, ASA

So what is the distinction for appraisal reports? Well, it appears that so long as the original choice of data (the comps) is subjectively determined, no amount of analysis, testing, replication, or statistics will make the end result objective. A subjective start can only end in a subjective result.

If the reproducer has no way of knowing what data was available or what data was chosen – no reproduction or replication is possible. (Except perhaps by accident). So we are relegated to being subjectively “reviewed,” or “skillfully” check listed.

The appraisal profession is being chipped at from many angles. Is it possible that reproducibility might be a good goal for valuation professionals? Can it put us more into the realm of objective researchers, instead of “opinion supporters”? Can we continue to say “Trust me; I know a good comp when I see it.”? Today’s data and computation make data selection reproducible. So, until we change appraisal to an objective process, we cannot achieve scientific reproducibility. We will forever be subject to review opinions, and automated checklists. Is it time to look? This topic is troubling. We’ll examine it further in future blogs. We appreciate your comments on this.