In an earlier post we noted that the ideal comparable data set is the complete CMS, the Competitive Market Segment^(c) , not just three comps.

Some readers disagreed, believing that three comps is enough.

A primary reason for the use of all sales which competed directly with the subject property is that it improves the reliability of the value prediction.  One Reader noted that presenting three comps (sometimes four) is just right.  I actually agree with that.

But there’s a catch.  There’s a distinction between human understandability, and analytical reliability.  We must separate the two intents.  Both are important.

One of the main points of Evidence Based Valuation^© is that the analysis must include the right data.  The new valuation modeling paradigm separates the analysis from the reporting.  (Just as is prescribed in the traditional model).

So in the sense of reporting to the client, it’s important to present the results in an understandable way.  I agree that it is easier for the typical reader of an appraisal report to understand the comparison of three or four comps.  (Or five or six for non-residential work).  This makes sense.  In fact, when I do work for litigation purposes, the presentation of a few comparable sales works well – for understanding.  Yes,    “It is our job to make the reading of the report simple and understandable, and not misleading.”  Participants in court are helped by presenting illustrative comps.  But what helps understanding may not improve reliability and believability.

Reporting is separate from analysis.  Reporting requires understandability.  For analysis, “three or four” or “five or six” has no theory behind it.   And this is only because our entire appraisal process was created in a 1930’s world of sparse data – not internet instantly available complete data sets.  Analysis requires an optimal data set.

For analysis there is an ideal.

More information is always better.  It improves sureness and trueness (precision and accuracy).  Up to a point.  That point is at which adding more data creates more variation, with no improvement in trueness.  In data science, econometrics, statistics (and common sense tells you) – it is called the bias-variance tradeoff.  At some point the GIGO rule sets in.  Garbage in, garbage out.

Common sense also tells you that one comp is not as good as two, and six is better than three.  Up to the point where they are no longer comps . . .  (The CMS – the Competitive Market Segment)

Data science, the new valuation modeling paradigm, and evidence-based valuation is “a blend of artfully applied scientific concepts and common sense analysis.”  Exactly what is argued.  We have no disagreement.  Science itself includes art.

In the traditional “1930s model” of appraisal, three comps made sense because just gathering three good comps was the challenge.  Today the challenge is providing a truer, surer, more reliable result.  You cannot get an objective result from a subjectively selected data set.

The old way says “Trust me. I know a good comp when I see it.”

The new way says “Here is the evidence.”