Quiz: How many comps are “just right?”
First we must ask: What is a comparable?
We can take two attitudes: 1) “I know a good comp when I see it!” or; 2) What does our accepted literature say? The Appraisal of Real Estate says . . .
“The data used for comparison … should come from properties that are similar…” (TARE, p.121)
“The comparable properties selected for analysis should be similar to the subject property in terms of zoning and other characteristics.” (p. 106)
“A good comparable sale is a competitive alternative …” (p.121)
So . . . What is “competitive?” The sentence continues:
“… a competitive alternative – i.e., a property that the buyer of the subject would also consider.”
Comparable = Similar = Competitive = Comparable
Fuzzy circular. But all we have to work with. For now, let’s say we need a cutoff. Is it a comp, or not a comp. Yes or no. (In the Valuemetrics.Info SGDS1 workshop, we use the much better, five-dimension method!)
So: How many comparable-competitive-similar-comparables are best? Is it three because that fits on a standard sheet of paper? Should it be six, because that fits well sideways on a spreadsheet? Or should it be the whole city or neighborhood?
More questions: Are two comps better than one? Are four comps better than two? What about 8 instead of 4, or 32 or 63? How about 12,448 comps? Is there an optimum amount of data?
The data science says yes . . . It involves a famous trade-off. It is called the bias-variance tradeoff. For appraisers, subjective bias may come from picking three comps. There is always some uncertainty in each of the sale prices, some uncertainty in the reported measurements. The result will always be too high or too low – just from whatever three are picked. This is analytic bias (not appraiser bias, of course).
Variance comes from the randomness also, but this randomness is noise error. As more data is added, some if it may be irrelevant, and simply not correlated with prices at all. Adding this ‘excess’ data simply increases the noise, and makes the result, less certain. Too little data creates bias. Too much data creates noise. Too much or too little are less than optimum. This is the technical explanation.
I found this – a five year old’s explanation: A person with high bias is someone who starts to answer before you can even finish asking. A person with high variance is someone who can think of all sorts of crazy answers.
Back to our question:
So what is the ‘sweet point’ of the number of comparables to use? We need to define one more term closely — Information is data made useful. Data becomes useful information in two ways — through selection/summarization or estimation/prediction.
In earlier appraisal history, the cost, the effort of identifying and confirming more than a handful of comps was a reasonable factor. “The selection of comparables is directed to some extent by the availability of data.” (TARE p.121)
Today, (in most appraisal situations) we can get all or substantially all the sales in an instant. What is different today is how we go about verifying our data: confirming, cleaning, and preparing. We attend to outliers as much as the ‘similars.’ The ideal data set is the complete CMS©, the Competitive Market Segment©. This is the first major goal of evidence-based valuation. Get the right data set: the CMS©.
September 13, 2017 @ 6:18 am
Ummm….George … What’s a “market segment?” 🙂 How do I know whether a sale belongs in THIS segment, or THAT one? Assuming, of course, that I know what a “market segment” is.
Enough teasing. You’re sneaking up on a method of data segmentation that data scientists call cluster analysis. There are other methods of segmenting data, but clustering is one of them. There are–according to one source–thirty different methods of calculating the number of segments and their respective boundaries.
As you’ve said in a prior post, we appraisers need to become good data scientists. This is NOT a simple problem.
September 15, 2017 @ 9:20 pm
Cluster analysis should be a part of any appraisal curriculum, and you don’t need to learn 30 different methods. It’s all about measuring distance between sale attributes or features. And measuring should be intuitive for appraisers.
September 13, 2017 @ 7:27 am
How about this, the number of comparable properties would be equal to the number of properties within a given market area that the buyer considered before purchasing the property that the buyer presently owns. The criteria would be price, location, GLA, condition, and any attendant amenities, in that order.
September 17, 2017 @ 2:23 pm
You did not answer your own question: How Many Comps Are Enough? You gave other alternative suggestions, but never gave your opinion on how many. In my 26 year experience, it is quite simple. Bracketing is key. If there is an adequate amount of data available, three closed sales are optimum – one slightly inferior, one similar, and one slightly superior to the subject. To support the comparable sales selection, provide a non-closed sale (pending or listing). This is the perfect scenario and is appraisal 101 – back to basics. There is no reason to provide an excessive amount of comparable data, just for the heck of it. Appraisal reports are supposed to be reader friendly, not overkill or overwhelming.
September 19, 2017 @ 7:02 pm
You say it’s simple, so I’m curious, how did you arrive at your 3 closed bracketing sales are optimum measure? Optimum in what way?
September 19, 2017 @ 8:12 pm
Unbelievable. What used to be a blend of artfully applied scientific concepts and common sense analysis has been supplanted by process alone. My favorite reads when I worked for IRS were the Business Enterprise Valuators / CPAs professional papers on “new” accounting methods that proved up was now down; left was really to the right if you go far enough, and white is the presence of all color while black is the absence of all color (OK Im cheating a smidge on that last). Two and two CAN be five, and so on.
You going to believe your lying eyes and obviously confused independent thought, or are you going to trust those experts who spend their whole life studying economic theory? The ones that are almost always wrong in the present but have 20/20 hindsight? The ones that create nice scientific terms (bias; deviation, etc.) to explain why they are rarely more accurate than 70% in a world that demands 95% to 98%?
I agree with Deena. Back to Basic Appraising 101
October 1, 2017 @ 3:27 pm
Charles, I did not say that the compilation of data and the analysis of that data was simple. However, it is our job to make the reading of the report simple and understandable, and not misleading. We are in control of the preparation and layout of the report. There is no reason to offer so many comparables that it overwhelms the reader — leaving them confused and open to challenging our selection of data and ultimate results. So, having the perfect scenario, narrowing it down to the three best sales is optimum (with a non-closed comparable thrown in for support and to show what the current trend is). Period. Appraisal 101. (This, of course, is mostly attainable when there is a nice pool of truly comparable data available.) So, my answer to George’s question of ‘How Many Comps Are Enough,’ is three.