Most of us grew up learning two types of statistics — descriptive and inferential.
Descriptive statistics provide summaries of any group, like the mean, the median, the range, the variance. It only got hard when we had to square a bunch of numbers, add them up, and then get out a slide rule (prior to 1972, handheld calculators couldn’t perform this function) to take a square root of the thing to get the standard deviation to compare the spread of two data sets. We may have even learned about skew or even a mysterious kurtosis with leptos or platos or even a meso- to be cleaned up.
In high school and college classes, we might’ve had a quick review of the summary statistics, but then quickly shifted forever into the world of inference — statistical inference. We never returned to descriptive summary statistics. Eventually the HP12c saved the day. In 1981 that is. Except for a few of us who had the forerunner, the HP38c.
From then on, in every statistics class, the problem was to describe a population. Why? Mostly, because back then, data was expensive to gather. We were taught if we could gather a few data points, then we could draw inference from the sample to the population. That is what we grew up with. Then it got harder. Samples had to be randomly picked. And even then, there was a bunch of approximation error. This error which comes from within the sample that randomly happened. “Randomness is important,” we were told. Then there were a bunch of statistics which assumed you took multiple samples, then took the average outcome of those, and calculated test statistics to tell how well the sample represented the population. These were p-values, chi-squared, confidence intervals, F-test, t-test, and more “statistics.”
Whew!
Then we became appraisers. We were taught to pick comps. No one told us this was also a sample. But it was a sample based on judgment. Good profession judgment. Judgment based on specialized education, experience in a market, and training. Judgment: subjective, but effective. Our entire professional “appraisal process” was built on the basis that a comparable is competitive to the subject. It is competitive if it is similar to the subject. And it is similar if it can be compared to the subject. Clear.
It worked well. It was the only solution possible in the days of sparse, incomplete data. Pick three or six comps. Compare them. Calculate rates and ratios. Compare those. Then use your professional training and experience to adjust, reconcile, and expertly opine on the value opinion. In some areas, it’s still the only reasonable way to appraise. In other areas, it’s not. We have quick data. We have complete data. We have big data. We have big computers in a microchip. We have software. And visual tools to ‘see’ data.
But we never, ever, used random samples. Ever.
Yet the first topical appraiser qualifying class, “Real Estate Finance, Statistics, and Valuation Modeling (2004)” – assumed that comps were randomly selected. Then these inferential ‘statistics’ were touted as evidence that your model is good.
Unfortunately, this misuse has permeated much of our curriculum, extending to ‘advanced’ quantitative methods. The problem was we forgot the beginning and intermediate part of “valuation modeling.”
It is a gross fallacy that statistical p-values can somehow prove your model is good. This misuse, however widespread and heavily taught, is wrong. You cannot use a measure of sample strength to tell you how good the model or approach is that you picked! This misuse has worked itself into appraisal litigation valuations, expert opinions, as well as many other places – where it does not belong.
Some good news to the rescue!
The American Statistical Association, for the first time in its history – decided that this abuse had become so widespread, that last year it passed an official statement denouncing this misuse. Among other things it states: “The p-value was never intended to be a substitute for scientific reasoning.” (See a summary here: http://www.amstat.org/asa/files/pdfs/P-ValueStatement.pdf)
In that statement it also carries a more positive note: “Good statistical practice is an essential component of good scientific practice,” the statement observes, and such practice, “emphasizes principles of good study design and conduct, a variety of numerical and graphical summaries of data, understanding of the phenomenon under study, interpretation of results in context, complete reporting and proper logical and quantitative understanding of what data summaries mean.”
In today’s world of appraisal – neither random nor judgment sampling is necessary. We usually have all or most all the available data. We need only improve, “clean” that data, and properly use what we learned in the third grade. Summary numbers, graphs, tables. This is the basis of evidence based valuation©. This is the basis of the future of the valuation profession.
This is the “new valuation modeling paradigm.”
September 20, 2017 @ 7:54 am
Thought provoking, as always . . . and many thanks for continuing the discussion about how we can (and should) better analyze the large quantity of data that is now available at relatively low cost. Below are a couple of points for consideration.
First is that a population of sales within a specified geographic area over a particular time period is just that – a population of sales. But it is not a population of all the properties within that same geographic area. And we usually are interested in that larger group, not just the smaller group of properties that may have sold. For example, we might be developing a model to predict prices, or we might be analyzing the impact of some detrimental condition. And in that case, the group of properties that sold is indeed a sample, a sample that we use to estimate the parameters of the larger population. It is most assuredly not a random sample, but it is a sample nonetheless, and is the best sample available, since we can’t control which properties sell and which ones don’t.
A second point concerns the reliability of statistical modeling generally. Modern valuation theory adopts some of the tenets of neoclassical economics, particularly some of the assumptions associated with the perfectly competitive market (check out some of the conditions imposed on the hypothetical market in some of our value definitions). Of course, the markets we study do not conform to the perfectly competitive market of neoclassical economics, and this is precisely why we have to exercise judgment about the use of statistical modeling. Although inefficient and far from perfect, some of our real estate markets (conforming tract homes, for example) tend to resemble the perfectly competitive market more than others; and it is these markets that are most amenable to statistical modeling. The more the market moves away from this end of the continuum, however, the more careful we have to be about our methods and techniques. Thin markets (think few buyers/sellers, large property variation and/or little available information) will rarely lend themselves to statistical methods, and one of the most important judgments we have to make is to decide what tools to use to solve the valuation problem.
September 20, 2017 @ 10:26 pm
Thank you for your post. You keep opening my mind and reminding me of things I need to remember. The quote from the ASA sounds like they could have been talking about appraisals specifically. I think it also applies to the R-Squared values that appraisers should forget about on simple regression.