Most of us grew up learning two types of statistics — descriptive and inferential.
Descriptive statistics provide summaries of any group, like the mean, the median, the range, the variance. It only got hard when we had to square a bunch of numbers, add them up, and then get out a slide rule (prior to 1972, handheld calculators couldn’t perform this function) to take a square root of the thing to get the standard deviation to compare the spread of two data sets. We may have even learned about skew or even a mysterious kurtosis with leptos or platos or even a meso- to be cleaned up.
In high school and college classes, we might’ve had a quick review of the summary statistics, but then quickly shifted forever into the world of inference — statistical inference. We never returned to descriptive summary statistics. Eventually the HP12c saved the day. In 1981 that is. Except for a few of us who had the forerunner, the HP38c.
From then on, in every statistics class, the problem was to describe a population. Why? Mostly, because back then, data was expensive to gather. We were taught if we could gather a few data points, then we could draw inference from the sample to the population. That is what we grew up with. Then it got harder. Samples had to be randomly picked. And even then, there was a bunch of approximation error. This error which comes from within the sample that randomly happened. “Randomness is important,” we were told. Then there were a bunch of statistics which assumed you took multiple samples, then took the average outcome of those, and calculated test statistics to tell how well the sample represented the population. These were p-values, chi-squared, confidence intervals, F-test, t-test, and more “statistics.”
Then we became appraisers. We were taught to pick comps. No one told us this was also a sample. But it was a sample based on judgment. Good profession judgment. Judgment based on specialized education, experience in a market, and training. Judgment: subjective, but effective. Our entire professional “appraisal process” was built on the basis that a comparable is competitive to the subject. It is competitive if it is similar to the subject. And it is similar if it can be compared to the subject. Clear.
It worked well. It was the only solution possible in the days of sparse, incomplete data. Pick three or six comps. Compare them. Calculate rates and ratios. Compare those. Then use your professional training and experience to adjust, reconcile, and expertly opine on the value opinion. In some areas, it’s still the only reasonable way to appraise. In other areas, it’s not. We have quick data. We have complete data. We have big data. We have big computers in a microchip. We have software. And visual tools to ‘see’ data.
But we never, ever, used random samples. Ever.
Yet the first topical appraiser qualifying class, “Real Estate Finance, Statistics, and Valuation Modeling (2004)” – assumed that comps were randomly selected. Then these inferential ‘statistics’ were touted as evidence that your model is good.
Unfortunately, this misuse has permeated much of our curriculum, extending to ‘advanced’ quantitative methods. The problem was we forgot the beginning and intermediate part of “valuation modeling.”
It is a gross fallacy that statistical p-values can somehow prove your model is good. This misuse, however widespread and heavily taught, is wrong. You cannot use a measure of sample strength to tell you how good the model or approach is that you picked! This misuse has worked itself into appraisal litigation valuations, expert opinions, as well as many other places – where it does not belong.
Some good news to the rescue!
The American Statistical Association, for the first time in its history – decided that this abuse had become so widespread, that last year it passed an official statement denouncing this misuse. Among other things it states: “The p-value was never intended to be a substitute for scientific reasoning.” (See a summary here: http://www.amstat.org/asa/files/pdfs/P-ValueStatement.pdf)
In that statement it also carries a more positive note: “Good statistical practice is an essential component of good scientific practice,” the statement observes, and such practice, “emphasizes principles of good study design and conduct, a variety of numerical and graphical summaries of data, understanding of the phenomenon under study, interpretation of results in context, complete reporting and proper logical and quantitative understanding of what data summaries mean.”
In today’s world of appraisal – neither random nor judgment sampling is necessary. We usually have all or most all the available data. We need only improve, “clean” that data, and properly use what we learned in the third grade. Summary numbers, graphs, tables. This is the basis of evidence based valuation©. This is the basis of the future of the valuation profession.
This is the “new valuation modeling paradigm.”