We clean, scrub, transform, wrangle, munge, map, shape, enrich, enhance, prepare, preprocess, improve, and … verify. Data scientists spend 80% of their time on the data. Those of us who continue to move into these more objective methods – find this to be true. In fact, once you have the data prepared and enhanced, the result falls out and seems obvious.
In USPAP, (Uniform Standards of Professional Appraisal Practice) – in appraisal, an appraiser “must collect, verify, and analyze all information necessary for credible assignment results.” The word “verify” is not defined nor explained – but you must do it.
The Appraisal of Real Estate, (ARE) 14th edition, recognizes that there are different levels of verification, which may or may not require you to “talk directly to a party to the transaction.” This may depend on the requirements of the client.
In the ARE chapter on The Sales Comparison Approach it does specify that you “Verify the information by confirming that the data is: factually accurate, and is arms-length. And should also “elicit additional information about:” motivation, economic characteristics, value component allocations and “other significant factors … to ensure that comparisons are credible.”
Generally, the focus of ARE and USPAP are to be credible (“worthy of belief”). Unfortunately, the legacy appraisal process comprises subjective judgment (picking comps, making adjustments, and reconciling the varying answers). Data science, and EBV© (Evidence Based Valuation) focus on analytical objectivity. The data selection, predictive methods, and results are objectively estimated. Any uncertainty is not buried in words or intentional ambiguity. The uncertainty itself is measured or estimated.
The focus is not on human (appraiser) unbiasedness. The focus is on analytical unbiasedness. This takes a different, modern set of skills. Skills that today are not commonly taught for licensing or designations.
So, what about “verifying” in the data science approach to asset assessment? Our modern approach actually does what USPAP and the ARE tell us to do:
- USPAP: use all the relevant sales “as are available” (SR 1-4)
- ARE: provide “detailed evidence [of] the best and most relevant data.” p. 96. 14th
In traditional appraisal — “verify” can include “confirm” or other ways to establish factual reliability. Modern (EBV©) methods embrace a deeper and more detailed evidence, as are required by standards and recommended practices.
Data science ‘wrangling’ of data includes verification and confirmation. It goes well beyond the requirements of good appraisal practice and standards. We started this note with 12 common words worked on data in data science. So, what’s different?
Evidence Based Valuation©:
- directs attention to where accuracy and precision are best improved;
- transforms data where it is more easily analyzed and understood;
- enables reproducibility, eliminating subjective reviews;
- delivers additional quantitative/logical results.
The days of subjective data selection and undefined levels of “verification” are nearly over. The appraisal profession can learn to produce modern results, or be pushed aside.
“Trust, but verify.”
August 12, 2020 @ 7:15 am
So how do we learn these science based methods? Can you recommend any books?
August 12, 2020 @ 11:26 am
Verification is actually defined in the Dictionary of Real Estate Appraisal – “the process of validating or establishing the truth about information from another source . . . a valuer . . . may confirm information directly with a party knowledgable about the property or the transaction . . . or with another credible source.” The debate in appraisal practice is whether that includes actually talking to someone (buyer, seller, agent, etc.) in connection with every comparable transaction used in an assignment.
In traditional appraisals with a limited number of comparables, that usually isn’t an onerous task, though I’d argue that depending on the data and the assignment, it probably isn’t always necessary. One of the reasons agents and brokers often don’t want to talk with appraisers is that many appraisers use the personal verification as simply an exercise in “hey, is the info in the MLS (or whatever source) correct?” Agents and brokers hate this, as well they should, because it wastes their time. But many appraisers do it because they think they are supposed to.
For some types of assignments (eminent domain comes to mind), this is often a critical task. For many other types of assignments, not so much. A careful examination of the data will tell the appraiser if there is something that needs further confirmation. Maybe the condition of the property is uncertain, there were unspecified concessions, or the transaction price appears high or low relative to other data considered. I’ve found that agents and brokers generally don’t mind those types of questions, because they are focused and pertinent. But simply doing a personal verification of every transaction because you think you are supposed to is a waste of everyone’s time.
Which brings us to Data Science, which often utilizes large datasets . . . sometimes hundreds or even thousands of observations. Personally verifying or confirming that quantity of data is simply not feasible. Nor is it necessary. One of the hallmarks of data science is the ability to identify outliers. These are the transactions that require attention, which frequently means making phone calls; sometimes outliers are excluded. It isn’t necessarily a bad idea, when dealing with large datasets, to talk to key players in the market to see if they have any observations or knowledge that might not show up in the data; a reality check, so to speak.
Key takeaway is to use personal verification judiciously. Those who use it indiscriminately will only serve to annoy the subject of their inquiries, making these folks less likely to take phone calls from any appraiser, particularly in cases where their information might be critically important.
August 12, 2020 @ 11:28 am
George teaches two excellent classes on this subject – Stats, Graphs & Data Science 1 and 2; I’ve taken both of them and they are well worth the time and money.
August 14, 2020 @ 12:44 pm
Michael,
Thank you for your insightful comments.
Especially those about the valuemetrics.info education.
We have both the classes you mention coming up in October in Oakland. And SGDS1 in Columbus Ohio.
September 10, 2020 @ 7:09 am
Hey George! I follow your blog and interested in taking your classes. Do you ever come to the Northeast? Also, do you offer any online?
On this subject – I find blending both approaches is the best. I’ll spend most of my time scrubbing and coding data to build a model, and while doing it I’m able to find and select better comparables for the traditional sales comparison. The reconciliation process is far easier when the data is aggregated and presented cleanly.
September 10, 2020 @ 8:52 am
Thank you Steve,
We do plan to do something next year in the northeast. Also working on some on line soon.
You might consider a TAAR subscription. The current edition provides the code for market analysis scatterplots, trend lines, and time adjustments.
And yes, the use of ‘both methods’ works. Better comps, better adjustments, as well as direct predictive methods. Report comps we call “illustrative”, as the human brain can comprehend 4 or 5, but 7+ not very well.
We have to find ways to subdivide, or summarize. (Which is what descriptive stats do!), in order to comprehend a market.