Big Data, What’s the Big Deal?

Big data is in the news.

Why is it a big deal?  What does big data mean to appraisers? And what of those who care about collateral risk, litigation damages, investment risk and portfolio risk.

Perhaps we are progressing out of the age of information to the “age of connection.”  Yes, we are connected by social media, cell phones, wearables, and The Internet of Things (email from your fridge that you need to buy milk).  It’s the age of machine-learning algorithms.  Soon, we’ll tell the car where to go.  It will take us there better and safer than if we are driving.

Big data is everything connected to today’s huge data sets – in different places, in different forms, intentionally and unintentionally collected, often with no purpose in mind.  It’s data waiting for someone to find a problem to solve.  We used to scratch together information to solve a problem.  Today we have masses of data, waiting for a problem to be asked.

Simply put, big data has to do with the huge, growing amount of electronic information available to you.  Data science focuses on the management of data, its analysis, and conveying results, decisions and recommendations.  Data science includes statistics, computation, and human interface to the machine.

The valuation process has come to the same place. 

It has undergone an evolution.  This evolution has been controlled by what data came to be available:

  • To the 1970s,: the data rummage era, based on scarce availability, using whatever we could find;
  • To the 1980s: the market analysis era, based on convenience, using ‘similar’ sales;
  • To the turn of the new century: the data discarding era, based on avoidance, deleting all but three or six sales;
  • Today: the data optimization era, based on the complete competitive market segment.

The data rummage and optimization eras optimized analyst/appraiser usefulness.  The middle two do not.

In the market analysis era, we evolved to defining and focusing on markets, but still used convenient information.  In the data discarding era, we happily took advantage of easy data, but ignored the possibility of more reliable and more credible results from using and analyzing all information.  We turned a blind eye, not seeing, and not hearing the words:

“An appraiser must collect, verify, and analyze all information necessary for credible assignment results.”  (USPAP SR1-4)

Is subjective ‘credible’ enough?  Or can we measure and maximize objective reliability?

Data science, EBV (Evidence Based Valuation)©  gives us higher objective reliability, and thus a far higher professional subjective reliability of our work.  Should our goal be maximal reliability, or is subjective ‘believability’ enough?

Stay tuned.  Same time.  Same channel.  We will explore this more.