Automated Valuation Models (AVMs) are becoming increasingly popular in the residential real estate market, offering fast and convenient property valuations. But how accurate are these models, and how can we be sure they're reflecting true market value? AVMs rely on a variety of data points to estimate property values. These data points can be categorized into several types like property-specific (property characteristics, location data, tax records), market-level (economic indicators, public records), or transaction-specific (recent sales prices of comparable properties). For this article, we will focus on two specific transaction-level benchmarks: pre-list and post-list.
First, Let’s Define These Two Benchmarks:
A postlist benchmark is a comparison of the AVM's predicted value to the actual sale price of a property after it has been sold. This comparison is fundamental for evaluating the automated valuation model's accuracy and identifying areas for model improvement.
A prelist benchmark is the valuation estimate generated by the model before a property is listed on the market. It serves as a baseline for comparison once the property is sold, allowing for an evaluation of the AVM's accuracy in predicting market value without knowledge of the list price. Prelist benchmarks also offer insights into how the AVM performs relative to the sold price, but with the advantage of being blind to the list price. In other words, since the AVM is generated prior to the property being listed, it does not take the list price into account, thus providing a more unbiased comparison to the actual sold price. This method eliminates any potential biases introduced by the list price, which often influences both the AVM estimate and the final sale price, particularly in cases where the sale price is closely aligned with the list price.
The Problem with Postlist Benchmarks
Traditionally, AVM accuracy has been measured by comparing the AVM estimate as it exists just prior to the property closing, to the final closed sale price of a property. This approach, however, has a significant flaw: the AVM observed the list price for a large proportion of observations in the sample.
More often than not, the sale price will be fairly close to the list price. By not accounting for this in the AVM benchmark, the AVM’s accuracy statistics calculated from the postlist observations overinflate the model performance when the AVM estimate is applied to off-market properties.
Prelist Benchmarks: A More Accurate Picture
Like a postlist benchmark, Prelist benchmarks compare the AVM estimate to the final sale price of a property. However, they get around the issue of the model observing the list price by using the automated valuation model estimate that existed before the property was listed for sale, and then comparing that to the final sale price.
By measuring the AVM’s performance with a pre-list benchmark, we get a more accurate picture of the model's ability to assess true market value when the model is not privy to the list price..
Benefits of Prelist Benchmarks
Here's why pre-list benchmarks are crucial for measuring AVM accuracy:
- Reduced bias: Pre-list automated valuation model estimates were established before the list price was set, offering a more objective measure of market value.
- Improved transparency: Focusing on pre-list benchmarks allows for a clearer understanding automated valuation model accuracy when applied to off-market properties Off-market properties represent 98-99% of the housing stock at any given point in time.
- Better insights for investors and lenders: More accurate AVM valuations based on pre-list benchmarks provide valuable insights for investors and lenders making informed decisions.
Why Prelist Benchmarks Matter
HouseCanary champions the use of prelist benchmarks because it provides a more reliable picture of AVM accuracy:
- Eliminates "snap-to-list price" bias: AVMs often incorporate logic that heavily weights the listing price, which leads to over inflated accuracy for on-market properties. However, these models’ performance often fall apart for off-market properties where a list price is unavailable..
- Focuses on true market signals: A model solely reliant on list price data might achieve high accuracy for on-market properties without actually considering other crucial market factors. Prelist benchmarks push AVMs to learn from a wider range of data points, resulting in a more robust and reliable model for both on and off-market properties.
The Takeaway
When evaluating AVM accuracy, prelisting benchmarks offer a more reliable and unbiased assessment of the model's ability to reflect true market value. By focusing on prelist data, we gain a clearer picture of the AVM's effectiveness in today's dynamic real estate market. Prelist benchmarks are a significant step towards ensuring AVM accuracy. However, it's important to remember that AVMs are still models, and their accuracy can be impacted by factors like data quality, completeness, and market volatility. It’s also important to remember that AVMs are only as good as the data they're trained on. High-quality, comprehensive data that includes recent sales figures, property characteristics, and local market trends is crucial for accurate valuations.
By embracing a prelist benchmark approach, HouseCanary ensures that our AVMs deliver the most reliable and unbiased property valuations, empowering you to make the most informed decisions against your SFR investments.
