Ensuring Trustworthy Valuations in the Age of Automated Valuation Models (AVMs)

The rise of Automated Valuation Models (AVMs) in the mortgage lending industry has ushered in a new era of efficiency and convenience. However, with this innovation comes the critical need for ensuring the accuracy and fairness of these automated valuation tools. This piece explores the recently proposed AVM quality control standards and how HouseCanary strives to ensure that our AVM meets these standards.

‍

The AVM quality control landscape

The Dodd-Frank Wall Street Reform and Consumer Protection Act (Dodd-Frank Act) established new regulations for AVMs used in mortgage lending. These regulations outline five key quality control standards that AVMs must meet:

1. Ensure a High Level of Confidence in the Estimates: AVMs must produce reliable and accurate valuations.
2. Protect Against Data Manipulation: Measures must safeguard AVM data from tampering or manipulation.
3. Avoid Conflicts of Interest: Institutions must mitigate potential conflicts that could influence AVM outputs.
4. Require Random Sample Testing and Reviews: Regular reviews and testing of AVMs are essential for identifying and addressing potential issues.
5. Comply with Nondiscrimination Laws: AVMs must not perpetuate bias or discrimination based on protected characteristics.

This document emphasizes that the ultimate responsibility for ensuring AVM quality falls on the users, namely lenders and secondary market participants. These institutions must conduct thorough due diligence on AVM providers to verify their compliance with the AVM Quality Control Standards listed above.

‍

HouseCanary and AVM quality control

HouseCanary is committed to transparency and works diligently to ensure our AVM meets the standards outlined in Section 1125 of the Dodd-Frank Wall Street Reform and Consumer Protection Act (Dodd-Frank Act). 

Here’s how we address each of the five quality control factors:

1. Ensure a High Level of Confidence in the Estimates: We utilize a comprehensive dataset encompassing public records, property characteristics, multiple listing records, and market trends. This data is continuously updated and refined to enhance valuation accuracy. Additionally, we employ a statistical measure called "Forecast Standard Deviation (FSD)" to estimate the valuation model uncertainty. The FSD informs the user of the model’s confidence (or lack thereof) in the estimated value for a given property.  
2. Protect Against Data Manipulation: We employ robust data security measures to safeguard ourdata from tampering. These measures may include encryption, access controls, and regular audits. By prioritizing data security, we help to ensure the integrity of its AVM outputs.
3. Avoid Conflicts of Interest: Our revenue model is independent of valuation outcomes, eliminating any potential incentive to inflate or deflate property value estimates. This ensures that the AVM remains objective and unbiased.
4. Require Random Sample Testing and Reviews: We conduct continuous testing of our valuation model using both internal and third-party monitoring. These tests analyze aggregate valuation error across various factors, such as property type, price tier, and geographic location. Additionally, we provide performance metrics that allow financial institutions to compare AVM performance to their established criteria. This transparency in testing and performance empowers lenders to make informed decisions about AVM usage.
5. Comply with Nondiscrimination Laws: We actively work to mitigate potential biases in our AVM development process. This includes excluding protected attributes, such as race, religion, and sex, from the training and prediction procedures. We also conduct racial disparate testing on a monthly basis to identify and address any potential disparities in valuations across different racial demographics.

‍

Beyond the AVM standards: security and transparency

Additionally, we conduct regular penetration testing, business continuity testing, and incident response testing to ensure data security and business continuity. Here's a summary of our practices:

• SOC 2 Audits: We undergo annual SOC 2 audits, which demonstrates the effectiveness of our control environment.
• ISO Certification: Our Information Security Management System adheres to the rigorous ISO/IEC 27001:2013 standard.
• Penetration Testing: Regular penetration testing identifies and eliminates vulnerabilities in web applications and APIs.
• Business Continuity and Incident Response: We ensure minimal disruption during unforeseen events through regular testing of their business continuity and incident response plans.
• Data Encryption: Customer data is encrypted using modern protocols to safeguard privacy.
• Full archival and transparency of historical performance metrics: This allows financial institutions to make data-driven decisions about the suitability of our AVM solution for their specific needs.

‍

Conclusion

While AVMs offer significant benefits for the mortgage lending industry, ensuring their accuracy, fairness, and security is paramount. Our commitment to AVM quality control standards, robust data security practices, and transparent testing methodologies position us as a reliable provider for lenders and secondary market participants seeking trustworthy AVM solutions. By working together, lenders, AVM providers, and regulators can foster a more transparent and trustworthy AVM ecosystem that benefits consumers and the housing market as a whole.