Out of 100 personal finance searches, 43% had misleading or incorrect information.
I have noticed this myself, I was not asking AI overview a question but it showed up on top of a search query. I was like that’s not right, and it’s not a hard question.
From the study:
Key Findings
The study evaluated AI overviews across various financial topics, including banking, credit, investing, taxes, and student loans.
The results showed:
- 57% of Google’s AI overviews were accurate
- 43% contained misleading or inaccurate information
- 12% were completely incorrect
- 31% were either misleading or missing crucial details
Queries Google AI Got Right
- Basic definitions and explanations: For example, “What is a wire transfer?” and “How does a credit card work?”
- Simple, straightforward questions: Such as “Do I have to pay back student loans?”
- Recent trending topics: Like “What was the Chase Glitch?”
- General insurance questions: For instance, “When should I get life insurance?”
Queries Google AI Got Wrong
- Complex tax topics: For example, “Can you use a 529 plan for a Roth IRA?” and “Does owning your house in an LLC help with taxes?”
- Nuanced financial products: Such as “Is an IUL better than a 401k?”
- Time-sensitive information: Like outdated student loan repayment plans or savings account rates.
- State-specific financial rules: For instance, misrepresenting California’s 529 plan rules.
- Questions requiring context-dependent answers: Such as “Can I file as independent for FAFSA?”
- Queries about financial limits or thresholds: For example, incorrect IRA contribution limits.
- Complex student loan topics: Particularly around forgiveness programs and repayment plans.
- Investment comparisons: Like “Are annuities better than CDs?”
What This Means
Google’s AI performs well at giving straightforward answers to factual queries.
On the other hand, it struggles with nuanced understanding, up-to-date information, and consideration of multiple factors.
This suggests that the AI can handle basic financial literacy topics, but it’s unreliable for complex financial decisions or advice.
Read full story
