AI in Financial Analysis: What Works, What Hallucinates, and Who Owns the Number

Why This Matters

Financial analysis is where AI looks the most magical and behaves the most dangerously. A bookkeeping error gets caught at reconciliation. A bad number in a valuation model can move a real investment, a lending decision, or a board vote before anyone checks the math. Bookkeeping records the past. Tax reports the past to an authority. Financial analysis makes claims about the future—and forecasts, valuations, and recommendations get acted on with real money.

The U.S. Government Accountability Office, in its May 2025 report on AI in financial services (GAO-25-107197), found that AI is now applied across automated trading, credit decisions, investment decisions, and risk management, with benefits including improved efficiency and reduced costs. But the same report warned that AI can amplify the risks already inherent in financial activity, driven by complex and dynamic models, poor-quality data, reliance on third parties, and new vulnerabilities such as hallucinations. It specifically flagged that AI models may produce false or misleading information about financial products and services, potentially harming consumers or investors.

That is the core tension of this entire page: the tasks where AI saves the most time are also the tasks where a quiet error does the most damage. We follow one continuous example—analyzing ABC Coffee Shop—so the concepts stay concrete.

Learning objective

By the end of this module, learners should be able to describe what AI genuinely does well in financial analysis, explain why a widely hyped earnings-prediction study was withdrawn and must not be cited as fact, interpret measurable hallucination evidence, apply SR 11-7 / CFA / SEC guardrails, and decide when to rely on, verify, or reject AI output.

What AI Genuinely Does Well

Used as an assistant rather than an oracle, AI is legitimately strong at several analyst tasks. These claims are supported by vendor documentation and peer-reviewed research, and each has documented limits covered later.

Summarizing long disclosures

AI can compress a 100-page 10-K or an hour-long earnings call into themes, then let you jump to the source. Microsoft’s Copilot in Excel documentation describes surfacing summaries, trends, and outliers from your data in natural language.

Generating formulas and analysis in the spreadsheet

Copilot in Excel can generate formula columns, create charts and PivotTables with editable links to source data, and answer data questions using Python-based analysis whose code you can expand and inspect.

Classifying financial sentiment at scale

Domain-specific models are measurably better than generic tools here. FinBERT, a finance-tuned language model, achieved 88.2% out-of-sample sentiment-classification accuracy on analyst-report sentences in one peer-reviewed study—versus 62.1% for the standard Loughran-McDonald finance dictionary and 71.9%–76.3% for common machine-learning methods. That figure is study-bound to that dataset and task.

First-draft narratives and variance explanations

AI is useful for turning “revenue up 8%, margin down 2 points” into readable commentary you then verify and correct.

The consistent pattern: AI accelerates transcription, summarization, drafting, and pattern-surfacing. It does not reliably replace judgment, verification, or accountability.

The Study Everyone Cites—and Why We Don't Cite Its Numbers

If you research this topic, you will quickly hit a widely repeated 2024 claim: that GPT-4 predicted the direction of companies' future earnings better than human analysts, roughly 60% versus 57%, in a University of Chicago working paper by Kim, Muhn, and Nikolaev titled Financial Statement Analysis with Large Language Models.

Unverified — paper withdrawn. The arXiv record shows the paper was formally withdrawn on February 20, 2025, with the authors' own note stating that "a co-author identified inconsistencies in the data and analyses while attempting to replicate past analyses," and that they had "temporarily withdrawn the working paper from circulation while we review the research findings." This page treats the specific "60% beats analysts" figure as unverified and does not rely on it.

This is not an attack on the researchers—self-correction is science working as intended. But it is a perfect, real illustration of this page's entire thesis: the most exciting AI-in-finance headline of 2024 turned out to rest on numbers the authors themselves could not reproduce. If a team of respected finance professors, working carefully, found data inconsistencies serious enough to retract, an analyst pasting a figure from a chatbot into a model has every reason to verify before relying.

What the Evidence Does Reliably Show

Where the peer-reviewed record is consistent and reproducible is on AI's tendency to state false things confidently— especially specifics like citations, figures, and reasoning chains.

Study	What was tested	Finding
Financial literature citations (arXiv, 2024)	150 citations requested from chatbots	ChatGPT-4o hallucinated references 20.0% of the time; o1-preview 21.3%; Gemini Advanced 76.7%
AI hallucination in financial research (IJEFI, 2026)	Repeated capital-budgeting prompt over 3 days on ChatGPT-4o	~half of cited studies were fictitious and unverifiable; recommendations reversed across days

Two lessons for analysts follow directly. First, AI-supplied references and figures must be traced to primary sources—roughly one in five financial citations from even a strong model was fabricated in the citation study. Second, AI is not deterministic: the same prompt produced contradictory financial recommendations on different days, so "it said so yesterday" is not evidence.

The GAO reached the same conclusion at the system level, listing hallucinations among the novel vulnerabilities AI introduces to financial institutions.

Worked Example: ABC Coffee Shop

Suppose you ask an AI assistant to "analyze ABC Coffee Shop's financials and tell me if it's a good investment." Three realistic failure modes show why the output needs an analyst on top.

1. The confident wrong number

The AI reports a current ratio of 2.1. Your workbook says 1.4. The AI transposed current liabilities. Because hallucination research shows models present false specifics with the same fluency as true ones, nothing in the tone of the answer warns you.

2. The fabricated benchmark

The AI claims “the average coffee-shop net margin is 12%” and cites a study. Citation studies show a meaningful share of such references are fictitious—so the benchmark you’d anchor a valuation on may not exist.

3. The unstable recommendation

Ask twice, get “attractive entry point” and “overvalued.” The reproducibility failure documented in the research means the recommendation itself can flip.

The fix is not to abandon AI—it is to use it for the first draft of the analysis and then verify every figure against the source data, exactly as the professional frameworks below require.

The Correct Workflow

1

Pull actual figures from primary sources

Use ABC Coffee Shop’s source records—or, for public companies, primary filings. The SEC’s free EDGAR APIs return structured XBRL financial facts directly from filings with no vendor in between.

2

Compute ratios with inspectable logic

Compute ratios yourself, or with AI-generated formulas whose logic you can expand and check.

3

Draft, then confirm each claim

Use AI to draft the narrative and flag anomalies, then confirm every claim against source data.

4

Document the trail

Record what AI produced, what you changed, and why.

Professional Guardrails Regulators Now Expect

Financial analysis sits inside a dense web of professional standards. None of them were written to ban AI—and none of them let you outsource responsibility to it.

Model risk management (SR 11-7)

Long before generative AI, banking regulators governed the risk of relying on models. The Federal Reserve and OCC's SR 11-7 guidance defines model risk management around robust development, independent validation, and governance—and its guiding principle is "effective challenge": critical analysis by objective, informed parties who can identify a model's limitations and assumptions. It calls for ongoing monitoring and "outcomes analysis" comparing model outputs to actual results. An AI model is still a model; SR 11-7's logic—validate it, challenge it, monitor it, document its limits—applies squarely.

CFA Institute ethics for investment professionals

The CFA Institute's framework for AI in investment management is built on data integrity, accuracy, transparency and interpretability, and accountability. It ties directly to the CFA Standards: professionals must have a reasonable basis supported by appropriate research for their analysis and recommendations (Standard V), must not act on material non-public information that AI tools might ingest (Standard II), and must disclose how AI is incorporated into the investment process (Standard III). Crucially, it advises preferring the less complex model when a simpler one delivers similar outcomes—a direct rebuke to using opaque AI for its own sake.

The SEC and "AI-washing"

Overstating AI is itself an enforcement risk. In March 2024, the SEC brought its first AI-washing cases, settling with investment advisers Delphia and Global Predictions for false or misleading statements about their use of AI, with civil penalties of $225,000 and $175,000 respectively. Global Predictions had, among other things, claimed its models "outperformed IMF forecasts by 34%" without documented substantiation. The takeaway for analysts and firms: any performance or capability claim involving AI must be substantiable, and descriptions of AI use must be accurate across filings, marketing, and websites.

Framework	Source	Core requirement for AI use
Model risk management	Fed/OCC SR 11-7	Validate, “effectively challenge,” monitor, and document model limits
Investment ethics	CFA Institute	Data integrity, accuracy, transparency, accountability; disclose AI use; reasonable research basis
Marketing/antifraud	SEC (Advisers Act)	AI claims must be truthful and substantiated; no “AI-washing”
Systemic/operational risk	GAO 2025	Manage data quality, third-party concentration, hallucination, cyber risk

Practice: Rely, Verify, or Reject?

For each AI output, decide whether to Rely, Verify, or Reject, then reveal the best response.

Scenario 1

AI summarizes ABC Coffee Shop’s lease footnote and links each claim to the paragraph it came from.

Scenario 2

AI states the specialty-coffee industry’s average EBITDA margin is 18% and cites “Journal of Food Economics, 2023.”

Scenario 3

AI generates an Excel formula column computing gross margin, and you can expand the underlying logic.

Scenario 4

AI recommends “buy” on a public coffee chain; asked again an hour later, it recommends “hold.”

Scenario 5

AI drafts a variance commentary explaining why ABC’s Q2 labor costs rose.

Score: 0/5 correct (0 reviewed)

Knowledge Check

Five questions on the withdrawn paper, citation hallucination rates, effective challenge / SR 11-7, FinBERT, and AI-washing.

Question 1: Why is the withdrawn Kim–Muhn–Nikolaev paper used as a teaching example here?

Question 2: Roughly how often did a strong model fabricate financial-literature citations in testing?

Question 3: What is the “effective challenge” principle, and where does it come from?

Question 4: Is a finance-tuned model actually better than a generic tool for sentiment analysis?

Question 5: What is “AI-washing,” and why should analysts care?

The Bottom Line

AI is a powerful analyst assistant for summaries, inspectable formulas, first-draft narratives, and—with domain-specific tooling like FinBERT—financial text classification that can outperform older methods in studied settings.
The most-hyped “AI beats analysts” earnings-prediction paper (Kim, Muhn & Nikolaev) was withdrawn by its authors for data inconsistencies; treat viral 60%/57% figures as unverified.
Hallucination of figures and citations is measurable and material—roughly one in five financial citations from a strong model was fabricated in one study; another found about half of cited studies fictitious and recommendations that flipped across days.
Regulators and professional bodies expect validation, effective challenge (SR 11-7), disclosure, and substantiation (CFA ethics; SEC AI-washing enforcement) whenever AI touches a financial decision.
The professional standard is AI for the first draft, a competent human for the judgment, and a documented trail proving which was which.

In financial analysis, the number is only as trustworthy as the person willing to sign their name under it.

Sources & Further Reading

Selected primary, research, and practice sources used in this module. Standards and tools change; recheck official Fed/OCC, CFA, SEC, and vendor docs when evaluating a tool or claim. Research accuracy figures remain tied to their study conditions. The Kim–Muhn–Nikolaev arXiv record is included with its withdrawn status clearly labeled.

Ready to Practice?

Take the rely / verify / reject habit from this lesson into the AI Practice Arena—judgment first, automation second.

Open AI Practice Arena

What's Next?

Next in the practice catalog is AI in Accounts Payable (coming soon)—or return to the AI hub to revisit Foundations and related practice topics.

➡️

AI in Accounts Payable

Next in the practice catalog (coming soon)

AI in Accounting Hub

Browse all AI pillar topics

Related Concepts

AI in Audit

Anomaly detection and fuller transaction testing

AI Limitations and Hallucinations

Why confident answers still need verification

Excel Copilot for Accountants

CLEAR prompts, reconciliations, and workbook controls

AI and the CPA Role

Professional responsibility as tools change