Introduction
Artificial Intelligence (AI) is revolutionizing various fields, with finance being one of the most impacted. OpenAI’s ChatGPT is a notable AI tool making significant contributions, particularly in financial analysis and reasoning. Research & developments, such as BloombergGPT, have also paved the way for AI-driven financial tools, while our paper on ChatGPT bias with Groupama Asset Management highlights concerns in this area.
This article will dive into how ChatGPT can be utilized for financial tasks, showcasing its strengths and areas for improvement. Please note that the content shared here is based solely on our internal research and experiences with clients in the financial industry.
ChatGPT in Financial Analysis
Since its debut in November 2022, ChatGPT has gained recognition for its utility in financial analysis. Researchers have employed it for tasks such as document sorting, sentiment analysis, and data extraction within financial settings. Unlike traditional methods that struggle with complex texts, ChatGPT has demonstrated an ability to handle these challenging tasks effectively.
Methodology
To aid finance professionals in understanding ChatGPT’s capabilities, we conducted a comparative analysis between ChatGPT and another tool, Microsoft Copilot, across various financial tasks. Figure 1 provides a detailed comparison of their performance.
Here are the findings:
ChatGPT’s Strengths:
- Detailed Financial Analysis: Excels in creating comprehensive financial reports.
- Income Statements and Ratio Analysis: Performs well in generating and analyzing financial statements.
- Balance Sheet Analysis: Efficiently extracts and analyzes balance sheet data.
- IFRS Standards and Discounted Cash Flow Calculations: Proficient in comparing international financial standards and calculating cash flows.
- Basic VAT Questions and Financial Restructuring: Capable of handling VAT queries and providing clear instructions for financial restructuring.
Copilot’s Strengths:
- Simple Financial Tasks: Adept at currency conversion and basic VAT questions.
- Up-to-Date Rates: Features a built-in tool for direct conversions with current rates.
However, Copilot struggled with more complex tasks and often required significant assistance, especially in generating income statements and managing financial restructuring.
Detailed ChatGPT Evaluation
We conducted a thorough evaluation to assess ChatGPT’s performance (Figure 01). We evaluated ChatGPT across different financial tasks, and performance was assessed based on a scoring system detailed below. The performance score was calculated by assigning a score based on accuracy, clarity of the results, and the tool’s ability to manage tasks autonomously.
Figure 1 : Detailed Comparison of ChatGPT and Copilot Performance in Financial Tasks
Source : AI-Driven Financial Analysis: Exploring ChatGPT’s Capabilities and Challenges. International Journal of Financial Studies, 12, 60.
The subsequent phase of our study focused exclusively on ChatGPT, testing it on tasks such as:
- Discounted Cash Flows
- Earnings Forecasts
- Valuing Private Equity Firms
- Financial Ratio Analysis
- Financial Restructuring (Mergers)
- IFRS Standards
Performance Highlights:
- Effective Communication: Provides clear and straightforward explanations.
- Accuracy with Detailed Instructions: Performs well when given specific instructions, though flexible instructions can lead to errors.
- External Validation: An external study corroborated our findings, emphasizing the importance of precise guidance.
Testing
ChatGPT-4o has been evaluated against human analysts on a range of tasks, from basic calculations to complex financial analyses. The tasks included present value calculations, ratio analysis, portfolio construction, and financial statement analysis. The researchers assessed its ability to understand tasks, generate ideas, calculate accurately, and think critically.
Multi-Step Reasoning Tasks
Figure 2 illustrates the results of these tasks. ChatGPT-4o performed comparably to human analysts in basic financial tasks, demonstrating logical and reasoned approaches. However, it faced challenges with tasks requiring deeper analysis and complex information management, such as calculating the internal rate of return (IRR).
Figure 2 : Multi-Step Reasoning Tasks Performance based on the Task Type
Source : AI-Driven Financial Analysis: Exploring ChatGPT’s Capabilities and Challenges. International Journal of Financial Studies, 12, 60.
Complex Reasoning Tasks
In advanced tasks like portfolio construction and capital budgeting analysis, ChatGPT-4o fell short of human analytical capabilities. As we can observe on Figure 2, while ChatGPT-4o could perform technical analyses and provide investment recommendations based on indicators like Bollinger Bands and MACD, it struggled with developing detailed templates for capital budgeting and financial statement analysis.
Figure 3 : Complex Reasoning Tasks Challenges Performance based on the Task Type
Source : AI-Driven Financial Analysis: Exploring ChatGPT’s Capabilities and Challenges. International Journal of Financial Studies, 12, 60.
Practical Takeaways
- Efficiency: ChatGPT-4o enhances financial analysis by automating repetitive calculations, freeing analysts to focus on strategic decisions.
- Technical Analysis: Its proficiency in technical analysis and portfolio construction aids in developing investment strategies.
- Improvement Areas: Figure 4 highlights areas where improvements in data accuracy, contextual understanding, and real-time updates are necessary for maximizing ChatGPT’s benefits in financial analysis.
Figure 4 : Overall Evaluation Metrics based on Six Key Criteria
Source : AI-Driven Financial Analysis: Exploring ChatGPT’s Capabilities and Challenges. International Journal of Financial Studies, 12, 60.
Conclusion
This study highlights ChatGPT-4o’s strengths and limitations in financial analysis. While it excels in basic and some complex tasks, it lacks the deep analytical capabilities required for more intricate financial analyses. Integrating ChatGPT-4o with specialized financial data tools could significantly benefit human analysts. Future research should aim to enhance AI’s deep thinking abilities and address ethical concerns for fair and objective financial analysis.
References
[1] Liu, L. X., Sun, Z., Xu, K., & Chen, C. (2024). AI-Driven Financial Analysis: Exploring ChatGPT’s Capabilities and Challenges. International Journal of Financial Studies, 12, 60. https://doi.org/10.3390/ijfs12030060 For further details, please refer to the complete study in the International Journal of Financial Studies(ijfs-12-00060-v2).
