ChatGPT has taken the world by storm — whether we want it to or not.
You know that saying, "When in Rome?" Well, we're right in the middle of it, and studies have already shown that workers across industries are using AI to improve their productivity by 66 percent.
But is it really all it's cracked up to be?
Accounting Today put ChatGPT to the test—or should we say the CPA exam. And in all four sections, the tool failed miserably.
In this article, we'll get to the bottom of:
- Why ChatGPT failed the CPA exam
- What ChatGPT can't do — and why it won't take your job
- What AI can do — and why you should try it anyway
- How to get the best of both worlds with AI
Let's get started.
Why ChatGPT failed the CPA exam
In collaboration with Surgent CPA review Accounting Today signed up ChatGPT for the CPA exam. And it failed in all four sections:
- REG: 39%
- AUD: 46%
- FAR: 35%
- BEC: 48%
🎓 A little about the experiment: To ensure a fair assessment, two separate laptops were used, each equipped with a distinct ChatGPT 3.5 Pro account. One laptop handled the BEC and FAR sections, while the other tackled the REG and AUD sections. ChatGPT's scores fell significantly below the passing threshold in each section, indicating a clear failure to meet CPA exam requirements.
This might sound like a surprise, considering half of us have heard that ChatGPT is coming for our jobs. Open AI has had its share of wins in other exams, like the Wharton MBA and Bar exams.
But the truth is, while AI has the potential to do a lot of different things, it's still just a tool with its own limits.
Here are some of them:
1. ChatGPT can't apply accounting principles
One major limit for ChatGPT in the CPA exam was its struggle with comprehending intricate accounting concepts.
The CPA tests for a candidate's knowledge of Generally Accepted Accounting Principles (GAAP) and Generally Accepted Auditing Standards (GAAS).
And apparently, ChatGPT struggled to consistently apply these standards — so its responses were inconsistent and often wrong.
2. ChatGPT doesn't have critical judgment
ChatGPT also fell short in its ability to apply critical judgment.
The CPA exam assesses candidates' ability to make sound decisions based on complex scenarios. ChatGPT's responses often lacked nuance and couldn't critically access the implications of different accounting scenarios.
3. ChatGPT can't thoroughly analyze complex data
According to Accounting Today: AI, including ChatGPT, currently falls short in its capacity to exhibit the level of critical analysis and judgment required for the CPA role.
Basically, it can't evaluate intricate financial data, identify potential risks and make sound decisions that align with professional standards.
The CPA certification isn't something easily earned. It validates your technical knowledge, ethical integrity and commitment to accounting standards and regulations. And it takes years of school and weeks of study to get it right.
Right now, a language model just isn't gonna cut it.
What ChatGPT can't do — and why it won't take your job
AI won't be replacing CPAs any time soon. Here's a little more of why:
1. A whole lot of inaccuracies
Like in the CPA exam, tools like ChatGPT produce a lot of errors — especially if the query is too complicated or ambiguous.
Even with future iterations like GPT-4, we can't count on 100 percent accuracy.
An experienced accountant said: "The reason I'm not worried for now is that when GPT-4 is wrong, it is confidently wrong, which makes it imperative for qualified CPAs to review the work with actual knowledge on the subject."
2. No real accountability
AI tools obviously lack personal accountability... so who is responsible when errors occur? In case of incorrect financial advice or decisions, you can't go without having someone to hold accountable.
Without a real person involved, we can't trace or resolve mistakes — leading to potential financial and reputational consequences for your firm.
3. Regulatory concerns
Like we mentioned before, ChatGPT struggled to apply the GAAP and GAAS standards on the exam.
Not only should the tool adhere to these regulations, it also needs to keep up with their many changes.
Plus, data privacy and security are a huge concern when it comes to AI platforms.
It's your priority to protect client confidentiality and comply with data protection laws — and its no different when it comes to using these tools. It's a serious issue, and there are many ongoing lawsuits involving AI tools like ChatGPT.
{{banner-short-v2="/banner-ads"}}
What AI can do — and why you should try it anyway
Now, before you swear off AI entirely, let's think for a second.
Yes, ChatGPT failed the exam and showcased its limitations. Mainly, it can't distill complicated information like a CPA needs to.
But guess what, that's your job.
By combining AI tools with your education and skill set, you can make yourself and your firm a powerful force to be reckoned with.
If you partner with it, there's a lot of potential for AI in accounting.
It can...
1. Automate routine tasks
ChatGPT is great at automating routine tasks—think data entry or invoice processing.
By using AI, your firm can free up time and resources from mundane admin work to focus on the tasks that actually matter.
For example, Scribe is an AI-powered tool that documents accounting processes for you. You can turn even the most complicated workflow into a visual step-by-step guide in seconds.
2. Build & templatize basic reports
ChatGPT can also assist in generating basic financial reports.
Its language processing abilities let it compile and summarize financial data — so you don't have to. Just review and analyze the information for further insights and decision-making.
3. Support data analysis
Emphasis on support here, but ChatGPT can process large datasets and help identify patterns. While you should never let it replace your expertise, it can create documentation fast and help you better understand the data you're working with.
How to get the best of both worlds with AI
Financial decision-making will always need human judgment, expertise and relationships. You need a qualified CPA to review and approve any AI tool's outputs.
Leverage AI tools to make your life easier by letting it do the redundant, routine and baseline tasks that take up all of your time.
First and foremost, you're the expert on your job. Let AI help you move faster and more efficiently.
Tools Scribe's free accounting document generator can handle the busy work — so you can do the work that matters.
“Such a great product for creating SOPs. I’m winning back multiple hours a week, and the guides look so much better than our previous SOP template."— Josie K.,Scribe Reviewer.
