糖心在线

• A three-part classroom exercise empowers students to evaluate generative AI’s capabilities through real-world finance scenarios that connect academic content to practical career applications.
• By interacting with AI as both clients and professionals, students develop essential analytical and evaluative skills and learn to view AI as a tool, not as a replacement for human expertise.
• The exercise equips students with a nuanced understanding of generative AI’s strengths and limitations and shows them how emerging technologies are transforming the business landscape.

 
Preparing students for success in business means bridging the gap between classroom concepts and real-world applications. As generative artificial intelligence (GenAI) increasingly influences business and finance careers, it is essential for students to explore its impact firsthand. In my classroom, I have developed a  that engages students in an exploration of how large language models (LLMs) such as ChatGPT can perform tasks relevant to business roles.

As students evaluate AI-generated insights, they refine their understanding of key concepts, identify GenAI’s strengths and limitations, and develop practical skills for their business careers. Additionally, this activity fosters critical thinking, enhances analytical skills, and prepares students to adapt to emerging technologies that are reshaping the industry.

Below, I share the structure and outcomes from this exercise. I also share recent academic research on LLMs in which I and a co-author further explain what these outcomes might mean for AI’s use in the workplace.

An Overview of the Teaching Approach

To provide a foundation for the exercise, I select a job relevant to the course material and discuss how GenAI might perform tasks required in that role. For example, in a personal finance class that I teach, I discuss how individuals might seek financial advice from ChatGPT instead of from a professional financial advisor. I point out that if ChatGPT can provide accurate financial advice, demand for this advice might be reduced, impacting future job prospects for students.

Then, I ask students to interact with ChatGPT through an activity divided into three stages: the Skills Test, the Replacement Test, and the Complement Test. Each stage requires students to prompt ChatGPT as they take on different roles (as either clients or financial advisors) and critically evaluate its responses, reinforcing their knowledge of class concepts while simultaneously exploring AI’s potential and limitations. 

Summary of Tests and Roles

Skills Test: Can AI Answer Basic Questions?

For this part of the exercise, students assess ChatGPT’s ability to respond to finance-related questions that might be asked of a financial advisor. Example prompts include:

• How much money should a 30-year-old invest in stocks versus bonds?
• What are the main differences between active and passive investment strategies?
• Is it better financially to defer money into a 401(k) or invest post-tax in a Roth IRA?

Students rate ChatGPT’s responses on a three-point scale: Excellent, Good, or Poor. This test reinforces classroom concepts by requiring students to analyze AI-generated financial advice critically. Additionally, students reflect on how well AI applies theoretical knowledge to practical scenarios, and they uncover potential inaccuracies or biases in AI-generated answers.

Students recognize that while ChatGPT provides detailed and generally accurate responses, it may lack the nuanced understanding or customization required for real-world financial decision-making.

When students are in the role of evaluator, they are forced to think critically about the responses the AI generates. As Ethan Mollick and Lilach Mollick note in a  in Harvard Business Impact’s Inspiring Minds, this thinking process increases the cognitive burden on students and improves their understanding of a topic.

Through the Skills Test, students recognize that while ChatGPT provides detailed and generally accurate responses, it may lack the nuanced understanding or customization required for real-world financial decision-making. This realization helps students appreciate the ongoing necessity of human judgment in finance-related roles, emphasizing that AI should be viewed as a tool, rather than a replacement for expertise.

Replacement Test: Can AI Perform This Job?

Here, students act as clients seeking financial advice, and ChatGPT is prompted to act as a financial advisor. Students enter a starter prompt that provides ChatGPT with the (fictitious) client’s financial details, such as age, marital status, number of children, home ownership status, amount of debt, and retirement account balances.

They are instructed to interact conversationally with ChatGPT, asking for financial advice based on the details they provided. After asking follow-up questions and interacting in an unstructured conversation, students assess whether ChatGPT can replace a human professional.

Students separately analyze the accuracy of the responses (whether the content was correct and suitable for the client) and the similarity of the language to that a human professional might use (whether the phrasing was presented in an understandable manner). Students generally rate the overall quality of ChatGPT’s responses in the Replacement Test as lower than in the Skills Test. 

Interestingly, students score ChatGPT much higher for the similarity of its responses to human-written advice than for its accuracy. This divergence suggests that the models do well in phrasing responses as a professional would, but the accuracy is still lower than a typical human professional would offer.

This step also allows students to reflect on the importance of human judgment, the value of personalized financial planning, and the potential ethical considerations of AI’s use in finance. A natural extension of this teaching exercise is to explore real-world cases and consider how AI-powered financial tools, such as robo-advisors, might perform relative to human professionals. This analysis can give students insight into the broader industry’s shift toward automation and the challenges involved in relying on AI for financial decision-making.

Complement Test: Can AI Assist Professionals?

Finally, students take on the role of a financial advisor and use ChatGPT as an assistant. They describe a class-related assignment to ChatGPT—creating a financial plan for a fictitious client—and seek the technology’s suggestions for improvement. They then benchmark ChatGPT’s feedback against that received from peers and the professor.

Students analyze whether AI provides actionable insights or simply reiterates common knowledge. Additionally, they consider how AI tools can streamline financial analysis, improve decision-making efficiency, and support professionals rather than replace them.

One compelling takeaway from this test is that ChatGPT can help generate alternative solutions or new perspectives, especially when students feel stuck in their analyses. For instance, students working on a portfolio diversification strategy might receive AI-generated suggestions that include asset classes they had not previously considered. While students still need to validate and refine these recommendations, AI serves as a brainstorming partner, fostering creativity and expanding analytical possibilities.

Research and Takeaways

I have now conducted this exercise with two cohorts of students, in 2024 and 2025. In that time, I have seen students’ ratings of ChatGPT’s abilities improve in all three tests. The most noticeable improvement has occurred in the Complement Test. When asked to evaluate the quality of ChatGPT’s suggestions to improve their plans, 41 percent of students in the 2025 cohort rated ChatGPT as “excellent” compared to only 22 percent of students in the 2024 cohort.

These high-level findings square with  on LLMs in finance, co-authored with D.J. Fairhurst and published in Financial Analysts Journal. In that paper, we discuss our process of prompting ChatGPT and other LLMs with more than 10,000 questions tied to finance licensing exams that are required for many jobs in the securities industry. We find that ChatGPT is proficient at answering broad-based or general questions such as those dealing with financial market trends.

However, we also find an increase in inaccurate responses for questions linked to more specialized topics, such as insurance and tax. Our research concludes that the appropriate use of LLMs is task-specific, not job-specific.

The exercise deepens students’ understanding of finance concepts, enhances their ability to evaluate and synthesize information critically, and makes them more comfortable using AI tools.

A practical takeaway from both the in-class exercise and our research is that LLMs are best deployed as helpful assistants whose output complements the work of professionals in business settings.

Materials, Learning Outcomes, Adaptability

From a professor’s point of view, presenting this exercise in the classroom is relatively straightforward. ChatGPT is accessible to anyone with a device that has internet access. There is no cost for a basic ChatGPT account, and students without an account can access it without logging in. While faculty commonly use ChatGPT for such assignments, alternatives such as Google’s Gemini are also available. Instructors can prepare suggested prompts to guide interactions.

In my class, I have found that the exercise provides three educational benefits. First, it deepens students’ understanding of finance concepts by requiring them to analyze AI-generated responses. Second, these interactions with AI enhance their ability to evaluate and synthesize information critically.

Finally, this exercise helps all students become more comfortable using AI tools, whether they are familiar with LLMs or have never interacted with these tools before. Students develop greater AI literacy, including the ability to:  

• Recognize AI’s capabilities and limitations in financial contexts.
• Use structured prompting techniques (such as , , and ).
• Evaluate financial advice and industry applications of AI.

This teaching exercise is highly adaptable. While this example focuses on the field of finance, instructors can modify it to fit other disciplines where AI is increasingly used to generate reports, analyze data, and assist in decision-making such as accounting, marketing, and management. 

Essential Skills for the Future

As my students engaged with AI, they obtained a deeper understanding of how the technology is transforming the finance profession. After completing this assignment, students expressed greater confidence in using AI. They also recognized its career relevance, noting that they expected that employers would demonstrate increasing interest in deploying AI tools.

By incorporating structured AI interactions into their courses, business faculty in all disciplines can help students bridge the gap between classroom learning and professional application. Through hands-on engagement with AI, students will be better prepared for the evolving landscape of AI, and they will develop essential skills that will enable them to adapt and thrive in their future careers.