In a groundbreaking study carried out by the University of Montana and its partners, it has been revealed that ChatGPT, a well-known artificial intelligence tool, can outperform 99% of human thinkers when evaluated through a standard creativity test.
Dr. Erik Guzik, an assistant clinical professor at UM's College of Business, led the research. For this, he and his team employed the Torrance Tests of Creative Thinking (TTCT), a widely recognized tool that has been utilized for several years to measure how creative people are.
The TTCT asks the test participants a series of thought-provoking questions carefully crafted to assess their level of creativity.
“Suppose it’s a basketball,” Dr. Guzik provides an example of the kind of prompt the TTCT might present to a test subject. “Try to think of as many different ways as you can to use a basketball.”
Some common answers to this question could include throwing the basketball into a hoop or playing catch with your buddies.
Guzik further mentioned that if individuals challenge themselves to think creatively and explore new possibilities, they might consider cutting up the basketball and repurposing it as a planter. Similarly, in the case of bricks, one could utilize them for construction purposes or employ them as functional paperweights. He also proposed the concept of grinding them into fine particles and reshaping them into something entirely new.
Guzik anticipated that ChatGPT would excel in generating ideas for a fluency assessment, as that is a primary function of generative AI. True to his expectations, ChatGPT demonstrated exceptional abilities in providing responses to prompts that were deemed relevant, beneficial, and worthwhile by the evaluators.
Following the TTCT test on ChatGPT, which involved a set of eight questions specifically designed to elicit innovative solutions, Guzik conducted the same test on 24 of his entrepreneurship and finance students. Additionally, the researchers gathered substantial data by collecting 2,700 responses from students who participated in a nationwide test in 2016. This extensive comparison data set significantly enhanced the scope of their analysis.
Subsequently, Guzik and his team submitted the answers to the Scholastic Testing Service (STS) for grading. An important detail to note is that the STS evaluators were unaware that AI was being used in the test and regarded all participants, including ChatGPT, as if they were humans. Furthermore, the researchers emphasize that the TTCT, being a proprietary material, is safeguarded, ensuring that ChatGPT couldn't search the internet for creative responses to its questions, eliminating any possibility of cheating.
As per the results, ChatGPT showcased an exceptional performance in the fluency category, which determines one’s capacity to produce a significant quantity of unique concepts, placing it in the top percentile. However, in terms of flexibility, which evaluates the capability to devise ideas across diverse types and categories, the AI slightly dropped to the 97th percentile.
In light of these findings, Dr. Erik Guzik expressed their achievement in demonstrating the outstanding performance of ChatGPT and GPT-4 for the first time in the top 1% for originality, which was a remarkable revelation.
Additionally, he was glad to observe that a few of his UM students had also scored in the top 1%. However, ChatGPT fared much better than most college students nationwide, showcasing its superior performance.
Read next: AI is Learning From Other AI, Here’s How That Could Ruin the Industry
Dr. Erik Guzik, an assistant clinical professor at UM's College of Business, led the research. For this, he and his team employed the Torrance Tests of Creative Thinking (TTCT), a widely recognized tool that has been utilized for several years to measure how creative people are.
The TTCT asks the test participants a series of thought-provoking questions carefully crafted to assess their level of creativity.
“Suppose it’s a basketball,” Dr. Guzik provides an example of the kind of prompt the TTCT might present to a test subject. “Try to think of as many different ways as you can to use a basketball.”
Some common answers to this question could include throwing the basketball into a hoop or playing catch with your buddies.
Guzik further mentioned that if individuals challenge themselves to think creatively and explore new possibilities, they might consider cutting up the basketball and repurposing it as a planter. Similarly, in the case of bricks, one could utilize them for construction purposes or employ them as functional paperweights. He also proposed the concept of grinding them into fine particles and reshaping them into something entirely new.
Guzik anticipated that ChatGPT would excel in generating ideas for a fluency assessment, as that is a primary function of generative AI. True to his expectations, ChatGPT demonstrated exceptional abilities in providing responses to prompts that were deemed relevant, beneficial, and worthwhile by the evaluators.
Following the TTCT test on ChatGPT, which involved a set of eight questions specifically designed to elicit innovative solutions, Guzik conducted the same test on 24 of his entrepreneurship and finance students. Additionally, the researchers gathered substantial data by collecting 2,700 responses from students who participated in a nationwide test in 2016. This extensive comparison data set significantly enhanced the scope of their analysis.
Subsequently, Guzik and his team submitted the answers to the Scholastic Testing Service (STS) for grading. An important detail to note is that the STS evaluators were unaware that AI was being used in the test and regarded all participants, including ChatGPT, as if they were humans. Furthermore, the researchers emphasize that the TTCT, being a proprietary material, is safeguarded, ensuring that ChatGPT couldn't search the internet for creative responses to its questions, eliminating any possibility of cheating.
As per the results, ChatGPT showcased an exceptional performance in the fluency category, which determines one’s capacity to produce a significant quantity of unique concepts, placing it in the top percentile. However, in terms of flexibility, which evaluates the capability to devise ideas across diverse types and categories, the AI slightly dropped to the 97th percentile.
In light of these findings, Dr. Erik Guzik expressed their achievement in demonstrating the outstanding performance of ChatGPT and GPT-4 for the first time in the top 1% for originality, which was a remarkable revelation.
Additionally, he was glad to observe that a few of his UM students had also scored in the top 1%. However, ChatGPT fared much better than most college students nationwide, showcasing its superior performance.
Read next: AI is Learning From Other AI, Here’s How That Could Ruin the Industry
