By OpenAI's own testing,NSPS-276 Hot JAV Porn Movie its newest reasoning models, o3 and o4-mini, hallucinate significantly higher than o1.
First reported by TechCrunch, OpenAI's system card detailed the PersonQA evaluation results, designed to test for hallucinations. From the results of this evaluation, o3's hallucination rate is 33 percent, and o4-mini's hallucination rate is 48 percent — almost half of the time. By comparison, o1's hallucination rate is 16 percent, meaning o3 hallucinated about twice as often.
SEE ALSO: All the AI news of the week: ChatGPT debuts o3 and o4-mini, Gemini talks to dolphinsThe system card noted how o3 "tends to make more claims overall, leading to more accurate claims as well as more inaccurate/hallucinated claims." But OpenAI doesn't know the underlying cause, simply saying, "More research is needed to understand the cause of this result."
OpenAI's reasoning models are billed as more accurate than its non-reasoning models like GPT-4o and GPT-4.5 because they use more computation to "spend more time thinking before they respond," as described in the o1 announcement. Rather than largely relying on stochastic methods to provide an answer, the o-series models are trained to "refine their thinking process, try different strategies, and recognize their mistakes."
However, the system card for GPT-4.5, which was released in February, shows a 19 percent hallucination rate on the PersonQA evaluation. The same card also compares it to GPT-4o, which had a 30 percent hallucination rate.
In a statement to Mashable, an OpenAI spokesperson said, “Addressing hallucinations across all our models is an ongoing area of research, and we’re continually working to improve their accuracy and reliability.”
Evaluation benchmarks are tricky. They can be subjective, especially if developed in-house, and research has found flaws in their datasets and even how they evaluate models.
Plus, some rely on different benchmarks and methods to test accuracy and hallucinations. HuggingFace's hallucination benchmark evaluates models on the "occurrence of hallucinations in generated summaries" from around 1,000 public documents and found much lower hallucination rates across the board for major models on the market than OpenAI's evaluations. GPT-4o scored 1.5 percent, GPT-4.5 preview 1.2 percent, and o3-mini-high with reasoning scored 0.8 percent. It's worth noting o3 and o4-mini weren't included in the current leaderboard.
That's all to say; even industry standard benchmarks make it difficult to assess hallucination rates.
Then there's the added complexity that models tend to be more accurate when tapping into web search to source their answers. But in order to use ChatGPT search, OpenAI shares data with third-party search providers, and Enterprise customers using OpenAI models internally might not be willing to expose their prompts to that.
Regardless, if OpenAI is saying their brand-new o3 and o4-mini models hallucinate higher than their non-reasoning models, that might be a problem for its users.
UPDATE: Apr. 21, 2025, 1:16 p.m. EDT This story has been updated with a statement from OpenAI.
Topics ChatGPT OpenAI
Ryzen 5 1600X vs. 1600: Which should you buy?
Introducing the Winners of the 2022 Whiting Awards by The Paris Review
‘Palworld’ by the numbers — and they're big numbers
Amazon Echo Show 8 deal: Save 40%
Today's Hurdle hints and answers for May 9, 2025
Claire Schwartz, Poetry by Claire Schwartz
Announcing the Winners of 92Y's 2022 Discovery Contest by The Paris Review
Sam's Club membership discount
No Time for a Negative Peace
A new Pope has been chosen. Here's what his X posts say about him.
Dyson V8 Plus cordless vacuum: $120 off at Amazon
Redux: Even a Fact Is Not a Fact by The Paris Review
The Family Is Finished: On Memory, Betrayal, and Home Decor by Menachem Kaiser
Megha Majumdar, Fiction by Megha Majumdar
A worthless juicer and a Gipper-branded server
The Sixties Diaries by Ted Berrigan
TikTok users can't log in through Twitter / X
Stealing It Back: A Conversation with Frida Orupabo by Maya Binyam
Virtual Reality: The True Cost of Admission (and Why It Doesn't Matter)
Redux: Like No One Else by The Paris Review
How to delete your Clubhouse accountVR has finally found its place: porn'To All the Boys: Always and Forever' is the perfect farewell to Lara JeanPSA: Instagram polls are NOT anonymous32 of the biggest dating app bio red flags, as told by usersJackie Chan's daughter comes out on Instagram"We live in a society": Explaining the edgelord meme in Justice League'To All the Boys: P.S. I Still Love You' was cute, but we must talk about the ending'Clarice' shows how deep 'The Silence of the Lambs' shadow goesSingle but never tried online dating? You're not alone.Border agents can search phones without a warrant, court rulesHow to spot signs of alien lifeRussia used 'blacktivist' social media to meddle in electionHeartbreaking 'New Yorker' cover honors victims of Las Vegas shootingBus driver gets so into his jam, he gets up and dances while speeding down the roadPrecious little superfan gets a unique Batman doll made just for herWatch Naomi Osaka's delightful butterfly moment, midA totally bizarre interview just started a feud between Ivana and Melania Trump5 changes Google Maps should make for better driving directionsHow to delete your Clubhouse account Scientists rushed to save lab specimens as PG&E cuts power Nobel Prize in Physics awarded to scientists, some rally behind one who never got one Here's a Halloween playlist for all your spooky music needs In weird ad, Kellyanne Conway tells people to buy Ivanka's line of clothes California’s climate dystopia comes true with PG&E power blackouts 'Nancy Drew' is a pile of mediocrity with one chance at redemption (Review) Everything we expect to see at Google's Pixel 4 launch event Why you should lie in your password Trump starring in weird ads for socks and pizza? Feels like a long time ago. VSCO girls: Shield your eyes while this hydraulic press crushes a Hydro Flask New Samsung Galaxy Fold teardown reveals a much Apple now sells a smart bike helmet with an LED turn signal Police say abandoned SD card labeled 'homicide' led to arrest of murder suspect Russian internet trolls focused on black Americans, Senate confirms Of course Amazon workers are looking at Cloud Cam footage Social media campaign aims to highlight persecution of LGBTQ people Facebook co Trump's 'Easy D' tweet took the internet straight to penis jokes Twitter has a grand old time with Trump's unfinished sentence George and Amal Clooney are expecting twins so that's something nice
2.306s , 8223.6640625 kb
Copyright © 2025 Powered by 【NSPS-276 Hot JAV Porn Movie】,Defense Information Network