Please%20read%20the%20text%20carefully.%20Identify%20the%20main%20topic%20and%20key%20points%20covered%20in%20the%20text.%20%0AWrite%20a%20concise%20summary%20of%20the%20text,%20focusing%20solely%20on%20the%20main%20topic%20and%20key%20points%20you%20identified.%20Your%20output%20should%20contain%20only%20the%20summary%20text%20itself.%20Do%20not%20include%20any%20explanations,%20introduction,%20or%20other%20extraneous%20text.%20The%20summary%20should%20be%20able%20to%20stand%20alone.%0AHere%20is%20the%20text%20to%20summarize:%0A%3Ctext%3E%0ASkip%20to%20content%0A%0ATHE%20DECODER%0AArtificial%20Intelligence:%20News,%20Business,%20Research%0A%0A%0AGradient%0AColor%20scheme%0AColor%20scheme%0A%0AAI%20in%20practice%0ANov%2010,%202024%0A%0AMatthias%20Bastian%0AOpenAI's%20new%20%22Orion%22%20model%20reportedly%20shows%20small%20gains%20over%20GPT-4%0AMidjourney%20prompted%20by%20THE%20DECODER%0A%0A%0A%0A%0A%0AContent%0A%0A%0ASummary%0ALanguage%20model%20development%20seems%20to%20have%20hit%20a%20ceiling.%20OpenAI's%20upcoming%20Orion%20model%20barely%20outperforms%20GPT-4,%20its%20predecessor,%20according%20to%20a%20new%20report.%20This%20slowdown%20affects%20the%20entire%20AI%20industry.%0AAd%0A%0AThe%20Information%20reports%20that%20OpenAI's%20next%20major%20language%20model,%20codenamed%20%22Orion,%22%20delivers%20much%20smaller%20performance%20gains%20than%20expected.%20The%20quality%20improvement%20between%20GPT-4%20and%20Orion%20is%20notably%20less%20significant%20than%20what%20we%20saw%20between%20GPT-3%20and%20GPT-4.%0AWhat's%20more,%20Orion%20doesn't%20consistently%20beat%20its%20predecessor%20in%20areas%20like%20programming,%20showing%20improvements%20only%20in%20language%20capabilities,%20according%20to%20The%20Information's%20sources.%20The%20model%20could%20also%20cost%20more%20to%20run%20in%20data%20centers%20than%20previous%20versions.%0ARunning%20out%20of%20training%20material%0AOpenAI%20researchers%20point%20to%20insufficient%20high-quality%20training%20data%20as%20one%20reason%20for%20the%20slowdown.%20Most%20publicly%20available%20texts%20and%20data%20have%20already%20been%20used.%20In%20response,%20OpenAI%20created%20a%20%22Foundations%20Team%22%20led%20by%20Nick%20Ryder,%20The%20Information%20reports.%0AAd%0A%0AThis%20aligns%20with%20CEO%20Sam%20Altman's%20statement%20in%20June%20that%20while%20data%20exists%20in%20sufficient%20quantities,%20the%20focus%20will%20shift%20to%20learning%20more%20from%20less%20data.%20The%20company%20plans%20to%20use%20synthetic%20data%E2%80%94training%20material%20generated%20by%20AI%20models%E2%80%94to%20help%20bridge%20this%20gap.%0AThe%20Information%20notes%20that%20Orion%20has%20already%20trained%20partially%20on%20synthetic%20data%20from%20GPT-4%20and%20OpenAI's%20new%20%22reasoning%22%20model%20o1.%20However,%20this%20approach%20runs%20the%20risk%20of%20the%20new%20model%20simply%20%22resembling%20those%20older%20models%20in%20certain%20aspects,%22%20according%20to%20an%20OpenAI%20employee.%0ALLM%20stagnation%20poses%20a%20challenge%20for%20the%20industry%0AThe%20slowdown%20in%20LLM%20progress%20goes%20beyond%20OpenAI.%20The%20Verge%20recently%20reported%20that%20Google's%20upcoming%20Gemini%202.0%20is%20falling%20short%20of%20internal%20targets.%20Anthropic%20is%20rumored%20to%20have%20halted%20development%20on%20version%203.5%20of%20its%20flagship%20Opus,%20releasing%20an%20improved%20Sonnet%20instead%20-%20possibly%20to%20avoid%20disappointing%20users%20and%20investors.%0AOpen-source%20models%20catching%20up%20to%20billion-dollar%20proprietary%20ones%20over%20the%20past%2018%20months%20further%20shows%20this%20industry-wide%20plateau.%20This%20progress%20would%20be%20unlikely%20if%20major%20tech%20companies%20could%20effectively%20convert%20their%20massive%20investments%20into%20better%20AI%20performance.%0A%0ADifferent%20language%20models%20now%20perform%20similarly.%20While%20earlier%20versions%20showed%20clear%20performance%20gaps,%20MMLU%20scores%20have%20converged%20since%202023,%20suggesting%20a%20temporary%20performance%20ceiling.%20%7C%20Image:%20Maxime%20Labonne%20via%20X%0ABut%20in%20a%20recent%20interview,%20OpenAI%20CEO%20Sam%20Altman%20remained%20optimistic.%20He%20said%20that%20the%20path%20to%20artificial%20general%20intelligence%20(AGI)%20is%20clear,%20and%20that%20what%20is%20needed%20is%20a%20creative%20use%20of%20existing%20models.%20Altman%20could%20be%20referring%20to%20the%20combination%20of%20LLMs%20with%20reasoning%20approaches%20such%20as%20o1%20and%20agentic%20AI.%0ANoam%20Brown,%20a%20prominent%20AI%20developer%20at%20OpenAI%20and%20former%20Meta%20employee%20who%20helped%20create%20o1,%20says%20Altman's%20statement%20reflects%20the%20views%20of%20most%20OpenAI%20researchers.%0AThe%20new%20o1%20model%20aims%20to%20create%20fresh%20scaling%20opportunities.%20It%20shifts%20focus%20from%20training%20to%20inference%E2%80%94the%20computing%20time%20AI%20models%20have%20to%20complete%20tasks.%20Brown%20believes%20this%20approach%20is%20a%20%22new%20dimension%20for%20scaling.%22%0ABut%20it%20will%20require%20billions%20of%20dollars%20and%20significant%20energy%20use.%20This%20is%20a%20key%20industry%20question%20for%20the%20coming%20months:%20Does%20building%20ever-more-powerful%20AI%20models%E2%80%94and%20the%20massive%20data%20centers%20they%20need%E2%80%94make%20economic%20and%20environmental%20sense%3F%20OpenAI%20seems%20to%20think%20so.%0AGemini-LLM%20in%20AlphaProof%20was%20%22basically%20cosmetic%22%0AGoogle%20AI%20expert%20Fran%C3%A7ois%20Chollet%20criticized%20scaling%20language%20models%20for%20mathematical%20tasks.%20He%20called%20it%20%22especially%20obtuse%22%20to%20cite%20progress%20in%20mathematical%20benchmarks%20as%20AGI%20proof.%0AAd%0A%0AChollet%20argues%20empirical%20data%20shows%20deep%20learning%20and%20large%20language%20models%20can't%20solve%20math%20problems%20independently.%20Instead,%20they%20need%20discrete%20search%20methods%E2%80%94systematic%20approaches%20that%20check%20various%20solution%20paths%20rather%20than%20predicting%20likely%20answers%20like%20language%20models%20do.%0AHe%20also%20criticized%20using%20%22LLM%22%20as%20a%20marketing%20term%20for%20all%20current%20AI%20advances,%20even%20when%20unrelated%20to%20language%20models.%20He%20pointed%20to%20Gemini's%20integration%20into%20Google%20Deepmind's%20AlphaProof%20as%20%22basically%20cosmetic%20and%20for%20marketing%20purposes.%22%0A%0AShare%0A%0ASupport%20our%20independent,%20free-access%20reporting.%20Any%20contribution%20helps%20and%20secures%20our%20future.%20Support%20now:%0A%0A%0ABank%20transfer%0A%0ASummary%0A%09%E2%80%A2%09OpenAI's%20next%20major%20language%20model,%20%22Orion%22,%20is%20reported%20to%20have%20lower%20than%20expected%20performance%20improvements%20over%20its%20predecessor,%20GPT-4.%20In%20some%20areas,%20such%20as%20programming,%20it%20is%20not%20consistently%20better.%20One%20reason%20for%20this%20is%20the%20lack%20of%20high-quality%20training%20data.%0A%09%E2%80%A2%09The%20stagnation%20in%20the%20development%20of%20large-scale%20language%20models%20affects%20the%20entire%20AI%20industry.%20Google's%20Gemini%202.0%20is%20also%20said%20to%20fall%20short%20of%20expectations.%20Anthropic%20has%20yet%20to%20release%20its%20long-awaited%20Opus%203.5.%0A%09%E2%80%A2%09Despite%20the%20challenges,%20OpenAI%20CEO%20Sam%20Altman%20remains%20confident.%20The%20new%20North%20Star%20could%20be%20OpenAI's%20o1%20model,%20which%20is%20expected%20to%20open%20a%20new%20scaling%20horizon%20by%20scaling%20computing%20power%20for%20inference%20instead%20of%20just%20training.%0ASources%0AThe%20Information%0A%0AMatthias%20Bastian%0A%0A%0AOnline%20journalist%20Matthias%20is%20the%20co-founder%20and%20publisher%20of%20THE%20DECODER.%20He%20believes%20that%20artificial%20intelligence%20will%20fundamentally%20change%20the%20relationship%20between%20humans%20and%20computers.%0AProfile%0AE-Mail%0A%0A%0AAI%20in%20practice%0ANov%2010,%202024%0ATeachers%20discuss%20smart%20ways%20to%20catch%20AI-generated%20homework%0A%0A%0A%0A%0AAI%20in%20practice%0ANov%209,%202024%0AChatGPT%20hits%203.7%20billion%20visits%20in%20October%20as%20growth%20accelerates%20to%20115%25%20year-over-year%0A%0A%0A%0A%0AAI%20in%20practice%0ANov%209,%202024%0AGoogle%20adds%20Vids%20AI%20video%20creation%20tool%20to%20Workspace%20suite%0A%0A%0A%0ALoad%20more%0A%0AGoogle%20News%0A%0AJoin%20our%20community%0AJoin%20the%20DECODER%20community%20on%20Discord,%20Reddit%20or%20Twitter%20-%20we%20can't%20wait%20to%20meet%20you.%0A%0A%0A%0A%0A%0A%0A%09%E2%80%A2%09The%20Decoder%0A%09%E2%80%A2%09About%0A%09%E2%80%A2%09Advertise%0A%09%E2%80%A2%09Publish%20with%20us%0A%09%E2%80%A2%09Topics%0A%09%E2%80%A2%09AI%20research%0A%09%E2%80%A2%09AI%20in%20practice%0A%09%E2%80%A2%09AI%20and%20society%0A%0ATHE%20DECODER%20Newsletter%0AThe%20most%20important%20AI%20news%20straight%20to%20your%20inbox.%0A%E2%9C%93%20Weekly%0A%E2%9C%93%20Free%0A%E2%9C%93%20Cancel%20at%20any%20time%0AE-Mail%20*%0A%0A%0A%0ATo%20top%0APrivacy%20Policy%20%7C%20Privacy%20Manager%20%7C%20Legal%20Notice%0A%0AThe%20Decoder%20by%20DEEP%20CONTENT%20%7C%20ALL%20RIGHTS%20RESERVED%202024%0A%0A%0A%3C/text%3E