Shopping cart

Texas News App is your reliable source for real-time updates across Texas, covering Local News, Politics, Business, Sports, and more. With a focus on all 15 Texas regions, we bring the stories that matter most to communities statewide. Stay informed and connected with an app designed to reach Texans wherever they are.

TnewsTnews
Business & Tech

AI leaders are starting to rethink the best way to advance AI

AI leaders are starting to rethink the best way to advance AI
Email :13
AI conversation bubbles
  • AI leaders are rethinking data-heavy training for large language models.
  • Traditional models scale linearly with data, but this approach may hit a dead end.
  • Smaller, more efficient models and new training methods are gaining industry support.

For years, tech companies like OpenAI, Meta, and Google have focused on amassing tons of data, assuming that more training material would lead to smarter, more powerful models.

Now, AI leaders are rethinking the conventional wisdom about how to train large language models.

The focus on training data arises from research showing that transformers, the neural networks behind large language models, have a one-to-one relationship with the amount of data they’re given. Transformer models “scale quite linearly with the amount of data and compute they’re given,” Alex Voica, a consultant at the Mohamed bin Zayed University of Artificial Intelligence, previously told Business Insider.

However, executives are starting to worry that this approach can only go so far, and they’re exploring alternatives for advancing the technology.

The money going into AI has largely hung on the idea that this scaling law “would hold,” Scale AI CEO Alexandr Wang said at the Cerebral Valley conference this week, tech newsletter Command Line reported. It’s now “the biggest question in the industry.”

Some executives say the problem with the approach is that it’s a little mindless. “It’s definitely true that if you throw more compute at the model, if you make the model bigger, it’ll get better,” Aidan Gomez, the CEO of Cohere, said on the 20VC podcast. “It’s kind of like it’s the most trustworthy way to improve models. It’s also the dumbest.”

Gomez advocates smaller, more efficient models, which are gaining industry support for being cost-effective.

Others worry this approach won’t reach artificial general intelligence — a theoretical form of AI that matches or surpasses human intelligence — even though many of the world’s largest AI companies are banking on it.

Large language models are trained simply to “predict the next token, given the previous set of tokens,” Richard Socher, a former Salesforce executive and CEO of AI-powered search engine You.com, told Business Insider. The more effective way to train them is to “force” these models to translate questions into computer code and generate an answer based on the output of that code, he said. This will reduce hallucinations in quantitative questions and enhance their abilities.

Not all industry leaders are sold that AI has hit a scaling wall, however.

“Despite what other people think, we’re not at diminishing marginal returns on scale-up,” Microsoft chief technology officer Kevin Scott said in July in an interview with Sequoia Capital’s Training Data podcast.

Companies like OpenAI are also seeking to improve on existing LLMs.

OpenAI’s o1, released in September, still relies on the token prediction mechanism Socher refers to. Still, the model is specialized to better handle quantitative questions, including areas like coding and mathematics — compared to ChatGPT, which is considered a more general-purpose model.

Part of the difference between o1 and ChatGPT is that o1 spends more time on inference or “thinking” before it answers a question.

“To summarize, if we were to anthropomorphize, gpt-4 is like your super know-it-all friend who when you ask them a question starts talking stream-of-consciousness, forcing you to sift through what they’re saying for the gems,” Waleed Kadous, a former engineer lead at Uber and former Google principal software engineer, wrote in a blog post. “o1 is more like the friend who listens carefully to what you have to say, scratches their chin for a few moments, and then shares a couple of sentences that hit the nail on the head.”

One of o1’s trade-offs, however, is that it requires much more computational power, making it slower and costlier, according to Artificial Analysis, an independent AI benchmarking website.

Read the original article on Business Insider



This article was originally published by Lakshmi Varanasi,Lloyd Lee at All Content from Business Insider – Read this article and more at (https://www.businessinsider.com/ai-leaders-are-starting-to-rethink-way-to-advance-ai-2024-11).

General Content Disclaimer



The content on this website, including articles generated by artificial intelligence or syndicated from third-party sources, is provided for informational purposes only. We do not own the rights to all images and have not independently verified the accuracy of all information presented. Opinions expressed are those of the original authors and do not necessarily reflect our views. Reader discretion is advised, as some content may contain sensitive, controversial, or unverified information. We are not responsible for user-generated content, technical issues, or the accuracy of external links. Some content may be sponsored or contain affiliate links, which will be identified accordingly. By using this website, you agree to our privacy policy. For concerns, including copyright infringement (DMCA) notices, contact us at info@texasnews.app.

Comments are closed

Related Posts

0
YOUR CART
  • No products in the cart.