HawkInsight

  • Contact Us
  • App
  • English

Zuckerberg "Annunciation": Training Llama 3, will buy 350,000 Nvidia chips this year

On January 18, local time, Zuckerberg posted on his social platform that Meta is training the next generation of large model Llama 3, while the company is also building a large-scale computing infrastructure.。

On January 18, local time, Zuckerberg posted on his social platform Instagram to share some of Meta's AI progress.。

Zuckerberg mentioned that Meta's long-term vision is to build universal intelligence, responsibly open source it, and make it widely available so that everyone can benefit.。"We are bringing our two major AI research efforts (FAIR and GenAI) closer together to support this."。"

扎克伯格

GenAI refers to last year's hot generative artificial intelligence。FAIR, on the other hand, is Meta's artificial intelligence lab, as well as Google's DeepMind and Microsoft's investment in OpenAI, both of which are among the top artificial intelligence labs in Silicon Valley technology companies.。

Zuckerberg also revealed the progress of the meta-large model: "We are currently training our next-generation large model Llama 3, and we are building a large-scale computing infrastructure to support our future roadmap.。"

In 2023, Meta will open source the first and second generation models, namely Llama 1 and Llama 2.。The move won Meta a lot of praise and became the benchmark for open source LLM。

Meta will open source the big model, so that many developers and start-ups benefit from it。After Zuckerberg, Meta's chief AI scientist Yann LeCun also confirmed the news of the upcoming Llama 3 on social media on the 18th.。"To accelerate progress, FAIR is now a sister organization to GenAI, the AI product division."。At the same time, he is also very supportive of Meta's decision to open source the model, saying that no open source AI means no AI startup ecosystem.。

Yann LeCun

In addition, in order to better train large models, Zuckerberg also revealed that by the end of 2024, Meta will purchase more than 350,000 NVIDIA H100 GPUs, which is currently the industry's most powerful chip for building generative artificial intelligence.。If you include the Nvidia A100 and other artificial intelligence chips, by the end of this year, Meta will have nearly 600,000 H100 chips with computing power.。

This huge chip and computing power reserve will make Meta one of the most powerful AI systems in the world.。In the third quarter of last year, Amazon said it was building an AI system with 100,000 Trainium2 chips, while Oracle revealed that it had an AI system with more than 30,000 Nvidia H100 chips.。

A Meta spokesperson declined to say which chip vendors the company uses to buy chips from in addition to Nvidia, but the company has previously publicly stated plans to use AMD's chips.。

However, Meta's purchase of Nvidia's top chips is expected to consume a lot of the company's money.。It is understood that the price of a single H100 GPU is 2.Between $50,000 and $30,000, even at the lower end of this price range, Meta will spend as little as $8.7 billion on chip purchases this year.。

In its third-quarter earnings report last year, Meta projected that the company's capital expenditures for the full year 2024 would be between $30-35 billion, mainly for the company's increased server construction, including investments in non-artificial intelligence and AI hardware and data centers.。Combined with the above calculations, Meta spent nearly 30 percent of its estimated annual spending on chip purchases this year.。

Although the purchase of chips spent a lot of money, but Meta believes that this expenditure is still not saved, because the company is now in need of chips to support AI research and development.。In addition to using top-of-the-line chips to speed up the training of large models to catch up with technology companies such as OpenAI and Google, Zuckerberg said, "I'm very excited about our progress in making new AI-centric computing devices, such as Ray-Ban Meta smart glasses.。"

It is reported that Ray-Ban Meta smart glasses use artificial intelligence technology, allowing users to make phone calls, send messages and shoot videos hands-free。

As Zuckerberg enthusiastically shared Meta's internal AI progress, Meta's stock hit another new high on Thursday (January 18)。18, Meta rose more than 2% to close at 376.13美元。

Meta's share price has been on the rise since the start of the year, with a total market capitalization of $966.6 billion, just one step away from joining the "trillion club."。

Meta

 

·Original

Disclaimer: The views in this article are from the original author and do not represent the views or position of Hawk Insight. The content of the article is for reference, communication and learning only, and does not constitute investment advice. If it involves copyright issues, please contact us for deletion.