On December 6, Google released the much-anticipated self-developed language model Gemini.。Previously, this product was expected to be Google's secret weapon against OpenAI, but due to various reasons, Gemini's release time has been repeatedly delayed。Introducing Gemini in its official blog, Google said Gemini is Google's largest and most capable artificial intelligence model。

Google's Gemini 1.0 is divided into three versions of Ultra, Pro and Nano, of which Ultra has the strongest ability and the highest complexity and can handle the most difficult tasks, Pro has a slightly weaker ability and can be used to handle multitasking, and Nano pays more attention to the processing power of the end side.。

Gemini is the result of a large-scale collaboration between Google teams including Google Research colleagues. It has been multimodal from the beginning, which means it can summarize and seamlessly understand, manipulate and combine different types of information. Including text, code, audio, images and video。

Gemini is also our most flexible model yet - able to run efficiently on everything from data centers to mobile devices.。Its state-of-the-art capabilities will greatly enhance the way developers and enterprise customers build and scale AI。

From December 13, developers will be able to access Gemini via Google Cloud's API。Google revealed that Gemini will appear in other Google products in the next few months, including Pixel 8 smartphones, generative search and Chrome browser, and Gemini's most powerful artificial intelligence version will be officially launched in 2024 after "extensive trust and security checks."。

How strong is Gemini?？

• basic ability

From natural image, audio, and video understanding to mathematical reasoning, Gemini Ultra outperformed current SOTA results in 30 of the 32 academic benchmark sets widely used for large language model development.。

In addition, Gemini Ultra scored 90 points in MMLU (Large Scale Multitask Language Understanding Dataset)..0%, surpassing human experts for the first time。The MMLU dataset contains 57 subjects, including mathematics, physics, history, law, medicine and ethics, to test the knowledge reserve and problem-solving skills of large models.。

According to data released by Google, Gemini Ultra is able to surpass GPT-4 in multiple areas of total score, reasoning, math and code, and only outperform its rivals in text processing.。

In addition, in the more difficult MMMU benchmark, Gemini Ultra also achieved 59.With a high score of 4%, the MMMU benchmark tests the model's ability to reason carefully across domains, one of the areas where GPT-4 has been criticized.。

• multi-modal capability

In the benchmark test of multi-modal capability, Gemini Ultra is the omni-directional rolling GPT-4V。

We designed Gemini to be natively multimodal, pre-trained on different modalities from the start.。We then fine-tune it with more multi-modal data to further refine its effectiveness。This helps Gemini to fundamentally and seamlessly understand and reason about various inputs, far outperforming existing multimodal models - with features that are state-of-the-art in almost every field.。

• Other capabilities

In addition, Gemini has advanced capabilities for many large models, including sophisticated multimodal reasoning, the ability to recognize and understand text, images, and audio, and advanced coding capabilities.。According to Google, Gemini is particularly good at explaining reasoning in complex disciplines such as mathematics and physics.。

However, in the data released on the same day, Google did not disclose the specific parameter sizes of Gemini Ultra and Gemini Pro, but explicitly stated that the parameters of the smallest Gemini Nano were 1.8 billion (Nano-1) and 32.500 million (Nano-2)。Rumor has it that the Gemini Ultra has trillions of parameters and uses more than five times the computing power of GPT-4 for training.。

On the same day, Google also announced the most powerful, efficient and scalable TPU system to date - Cloud TPU v5p, designed for training cutting-edge artificial intelligence models.。Next-generation TPUs will accelerate Gemini's growth, help developers and enterprise customers train mass-generative AI models faster, and enable new products and features to meet customers faster。

Can Gemini really surpass GPT-4?？

Although Google has basically written hard OpenAI on its face this time, and has shown Gemini to be better than its competitors in a number of published results, this may not be the case.。

Google's DeepMind CEO Demis Hassabis did say that Gemini is better than OpenAI's GPT-4 on a range of metrics, while the fact is that there are three versions of Gemini, Ultra, Pro and Nano.。Google's product for comparison this time is only Gemini Ultra, which is a high-end version of Gemini.。

And according to Google's official blog, Gemini Ultra hasn't met with you so soon and won't be available until at least 2024, and before Gemini Ultra goes public, the version that Google will open up is Gemini Pro, which users can access through Google's Bard chatbot.。

This means that even though Google has made an official assessment and has hammered GPT-4, it has nothing to do with the user.。Because in the future, users will not be able to experience this "powerful" Ultra product at all, and will not be able to verify Google's official reviews.。

How does Gemini Pro compare to GPT-4?？According to this published Google Tech paper, Gemini Pro outperforms GPT-3 on most metrics.5, but failed to beat OpenAI's GPT-4。

The order should be this: Gemini Ultra > GPT-4 > Gemini Pro。But even when Gemini Ultra is released next year, there will be a significant number of people who won't use this powerful product, as Hassabis has already revealed in a blog post that Ultra will be limited to Bard Advanced users.。

Regarding why Gemini Ultra needs to be delayed, according to the media, the main reason is that the product has encountered difficulties in handling English language prompts。

How to evaluate Gemini？mixed reviews

Google CEO Sundar Pichai said: "Now we are taking a new step on the road to Gemini, our most powerful and versatile model to date, with the most advanced performance in many leading benchmarks.。

In this regard, many people are cheering, which seems to be another milestone in the history of artificial intelligence。Ethan Mollick, a professor at the University of Pennsylvania's Wharton School of Business, wrote excitedly on X: "Most importantly, it seems to be the first model to beat GPT-4.。

However, there are also many people who are skeptical of Gemini's "immensely powerful," mainly because Google provides too little data in the official blog.。According to Yacine Jernite, a researcher at artificial intelligence company Hugging Face, the fact that only two paragraphs of data appear in the 60-page report seems somewhat irrational and amounts to forcing the market to believe Google。

Jesse Dodge, a research scientist at the Allen Institute for AI, also said that while Google called Gemini's training data key to its performance, the company "provided very little information about how the data was made, how it was sifted, and what the data contained."。