Sign In
Learn bits
Science & Tech.
Mahesh

19/02/24 08:24 AM IST

Gemini Pro 1.5 with 1 million tokens surpasses GPT-4 Turbo

In News
  • Google has unveiled its latest next-generation AI model, the Gemini 1.5 Pro.
  • The new model is built on MoE architecture and is claimed to be far more advanced than its contemporaries.
Gemini 1.5 Pro
  • Google claims that the Gemini 1.5 Pro is capable of achieving similar results as the recently launched Gemini 1.0 Ultra, albeit with much less computing power.
  • And, the most outstanding aspect of the Gemini 1.5 Pro is its ability to process the amount of information by up to one million tokens consistently.
  • This is certainly the longest context window for any large-scale foundation model developed yet.
  • To put into perspective, the Gemini 1.0 models have a context window of up to 32,000 tokens, GPT-4 Turbo has 1,28,000 tokens and Claude 2.1 has 2,00,000 tokens.
  • While the model comes with a standard 1,28,000 token context window, Google is allowing a limited number of developers and enterprise customers to try it with a context window of up to one million tokens.
  • The Gemini 1.5 Pro is currently in preview mode and developers can test the model using Google’s AI Studio and Vertex AI.
  • Google has claimed that since it launched the Gemini 1.0, they have been consistently testing, refining, and enhancing its capabilities — and 1.5 Pro is an outcome of its efforts.
  • The MoE architecture can be understood as a collective approach where the whole problem is divided into numerous sub-tasks that are later trained by a cluster of experts on each sub-task.
  • This is a step change in Google’s approach which builds upon research and engineering innovations across nearly every part of its foundational model development and infrastructure.
  • Google claims the new MoE architecture makes the Gemini 1.5 Pro more efficient to train and serve.
Benefits
  • The Gemini 1.5 Pro can reportedly ingest up to 7,00,000 words or about 30,000 lines of code.
  • This is 35 times more than what Gemini 1.0 Pro can take in. Besides, the Gemini 1.5 Pro can process up to 11 hours of audio and 1 hour of video in a wide range of languages.
  • The demo videos posted on Google’s official YouTube channel showed the long context understanding of the model by using a 402-page-long PDF.
  • The demo also showed a live interaction with the model based on the PDF file as prompt, which was 3,26,658 tokens and had 256 tokens worth of images.
  • The demo used a total of 3,27,309 tokens.
Significance
  • Gemini 1.5 Pro with a 1 million-token context window will be free to use. Google may introduce pricing tiers in the future on the model that starts at 1,28,000 context windows and will scale up to 1 million tokens.
  • Gemini 1.5 Pro is a new frontier in Google’s AI developments.
  • In December last year, Google introduced its most flexible AI model Gemini 1.0 in three different sizes, including Gemini Ultra, Gemini Pro, and Gemini Nano.
  • At the time of launch, Google claimed that its Gemini 1.0 surpassed several state-of-the-art performances on a range of benchmarks including coding and text.
  • The Gemini series has been known for its next-generation capabilities and sophisticated reasoning. All Gemini sizes have been known for their multimodality — the ability to understand text, images, audio and more.
Source- Indian Express

More Related Current Affairs View All

21 Sep

India Achieves Tier 1 Status in Global Cybersecurity Index 2024

' India has secured Tier 1 status in the Global Cybersecurity Index (GCI) 2024, published by the International Telecommunication Union (ITU).' The GCI is a comprehensive assessm

Read More

21 Sep

A new White Revolution

'Operation Flood, launched in 1970, ushered in the White Revolution and transformed the dairy sector in India.' Dairy cooperatives procured 660 lakh kg of milk per day in 2023-2

Read More

21 Sep

Changes in Canada’s international work permit system

'Canadian Prime Minister Justin Trudeau announced changes to the country’s international work permit system, aiming to reduce the number of immigrant workers in Canada.'

Read More

India’s First Ai-Driven Magazine Generator

Generate Your Custom Current Affairs Magazine using our AI in just 3 steps