Sign In
Learn bits
Science & Tech.
Mahesh

19/02/24 08:24 AM IST

Gemini Pro 1.5 with 1 million tokens surpasses GPT-4 Turbo

In News
  • Google has unveiled its latest next-generation AI model, the Gemini 1.5 Pro.
  • The new model is built on MoE architecture and is claimed to be far more advanced than its contemporaries.
Gemini 1.5 Pro
  • Google claims that the Gemini 1.5 Pro is capable of achieving similar results as the recently launched Gemini 1.0 Ultra, albeit with much less computing power.
  • And, the most outstanding aspect of the Gemini 1.5 Pro is its ability to process the amount of information by up to one million tokens consistently.
  • This is certainly the longest context window for any large-scale foundation model developed yet.
  • To put into perspective, the Gemini 1.0 models have a context window of up to 32,000 tokens, GPT-4 Turbo has 1,28,000 tokens and Claude 2.1 has 2,00,000 tokens.
  • While the model comes with a standard 1,28,000 token context window, Google is allowing a limited number of developers and enterprise customers to try it with a context window of up to one million tokens.
  • The Gemini 1.5 Pro is currently in preview mode and developers can test the model using Google’s AI Studio and Vertex AI.
  • Google has claimed that since it launched the Gemini 1.0, they have been consistently testing, refining, and enhancing its capabilities — and 1.5 Pro is an outcome of its efforts.
  • The MoE architecture can be understood as a collective approach where the whole problem is divided into numerous sub-tasks that are later trained by a cluster of experts on each sub-task.
  • This is a step change in Google’s approach which builds upon research and engineering innovations across nearly every part of its foundational model development and infrastructure.
  • Google claims the new MoE architecture makes the Gemini 1.5 Pro more efficient to train and serve.
Benefits
  • The Gemini 1.5 Pro can reportedly ingest up to 7,00,000 words or about 30,000 lines of code.
  • This is 35 times more than what Gemini 1.0 Pro can take in. Besides, the Gemini 1.5 Pro can process up to 11 hours of audio and 1 hour of video in a wide range of languages.
  • The demo videos posted on Google’s official YouTube channel showed the long context understanding of the model by using a 402-page-long PDF.
  • The demo also showed a live interaction with the model based on the PDF file as prompt, which was 3,26,658 tokens and had 256 tokens worth of images.
  • The demo used a total of 3,27,309 tokens.
Significance
  • Gemini 1.5 Pro with a 1 million-token context window will be free to use. Google may introduce pricing tiers in the future on the model that starts at 1,28,000 context windows and will scale up to 1 million tokens.
  • Gemini 1.5 Pro is a new frontier in Google’s AI developments.
  • In December last year, Google introduced its most flexible AI model Gemini 1.0 in three different sizes, including Gemini Ultra, Gemini Pro, and Gemini Nano.
  • At the time of launch, Google claimed that its Gemini 1.0 surpassed several state-of-the-art performances on a range of benchmarks including coding and text.
  • The Gemini series has been known for its next-generation capabilities and sophisticated reasoning. All Gemini sizes have been known for their multimodality — the ability to understand text, images, audio and more.
Source- Indian Express

More Related Current Affairs View All

05 May

Key objectives of caste count

'The Union Cabinet has decided that the next Census will include questions on caste to its enumeration of India’s population, for the first time in almost a century. ' T

Read More

05 May

2 new genome-edited rice varieties

'Agriculture Minister Shivraj Singh Chouhan recently released two genome-edited varieties of rice, the first achievement of its kind in the country.' These two varieties, which

Read More

05 Mar

First-ever comprehensive survey of India’s river dolphins

'Prime Minister Narendra Modi released the results of the first-ever comprehensive population estimation of riverine dolphins – Gangetic and Indus dolphins – done in In

Read More

India’s First Ai-Driven Magazine Generator

Generate Your Custom Current Affairs Magazine using our AI in just 3 steps