Sign In
Learn bits
Science & Tech.
Mahesh

19/02/24 08:24 AM IST

Gemini Pro 1.5 with 1 million tokens surpasses GPT-4 Turbo

In News
  • Google has unveiled its latest next-generation AI model, the Gemini 1.5 Pro.
  • The new model is built on MoE architecture and is claimed to be far more advanced than its contemporaries.
Gemini 1.5 Pro
  • Google claims that the Gemini 1.5 Pro is capable of achieving similar results as the recently launched Gemini 1.0 Ultra, albeit with much less computing power.
  • And, the most outstanding aspect of the Gemini 1.5 Pro is its ability to process the amount of information by up to one million tokens consistently.
  • This is certainly the longest context window for any large-scale foundation model developed yet.
  • To put into perspective, the Gemini 1.0 models have a context window of up to 32,000 tokens, GPT-4 Turbo has 1,28,000 tokens and Claude 2.1 has 2,00,000 tokens.
  • While the model comes with a standard 1,28,000 token context window, Google is allowing a limited number of developers and enterprise customers to try it with a context window of up to one million tokens.
  • The Gemini 1.5 Pro is currently in preview mode and developers can test the model using Google’s AI Studio and Vertex AI.
  • Google has claimed that since it launched the Gemini 1.0, they have been consistently testing, refining, and enhancing its capabilities — and 1.5 Pro is an outcome of its efforts.
  • The MoE architecture can be understood as a collective approach where the whole problem is divided into numerous sub-tasks that are later trained by a cluster of experts on each sub-task.
  • This is a step change in Google’s approach which builds upon research and engineering innovations across nearly every part of its foundational model development and infrastructure.
  • Google claims the new MoE architecture makes the Gemini 1.5 Pro more efficient to train and serve.
Benefits
  • The Gemini 1.5 Pro can reportedly ingest up to 7,00,000 words or about 30,000 lines of code.
  • This is 35 times more than what Gemini 1.0 Pro can take in. Besides, the Gemini 1.5 Pro can process up to 11 hours of audio and 1 hour of video in a wide range of languages.
  • The demo videos posted on Google’s official YouTube channel showed the long context understanding of the model by using a 402-page-long PDF.
  • The demo also showed a live interaction with the model based on the PDF file as prompt, which was 3,26,658 tokens and had 256 tokens worth of images.
  • The demo used a total of 3,27,309 tokens.
Significance
  • Gemini 1.5 Pro with a 1 million-token context window will be free to use. Google may introduce pricing tiers in the future on the model that starts at 1,28,000 context windows and will scale up to 1 million tokens.
  • Gemini 1.5 Pro is a new frontier in Google’s AI developments.
  • In December last year, Google introduced its most flexible AI model Gemini 1.0 in three different sizes, including Gemini Ultra, Gemini Pro, and Gemini Nano.
  • At the time of launch, Google claimed that its Gemini 1.0 surpassed several state-of-the-art performances on a range of benchmarks including coding and text.
  • The Gemini series has been known for its next-generation capabilities and sophisticated reasoning. All Gemini sizes have been known for their multimodality — the ability to understand text, images, audio and more.
Source- Indian Express

More Related Current Affairs View All

15 Nov

Government issues guidelines to curb misleading ads by coaching centres

'The central Government issued new guidelines aimed at curbing misleading advertisements by coaching institutes, specifically prohibiting false promises such as "100 per cent selec

Read More

15 Nov

Janjatiya Gaurav Divas

'Every year on November 15th, Janjatiya Gaurav Divas is celebrated to honor the contributions of these communities, especially in India’s freedom struggle.' 5th November

Read More

15 Nov

Supreme Court’s order on mandatory accessibility standards

'A bench of the Supreme Court last week ordered the Union government to frame mandatory rules for ensuring the accessibility of public places and services to persons with disabilit

Read More

India’s First Ai-Driven Magazine Generator

Generate Your Custom Current Affairs Magazine using our AI in just 3 steps