Sign In
Learn bits
Science & Tech.
Mahesh

19/02/24 08:24 AM IST

Gemini Pro 1.5 with 1 million tokens surpasses GPT-4 Turbo

In News
  • Google has unveiled its latest next-generation AI model, the Gemini 1.5 Pro.
  • The new model is built on MoE architecture and is claimed to be far more advanced than its contemporaries.
Gemini 1.5 Pro
  • Google claims that the Gemini 1.5 Pro is capable of achieving similar results as the recently launched Gemini 1.0 Ultra, albeit with much less computing power.
  • And, the most outstanding aspect of the Gemini 1.5 Pro is its ability to process the amount of information by up to one million tokens consistently.
  • This is certainly the longest context window for any large-scale foundation model developed yet.
  • To put into perspective, the Gemini 1.0 models have a context window of up to 32,000 tokens, GPT-4 Turbo has 1,28,000 tokens and Claude 2.1 has 2,00,000 tokens.
  • While the model comes with a standard 1,28,000 token context window, Google is allowing a limited number of developers and enterprise customers to try it with a context window of up to one million tokens.
  • The Gemini 1.5 Pro is currently in preview mode and developers can test the model using Google’s AI Studio and Vertex AI.
  • Google has claimed that since it launched the Gemini 1.0, they have been consistently testing, refining, and enhancing its capabilities — and 1.5 Pro is an outcome of its efforts.
  • The MoE architecture can be understood as a collective approach where the whole problem is divided into numerous sub-tasks that are later trained by a cluster of experts on each sub-task.
  • This is a step change in Google’s approach which builds upon research and engineering innovations across nearly every part of its foundational model development and infrastructure.
  • Google claims the new MoE architecture makes the Gemini 1.5 Pro more efficient to train and serve.
Benefits
  • The Gemini 1.5 Pro can reportedly ingest up to 7,00,000 words or about 30,000 lines of code.
  • This is 35 times more than what Gemini 1.0 Pro can take in. Besides, the Gemini 1.5 Pro can process up to 11 hours of audio and 1 hour of video in a wide range of languages.
  • The demo videos posted on Google’s official YouTube channel showed the long context understanding of the model by using a 402-page-long PDF.
  • The demo also showed a live interaction with the model based on the PDF file as prompt, which was 3,26,658 tokens and had 256 tokens worth of images.
  • The demo used a total of 3,27,309 tokens.
Significance
  • Gemini 1.5 Pro with a 1 million-token context window will be free to use. Google may introduce pricing tiers in the future on the model that starts at 1,28,000 context windows and will scale up to 1 million tokens.
  • Gemini 1.5 Pro is a new frontier in Google’s AI developments.
  • In December last year, Google introduced its most flexible AI model Gemini 1.0 in three different sizes, including Gemini Ultra, Gemini Pro, and Gemini Nano.
  • At the time of launch, Google claimed that its Gemini 1.0 surpassed several state-of-the-art performances on a range of benchmarks including coding and text.
  • The Gemini series has been known for its next-generation capabilities and sophisticated reasoning. All Gemini sizes have been known for their multimodality — the ability to understand text, images, audio and more.
Source- Indian Express

More Related Current Affairs View All

10 Jan

Rural landowners in Delhi want repeal of Sections 33 and 81 of Delhi Land Reforms Act

'Both sections dealing with the use and sale of agricultural land have come under the spotlight ahead of the Delhi Assembly elections.' This can only be done by the Central gove

Read More

10 Jan

Pravasi Bharatiya Diwas

'Prime Minister Narendra Modi inaugurated the 18th edition of the Pravasi Bharatiya Diwas  in Bhubaneswar.' The event is held once every two years to “honour the cont

Read More

10 Jan

Deciphering the Indus script

'Recently, Tamil Nadu Chief Minister M.K. Stalin announced a $1-million prize for experts or organisations in the event of their success in deciphering the scripts of the Indus Val

Read More

India’s First Ai-Driven Magazine Generator

Generate Your Custom Current Affairs Magazine using our AI in just 3 steps