Sign In
Learn bits
Science & Tech.
Mahesh

19/02/24 08:24 AM IST

Gemini Pro 1.5 with 1 million tokens surpasses GPT-4 Turbo

In News
  • Google has unveiled its latest next-generation AI model, the Gemini 1.5 Pro.
  • The new model is built on MoE architecture and is claimed to be far more advanced than its contemporaries.
Gemini 1.5 Pro
  • Google claims that the Gemini 1.5 Pro is capable of achieving similar results as the recently launched Gemini 1.0 Ultra, albeit with much less computing power.
  • And, the most outstanding aspect of the Gemini 1.5 Pro is its ability to process the amount of information by up to one million tokens consistently.
  • This is certainly the longest context window for any large-scale foundation model developed yet.
  • To put into perspective, the Gemini 1.0 models have a context window of up to 32,000 tokens, GPT-4 Turbo has 1,28,000 tokens and Claude 2.1 has 2,00,000 tokens.
  • While the model comes with a standard 1,28,000 token context window, Google is allowing a limited number of developers and enterprise customers to try it with a context window of up to one million tokens.
  • The Gemini 1.5 Pro is currently in preview mode and developers can test the model using Google’s AI Studio and Vertex AI.
  • Google has claimed that since it launched the Gemini 1.0, they have been consistently testing, refining, and enhancing its capabilities — and 1.5 Pro is an outcome of its efforts.
  • The MoE architecture can be understood as a collective approach where the whole problem is divided into numerous sub-tasks that are later trained by a cluster of experts on each sub-task.
  • This is a step change in Google’s approach which builds upon research and engineering innovations across nearly every part of its foundational model development and infrastructure.
  • Google claims the new MoE architecture makes the Gemini 1.5 Pro more efficient to train and serve.
Benefits
  • The Gemini 1.5 Pro can reportedly ingest up to 7,00,000 words or about 30,000 lines of code.
  • This is 35 times more than what Gemini 1.0 Pro can take in. Besides, the Gemini 1.5 Pro can process up to 11 hours of audio and 1 hour of video in a wide range of languages.
  • The demo videos posted on Google’s official YouTube channel showed the long context understanding of the model by using a 402-page-long PDF.
  • The demo also showed a live interaction with the model based on the PDF file as prompt, which was 3,26,658 tokens and had 256 tokens worth of images.
  • The demo used a total of 3,27,309 tokens.
Significance
  • Gemini 1.5 Pro with a 1 million-token context window will be free to use. Google may introduce pricing tiers in the future on the model that starts at 1,28,000 context windows and will scale up to 1 million tokens.
  • Gemini 1.5 Pro is a new frontier in Google’s AI developments.
  • In December last year, Google introduced its most flexible AI model Gemini 1.0 in three different sizes, including Gemini Ultra, Gemini Pro, and Gemini Nano.
  • At the time of launch, Google claimed that its Gemini 1.0 surpassed several state-of-the-art performances on a range of benchmarks including coding and text.
  • The Gemini series has been known for its next-generation capabilities and sophisticated reasoning. All Gemini sizes have been known for their multimodality — the ability to understand text, images, audio and more.
Source- Indian Express

More Related Current Affairs View All

28 Aug

IADT-1

'Recently, the Indian Space Research Organisation (ISRO) successfully carried out its first Integrated Air Drop Test (IADT-1), a crucial milestone in the preparation for the countr

Read More

28 Aug

H-1B visa changes

'H-1B visa changes 2025: United States Commerce Secretary Howard Lutnick called the popular H-1B visa program a “scam”, saying that American businesses should be focuse

Read More

28 Aug

UGC’s Learning Outcomes-based Curriculum Framework

'Recently, the University Grants Commission (UGC) issued a public notice on its development of a draft Learning Outcomes-based Curriculum Framework (LOCF).' LOCF is intended to

Read More

India’s First Ai-Driven Magazine Generator

Generate Your Custom Current Affairs Magazine using our AI in just 3 steps