Everything You Need To Know About Google’s Gemini AI

Google I/O unveiled significant advancements in its Gemini AI model, showcasing enhanced image and video search capabilities, deeper Workspace integration, improved conversational AI, and the introduction of innovative tools like Gems.

Recently concluded Google I/O included a plethora of AI announcements. The keynote addressed Google’s Gemini AI models and how they’re incorporated into programs like Workspace and Chrome, as was to be expected.

If you were unable to watch the event in real-time, you can catch up on Google’s latest news in the roundup below.

Google Lens now lets you search by recording a video

And you’ll also be able to ask questions with video, right in Search. Coming soon. #GoogleIO pic.twitter.com/zFVu8yOWI1

— Google (@Google) May 14, 2024

You can already conduct image-based searches with Google Lens, but now you can also conduct video-based searches, which is a big step forward for Google. In other words, you can record a video of something you want to look up and ask a question during the video, and Google’s artificial intelligence will try to find pertinent web results.

Gemini will answer questions about your photo library

This summer, Google is launching a new tool that may come in handy for anyone who has years, or even more than a decade,’s worth, of images to go through. “Ask Photos” allows Gemini to search through your Google Photos collection to get the answers to your queries; it’s not just for finding images of cats and dogs, though. To make his point, CEO Sundar Pichai asked Gemini for his license plate number. The number itself was the response, followed by a photo of it so he could confirm that it was correct.

Google’s flagship Gemini model gets faster and more capable

Google has added Gemini 1.5 Flash, a new AI model, to its portfolio. Although the new multimodal model is tailored for “narrow, high-frequency, low-latency tasks,” it is just as powerful as the Gemini 1.5 Pro. This improves its ability to produce quick answers. Google claims that the modifications it made to Gemini 1.5 will enhance its reasoning, coding, and translation capabilities. Additionally, Google claims to have quadrupled the context window (or amount of data) of Gemini 1.5 Pro from 1 million to 2 million tokens.

Gemini joins users in Workspace

Google is gradually adding Gemini 1.5 Pro, its most recent mainstream language model, to the sidebar of its Drive, Gmail, Docs, Sheets, and Slides applications. It will become more of a general-purpose assistant within Workspace when it launches for paying customers next month, pulling information from all of your Drive’s content no matter where you are. Additionally, it can compose emails for you by leveraging information from a document you are currently viewing or schedule a reminder for you to reply to an email you are reading at a later time. Although some early testers have previously had access to these functionalities, Google says that starting next month, all paid Gemini subscribers will be able to utilize them.

Project Astra is Google’s Star Trek AI future

View this post on Instagram

A post shared by The Verge (@verge)

Google’s Project Astra is a multimodal AI helper that it believes will grow into a comprehensive virtual assistant that can remember where your belongings are, monitor and comprehend what it sees through your device’s camera, and perform tasks on your behalf. The company’s goal it is to be a true AI agent that can conduct tasks on your behalf in addition to talking to you; it powers several of the most amazing demos from this year’s I/O.

Veo brings Sora-style video generation to creators

With a new generative AI model that can produce 1080p video in response to text, image, and video-based requests, Google has responded to OpenAI’s Sora. More prompts can be used to adjust videos in a multitude of genres, such as timelapses and aerial shots. In addition to pushing Veo to Hollywood for use in motion pictures, the company is already providing it to certain creators for use in YouTube videos.

Gems bring custom chatbot creation to Gemini

Whether you need a yoga bestie or calculus tutor, in the coming months you’ll be able to customize Gemini, saving time when you have specific ways you interact with Gemini again and again. We’re calling these Gems. #GoogleIO pic.twitter.com/YQOHsUbMWE

— Google (@Google) May 14, 2024

Google is launching Gems, a proprietary chatbot builder. Similar to OpenAI’s GPTs, Gemini can be instructed by users through Gems to alter its response and area of expertise. You can soon (if you’re a Gemini Advanced subscription) be able to have it be a relentless, upbeat running coach with daily inspiration and training plans—my worst nightmare.

Gemini becomes a better conversation partner

Making voice conversations with Gemini feel more organic is the goal of the new Gemini Live function. Users will be able to stop the chatbot in mid-sentence or ask it to observe through their smartphone camera and provide real-time commentary about what it sees. The chatbot’s voice will also be enhanced with some added personality. Additionally, Gemini is gaining new integrations that allow it to use multimodal features (e.g., adding information from a flyer to your calendar) to update or get data from Google Calendar, Tasks, and Keep.

Circle to Search can help solve math problems now

You may now circle a math issue on your Android phone or tablet to get assistance solving it. Although Google’s AI can’t solve the problem for you and won’t assist kids in cheating on their schoolwork, it can break it down into manageable steps.

Google Search gets an AI overhaul

This week, Google will make “AI Overviews”—a mouthful that was previously known as “Search Generative Experience”—available to all US users. Now, a “specialized” Gemini model will create and add condensed responses from the internet to results pages (think of them as AI search tools like Arc Search or Perplexity).

Android gets AI-powered scam detection

This week, Google will make “AI Overviews”—a mouthful that was previously known as “Search Generative Experience”—available to all US users. Now, a “specialized” Gemini model will create and add condensed responses from the internet to results pages (think of them as AI search tools like Arc Search or Perplexity).

Android devices are about to get smarter AI

According to Google, users will soon be able to ask Gemini questions regarding on-screen videos, and the system will respond by using the subtitles that automatically appear. In addition, it can absorb PDFs and provide information for Gemini Advanced users who pay for it. Over the next months, Gemini for Android will receive these and other multimodal updates.

Google Chrome is getting an AI assistant

Google revealed that Chrome for desktop users will now support the lightweight Gemini Nano variant. With the help of on-device AI, the built-in assistant in Google Chrome will assist you in creating text for product evaluations, social media postings, and other purposes.

Google upgrades its SynthID AI watermarking

Google states that SynthID’s capabilities are growing. It will now be able to recognize movies produced by artificial intelligence (AI) and will incorporate watermarking into content made with its new Veo video generator.

Recently reported by GreatGameIndia, Google’s AI faced criticism for historical inaccuracies and woke biases. Google co-founder Sergey Brin admitted errors in Gemini, citing insufficient testing at the Gemini Hackathon.