Google is brining homework help and a new Gemini interface to Android

During the heavy slate of AI announcements in Tuesday’s Google I/O keynote, the company revealed several features headed to Android — most of them tied to Gemini Nano, the mobile-oriented version of its generative AI tech. For students, though, the biggest news may be the addition of homework support in Circle to Search. This makes it possible to solve some word problems in a matter of seconds.

Starting today — on Android devices that support Circle to Search — you can highlight a variety of onscreen math- and physics-related word problems and get step-by-step instructions for solving them. This is thanks to advancements in the education-oriented LearnLM AI models, Google explains. You’ll have to wait until later in 2024, however, to get answers to anything involving diagrams, graphs, or symbolic formulas. It’s not clear if the company is waiting for Android 15 this fall, or simply more AI development behind the scenes.

All the Gemini Android upgrades

Google is meanwhile promising a more convenient Gemini overlay interface for Android, simplifying tasks like dropping generated images into apps. You’ll be able to ask Gemini to extract info from YouTube videos, or — if you’re subscribed to Gemini Advanced — answer questions based on the content of PDF files. For video, Google used the example of Gemini explaining the two-bounce rule based on a “How to Play Pickleball” clip. The new overlay will make additional dynamic suggestions too, but all of these enhancements are only rolling out “over the next few months,” according to Google.

Beginning with Pixel phones later this year, Google is launching Gemini Nano with Multimodality. That means the AI will be able to interpret sights and sounds, including spoken language, much like OpenAI’s new GPT-4o model. You can switch between text, image, and audio modes via a toggle. The company was quiet on whether people will need the unannounced Pixel 9, and how this might tie into Project Astra, an effort to deliver AI results based on real-time video. Google may be preparing an AI-powered successor to its defunct Glass devices.

Minor upcoming additions to Gemini Nano include real-time scam detection during phone calls, and multimodal support in TalkBack, which will help better describe images to the visually impaired, with or without a network connection. The TalkBack update is coming later this year. Scam detection is currently in testing, and we should learn more about launch plans by the end of 2024. One thing Google has confirmed is that it’ll be opt-in, presumably to address worries about AI snooping on private phone conversations.

Google is set to reveal more about Android 15 on Wednesday. Most likely the company will go beyond AI to talk about features that have been teased, hinted, or otherwise flown under the radar, such as satellite messaging. The OS is expected to launch this fall in tandem with new Pixel devices.