At Google I/O, we previewed Gemma 3n with text and image inputs and launched the Google AI Edge Gallery app on GitHub. This open-source, interactive playground app is designed to inspire and enable developers by providing practical examples, transparent performance metrics, and direct links to the documentation you need to start building experience powered by on-device AI models. The developer community’s response has been fantastic, reaching 500,000 APK downloads in just two months, demonstrating the community’s excitement for powerful, private, on-device generative AI.
Today, we’re thrilled to take two big steps forward: adding audio modality to the Google AI Edge stack and bringing the Google AI Edge Gallery to the Google Play Store.
New audio capabilities with Gemma 3n
Beyond text and vision, the Google AI Edge stack now supports audio. Our first model with this capability is Gemma 3n, accessible through the MediaPipe LLM Inference API for Android and for Web. Audio understanding unlocks powerful new on-device features, including:
- High-Quality Speech-to-Text: Transcribe audio to text from a variety of spoken languages.
- Speech-to-Translated-Text: Translate spoken audio into text in another language.
In this initial release, the MediaPipe LLM Inference API supports audio batch inference for clips up to 30 seconds long. Streaming audio support is next on our roadmap.
Try it yourself in the Google AI Edge Gallery
To let you experience this new modality firsthand, we’ve added a new „Audio Scribe” capability to the Google AI Edge Gallery. With Audio Scribe, you can upload an audio clip or use your device’s microphone to record audio, and watch as Gemma 3n transcribes it directly on your phone, no internet connection required.
Google AI Edge Gallery in Play Store
To make these powerful demonstrations more accessible than ever, the Google AI Edge Gallery is now available in open beta on the Google Play Store. The code will continue to be open-sourced on Github, giving you the best of both worlds: the easiest way to experience the demos via the Play Store, and the complete source code to explore on GitHub.
What’s Next
This is just the beginning. Our goal is to make the Google AI Edge Gallery the most inspiring and helpful showcase for on-device AI. In the coming months, we plan to:
- Bring the app to iOS users.
- Showcase more examples using Google AI Edge Generative AI Tasks like Retrieval Augmented Generation (RAG) and on-device function calling.
- Work with the community to convert, optimize, and publish more open-source models to the Hugging Face LiteRT community and integrate them into the app.
Download the app from the Play Store or check out the open-source project on Github. We can’t wait to see what you build!
Acknowledgements
We’d like to extend a special thanks to our key contributors for their foundational work on this project: Grant Jensen, Ho Ko, Ivan Grishchenko, Jae Yoo, Jing Jin, Marissa Ikonomidis, Tyler Mullen, Weiyi Wang, and Yu-hui Chen
We also gratefully acknowledge the significant contributions from the following team members: Chandramouli Amarnath, Chunlei Niu, Deepak Nagaraj Halliyavar, Dillon Sharlet, Fengwu Yao, Ireneu Pla, Joe Zou, Kris Wright, Lin Chen, Mark Sherwood, Raman Sarokin, Renjie Wu, Rishika Sinha, Ronghui Zhu, Suleman Shahid, Suril Shah, Vladimir Kirilyuk, Wai Hon Law, Yasir Modak, and Zi Yuan
This effort was made possible by the guidance and support from our leadership: Cormac Brick, Jianing Wei, Juhyun Lee, Karthik Raveendran, Lu Wang, Matthias Grundmann, Ram Iyengar, and Sachin Kotwani
Add comment