How to Build With the Chromes Latest Built-in AI
Source: tai_blog

Gemini Nano  built-in AI by Google is picking up steam lately. Google first announced built-in AI in this years I/O event. The model was subsequently launched in the newest Canary release and Dev channel.   The current default for building AI features for the web is server-side solutions. OpenAI and Anthropic are among the main players dominating the market. Other key players like Google seemed lagging behind. But it is changing now.   My first impression of Gemini Nano is finesse.   Local  private  and offline models are the future. We already have some tools that provide this to a certain extent  like LM Studio and Ollama. But ordinary users dont bother downloading models to run things locally. Thats where built-in AI comes in.   You can bring top-notch LLM capabilities to your users without compromising their privacy  with no middleman involved  and you can deliver a snappy user experience because you are eliminating network round trips.   In some cases  you can build offline first products where your users can access built-in AI even when they are not connected to the internet.   Setting up Gemini NanoYou need at least Windows 10 or MacOS 13  integrated GPU  and 22GB of storage (but the model doesnt take that much space  its just to make sure you have enough storage margin).   Gemini Nano is still in early preview. Therefore  you need to download the Chrome Dev channel or Canary channel and confirm that your version is equal to or newer than 128.0.6545.0.   After installing Canary or dev channel  go to chrome://flags/#optimization-guide-on-device-model and select Enabled BypassPerfRequirement.   Then go to chrome://flags/#prompt-api-for-gemini-nano and select Enabled.   After that  restart Chrome.   To test that you have configured everything correctly  open DevTools and enter this in the console:   await window.ai.canCreateTextSession();If you got readily then you are all set.   Now we are ready to hack.   Lets cook something...As a developer  I think built-in AI will help my users quickly skim information on my website.   So  I am going to implement Gemini Nano on my site for AI jobs to help job seekers quickly get some info from job descriptions such as salary details  location restrictions  and minimum qualifications  all without needing to read through the 300-word description. This can help improve the productivity of job seekers.   I assume similar use cases for many other sites  such as recipe websites  product reviews  tutorials  etc. It is also for content creation-related tasks such as proofreading  rephrasing  and grammar correction.   I am using Next.js for the site  so the following code will be on Next.js.   The basic idea is to show a chat widget on the corner for users to chat with the job description. To do that  we need to pass the job description to Gemini Nano and direct user questions to it to answer based on the job description.   Next.js components are RSC by default  so we need to declare a client component.   First  we need a client component to access the built-in AI because the Gemini Nano is on the users device. To mark a component as a client component  we need to use a directive called use client.   To do so  put this at the beginning of the component  above any imports or other code:   'use client';After that  we need to make sure the user device can run Gemini Nano and is ready to receive prompts  otherwise  they cant use the chat feature and we shouldnt display the chat widget for them.   Now lets write the code to receive questions from users and send them to built-in AI:   We are using the react-markdown library to render results from Gemini Nano as markdown.   To put it all together:   ConclusionWhile all this is crazy  the Chrome team recently shipped session persistence. The best part about this is you dont need to keep track of the entire conversation history as we do with OpenAIs chat completion endpoint. Chrome will preserve the context for you. If you need to destroy the context and start an entirely new conversation  you can do:   session.destroy();Its pretty awesome to see where built-in AI is headed  and excited to see what other developers will build with this.   Thats it for this post. Please let me know in the comments below if you have any questions. I would love to help.   Thank you for reading. I build my products in public on Twitter and here. Please make sure to follow me if you are interested in building stuff together.