Select to AI
I grew a Chrome extension to 1,000 users and a Featured badge on the Chrome Web Store, then turned it into a browser agent that plans a task, carries it out and checks every step actually worked.
01The problem
Using AI while you read the web means copying text, switching tabs, pasting, and switching back. I built Select to AI so you can highlight text and ask about it right there. Version 3 goes further: you give the browser a task, like "find a USB-C cable under $10", and it does it for you.
02What I did
All of it: the permissions, the scripts that run on each page, the side panel, popup and tab views, the connections to each AI provider, the agent itself, the store listing, releases and user support.
03What made it hard
Chrome's Manifest V3 limits what extensions can do in the background and what they can access. I also set myself a rule that users' API keys and pages never touch a server I run. And an agent that acts on real websites has to prove each step worked, not just claim it did.
04How it works
You type "ai" in the address bar and it decides whether to open a site, ask the AI a question or start the agent. I raised the accuracy of that decision from 68.0% to 94.7% on a held-out set. The agent then works in a loop: a planner writes a checklist, each step clicks or types on the page, and I measure whether the page really changed (a new URL, new content) instead of trusting the model. After two failed steps it stops to rethink, and a final check confirms the answer against what is on the page before it finishes.
05Other options I ruled out
A server in the middle would have made connecting to providers easier, but it would have meant holding users' keys, so I ruled it out. Trusting the model when it says a click worked would have been simpler, but models often get that wrong. I also made the agent stop and ask the user at logins, CAPTCHAs and anything unclear, rather than guess.
06How I tested it
Everything is in the public repo: 149 unit tests, 10 attack cases that try to trick it into sending data to the wrong place (all blocked before each release), and an end-to-end test that runs the real agent through 4 tasks: a search, filling in a form, hitting a login wall and correctly asking the user, and comparing two pages.
07Results
1,000 users and a Featured badge on the Chrome Web Store. The agent passes all 4 end-to-end tasks.
08What's next
The store still serves the earlier highlight-and-ask version, so publishing the agent version is next. The agent also remembers tasks it completed successfully, and I'm still watching how much that helps over time.