{"id":3570,"date":"2025-09-03T11:59:24","date_gmt":"2025-09-03T09:59:24","guid":{"rendered":"https:\/\/blog.tomayac.com\/2025\/09\/03\/for-all-thats-holy-can-you-just-leverage-the-web-please\/"},"modified":"2025-09-03T11:59:24","modified_gmt":"2025-09-03T09:59:24","slug":"for-all-thats-holy-can-you-just-leverage-the-web-please","status":"publish","type":"post","link":"https:\/\/tomayac.com\/wordpress\/2025\/09\/03\/for-all-thats-holy-can-you-just-leverage-the-web-please\/","title":{"rendered":"For all that’s holy, can you just leverage the Web, please?"},"content":{"rendered":"\n\t\t\t

When I moved in with my wife Laura in 2005, we lived in a shared apartment in\nBarcelona that had an ancient washing machine that was just there already, no\nidea who initially bought it. I managed to break the washing machine door's\nclosing mechanism some time in 2006, so for a few weeks, whenever we did the\nwashing, we had to lean a chair against the door so it wouldn't open. At the\ntime, we were both students and living on a small budget.<\/p>\n

Eventually, later in the same year, we bought an Electrolux machine that has\naccompanied us ever since. First on our move to Hamburg, then there through\nthree apartments, and finally back to Spain, where we live now in the Catalonian\ncountryside. Anyway, the washing machine had a motor damage last week, so after\nalmost 20 years, it was time for a new one. I ordered it online (another\nElectrolux, without<\/em> Internet nor WiFi), it was delivered swiftly, and I\ninstalled it hopefully correctly.<\/p>\n

\"Our<\/p>\n

The washing machine came with a voluntary 10 year warranty if you registered it.\nThe brochure where this offer was announced featured a free telephone number and\na QR code that pointed at the number (in plain text, not making use of the\ntel:<\/code> protocol). I called the number, and to my absolute surprise<\/em> there were\ncurrently more callers than usual. After about 20 minutes, I had an agent on the\nphone, but after saying what I wanted, they just hung up on me (or the\nconnection cut, whatever). Fine, I called again, but now, the call center was\nover capacity and they didn't even let me enter in the wait loop.<\/p>\n

They did offer to send me a link to a chat service on their website via SMS,\nthough, so I went for that option. The SMS literally pointed me at something\nlike https:\/\/www.<\/code> broken up by a space and then example.com\/gc\/<\/code>. When I\nclicked the linkified example.com\/gc\/<\/code>, I ended up on a broken site whose\ncertificate wasn't trusted. After fixing the link manually and prepending the\nhttps:\/\/www.<\/code> part, the page didn't load.<\/p>\n

At this point I was close to giving up, but I had one last card that I wanted to\nplay: I searched Google for "electrolux warranty register", and it pointed me at\na site https:\/\/www.example.com\/mypages\/register-a-product\/<\/code> as the first\nresult. This looked promising. The mypages<\/code> already suggested that this was\ngated behind a login, so I created an account, which was painless. (Turns out,\nafter having an account and being logged in, the chat URL also worked\u2014what an\noversight on their part.) On the page, they had a field where you could enter\nthe washing machine's product number from the identification plate on the door\nof the washing machine, together with helpful information where to find the\ndata.<\/p>\n

\"Annotated<\/p>\n

But even better, they offered a service where you could just upload a picture of\nthe identification plate, and some AI on their server then extracted the product\nnumber and let you register the product with two clicks. What a fantastic\nexperience compared to the crappy (and likely for the operator way more\nexpensive) call center experience.<\/p>\n

\"Electrolux<\/p>\n

Why they didn't just put this URL on the brochure and the QR code is beyond me.\nAs the title suggests: For all that's holy, can you just leverage the Web,\nplease?<\/strong> Don't make me talk to people! They could still offer to register the\nmachine by telephone as an alternative, but in 2025, the default for such things\nshould just be the Web.<\/p>\n

Bonus \ud83d\udd17<\/a><\/h2>\n

Since I work on built-in AI<\/a> as\nmy day job in the Chrome team at Google, I could not not<\/em> notice this "extract\nthe product number from this identification plate"<\/em> use case for client-side AI.\nI coded up a quick\ndemo<\/a>\nusing the Prompt API<\/a> embedded\nbelow that shows this in action. Here's a quick walkthrough of the code:<\/p>\n

    \n
  1. Create a session with the LanguageModel<\/code>, informing the user of download\nprogress if the model needs to be downloaded, and telling the model about the\nto-be-expected inputs (English texts and images) and outputs (English texts).\nIn the system prompt, I tell the model what its overall task is (identify\nproduct numbers from photos of identification plates).<\/li>\n
  2. Prompt the model using the promptStreaming()<\/code> method with a multimodal\nprompt, one textual and one image. The Prompt API supports\nstructured output<\/a>\nin the form of a JSON Schema or regular expression. Product numbers have nine\ndigits, so I pass the regular expression \/\\d{9}\/<\/code> as the\nresponseConstraint<\/code> option.<\/li>\n
  3. Iterate over the chunks of the response. Since I'm just expecting nine\ndigits, this is probably a bit overkill, but, hey\u2026<\/li>\n
  4. (Not shown) On the server, verify that the recognized product number actually\nexist. Companies typically have some sort of verification rules like\nchecksums, or washing machine product numbers always start with 91<\/code> or\nsomething. If you know those rules, you can of course make them part of the\nresponseConstraint<\/code>, but you always need to verify untrusted user input\n(which the output of an LLM counts as) on the server.<\/li>\n<\/ol>\n
    const<\/span> session =<\/span> await<\/span> LanguageModel.<\/span>create<\/span>(<\/span>{<\/span>\n  monitor<\/span>(<\/span>m<\/span>)<\/span> {<\/span>\n    m.<\/span>addEventListener<\/span>(<\/span>'downloadprogress'<\/span>,<\/span> (<\/span>e<\/span>)<\/span> =><\/span> {<\/span>\n      console.<\/span>log<\/span>(<\/span>`<\/span>Downloaded <\/span>${<\/span>e.<\/span>loaded *<\/span> 100<\/span>}<\/span><\/span>%.<\/span>`<\/span><\/span>)<\/span>;<\/span>\n    }<\/span>)<\/span>;<\/span>\n  }<\/span>,<\/span>\n  expectedInputs<\/span>:<\/span> [<\/span>{<\/span> type<\/span>:<\/span> 'text'<\/span>,<\/span> languages<\/span>:<\/span> [<\/span>'en'<\/span>]<\/span> }<\/span>,<\/span> {<\/span> type<\/span>:<\/span> 'image'<\/span> }<\/span>]<\/span>,<\/span>\n  expectedOutputs<\/span>:<\/span> [<\/span>{<\/span> type<\/span>:<\/span> 'text'<\/span>,<\/span> languages<\/span>:<\/span> [<\/span>'en'<\/span>]<\/span> }<\/span>]<\/span>,<\/span>\n  initialPrompts<\/span>:<\/span> [<\/span>\n    {<\/span>\n      role<\/span>:<\/span> 'system'<\/span>,<\/span>\n      content<\/span>:<\/span>\n        'Your task is to identify product numbers from photos of identification plates.'<\/span>,<\/span>\n    }<\/span>,<\/span>\n  ]<\/span>,<\/span>\n}<\/span>)<\/span>;<\/span>\n\nconst<\/span> stream =<\/span> session.<\/span>promptStreaming<\/span>(<\/span>\n  [<\/span>\n    {<\/span>\n      role<\/span>:<\/span> 'user'<\/span>,<\/span>\n      content<\/span>:<\/span> [<\/span>\n        {<\/span>\n          type<\/span>:<\/span> 'text'<\/span>,<\/span>\n          value<\/span>:<\/span>\n            'Extract the product number from this identification plate. It has nine digits and appears after the text \"Prod.No.\".'<\/span>,<\/span>\n        }<\/span>,<\/span>\n        {<\/span> type<\/span>:<\/span> 'image'<\/span>,<\/span> value<\/span>:<\/span> image }<\/span>,<\/span>\n      ]<\/span>,<\/span>\n    }<\/span>,<\/span>\n  ]<\/span>,<\/span>\n  {<\/span>\n    responseConstraint<\/span>:<\/span> \/<\/span>\\d{9}<\/span>\/<\/span><\/span>,<\/span>\n  }<\/span>\n)<\/span>;<\/span>\n\nfor<\/span> await<\/span> (<\/span>const<\/span> chunk of<\/span> stream)<\/span> {<\/span>\n  console.<\/span>log<\/span>(<\/span>chunk)<\/span>;<\/span>\n}<\/span><\/code><\/pre>\n