Sometimes, even as a technology reporter, you may be surprised at how quickly technology improves. Case in point: I only learned today that my iPhone offers a feature I’ve been wanting for a long time: the ability to identify plants and flowers from a photo.
True, several third-party apps have been offering this feature for years, but the last time I tried them, I was disappointed by their speed and accuracy. And yes, there are Google Lens and Snapchat Scan, but it’s always less convenient to open an app that you wouldn’t use otherwise.
But since the introduction of iOS 15 last September, Apple has offered its own version of this visual search feature. It’s called Visual Look Up, and it’s great.
It works very easily. Just open a photo or screenshot in the Photos app and look for the blue “i” icon below. If you have a little shiny ring around, then iOS has found something in the photo that you can identify by machine learning. Tap the icon, then click “Search Up” and it will try to retrieve useful information.
Touching the “i” icon usually gives you more information about when you took the photo and the camera settings. If the ring shines, however, there is also Visual Look Up data to see. After clicking on the “i” icon, you will have the option to search for more information based on selected categories.
Nor does it work only for plants and flowers, but for landmarks, art, pets, and “other objects.” It’s not perfect, of course, but it surprised me more than the times that it didn’t disappoint me. Here are some more examples just from my camera cart:
Visual Look Up works for landmarks, animals and art, as well as plants and flowers. Image: The Verge
Although Apple announced this feature last year at WWDC, it has not been precisely announcing its availability. (I saw it through a link in one of my favorite tech newsletters, The Overspill.) United States “and then listing other supported regions elsewhere. page.
Visual Look Up still has limited availability, but access has been expanded since launch. It is now available in English in the US, Australia, Canada, the UK, Singapore and Indonesia; in French in France; in German in Germany; in Italian in Italy; and in Spanish in Spain, Mexico and the USA.
It’s a great feature, but it also makes me wonder what else visual search could do. Imagine taking a picture of your new indoor plant, for example, just for Siri to ask “Do you want to set reminders for an irrigation program?” – or, if you’re taking a photo of a holiday milestone, have Siri search the web for opening hours and where to buy tickets.
I learned a long time ago that it is nonsense to put your hopes in Siri by doing something too advanced. But these are the kinds of features we could get with future AR or VR headsets. Hopefully, if Apple introduces this type of functionality, it will add a bigger touch.