Many of the changes brought on by Apple Intelligence are centered around the camera and images on your iPhone, like making a movie out of your stored photos or generative image editing. But natural language search, which lets you find photos by describing them using regular language, was one of our favorites.
In a new blog post though, developer Jeff Johnson has shed light on some privacy concerns around this feature which was intended to help people use their iPhones to more easily search for landmarks and other points of interest. Many Apple users have have already updated their devices to iOS 18 without being aware that it comes equipped with a photo feature called Enhanced Visual Search.
Here’s how it works: the device detects a point of interest in a photo – the example getting thrown around is the Eiffel Tower – then on-device AI creates a mathematical fingerprint or what Apple has described as “a vector embedding is calculated for that region of the image.”
This vector embedding or fingerprint is then sent to Apple’s servers, with a relay in the middle for privacy alongside fake queries so the server doesn’t know which one is legit. Cupertino’s servers then return possible matches in an encrypted form.
While Apple’s explanations have assured users this is all done while keeping photo data private, Johnson sees this as a privacy violation that was introduced silently and without consent. He writes in his blog: “I never requested that my on-device experiences be ‘enriched’ by phoning home to Cupertino… If something happens entirely on my computer, then it’s private, whereas if my computer sends data to the manufacturer of the computer, then it’s not private, or at least not entirely private.”
The Enhances Visual Search feature is part of the iOS 18 and macOS 15 release from September 16, 2024 and the company’s overview of the features at that time did not explain or mention this tradeoff.
They did say that search would now include “natural language queries and expanded understanding” but that doesn’t really explain to users that their photo data is being sent back to Apple’s servers by default.
On October 24th though, there was a brief explanation in a highly technical Apple document on machine learning and homomorphic encryption with a reference to EVS and privacy, explaining the feature allows for searching for landmarks or points of interest. There’s also a legal document which offers more information. However, the reality is that few users access or read these notices and fewer yet understand the information contained within them.
How to turn the Enhanced Visual Search (EVS) feature off
If you’re worried about one of the best iPhones sending your photos and the data they contain back to Apple, Enhanced Visual Search is easy enough to turn off.
Keep in mind though, this feature comes enabled by default, so you may also want to let your privacy-conscious friends and family know about it too. You may have to provide tech support for some of your older relatives but here’s how to turn it off on your iPhone:
You can opt out of the EVS feature anytime by going to Settings > Apps > Photos. From there, when you scroll all the way to the bottom you will see an option to toggle Enhanced Visual Search on or off.
If you’re using one of the best MacBooks or even a Mac mini M4, you can do the same thing by opening the Photos app, clicking on Settings within the app and in the General tab you’ll find the option to toggle Enhanced Visual Search on or off.
Keep in mind though, that if there is a landmark or other point of interest you do want to use this feature with, you can always reenable it by following the steps detailed above in reverse.
Hopefully, we’ll see Apple address the fact that Enhanced Visual Search sends photo data back to its servers sometime soon. In the meantime though, at least you now know how to to turn off this feature that comes enabled by default in iOS 18.
