Windows’ new Recall feature: A privacy and security nightmare?

Microsoft has announced the Copilot+ line of Windows 11-powered PCs that, among other things, will have Recall, a feature that takes screenshots every few seconds, encrypts them, saves them, and leverages AI to allow users to search through them for specific content that has been viewed in apps, websites, documents, etc.

What could possibly go wrong?

About Windows Recall

“Once you find the snapshot that you were looking for in Recall, it will be analysed and offer you options to interact with the content. What actions you can take depend on the content and the chat provider capabilities in Copilot in Windows. For example, you may highlight a block of text and decide to summarise it, translate it, or open it with a text editor like Word or Notepad. If you highlight an image, you will be able to edit it or use your chat provider in Copilot in Windows to find or create a similar image,” Microsoft explains.

“Recall will also enable you to open the snapshot in the original application in which it was created, and, as Recall is refined over time, it will open the actual source document, website or email in a screenshot.”

Windows 11 Recall settings

These are the most salient details related to this new feature (as outlined by Microsoft):

• The screenshots are stored only on the device, and are protected “using data encryption on your device and (if you have Windows 11 Pro or an enterprise Windows 11 SKU) BitLocker”
• The screenshots are not sent to or used by Microsoft for ad targeting
• “Screenshots are only available to the person whose profile was used to sign in to the device. If two people share a device with different profiles they will not be able to access each other’s screenshots. If they use the same profile to sign in to the device then they will share a screenshot history.”
• Users can prevent Recall from collecting content from specific apps or websites visited in a supported browser. Recall does not take snapshots of InPrivate web browsing sessions in Microsoft Edge, DRM-protected content, but “will not hide information such as passwords or financial account numbers, (…) especially when sites do not follow standard internet protocols like cloaking password entry.”
• Users can pause the creation of snapshots, delete them, and switch Recall off

The possible security and privacy pitfalls of Windows Recall

Security and privacy professionals, who are used to viewing technological solutions and new features through an attacker’s lens, have immediately pinpointed its possible disadvantages.

Kevin Beaumont thinks the feature will be a boon for criminals using information-stealing malware. “The whole thing is a truly terrible idea that will allow AI to super power fraudsters – just steal the Recall database, instead of just the local browser password database,” he noted.

Even if the database can’t be exfiltrated, there are info-stealers out there that use optical character recognition (OCR) to extract sensitive text from images/screenshots.

Beaumont also pointed out that while Recall can be switched off, it can also be covertly turned on by threat actors with Powershell.

“Microsoft are inventing a new security nightmare using Copilot, which will undoubtedly lead to increased fraud for consumers and other woes for businesses,” he opined.

But there are also other potential security, safety and privacy issues that may arise.

For one: How much do you trust your partner / family / roomate / coworkers (or even your government)? If they manage to get your computer password, they can easily search through your snapshots for other passwords, sensitive data, etc.

Enterprises should also consider that the feature may “recall” confidential business data.

And then: How much do you trust Microsoft? The company is not using Recall snapshots now, but that can easily change in the future. They might want to use the feature to train AI, for example.

Earlier this month, Microsoft CEO Satya Nadella sent a memo to employees asking them to prioritize security above all else, even releasing new features.

I guess time will tell whether it was just an effort to score some good publicity in the wake of the breaches the company suffered by the hands of suspected state-sponsored hackers in past years, and the scathing report on the company’s defenses by the CISA’s Cyber Safety Review Board.