How AI is changing the in-store checkout experience

One common thread behind the new wave of checkout technologies is computer vision, or vision AI. This type of artificial intelligence that allows computers to recognition based on images. Computer vision lays the groundwork for systems that can recognize a handprint, or cameras that can identify a specific item.

Al Sambar, general partner at XRC Ventures and manager of the Opportunity Fund that invests in early stage companies in the checkout space, said its hard to find a retailer that’s not experimenting with contactless payment in some form. But he’s betting on vision AI-based systems as a leading technology choice, because of its ability to recognize something that doesn’t have a barcode, eliminating the need for scanning.

“We start making our investments early, even if it takes five or six years to where it can achieve mass adoption,” he said.

Though one technology has yet to dominate the retail sector, there’s ample room for experimentation: Only about 13% of shoppers say they use cash for most purchases, according to a Gallup survey in August 2022. And more than six in 10 shoppers say they believe the U.S. will become cashless in their lifetime. 

The transition is full of experimental technologies from both startups with bold ideas and major players deploying new products. Nili Klenoff, svp global head of e-commerce and in-store authentication solutions at Mastercard, said that contactless checkout represents at least 60% of all in-person transactions, more than double what it was three years ago.  

In response to the rising interest, Mastercard last fall piloted a biometric payment technology at five supermarkets in Brazil that allowed customers to check out using their smile, palm or eye. This year, it’s furthering testing in the Middle East and Asia, Klenoff said.

“I see so much potential for biometrics in the in payments and beyond payments,” Klenoff said. “That’s the physical world, the digital world or the world of tomorrow, like the metaverse.”

Here’s how vision AI-based systems — as well as other emerging technology like biometrics — are changing the checkout landscape.

Checking out via biometrics

In general, biometrics work by a person scanning a part of their body to recognize who they are, which is connected to their payment method. But this can raise privacy concerns — about 71% of customers, according to Mastercard in June 2022, had concerns about their information being saved. In response, Mastercard’s system deletes the scan immediately in order to protect a shopper’s identity, Klenoff said, meaning in the event of a data breach, the payment information could not be connected to a specific user.

“That was a program that really set standards for security and data privacy and biometric performance for a frictionless biometric checkout experience,” Klenoff said.

Ultimately, the more biometrics uses that crop up, the more comfortable shoppers will get with them. Klenoff points to how unlocking a phone with biometrics — such as Apple’s FaceID — makes the technology commonplace.

“Those are the the experiences that consumers have been using for a long time. And we think those are going to be those tipping points for why consumers are going to embrace biometrics as a regular form of payment in store,” she said. “They’ve already been using it.”

Shoppers can expect to see more use cases rolling out in the future; this month, Amazon One announced a rollout in 500 Whole Foods Market stores. The “palm recognition service” uses computer vision algorithms to scan the custom’s palm image, capturing a unique signature that’s associated with their payment information. The expansion more than doubles the availability of Amazon One, which is used in about 400 locations including Amazon Fresh stores, plus Panera Bread locations and Coors Field in Denver.

So far, the service has been used more than three million times, according Amazon.

AI powers retailers’ emerging tech

Sambar from XRC Ventures said that as important as it is for shoppers to warm up to contactless payments, retailers need to be able to install the systems easily in order to have them used.

“In part, the entire barrier to adoption is that the enterprises had invested hundreds of millions or billions into the point of sale system,” he said. “You need solutions that don’t require them to replace all those in one shot.”

One of the Opportunity Fund’s investments is in Veeve, a smart cart checkout system that uses cameras enabled with vision AI to sense what’s being placed in the cart. The device clips onto a shopping cart, automatically scans what shoppers are putting in the cart, and can accept payments. Aggregated data from Veeve shows that shoppers tend to spend an additional 10% to 15% more when using the technology.

Meanwhile, delivery service Instacart is also relying on computer vision to fuel its a Connected Stores platform. Launched in September 2022, Instacart’s Caper Cart uses a system of cameras and scales in its AI-powered technology. Customers checkout by placing items in the cart that to sense what items are added or removed.

David McIntosh, vp and gm of Connected Stores at Instacart, said the platform was launched based on feedback from users who liked the personalization aspects of online delivery, but still wanted to go the store. A third iteration of the Caper Cart, which is larger and lighter than previous models, debuted this summer at a ShopRite in New Jersey and is now rolling out in Schnucks locations.

McIntosh said checkout experiences like Caper Cart can also provide broader benefits to retailers by becoming a tool for omnichannel shopping. For example, customers can enter their phone numbers for a loyalty program, and have their shopping history data saved for future online purchases. 

“This is really all about unifying that in store and online experience and putting those pieces together.”

But retailers must be able to integrate the new methods into their stores in a way that works for associates, McIntosh said, or else the systems won’t take off. One advent in the new version of the Caper Cart is a connected charging system, where the carts can simultaneously charge when pushed into one another. The system also alerts associates when it senses suspicious behavior, like someone blocking the cameras. 

“If you don’t have an experience that plugs into the operations of the store and can sort of easily retrofit the store, it’s just not going to scale,” he said.