INDUSTRY : Retail and Consumer


Object recognition technology uses machine learning and pattern recognition to detect and categorize specific objects within an image. The ability to identify specific objects in video or images opens up a myriad of opportunities for businesses, with potential uses ranging from purchasing, image classification, security and manufacturing production to self-driving cars and more.


More than three quarters (77 percent) of adults now own smartphones,1 and it is projected that more than 50 percent of consumers will have made purchases using their smartphones2 in 2017. Over the past several years, the increasing use of cameras on smartphones, tablets and other handheld devices in our everyday lives has changed how we interact both with the world around us and with the products we use. The act of photographing nearly every activity and product we consume has become ubiquitous. This has created an opportunity to disrupt the traditional customer purchasing experience and evolve to a new process by taking advantage of an activity most customers already do daily: snapping a picture.


PK built a Proof of Concept that demonstrates how object recognition technology and artificial intelligence can enhance the customer experience by enabling visual search using a smartphone or tablet camera. The “image-to-cart” technology used in the solution allows users to take a picture of an object, match it to a product for sale and then add it to their shopping cart or purchase it automatically.

By training the solution on a vast database of existing object images using convolutional neural networks, a machine can recognize objects within a photo taken by a customer and match it to the database. The solution then advances the customer to the designated next step of the purchasing process – adding it to the shopper’s cart or fully completing the purchase, for example.

With the global cart abandonment rate hovering around 75 percent3, removing several steps from the digital purchasing process has the potential to increase product sales by facilitating instant purchase. The PK solution demonstrates how the purchasing process can be simplified for those who do not know the name of an item, do not have time to search for the correct product or must rely on images from employees in the field to determine which part to purchase. As the purchasing process becomes increasingly digital, PK’s solution offers the opportunity for organizations to harness a common activity – taking a picture – to increase sales opportunities.

1 “10 facts about the smartphone as the iPhone turns 10.”  

2 “Most digital buyers will make purchases via a smart phone in 2017.”  

3 “How is cart abandonment and remarketing changing in 2017?”