You only get one shot at a first impression, and listings with a clean white background have the potential to convert better and sell faster. Today, we’re introducing a new feature that uses computer vision technology to make sellers’ listing photos easier on the eyes and more effective in Google Shopping for both Android and iOS. The feature removes the background of a seller’s listing photo and replaces it with a white background, optimizing listings for Google Shopping, and improving the shopping experience for buyers by making search look and feel more streamlined.
eBay is an open marketplace where sellers can take their own photos and post them to the platform. We have millions of C2C and small business sellers who may not be able to use a professional photographer or have time for photo-editing software. Because of that, not all inventory is listed and photographed with optimal lighting or a clean white background. The resulting background — often clutter on a kitchen table, closet door or store shelf — creates noise and impacts results when a buyer shops using Image Search to find an item.
How It Works
Sellers can easily use this feature by going into the Sell flow from their mobile device. After they take or upload their photos within the eBay listing flow on their Android or iOS device, the seller can leverage the background removal tool to make a first pass at adding the white background for any of their photos. The seller can also touch up any missing areas or use the photo as is.
The Technology Behind Image Clean-up
Our computer vision algorithm processes the photo completely, using the processor on your mobile phone, to separate the foreground from the background clutter. Doing so enables us to change the background to a uniform white for a consistent look and feel.
The current approach is based on these assumptions:
● Pixels along the image border are predominantly background
● The foreground and background have sufficient contrast to indicate different coloring
We built color models for the foreground and background and solved for unknown pixels in the mask using conditional random fields. The output of the algorithm is a mask made up of the probability of foreground for each pixel. For example, when this is 100%, the pixel is entirely associated with foreground and when it is 0%, it is entirely associated with the background. This mask is then used to blend the foreground with a white image resulting in the desired image.
The confidence in background removal is measured by a factor that we call “separability,” which estimates how difficult it is to separate foreground from background. The closer the separability score is to the maximum value of 100%, the more likely the algorithm can easily separate foreground from the background. We use this to guide the tool on whether it should show an auto-cleanup result to the user or let the user do manual touchup to remove the background. Low contrast images and images with background clutter will typically produce a low separability score.
Images that score close to 100% on separability will trigger a bonus flow through Automatic Cleanup, in which the background removal tool will attempt to remove the background without seller input. The seller can then choose whether they want to use the photo as is, or make edits and do touch ups of their own.
The idea for Image Cleanup was conceived during eBay’s Hack Week, an annual company-wide competition challenging our technologists to innovate and reimagine the ecommerce experience. Leveraging the latest advances in computer vision and AI, we continue to work on additional features that will make our users’ lives easier.
This feature is now rolling out on Android and iOS devices in U.S., U.K. Germany and Australia and will be rolling out to all other regions in the next month.