User pain points
Wish has a large assortment of products; however, too many product offerings can exhaust customers, leading to frustration and choice overload. From the App Store reviews, “Filters” and “categories” are the top two feature requests in improving the shopping experience.
Experiment with filters and categories (V1)
To quantify the effectiveness of filters and categories, the features were added and I ran multiple experiments on the production system before the full deployment.
V1 failed: Gross merchandise volume (GMV) dropped
Why did filters fail?
When shoppers land on the homepage, they haven’t decided what they want to buy. From the data collected, filters only improve the GMV slightly
when shoppers already narrow down to a specific set of products.
Why did categories fail?
Categories are not effective in narrowing down millions of product offerings. Choice fatigue hurts sales.
V1 setbacks
With these setbacks, I collected and analyzed the conversion funnel and the analytical data further. The key objective was to dig through the production data and discover how
Wish shoppers found their products.
Searching is a popular channel for finding products in Wish
Searching is a natural starting point for Wish shoppers to find products.
- 46% of shoppers perform a search.
- The add-to-cart click-through rate for the search function is higher than browsing.
Shoppers perform 3 manual searches per session on average. However, many shoppers say the displayed search results are not what they want.
Analyze shopper searching patterns
To find further insight, I analyzed the raw queries of each user. I studied what they search, how they search, and the correlations between queries.
This study also generated a detailed report on how products should be grouped and organized. It also challenged what should be displayed on the top-level menus.
Tree-structure v.s. net-structure browsing
The analysis indicates the shopper searching pattern resembles a net-structure. This reinforces why v1 failed. Category follows a tree-structure and creates friction that causes some shoppers to drop off. So, how can we browse from one node to another in a net-structure effectively?
A net-based browsing UI: Tags
The real challenge was how to do it as simple as possible. Therefore, I introduced Search Tags similar to the Google Image search tags. After a shopper types in a search query, the UI also displays a sequence of tags that shoppers may be interested in. This created a UX flow similar to shopper browsing patterns.
Partnered with machine learning team
Unfortunately, the search tags generated in the first prototype by the backend server team was too similar to the category. Instead, I got the buy-in from the machine learning team and initiated a joint co-design project to tackle the problem. Our machine learning team knows well about applying state-of-the-art algorithms. But that is not enough. By being a strong advocate for users and the business owners, I ensured the joint-designed product was what shoppers and the business wanted. My first recommendation was to mine the search queries in shopping sessions to generate search tags.
Launched Search Tag (V2)
V2 machine learning model used historical searching queries to generate the search tags. The key objective of this release was to iterate fast and learn fast.
The following are the results of the quantitative and qualitative analysis:
Quantitative Metrics- The gross merchandise volume was slightly positive.
- Only 15% of first-time users noticed the search tag.
- This increased to 50% after users searched a few more times.
Qualitative Feedback through User Interviews
- Most users didn't see the search tags.
- Some search tags were not relevant. I searched “Watches for women,” but search results returned “Men watches” and “Solar gadgets.”
- Some displayed products were not relevant; for example, it showed many men’s watches when I searched for women’s watches.
Co-designed with machine learning engineers
With the positive v2 result, I would review the quality of the search tags and the search results in each model training iteration. I would identify high ROI issues and recommended changes accordingly. For example, I suggested the search results be regional and season sensitive. I also suggested finding products with titles that are semantically similar rather than using the traditional keyword match. This motivated the engineers to change the algorithm towards Natural Language Processing (NLP) based solutions. I also suggested the application of computer vision algorithms in reducing the number of searching results that are visually similar.
As a demonstration of my involvement, here is one concept that I pitched in generating higher quality search tags.
Tag UI explorations
To improve the visibility of the search tag, I experimented with different visual elements and where it displays. For example, I worked with data analytics to locate where mobile users were dropped off and placed the search tags there.
Benchmark
To gauge the quality of our machine learning model, I decided to establish an external benchmark to measure the quality of the search tags. For the same search query, I present the search tags from the Google Image Search and the Wish model to shoppers for comparison. In general, shoppers preferred the tags from Google, in particular, they were more relevant and less redundant. I also ran experiments using the tags generated by Google instead of Wish in the production. While the click rate increased, the GMV dropped. This was because the Wish model was not trained to make good product recommendations for these new search tags yet. Nevertheless, all these findings gave me valuable information on how to improve the machine learning model.
Google Image Tag
Wish Tag
