QBO 3p ideal state screen

WISH RECOMMENDATION

User Goal

Discover products "quick & easy".

Business Goal

Increase add-to-cart volume, conversion rate, and gross merchandise value.

Contribution

Pitched the concept to the leadership team; guided the execution strategy in risk mitigation; executed user research, data analysis, UI/UX design; and refined the recommendation model with machine learning engineers.

Results

Increased 3+% gross merchandise value.

User pain points

Wish has a large assortment of products; however, too many product offerings can exhaust customers, leading to frustration and choice overload. From the App Store reviews, “Filters” and “categories” are the top two feature requests in improving the shopping experience.

App Store Review

Experiment with filters and categories (V1)

To quantify the effectiveness of filters and categories, the features were added and I ran multiple experiments on the production system before the full deployment.

Wish Filters Wish categories

V1 failed: Gross merchandise volume (GMV) dropped

Why did filters fail?
When shoppers land on the homepage, they haven’t decided what they want to buy. From the data collected, filters only improve the GMV slightly when shoppers already narrow down to a specific set of products.

Why did categories fail?
Categories are not effective in narrowing down millions of product offerings. Choice fatigue hurts sales.

V1 setbacks
With these setbacks, I collected and analyzed the conversion funnel and the analytical data further. The key objective was to dig through the production data and discover how Wish shoppers found their products.

Searching is a popular channel for finding products in Wish

Searching is a natural starting point for Wish shoppers to find products.

Shoppers perform 3 manual searches per session on average. However, many shoppers say the displayed search results are not what they want.

pie chart

Analyze shopper searching patterns

To find further insight, I analyzed the raw queries of each user. I studied what they search, how they search, and the correlations between queries.

Wish Shopper Search Pattern

This study also generated a detailed report on how products should be grouped and organized. It also challenged what should be displayed on the top-level menus.

Tree-structure v.s. net-structure browsing

The analysis indicates the shopper searching pattern resembles a net-structure. This reinforces why v1 failed. Category follows a tree-structure and creates friction that causes some shoppers to drop off. So, how can we browse from one node to another in a net-structure effectively?

Search pattern Findings

A net-based browsing UI: Tags

The real challenge was how to do it as simple as possible. Therefore, I introduced Search Tags similar to the Google Image search tags. After a shopper types in a search query, the UI also displays a sequence of tags that shoppers may be interested in. This created a UX flow similar to shopper browsing patterns.

 Competitor UI

Partnered with machine learning team

Unfortunately, the search tags generated in the first prototype by the backend server team was too similar to the category. Instead, I got the buy-in from the machine learning team and initiated a joint co-design project to tackle the problem. Our machine learning team knows well about applying state-of-the-art algorithms. But that is not enough. By being a strong advocate for users and the business owners, I ensured the joint-designed product was what shoppers and the business wanted. My first recommendation was to mine the search queries in shopping sessions to generate search tags.

Launched Search Tag (V2)

V2 machine learning model used historical searching queries to generate the search tags. The key objective of this release was to iterate fast and learn fast.

Launched V1

The following are the results of the quantitative and qualitative analysis:

Quantitative Metrics
Qualitative Feedback through User Interviews

Co-designed with machine learning engineers

With the positive v2 result, I would review the quality of the search tags and the search results in each model training iteration. I would identify high ROI issues and recommended changes accordingly. For example, I suggested the search results be regional and season sensitive. I also suggested finding products with titles that are semantically similar rather than using the traditional keyword match. This motivated the engineers to change the algorithm towards Natural Language Processing (NLP) based solutions. I also suggested the application of computer vision algorithms in reducing the number of searching results that are visually similar.

Machine Learning

As a demonstration of my involvement, here is one concept that I pitched in generating higher quality search tags.

Machine Learning involvement

Tag UI explorations

To improve the visibility of the search tag, I experimented with different visual elements and where it displays. For example, I worked with data analytics to locate where mobile users were dropped off and placed the search tags there.

UI1 UI2 UI3

Benchmark

To gauge the quality of our machine learning model, I decided to establish an external benchmark to measure the quality of the search tags. For the same search query, I present the search tags from the Google Image Search and the Wish model to shoppers for comparison. In general, shoppers preferred the tags from Google, in particular, they were more relevant and less redundant. I also ran experiments using the tags generated by Google instead of Wish in the production. While the click rate increased, the GMV dropped. This was because the Wish model was not trained to make good product recommendations for these new search tags yet. Nevertheless, all these findings gave me valuable information on how to improve the machine learning model.

Google Image Tag

Google tags VS Wish Tags

Wish Tag

Google tags VS Wish Tags

Launched V3 with better quality search tags

Results

Future improvement

Key focus in the future will be:

Takeaways

Don’t just pass off responsibility to the machine learning engineers. Instead, work collaboratively to improve the models iteratively. The success of the project depends on it.

Takeaways

Previous Project                     Next Project