Prof. Ravi Bapna is a Curtis L. Carlson Chair Professor in Business Analytics and Information Systems at Carlson School of Management, University of Minnesota, Minneapolis. Detailed profile can be found at here.
Carlson School of Management, University of Minnesota, Minneapolis
January 17, 2024
Design and Evaluation of New Product Category Recommendations: Evidence from a Randomized Field Experiment.
Recommending new product categories to existing consumers (i.e., categories that they have not yet pur- chased) can be useful for increasing customer lifetime value as well as for reducing risks from category-specific supply shocks and category-specific competition. In this paper, we design category-introduction-oriented recommendation methods to increase customers’ purchases from new product categories. We focus on appli- cation settings where the sales are highly concentrated, i.e., where the new category recommendation is particularly challenging. We use granular consumer journey data, employ comprehensive feature engineering and selection, and compare 15 recommendation models designed for new category introduction with robust offline evaluations. Then we estimate the causal economic impact of new category recommendations using a large-scale randomized controlled trial (RCT).
We find that the new product category recommendation can increase the purchase probability by up to 35% compared with no recommendation. We also explore two dimensions, namely, (i) increasing the choice in recommended new categories and (ii) providing personalized (as opposed to non-personalized) recommendations. We find that increasing choices further increases the sales in the recommended categories by up to 9% as compared to recommending a single category, and per- sonalized new category recommendation leads to 11% more purchases than recommending the most popular (non-personalized) new category. However, when recommending personalized new categories, more choices do not further increase sales as compared to recommending only one category. Finally, we go beyond standard average treatment effect analysis to discover customer heterogeneity. We find that the most recent visitors (who visit the platform within last a couple of days before the new category recommendation) are most responsive to multiple choices. In contrast, personalizing recommendations is more effective for not-so-recent customers, who visit the platform within three months before the treatment. A conditional average treat- ment effect treatment policy, which deploys the best treatment for different user segments, shows favorable lift in profit.
