Enhance Recommendations Based on Embeddings

Enhance Recommendations Based on Embeddings

Table of Contents

1. Introduction
2. Embeddings in Recommendation Systems
   • 2.1 Why Use Embeddings for Recommendations?
   • 2.2 Types of Embedding Models
3. The Frequency Both Together Recommendation Approach
   • 3.1 Challenges in Recommendation Diversity
   • 3.2 Dealing with Scale in Recommendations
4. The Vertweight Model for Embedding Generation
   • 4.1 Data Preparation for Embedding Generation
   • 4.2 Important Parameters in Vertweight Model
5. Evaluation Metrics for Embedding Models
   • 5.1 Offline Metrics
   • 5.2 Online Metrics
   • 5.3 Tracking Model Performance with MLflow
6. Using Embeddings for Arithmetic Operations
   • 6.1 Calculating Brand Similarities
   • 6.2 Caveats of Using Embeddings for Category Representations
7. The Architecture of a Recommendation System
   • 7.1 Experimental Phase
   • 7.2 Production Phase
8. Tricks for Efficient Model Training and Indexing
   • 8.1 Parallelizing Embedding Creation with PySpark and Pandas UDF
   • 8.2 Choosing the Nearest Neighbor Algorithm Library
   • 8.3 Considerations for Building Nearest Neighbor Trees
9. Implementing Post Filtering in the Serving Layer
   • 9.1 Importance of Post Filtering in Recommendations
   • 9.2 Sample Filtering Functions
10. Ensuring Performance and Scalability in the Serving Layer
    • 10.1 Load Testing for Low Latency and High Traffic Handling
    • 10.2 Evaluating Success with Conversion, Coverage, and Revenue Metrics
11. Conclusion and Final Thoughts

Introduction

Welcome to our presentation on the production use case of embedding basic recommendations. In this article, we will discuss the modeling stage, general architecture, and implementation details of a recommendation system that utilizes embeddings for generating relevant and complementary product recommendations.

Embeddings in Recommendation Systems

2.1 Why Use Embeddings for Recommendations?

Embeddings offer a way to represent items in a low-dimensional space, allowing us to capture the semantical relationships between items. By employing embeddings, we can overcome the limitations of co-occurrence-based recommendations and generate more accurate and diverse recommendations.

2.2 Types of Embedding Models

There are different methods to generate embeddings, such as ResNet, Verbatim Drag Bear, etc. These models are useful for content-based recommendations where image or text data is used. Additionally, embeddings can be concatenated to represent different dimensions of the same item, combining embeddings generated from image, text, and behavior data.

The Frequency Both Together Recommendation Approach

3.1 Challenges in Recommendation Diversity

One of the main goals of the frequency both together recommendation approach is to recommend complementary products rather than similar ones. However, the diversity of sequences poses a challenge as products from semantically different categories are included. Post-filtering is necessary to eliminate recommendations that may seem unrelated based on brand, price, gender, and category.

3.2 Dealing with Scale in Recommendations

When generating recommendations using more than 30 million products distributed over 40 categories, handling scale becomes crucial. Limiting the length of sequences and choosing optimal parameters for index creation and performance metrics are important considerations to ensure relevancy and scalability in recommendation generation.

The Vertweight Model for Embedding Generation

4.1 Data Preparation for Embedding Generation

User behavior data such as product views, adds to cart, and orders are considered as sentences in the data preparation phase. Each set of purchased items corresponds to an observation in the training set. Sequences containing products from diverse categories are split into subsequences to decrease noise and support recommendation relevancy.

4.2 Important Parameters in Vertweight Model

Parameters such as mean count, network size, and window affect coverage, storage, and computational cost. Careful consideration of these parameters is required to strike a balance between coverage and resource consumption. Evaluation metrics like precision, recall, and hit rate are used to measure the performance of the model.

Evaluation Metrics for Embedding Models

5.1 Offline Metrics

Offline metrics are measured during the training and indexing phases of the model. Precision, recall, and hit rate are commonly used metrics to evaluate the performance of embedding models. Standard evaluation metrics reflect the general picture of the model's performance and help in decision-making.

5.2 Online Metrics

Online metrics are measured after the production phase to track the real effect of the recommendation model. Continuous tracking of online metrics is essential for performance enhancements and improvement in recommendations. Superset and Apache Hive can be used to visualize and analyze online metrics.

5.3 Tracking Model Performance with MLflow

MLflow provides a user-friendly interface to track parameters and evaluation metrics of the model. It allows teams to collaborate effectively and store experiment information in a central server. MLflow's integration enables easy evaluation and enhancement of the model's performance.

Using Embeddings for Arithmetic Operations

6.1 Calculating Brand Similarities

Embeddings can be used to calculate brand similarities by performing arithmetic operations on the embeddings of brand and product. However, it is important to note that the accuracy of brand similarities depends on the representation of categories in the embeddings. Brands with homogeneous products yield more accurate results compared to brands with diverse categories.

6.2 Caveats of Using Embeddings for Category Representations

Representing categories using embeddings of low-level entities, such as product embeddings, can lead to decreased accuracy. Careful consideration of category representations is required to ensure accurate recommendations. Embeddings based on average product embeddings might yield better results for category representations.

The Architecture of a Recommendation System

7.1 Experimental Phase

The recommendation system follows a two-phase approach: the experimental phase and the production phase. During the experimental phase, data preparation, model training, offline metrics measurement, and metadata generation are performed. Metadata includes category hierarchy, gender, price, and brands, which are used for post-filtering actions.

7.2 Production Phase

After successful modeling and indexing, the recommendation model is put into the production system. The production phase entails the creation of a binary file from the indexing process. This binary file is used by the API to query recommendations. Continuous tracking of online metrics is done through Superset and Apache Hive to evaluate the model's performance in the production environment.

Tricks for Efficient Model Training and Indexing

8.1 Parallelizing Embedding Creation with PySpark and Pandas UDF

To overcome the time consumption in creating embeddings, parallelization using PySpark and Panda UDF is utilized. This approach distributes the embedding creation process across multiple nodes, significantly reducing the time required for embedding generation.

8.2 Choosing the Nearest Neighbor Algorithm Library

The choice of the nearest neighbor algorithm library is crucial for efficient recommendation generation. The benchmarks and performance overview of different libraries, such as HNSW, help in making an informed decision. Considerations like complexity, recall value, and resource consumption play a vital role in choosing the optimal library.

8.3 Considerations for Building Nearest Neighbor Trees

Building the nearest neighbor tree using HNSW library requires consideration of various parameters. Simplifying the tree can reduce resource consumption but leads to a poor recall value. Increasing the complexity of the tree improves recall but may cause a waste of time and resources. Finding the optimal balance is essential for efficient recommendation generation.

Implementing Post Filtering in the Serving Layer

9.1 Importance of Post Filtering in Recommendations

Post filtering is crucial in recommendation systems as it allows the elimination of recommendations that may seem unrelated based on brand, price, gender, and category. Including metadata in the canon index enables the application of various post-filtering techniques to enhance the relevancy of recommendations.

9.2 Sample Filtering Functions

Implementing post-filtering functions based on metadata, such as gender, category, brand, and price, helps in refining the recommendation list. Filtering recommendations based on the recommendation context and user preferences improves the overall recommendation quality.

Ensuring Performance and Scalability in the Serving Layer

10.1 Load Testing for Low Latency and High Traffic Handling

For recommendation systems serving millions of customers, performance and scalability are paramount. Load testing of the API application ensures low latency and high traffic handling capabilities. Monitoring performance metrics and response times help in identifying bottlenecks and optimizing the serving layer.

10.2 Evaluating Success with Conversion, Coverage, and Revenue Metrics

Evaluation of a recommendation system requires measuring conversion rates, coverage, and revenue related metrics. Grouping metrics based on different dimensions, such as placement, title, channel, and gender, helps understand the impact of the recommendations accurately. Careful analysis of popular products and category context is necessary for effective evaluation.

Conclusion and Final Thoughts

Embeddings play a vital role in recommendation systems by capturing semantical relationships between items. The frequency both together recommendation approach ensures the provision of complementary products rather than similar ones. Effective model training, optimum parameter tuning, and post-filtering techniques enhance the accuracy and relevancy of recommendations. Continuous evaluation and tracking of offline and online metrics help in improving the recommendation system's performance and scalability.