Embedding
Dense, lower-dimensional vector representations that capture latent similarity
Intent & Description
🎯 Intent
Convert high-cardinality categorical data, text, images, or similar inputs into dense, lower-dimensional, continuous vectors that capture latent similarity structure.
📋 Context
One-hot or sparse encodings treat every distinct value as equally different and cannot represent similarity between categories. For high-cardinality features or complex inputs like text and images, sparse representations are large and inefficient.
💡 Solution
Map discrete or high-dimensional inputs into dense, lower-dimensional, continuous vectors, typically learned jointly with the rest of the model. Inputs that behave similarly end up close together in the vector space, compressing the representation while capturing similarity that raw sparse encodings can’t express.
Real-world Use Case
- Word embeddings for NLP tasks
- Product ID embeddings in recommendation systems
- User embeddings for personalization
- Category embeddings with semantic relationships
Source
📌 TL;DR
Learn dense vector representations that capture similarity between discrete inputs, compressing high-cardinality data while preserving semantic relationships
Advantages
- Compresses high-dimensional sparse data into dense vectors
- Captures latent similarity structure between inputs
- Enables generalization across similar categories
- More efficient than one-hot for high-cardinality features
Disadvantages
- Adds parameters to the model
- Needs sufficient examples per distinct value to learn meaningful vectors
- Less directly interpretable than raw features
- Requires tuning of embedding dimensionality