Sharding Strategies
How you partition data across nodes determines hotspot risk, query complexity, and rebalancing cost. Range, hash, directory, consistent hashing, and geo sharding.
Intent & Description
🎯 Intent
Choose the right data partitioning strategy to balance load distribution, query performance, and operational complexity.
📋 Context
Sharding determines how data is distributed across nodes. Range sharding enables efficient range queries but creates hotspots on monotonic keys. Hash sharding distributes evenly but requires scatter-gather for range queries. Directory sharding offers flexible routing but creates a lookup bottleneck. Consistent hashing minimizes reshuffling. Geo sharding provides low latency for regional users.
💡 Solution
Choose shard key before writing data — nearly impossible to change later. Avoid monotonically increasing keys under write-heavy loads — add prefix hash. Plan for rebalancing from day one with consistent hashing and virtual nodes. Use composite keys for complex access patterns.
Real-world Use Case
📌 TL;DR
Sharding strategy determines data distribution and query patterns. Hash = even distribution, no range queries. Range = efficient ranges, hotspots risk. Consistent hashing = minimal rebalancing. Choose based on access patterns.
Advantages
- Each strategy optimized for different access patterns
- Hash sharding prevents hotspots
- Range sharding enables efficient range queries
- Consistent hashing minimizes rebalancing impact
Disadvantages
- Shard key choice is critical and hard to change
- Range sharding creates hotspots on monotonic keys
- Hash sharding prevents efficient range queries
- Rebalancing is complex and operationally challenging
// Sharding Strategies
class ShardManager {
// Hash Sharding: Even distribution
getHashShard(key, numShards) {
const hash = this.hash(key);
return hash % numShards;
}
// Range Sharding: Efficient range queries
getRangeShard(key, ranges) {
return ranges.find(r => key >= r.start && key <= r.end).shardId;
}
// Consistent Hashing: Minimal reshuffling
getConsistentHashShard(key, ring) {
const hash = this.hash(key);
// Find next node in ring clockwise
for (const node of ring.sort((a, b) => a.position - b.position)) {
if (hash <= node.position) return node.shardId;
}
return ring[0].shardId; // Wrap around
}
// Composite Key: Hash + Range for hybrid access
getCompositeShard(key, timestamp, numShards) {
const hashPrefix = this.hash(key) % (numShards / 10);
const timeRange = Math.floor(timestamp / 86400000) % 10;
return hashPrefix * 10 + timeRange;
}
}