Materialized Views
Automate denormalization and enable flexible querying without manual table duplication!
📖 The Story: Emma's Denormalization Nightmare
Emma built a movie review platform with millions of reviews. Users want to view reviews by movie AND by user. She tried three approaches to handle this dual-access pattern...
❌ Attempt 1: Secondary Index (Too Slow!)
Problems:
- 🔥 Coordinator contacts ALL 50 nodes
- ⏱️ Query takes 2-3 seconds under load
- 💥 Timeout errors during peak traffic
- 😡 Users complain about slow page loads
⚠️ Attempt 2: Manual Denormalization (Maintenance Hell!)
Problems:
- 🐛 Application code becomes complex (BATCH everywhere)
- 💥 Bugs! Forgot to update one table → data inconsistency!
- 📝 Every developer must remember to update both tables
- ⏱️ Code reviews become nightmares
- 🔥 Production incident: Data out of sync!
✅ The PERFECT Solution: Materialized Views!
Benefits:
- ✅ Automatic updates: Cassandra maintains views!
- ✅ Simple code: Write to base table only!
- ✅ Always consistent: No bugs from forgetting to update!
- ✅ Blazing fast: Both queries return in 5ms!
- ✅ No application complexity: Database handles it!
Emma's app is now fast, reliable, and easy to maintain! 🎉
🎬 What are Materialized Views?
Automatically maintained denormalized tables that stay in sync with the base table!
Simple Definition
Materialized View (MV): A read-only table that Cassandra automatically updates when the base table changes.
Think of Materialized Views as:
- 🪞 Magic Mirror: Automatically reflects changes from base table
- 🔄 Auto-Sync Tables: Cassandra handles all updates for you
- 📋 Smart Copy: Different primary key, always up-to-date
- 🎯 Query Shortcuts: Fast access patterns without code complexity
How Materialized Views Work
When to Use Materialized Views
Perfect For
- Multiple Access Patterns: Query same data by different keys
- Read-Heavy Workloads: More reads than writes
- Simple Denormalization: Just changing primary key
- Automatic Consistency: Need guaranteed sync
- Reduce Code Complexity: Avoid manual BATCH statements
Avoid For
- Write-Heavy Tables: High INSERT/UPDATE volume (slow!)
- Many MVs on One Table: > 3 MVs = performance hit
- Frequent Schema Changes: MVs harder to modify
- Complex Transformations: Can't filter or aggregate
- Large Base Tables: Initial MV build takes time
Trade-offs
- Write Performance: -20% to -40% on base table
- Storage: Each MV doubles storage for that data
- Consistency: Eventually consistent (slight delay)
- Limitations: Can't filter rows, only reorganize
- Build Time: Large tables take hours to populate
🔨 Creating Materialized Views
Master the syntax and rules for creating materialized views!
Basic Syntax
Critical MV Rules
- Include ALL Primary Key Columns: Base table PK must be in MV WHERE and PRIMARY KEY
- IS NOT NULL Required: All PK columns must have IS NOT NULL in WHERE
- Static Columns: Can only be included if partition key is the same
- No Filtering: Can't use WHERE with conditions other than IS NOT NULL
- No Aggregations: Can't use COUNT, SUM, AVG, etc.
Complete Example: E-Commerce Orders
Selecting Specific Columns
Dropping Materialized Views
🏗️ Real-World Materialized View Examples
Production-ready MV patterns from real applications!
Example 1: Social Media - Posts by User and Hashtag
Example 2: IoT Sensors - Data by Sensor and by Location
Example 3: E-Learning - Courses by Instructor and Category
Example 4: Event Tracking - Events by User and by Type
⭐ Materialized View Best Practices & Limitations
Learn what works, what doesn't, and how to optimize MVs!
DO's
- Limit MVs per Table: Max 3-5 MVs (performance!)
- Use for Read-Heavy: More reads than writes
- Include Base PK: Always in WHERE and PRIMARY KEY
- Monitor Build Progress: Large tables take time
- Test Performance Impact: Measure write latency
- Use Appropriate Replication: Same or less than base
- Document Your MVs: Explain query patterns
DON'Ts
- Don't Create Too Many: > 5 MVs = slow writes
- Don't Use for Writes: MVs are read-only!
- Don't Filter Rows: Can't use WHERE col = value
- Don't Aggregate: No COUNT, SUM, AVG
- Don't Use on Write-Heavy: > 10K writes/sec
- Don't Expect Instant: Eventually consistent
- Don't Ignore Write Cost: -20% to -40% throughput
Pro Tips
- Build During Low Traffic: Initial population is heavy
- Select Only Needed Columns: Smaller = faster
- Use Compound Keys: (status, date) for bucketing
- Monitor Repair: Ensure MVs stay in sync
- Consider Manual Denorm: For write-heavy tables
- Test on Production Load: Simulate real traffic
- Plan for Rebuilds: DROP + CREATE if corrupt
Major Limitations
❌ Limitation #1: Cannot Filter Rows
You can only reorganize data, not filter it!
❌ Limitation #2: Cannot Aggregate Data
No COUNT, SUM, AVG, GROUP BY, or computed columns!
❌ Limitation #3: Write Performance Impact
Each MV adds 20-40% overhead to write operations!
Performance Impact
- 1 MV: -20% write throughput
- 2 MVs: -35% write throughput
- 3 MVs: -50% write throughput
- 5+ MVs: -70%+ write throughput (not recommended!)
Solution: Limit to 3-5 MVs maximum per table.
❌ Limitation #4: Eventually Consistent
MVs update asynchronously - slight delay possible!
❌ Limitation #5: Static Column Restrictions
Can only include static columns if partition key is the same!
⚡ Performance Considerations
Optimize MV performance for production workloads!
📊 Write Performance Impact
Every MV adds overhead because Cassandra must:
- Parse the write: Determine which MVs are affected
- Generate MV mutations: Create writes for each MV
- Write to base table: Normal write operation
- Write to MV tables: Additional writes (async but still overhead)
- Maintain consistency: Track MV updates via batchlog
Building MVs on Large Tables
MV Build Time Estimates
| Table Size | Rows | Build Time |
|---|---|---|
| Small | < 1M rows | 1-5 minutes |
| Medium | 1M - 10M rows | 10-60 minutes |
| Large | 10M - 100M rows | 1-6 hours |
| Huge | > 100M rows | 6+ hours |
During build:
- High CPU usage across cluster
- Increased disk I/O
- MV is queryable but incomplete
- Don't create during peak traffic!
Monitoring MV Health
Optimization Strategies
Fast MVs
- Limit to 3 MVs: Per table maximum
- Select Only Needed: Fewer columns = faster
- Appropriate CL: LOCAL_QUORUM for writes
- Regular Repair: Keep MVs in sync
- Monitor Lag: Alert if > 1 second behind
Slow MVs
- Too Many MVs: > 5 per table kills writes
- SELECT *: Unnecessary data slows down
- No Monitoring: Don't notice sync issues
- Wrong CL: ALL consistency = very slow
- No Repair: MVs drift out of sync
When NOT to Use MVs
Consider manual denormalization instead if:
- Write throughput > 10,000 writes/second per table
- Need more than 5 different access patterns
- Write latency is critical (< 5ms required)
- Table has frequent schema changes
- Need to filter or aggregate data
🖥️ Interactive Materialized View Console
Practice creating materialized views in our simulator!
Try the examples or create your own MV...
Available Examples:
• Example 1: Basic MV creation
• Example 2: MV with clustering order
• Example 3: Multiple MVs on same table
💼 Interview Questions & Expert Answers
Master materialized views for your next Cassandra interview!
Answer: Materialized views are automatically maintained denormalized tables, while secondary indexes are lookups that enable filtering on non-primary key columns.
Materialized View:
- What: A complete table with different primary key
- Data: Full copy of data, automatically synchronized
- Queries: Fast, single-partition reads
- Writes: Cassandra updates automatically
- Performance: Read-optimized, write overhead 20-40%
Secondary Index:
- What: Lookup structure (column_value → partition_key)
- Data: No data duplication, just pointers
- Queries: Slower, scatter-gather across all nodes
- Writes: Automatic index updates
- Performance: Write overhead ~10%, reads slower than MV
| Feature | MV | Index |
|---|---|---|
| Read Speed | ⚡ Fast (5ms) | 🐌 Slower (50ms+) |
| Write Impact | -20% to -40% | -10% |
| Storage | 2x (full copy) | +10-20% |
| Best For | Known query patterns | Ad-hoc queries |
Answer: MVs have five major limitations: no row filtering, no aggregations, write performance impact, eventual consistency, and static column restrictions.
1. Cannot Filter Rows
You can only use WHERE with IS NOT NULL, not actual filtering.
2. Cannot Aggregate Data
No COUNT, SUM, AVG, MAX, MIN, or GROUP BY allowed.
3. Write Performance Impact
- Each MV adds 20-40% write overhead
- 3 MVs = 50% slower writes
- Cassandra must update base table + all MVs
4. Eventually Consistent
- MV updates are async (usually < 10ms delay)
- Read immediately after write may miss data
- Not suitable for strict consistency requirements
5. Static Column Restrictions
Can only include static columns if partition key remains the same in MV.
Workarounds:
- For filtering: Reorganize then filter in query
- For aggregations: Use counter tables or application-level
- For consistency: Query base table directly
- For heavy writes: Consider manual denormalization
Answer: Cassandra uses a batchlog-based approach to ensure MVs are updated atomically with the base table.
The MV Update Process:
- Write Received: Client writes to base table
- MV Detection: Coordinator identifies which MVs are affected
- Mutation Generation: Creates write mutations for each MV
- Batchlog Entry: Writes to distributed batchlog for durability
- Parallel Writes: Sends writes to base table + all MVs
- Async Completion: MVs update asynchronously
- Batchlog Cleanup: Removes batchlog entry when complete
Why Batchlog?
- Crash Recovery: If node fails mid-write, batchlog ensures MV updates complete
- Consistency: Prevents base table and MVs from getting out of sync
- Replay: Other nodes replay batchlog if coordinator fails
Trade-off:
Batchlog adds overhead (extra write + disk space), which is why MVs impact write performance.
Answer: Use MVs for read-heavy workloads with simple denormalization needs. Use manual denormalization for write-heavy workloads or when you need more control.
Use Materialized Views When:
- ✅ Read-Heavy: 80%+ reads, < 10K writes/sec
- ✅ Simple Reorganization: Just changing primary key structure
- ✅ Reduce Complexity: Don't want BATCH logic in application
- ✅ Few Access Patterns: 2-3 different query patterns max
- ✅ Automatic Sync: Need guaranteed consistency without code
- ✅ Development Speed: Faster to implement than manual
Use Manual Denormalization When:
- ✅ Write-Heavy: > 10K writes/sec
- ✅ Many Access Patterns: > 5 different ways to query
- ✅ Need Filtering: Want to filter rows (MVs can't)
- ✅ Need Aggregation: Want computed columns
- ✅ Custom Logic: Complex transformation rules
- ✅ Performance Critical: Can't afford 20-40% write overhead
- ✅ Batch Control: Want fine-grained control over when tables sync
Decision Matrix:
If (read_heavy AND simple_reorg AND < 5 patterns) → Use MV
Else → Use manual denormalization
Answer: Drop and recreate the MV, or use nodetool repair for minor inconsistencies.
Option 1: Full Rebuild (Recommended for Major Issues)
Option 2: Repair (For Minor Inconsistencies)
Common Causes of Inconsistency:
- Node Failures: Node crashed during MV update
- Network Partitions: Split brain scenarios
- Disk Corruption: SSTable corruption on MV
- Upgrade Issues: Bug in Cassandra version
- No Regular Repair: Replicas drifted over time
Prevention Best Practices:
- ✅ Run nodetool repair weekly on MVs
- ✅ Monitor MV lag (base table vs MV timestamps)
- ✅ Use LOCAL_QUORUM consistency level
- ✅ Monitor batchlog size (shouldn't grow unbounded)
- ✅ Test MV queries after cluster maintenance
Important Notes
- Rebuild Time: Large base tables take hours to rebuild MV
- During Rebuild: MV is queryable but incomplete
- Traffic Impact: Rebuild causes high CPU/disk I/O
- Recommendation: Rebuild during low-traffic windows
🎓 Chapter Summary: Materialized View Mastery
Congratulations! You now understand Materialized Views at a production level!
Key Concepts Mastered:
- Automatic Denormalization: Cassandra maintains MVs for you
- Different Primary Key: Same data, organized differently
- Eventually Consistent: Slight delay (usually < 10ms)
- Write Overhead: 20-40% per MV
- Read Performance: Fast as regular tables (5ms)
The Golden Rules:
- Limit to 3-5 MVs: Per table maximum
- Include Base PK: Always in WHERE and PRIMARY KEY
- IS NOT NULL Only: Can't filter by value
- Read-Heavy Workloads: Best use case
- Monitor Write Impact: Track latency
When to Use What:
| Scenario | Solution |
|---|---|
| Read-heavy, simple reorg | ✅ Use Materialized View |
| Write-heavy (> 10K/sec) | ❌ Manual Denormalization |
| Need to filter rows | ❌ Manual Denormalization |
| Need aggregations | ❌ Counter Tables |
| 2-3 access patterns | ✅ Use Materialized View |
| > 5 access patterns | ❌ Manual Denormalization |
Major Limitations to Remember:
- ❌ Cannot filter rows (only IS NOT NULL)
- ❌ Cannot aggregate data (no COUNT, SUM, AVG)
- ❌ Write overhead: 20-40% per MV
- ❌ Eventually consistent (slight delay)
- ❌ Static column restrictions
- ❌ Build time on large tables (hours)
Production Checklist:
- ✅ Base table PK included in MV WHERE and PRIMARY KEY
- ✅ All PK columns have IS NOT NULL
- ✅ Limited to 3-5 MVs per table
- ✅ Write performance tested (measured impact)
- ✅ MV build scheduled during low traffic
- ✅ Regular nodetool repair on MVs
- ✅ Monitoring for MV lag/inconsistencies
- ✅ Documentation of access patterns
Quick Command Reference:
🚀 You're now ready to use Materialized Views in production!
Remember Emma's story: MVs = Automatic denormalization without code complexity! 🎉
Responsive Ad