Section 7: Advanced MongoDB

⚡ Performance Optimization

Transform slow queries into lightning-fast responses - Master the art of making MongoDB blazing fast

1000x

Faster Queries

15+

Techniques

Interview Q&A

📖 Sarah's E-Commerce Nightmare

😱 The Problem

Sarah built an e-commerce website with MongoDB. It worked perfectly during testing with 100 products. But after launch with 50,000 products and 10,000 users, disaster struck:

🐌

15s

Search time

💸

60%

Cart abandonment

😡

₹50K

Daily loss

🔍 The Investigation

// Analyzing slow query db.products.find({ category: "shoes", color: "blue" }).explain("executionStats") // Result: ❌ COLLSCAN (Collection Scan) ❌ Examined: 50,000 documents ❌ Returned: 23 documents ❌ Time: 15,234 ms

Problem: MongoDB scanned ALL 50,000 products to find 23 blue shoes!

✨ The Solution

// Create compound index db.products.createIndex({ category: 1, color: 1 }) // New result: ✅ IXSCAN (Index Scan) ✅ Examined: 23 documents ✅ Returned: 23 documents ✅ Time: 12 ms

⚡

12ms

1000x faster!

📈

95%

Conversion up

💰

₹2L

Daily revenue

💡 Performance optimization = Better UX + More Revenue + Business Success!

🎯 What is Database Performance?

Performance = How quickly your database reads and writes data. Fast database = organized library. Slow database = messy room where you search everything.

❌ Without Index (COLLSCAN)

📚📚📚📚📚

Reading EVERY document

Time: 15 seconds 🐌

✅ With Index (IXSCAN)

📑 → 📖

Going DIRECTLY to data

Time: 12 milliseconds ⚡

📊 Key Performance Metrics

⏱️

Query Time

Goal: <100ms

📈

Throughput

Goal: 1000+ ops/sec

💻

CPU/RAM

Goal: <70%

🌐

Latency

Goal: <50ms

❌ Causes of Slow Performance

1️⃣ No Indexes

Scans all documents

2️⃣ Bad Queries

Fetches too much data

3️⃣ Poor Schema

Not optimized for queries

4️⃣ Hardware Limits

Low RAM, slow disks

🔑 Indexing: The #1 Performance Booster

Indexes are like a book's table of contents - they help MongoDB find data without scanning every document.

📚 Types of Indexes

1️⃣

Single Field Index

Index on ONE field. Most common type.

db.users.createIndex({ email: 1 })

Best for: Single field queries

2️⃣

Compound Index

Index on MULTIPLE fields together.

db.products.createIndex({ category: 1, price: -1 })

Best for: Multi-condition queries

📝

Text Index

For full-text search functionality.

db.articles.createIndex({ content: "text" })

Best for: Search features

🌍

Geospatial Index

For location-based queries.

db.stores.createIndex({ location: "2dsphere" })

Best for: Maps, "near me"

🔐

Unique Index

Ensures no duplicate values.

db.users.createIndex({ email: 1 }, { unique: true })

Best for: Emails, usernames

⏰

TTL Index

Auto-deletes documents after time.

db.logs.createIndex({ createdAt: 1 }, { expireAfterSeconds: 86400 })

Best for: Sessions, logs

💻 Essential Index Commands

MongoDB Shell

// CREATE index (1=ascending, -1=descending)
db.products.createIndex({ category: 1 })

// CREATE compound index
db.products.createIndex({ category: 1, price: -1 })

// VIEW all indexes
db.products.getIndexes()

// DROP an index
db.products.dropIndex({ category: 1 })

// INDEX usage stats
db.products.aggregate([{ $indexStats: {} }])
        

ESR Rule for Compound Indexes

Order fields: Equality (exact match) → Sort → Range ($gt, $lt). This maximizes index efficiency!

Don't Over-Index!

Each index uses disk space and slows writes. Rule of thumb: 5-7 indexes max per collection.

🔧 Query Optimization Techniques

🔍 Use explain() to Analyze Queries

Query Analysis

// Basic explain
db.products.find({ category: "shoes" }).explain()

// With execution stats (recommended)
db.products.find({ category: "shoes" }).explain("executionStats")
        

Field	Good ✅	Bad ❌	Meaning
stage	IXSCAN	COLLSCAN	Using index vs full scan
docsExamined	≈ nReturned	>> nReturned	Docs looked at
executionTimeMillis	<100	>1000	Query time

✨ Best Practices

✅ Use Projection

❌ Bad

db.users.find({ city: "Mumbai" })

✅ Good

db.users.find({ city: "Mumbai" }, { name: 1, email: 1 })

✅ Use limit()

❌ Bad

db.products.find({})

✅ Good

db.products.find({}).limit(20).skip(0)

⚡ Speed Comparison

Without Index15,234ms

15.2 seconds 🐌

With Index234ms

234ms

Index + Projection12ms ⚡

12ms

🚀 1000x faster with optimization!

🏗️ Schema Design for Performance

The key decision: Embed or Reference?

📦 Embedding

Store related data TOGETHER

{
  _id: "order123",
  customer: "John",
  items: [
    { name: "Shoes", price: 500 },
    { name: "Shirt", price: 300 }
  ]
}
                

✅ Pros: Single query, faster reads, no JOINs
❌ Cons: Can get too large, duplicate data

Best for: 1:Few relationships

🔗 Referencing

Store references (IDs) to related data

// orders collection
{ _id: "order123", customerId: "user456" }

// users collection
{ _id: "user456", name: "John" }
                

✅ Pros: No duplicates, smaller docs, flexible
❌ Cons: Multiple queries, slower reads

Best for: 1:Many, Many:Many

Golden Rule

"Data accessed together should be stored together." Design schema based on your most common queries!

🖥️ Hardware Optimization

🧠

RAM

Keep working set in memory

Minimum: 16GB+

💾

SSD Storage

10-100x faster than HDD

Use: NVMe SSD

⚡

CPU

Multi-core for parallel ops

Minimum: 4+ cores

🌐

Network

For replica sets

Use: 1Gbps+

📊 Monitoring & Profiling

Monitoring Commands

// Server status - overall health
db.serverStatus()

// Current operations
db.currentOp()

// Collection stats
db.products.stats()

// Enable profiler (logs slow queries)
db.setProfilingLevel(1, { slowms: 100 })

// View slow queries
db.system.profile.find().sort({ ts: -1 }).limit(5)
        

🔍 What to Monitor

Query Performance

Slow queries, COLLSCAN

Resource Usage

CPU, RAM, Disk I/O

Connections

Active, available

Replication Lag

Secondary delay

🌍 Real-World Examples

📱 Example 1: Social Media App

Problem: User feed loading in 8 seconds

// Before: No index
db.posts.find({ userId: { $in: followingList } }).sort({ createdAt: -1 })
// Time: 8000ms

// After: Compound index
db.posts.createIndex({ userId: 1, createdAt: -1 })
// Time: 45ms ⚡
            

🛒 Example 2: E-Commerce Search

Problem: Product search with filters slow

// Optimized compound index for common searches
db.products.createIndex({ 
    category: 1, 
    brand: 1, 
    price: 1,
    rating: -1 
})

// Query with projection
db.products.find(
    { category: "phones", brand: "Samsung", price: { $lt: 50000 } },
    { name: 1, price: 1, rating: 1, image: 1 }  // Only needed fields
).sort({ rating: -1 }).limit(20)
            

📊 Example 3: Analytics Dashboard

Problem: Aggregation queries timing out

// Solution: Pre-aggregated collections + indexes
db.dailyStats.createIndex({ date: -1, metric: 1 })

// Use $match early in pipeline
db.events.aggregate([
    { $match: { date: { $gte: startDate } } },  // Filter first!
    { $group: { _id: "$category", count: { $sum: 1 } } }
])
            

🎯 Interview Questions & Answers

20 most commonly asked MongoDB performance questions. Click to expand answers!

1 What is an index in MongoDB and why is it important? ▼

Answer: An index is a special data structure that stores a small portion of the collection's data in an easy-to-traverse form. It's like a book's table of contents.

Importance:

Without index: MongoDB scans EVERY document (COLLSCAN) - O(n) complexity
With index: MongoDB directly locates documents (IXSCAN) - O(log n) complexity
Can make queries 100-1000x faster

Example: db.users.createIndex({ email: 1 })

2 What is the difference between COLLSCAN and IXSCAN? ▼

COLLSCAN (Collection Scan):

Scans every document in the collection
Very slow for large collections
Indicates missing index

IXSCAN (Index Scan):

Uses index to find documents
Much faster, especially for large collections
Indicates proper indexing

Check with: db.collection.find({}).explain("executionStats")

3 What is a compound index? When should you use it? ▼

Compound Index: An index on multiple fields.

db.products.createIndex({ category: 1, price: -1 })

Use when:

Queries filter on multiple fields together
Queries filter on one field and sort by another
You want to cover multiple query patterns with one index

Important: Field order matters! The index can support queries on category alone, or category + price, but NOT price alone (leftmost prefix rule).

4 What is a covered query? ▼

Covered Query: A query that can be satisfied entirely using an index, without accessing the actual documents.

Requirements:

All query fields are part of the index
All returned fields are part of the index
No field in the query equals null

Example:

db.users.createIndex({ email: 1, name: 1 })

db.users.find({ email: "test@test.com" }, { name: 1, _id: 0 })

This is the FASTEST type of query!

5 What is the ESR rule for compound indexes? ▼

ESR = Equality, Sort, Range

Order fields in compound index as:

Equality: Fields with exact match (field: "value")
Sort: Fields used in sort()
Range: Fields with range queries ($gt, $lt, $in)

Example:

Query: find({ status: "active", age: { $gte: 18 } }).sort({ name: 1 })

Optimal index: { status: 1, name: 1, age: 1 }

6 How do you identify slow queries in MongoDB? ▼

Methods:

Database Profiler: db.setProfilingLevel(1, { slowms: 100 })
View slow queries: db.system.profile.find().sort({ ts: -1 })
explain(): db.collection.find({}).explain("executionStats")
MongoDB logs: Check mongod logs for slow operations
MongoDB Atlas: Performance Advisor, Real-Time Performance Panel

7 What is the difference between embedding and referencing? ▼

Embedding (Denormalization):

Store related data in same document
Single query retrieves all data
Best for: 1:1, 1:Few relationships

Referencing (Normalization):

Store reference (ObjectId) to related document
Requires multiple queries or $lookup
Best for: 1:Many, Many:Many relationships

Rule: "Data accessed together should be stored together"

8 What is a TTL index? ▼

TTL (Time-To-Live) Index: Automatically deletes documents after a specified time period.

Use cases:

Session data
Log entries
Temporary data
Cache entries

Example:

db.sessions.createIndex({ createdAt: 1 }, { expireAfterSeconds: 3600 })

Documents will be deleted 1 hour after their createdAt time.

9 How does MongoDB use RAM for performance? ▼

WiredTiger Cache: MongoDB's storage engine keeps frequently accessed data and indexes in RAM.

Working Set: The portion of data actively being used.

Best Practice:

Working set should fit in RAM
Default cache: 50% of RAM - 1GB
Monitor cache hit ratio

Check: db.serverStatus().wiredTiger.cache

10 What is projection and why is it important? ▼

Projection: Specifying which fields to return in query results.

Why important:

Reduces network bandwidth
Reduces memory usage
Can enable covered queries
Faster response times

Example:

db.users.find({ city: "Mumbai" }, { name: 1, email: 1, _id: 0 })

Only returns name and email fields.

11 What are the disadvantages of having too many indexes? ▼

Disadvantages:

Slower writes: Every insert/update must update all indexes
More disk space: Each index consumes storage
More RAM usage: Indexes should fit in memory
Longer index builds: Creating indexes takes time

Best Practice: 5-7 indexes maximum per collection. Only create indexes for actual query patterns.

12 How do you optimize aggregation pipeline performance? ▼

Optimization techniques:

$match early: Filter documents at the start
$project early: Remove unnecessary fields
Use indexes: $match and $sort can use indexes
$limit early: Reduce documents processed
Avoid $lookup when possible: Consider embedding
allowDiskUse: For large aggregations

Example:

db.orders.aggregate([ { $match: { status: "completed" } }, // Filter first { $project: { total: 1, date: 1 } }, // Reduce fields { $group: { _id: "$date", sum: { $sum: "$total" } } } ])

13 What is sharding and when should you use it? ▼

Sharding: Distributing data across multiple machines (horizontal scaling).

When to use:

Data too large for single server
Write throughput exceeds single server capacity
Working set doesn't fit in RAM
Typically when data exceeds 1-2 TB

Shard Key: Field used to distribute data. Choose wisely - affects query performance!

14 What is the $lookup operator and its performance impact? ▼

$lookup: Performs a left outer join to another collection.

Performance considerations:

Can be slow for large collections
Creates index on foreign field for better performance
Consider embedding if $lookup is frequent
Use with $match to reduce documents first

Optimization:

db.orders.aggregate([ { $match: { status: "pending" } }, // Filter first! { $lookup: { from: "users", localField: "userId", foreignField: "_id", as: "user" } } ])

15 How do you handle large bulk operations efficiently? ▼

Techniques:

bulkWrite(): Batch multiple operations
ordered: false: Continue on errors, faster
Batch size: 1000-5000 operations per batch
insertMany(): For bulk inserts

Example:

db.products.bulkWrite([ { insertOne: { document: {...} } }, { updateOne: { filter: {...}, update: {...} } }, { deleteOne: { filter: {...} } } ], { ordered: false })

16 What is the 16MB document size limit and how to handle it? ▼

Limit: Maximum BSON document size is 16MB.

Solutions:

GridFS: For files larger than 16MB
Reference pattern: Store related data in separate collection
Bucket pattern: Group related documents
Subset pattern: Store only recent/relevant data

Best Practice: Keep documents under 1MB for optimal performance.

17 What is read preference and how does it affect performance? ▼

Read Preference: Determines which replica set members to read from.

Options:

primary: Always read from primary (consistent)
primaryPreferred: Primary, fallback to secondary
secondary: Read from secondaries (scale reads)
secondaryPreferred: Secondary, fallback to primary
nearest: Lowest latency member

Performance tip: Use secondary reads for analytics queries to reduce primary load.

18 What is write concern and its performance trade-off? ▼

Write Concern: Level of acknowledgment for write operations.

Options:

w: 0 - No acknowledgment (fastest, risky)
w: 1 - Primary acknowledged (default)
w: "majority" - Majority acknowledged (safest)
j: true - Written to journal

Trade-off: Higher safety = slower writes. Choose based on data criticality.

19 How do you optimize MongoDB for high write throughput? ▼

Techniques:

Minimize indexes: Each index slows writes
Use bulk operations: bulkWrite(), insertMany()
Lower write concern: w: 1 instead of majority
Shard collection: Distribute writes
Use SSDs: Faster disk I/O
Pre-split chunks: For sharded collections

20 What tools does MongoDB provide for performance monitoring? ▼

Built-in Tools:

explain(): Query execution analysis
db.serverStatus(): Server metrics
db.currentOp(): Current operations
Database Profiler: Slow query logging
mongostat: Real-time stats
mongotop: Collection-level stats

MongoDB Atlas:

Performance Advisor
Real-Time Performance Panel
Query Profiler
Index Suggestions