🔄 Read/Write Path: Data Journey Through MongoDB

Answer:

Write Path Flow:

1. Application (0ms): Issues insert/update command via MongoDB driver
2. MongoDB Driver (~0.5ms): Sends request to mongod process over network
3. mongod Process (~1ms): Parses query and routes to storage engine
4. WiredTiger Cache (~1ms): Writes to in-memory cache (RAM) - Write acknowledged here!
5. Journal (~50ms): Operation logged to write-ahead log (WAL) on disk for crash recovery
6. Checkpoint (~60 seconds): Dirty pages flushed from cache to .wt data files on disk

Key Points:

• Write is acknowledged after step 4 (~1ms) for speed
• Journal ensures durability - on crash, replay journal to recover uncommitted writes
• Checkpoint creates consistent snapshot every 60 seconds
• Trade-off: Speed (cache) vs Durability (journal + checkpoints)

Answer:

Cache Hit (Fast ⚡):

• Data found in WiredTiger Cache (RAM)
• Return immediately without disk access
• Latency: ~100 microseconds (0.1ms)
• Example: Reading a user profile that was recently accessed

Cache Miss (Slower 🐢):

• Data NOT in cache → must read from disk (.wt files)
• After reading from disk, load into cache for future reads
• Latency: ~10-50 milliseconds (100-500x slower than cache hit)
• Example: Reading a document that hasn't been accessed in hours

Why Cache Hit Ratio Matters:

• Goal: Keep cache hit ratio >90%
• 90% hit rate = 10x faster average reads than 50% hit rate
• Increase cache size (more RAM) to improve hit ratio

Answer:

Write concern controls when MongoDB acknowledges a write operation. It's a trade-off between speed and durability.

w: 1 (Default - Acknowledged):

• Behavior: Acknowledged when written to primary node's cache
• Speed: Fast (~1ms)
• Durability: Good - journal ensures recovery on crash
• Risk: If primary fails before replicating, write could be lost
• Use case: Most applications, balanced approach

w: "majority" (Replica Set):

• Behavior: Acknowledged when written to majority of nodes (e.g., 2 out of 3)
• Speed: Slower (~10-50ms depending on network)
• Durability: Excellent - survives single node failures
• Risk: Very low - write is durable even if primary fails
• Use case: Critical data (banking, healthcare, financial transactions)

Example Configuration:

// Fast, default
db.users.insertOne(doc, { writeConcern: { w: 1 } })

// Highly durable, slower
db.transactions.insertOne(doc, { 
  writeConcern: { w: "majority", j: true } 
})

Answer:

Read preference determines which replica set member handles read operations. Trade-off between consistency and performance.

primary (Default):

• Behavior: All reads from primary node
• Consistency: Strongest - always latest data
• Performance: Primary can become bottleneck
• Use case: When you need guaranteed latest data (user profile updates, inventory)

secondary:

• Behavior: All reads from secondary nodes
• Consistency: Eventual - may be stale (lag 0-few seconds)
• Performance: Distributes read load across secondaries
• Use case: Analytics, reporting, search, dashboards (where stale data is acceptable)

When to use secondary:

• Read-heavy workloads that don't need latest data
• Reporting and analytics queries
• Search functionality
• Background batch processing

When to use primary:

• Transactional data requiring latest state
• Read-your-own-writes scenarios
• Critical operations (money transfers, inventory checks)

Example:

// Primary for critical reads
db.inventory.findOne({ sku: "ABC123" }).readPref("primary")

// Secondary for analytics
db.orders.aggregate([...]).readPref("secondary")

Answer:

MongoDB uses multiple mechanisms to ensure data durability:

1. Journal (Write-Ahead Log):

• All write operations logged to journal before applying
• Journal synced to disk every 50ms (configurable)
• On crash: Replay journal entries to recover uncommitted writes
• Location: dbPath/journal/ directory

2. Checkpoints:

• Every 60 seconds, flush WiredTiger cache to disk
• Creates consistent snapshot of all data
• After checkpoint, journal entries before that point can be deleted

3. Replica Sets (w: "majority"):

• Write acknowledged only after majority of nodes have it
• Survives single node failures
• Example: 3-node cluster needs 2 nodes to acknowledge

Crash Recovery Process:

1. MongoDB crashes at 10:30:45
2. Last checkpoint: 10:30:00
3. On restart:
   - Load checkpoint (consistent state at 10:30:00)
   - Replay journal (10:30:00 to 10:30:45)
   - All committed writes recovered! ✅

Trade-offs:

• More durability = Higher latency (journal sync, replication)
• Less durability = Lower latency (cache-only writes)
• Configure via writeConcern based on use case

Answer:

Read Optimizations:

• Indexes: Create indexes on frequently queried fields

  db.users.createIndex({ email: 1 })  // 100x faster lookups

• Projections: Fetch only needed fields

  db.users.find({}, { name: 1, email: 1 })  // Less data transferred

• Covered Queries: Query + projection covered entirely by index (no document fetch)

  db.users.createIndex({ email: 1, name: 1 })
  db.users.find({ email: "x" }, { name: 1, _id: 0 })  // Covered!

• Increase Cache Size: More RAM = higher cache hit ratio

  wiredTiger.cacheSizeGB: 16  // Larger cache

• Read from Secondaries: Distribute load for analytics/reporting

  db.logs.find().readPref("secondary")

Write Optimizations:

• Batch Operations: Insert many docs at once

  db.users.insertMany(docs)  // 100x faster than individual inserts

• Lower Write Concern: For non-critical writes

  db.logs.insertOne(doc, { w: 1 })  // Faster than w: "majority"

• Reduce Index Count: Each index slows writes

  // Remove unused indexes
  db.users.dropIndex("oldIndex")

• Connection Pooling: Reuse connections

  const client = new MongoClient(uri, { maxPoolSize: 100 })

General Optimizations:

• Shard for Scale: Distribute data across servers
• Monitor Cache Hit Ratio: Keep >90%
• Use Profiler: Identify slow queries