🎭 MongoDB Interview Scenarios

// Check query execution
db.products.find({name: /laptop/i}).explain("executionStats")

// Results show:
// - COLLSCAN (collection scan)
// - totalDocsExamined: 10,000,000
// - nReturned: 150
// - executionTimeMillis: 8500

Problem: Full collection scan of 10M documents!

// Create text index
db.products.createIndex({ name: "text", description: "text" })

// Update query to use text search
db.products.find({ $text: { $search: "laptop" } })

// Results:
// - IXSCAN (index scan)
// - executionTimeMillis: 45ms
// - 200x faster!

Pros:
✓ Purpose-built for text search
✓ Case-insensitive by default
✓ Supports stemming ("laptop" matches "laptops")
✓ Can search multiple fields
✓ Relevance scoring available

Cons:
✗ Exact phrase matching requires quotes
✗ Only one text index per collection
✗ Slightly more disk space

// Create regular index
db.products.createIndex({ name: 1 })

// Use with collation for case-insensitive
db.products.find({ name: "laptop" })
  .collation({ locale: "en", strength: 2 })

Pros:
✓ Works for exact match and prefix
✓ Simpler than text index

Cons:
✗ Regex still slow with index
✗ Doesn't support full-text features
✗ Must use collation for case-insensitive

// Sync MongoDB to Elasticsearch
// Use Elasticsearch for search queries
// MongoDB for transactional data

Pros:
✓ Most powerful search capabilities
✓ Advanced features (autocomplete, fuzzy, analytics)
✓ Scales independently

Cons:
✗ Additional infrastructure
✗ Complexity of maintaining sync
✗ Higher cost

// Create a newsfeed collection
db.newsfeeds.insertOne({
  _id: userId,
  posts: [
    { postId: "p1", userId: "u1", timestamp: ISODate(...), content: "..." },
    { postId: "p2", userId: "u5", timestamp: ISODate(...), content: "..." },
    // ... latest 100 posts
  ]
})

// When user creates post, push to all followers' feeds
db.newsfeeds.updateMany(
  { _id: { $in: followerIds } },
  { 
    $push: { 
      posts: {
        $each: [newPost],
        $position: 0,
        $slice: 100  // Keep only latest 100
      }
    }
  }
)

// Reading newsfeed becomes instant
db.newsfeeds.findOne({ _id: currentUserId })

// Performance:
// - Read: O(1) - single document lookup
// - Write: O(followers) - but asynchronous
// - Query time: ~5ms vs 2000ms!

// For users with many followers (celebrities):
// - Don't fan-out to all followers
// - Followers query celebrity posts directly
// - Cache results

// For regular users:
// - Use fan-out approach

function generateFeed(userId) {
  const user = db.users.findOne({ _id: userId });
  
  if (user.followerCount > 10000) {
    // Celebrity - followers query directly
    return queryCelebrityPosts(user.following);
  } else {
    // Regular user - use pre-generated feed
    return db.newsfeeds.findOne({ _id: userId });
  }
}

// DON'T DO THIS
{
  _id: "post1",
  title: "My Blog Post",
  comments: [
    { text: "Great!", user: "Alice", replies: [...] },
    { text: "Nice!", user: "Bob", replies: [...] },
    // ... 100,000 comments
  ]
}

Problems:
✗ Exceeds 16MB document limit
✗ Cannot paginate comments efficiently
✗ Must load entire document to read post
✗ Slow to add new comments (updates whole doc)

// Posts collection
{
  _id: "post1",
  title: "My Blog Post",
  content: "...",
  commentCount: 1523,
  // Embed ONLY latest 50 comments for quick display
  recentComments: [
    {
      _id: "c1",
      text: "Great post!",
      user: "Alice",
      timestamp: ISODate(...),
      replyCount: 3
    },
    // ... latest 50
  ]
}

// Comments collection (for ALL comments)
{
  _id: "c1",
  postId: "post1",
  text: "Great post!",
  user: "Alice",
  timestamp: ISODate(...),
  parentId: null,  // null for top-level, commentId for replies
  depth: 0         // 0 for top-level, 1 for first reply, etc.
}

// Create indexes
db.comments.createIndex({ postId: 1, timestamp: -1 })
db.comments.createIndex({ parentId: 1, timestamp: 1 })

// 1. Display post with recent comments (fast!)
db.posts.findOne({ _id: "post1" })
// Recent comments already embedded, no extra query needed

// 2. Paginate through all comments
db.comments.find({ postId: "post1", parentId: null })
  .sort({ timestamp: -1 })
  .skip(50)
  .limit(50)

// 3. Load replies for a comment
db.comments.find({ parentId: "c1" })
  .sort({ timestamp: 1 })

// 4. Add new comment
db.comments.insertOne({
  postId: "post1",
  text: "New comment",
  // ...
})

// Update post's recent comments (can be async)
db.posts.updateOne(
  { _id: "post1" },
  {
    $push: {
      recentComments: {
        $each: [newComment],
        $position: 0,
        $slice: 50
      }
    },
    $inc: { commentCount: 1 }
  }
)

// Assuming users collection
// Option 1: Hashed userId (RECOMMENDED)
sh.shardCollection("app.users", { userId: "hashed" })

Why hashed userId:
✓ Even distribution (no hotspots)
✓ High cardinality (100M unique values)
✓ Queries by userId are targeted

// Option 2: Compound key (if geographic distribution matters)
sh.shardCollection("app.users", { region: 1, userId: 1 })

Why NOT sequential _id:
✗ All new writes go to one shard (hotspot)
✗ Uneven distribution

// Start with 4 shards (each is a replica set)
// 
// Shard 1: 1.25TB data, handles userId hash 0-25%
// Shard 2: 1.25TB data, handles userId hash 25-50%
// Shard 3: 1.25TB data, handles userId hash 50-75%
// Shard 4: 1.25TB data, handles userId hash 75-100%

// Each shard:
// - 1 Primary + 2 Secondaries
// - 64GB RAM per server (working set: 25GB - fits in RAM!)
// - Handles 2.5K writes/sec (manageable)

// Config Servers: 3-node replica set for metadata
// Mongos: 4+ query routers (stateless, can scale independently)

// Phase 1: Setup (Week 1)
1. Provision 4 shard replica sets
2. Setup config servers
3. Deploy mongos routers
4. Test in staging

// Phase 2: Enable Sharding (Week 2)
1. Enable sharding on database
2. Shard the collection
3. MongoDB starts splitting and balancing chunks
4. Monitor balancer progress

// Phase 3: Cutover (Week 3)
1. Update application to connect to mongos (not replica set)
2. Monitor performance
3. Gradual traffic migration
4. Rollback plan ready

// Phase 4: Optimization (Week 4+)
1. Tune chunk size if needed
2. Add more shards if necessary
3. Optimize queries to include shard key

const session = client.startSession();

try {
  session.startTransaction({
    readConcern: { level: 'snapshot' },
    writeConcern: { w: 'majority' }
  });

  // Step 1: Check source account balance
  const sourceAccount = await db.accounts.findOne(
    { _id: "account1" },
    { session }
  );

  if (sourceAccount.balance < 100) {
    throw new Error("Insufficient funds");
  }

  // Step 2: Debit source account
  await db.accounts.updateOne(
    { _id: "account1", balance: { $gte: 100 } },
    { $inc: { balance: -100 } },
    { session }
  );

  // Step 3: Credit destination account
  await db.accounts.updateOne(
    { _id: "account2" },
    { $inc: { balance: 100 } },
    { session }
  );

  // Step 4: Record transaction for audit
  await db.transactions.insertOne({
    from: "account1",
    to: "account2",
    amount: 100,
    timestamp: new Date(),
    status: "completed"
  }, { session });

  // Commit transaction
  await session.commitTransaction();
  
  console.log("Transfer successful");

} catch (error) {
  // Rollback on any error
  await session.abortTransaction();
  console.error("Transfer failed:", error);
  throw error;
  
} finally {
  session.endSession();
}

// Step 1: Create pending transaction
db.transactions.insertOne({
  _id: "txn1",
  from: "account1",
  to: "account2",
  amount: 100,
  state: "pending"
})

// Step 2: Apply operations with transaction reference
db.accounts.updateOne(
  { _id: "account1" },
  { $inc: { balance: -100, pendingDebits: 100 } }
)

// Step 3: If successful, commit
db.transactions.updateOne(
  { _id: "txn1" },
  { $set: { state: "committed" } }
)

// Step 4: Finalize accounts
db.accounts.updateOne(
  { _id: "account1" },
  { $inc: { pendingDebits: -100 } }
)

// Complex and error-prone! Use native transactions instead.

async function cancelRide(rideId, userId) {
  // Atomic: check status and update in single operation
  const result = await db.rides.findOneAndUpdate(
    { 
      _id: rideId, 
      status: 'active'  // Only update if still active
    },
    { 
      $set: { 
        status: 'cancelled',
        cancelledAt: new Date(),
        cancelledBy: userId
      } 
    },
    { 
      returnDocument: 'after' 
    }
  );

  // If result is null, ride was already cancelled
  if (!result.value) {
    console.log("Ride already cancelled");
    return { success: false, reason: "already_cancelled" };
  }

  // Process refund only if update succeeded
  await processRefund(userId, result.value.fare);
  
  return { success: true };
}

// Guarantees: Only ONE call can change status from active to cancelled
// All other concurrent calls will get null result

// Add idempotency key to ride cancellations
async function cancelRide(rideId, userId, idempotencyKey) {
  // Check if we've already processed this request
  const existing = await db.cancellations.findOne({
    idempotencyKey: idempotencyKey
  });

  if (existing) {
    // Already processed, return previous result
    return existing.result;
  }

  // Process cancellation
  const result = await db.rides.findOneAndUpdate(
    { _id: rideId, status: 'active' },
    { $set: { status: 'cancelled' } }
  );

  if (result.value) {
    await processRefund(userId, result.value.fare);
  }

  // Store result with idempotency key
  await db.cancellations.insertOne({
    idempotencyKey: idempotencyKey,
    rideId: rideId,
    processedAt: new Date(),
    result: { success: true }
  });

  return { success: true };
}

// Client includes idempotency key with request
// cancelRide('ride123', 'user456', 'cancel-uuid-1234')

// Application is using secondary reads
const client = new MongoClient(uri, {
  readPreference: 'secondary'  // This causes stale reads!
});

Timeline:
t=0: User updates profile (write goes to PRIMARY)
t=1: User reads profile (read goes to SECONDARY - not yet replicated)
t=1: User sees OLD data!
t=2: Secondary catches up (replication lag)

Replication lag during high load: 100-500ms typical

// Change read preference to primary
const client = new MongoClient(uri, {
  readPreference: 'primary'
});

// Or per-query
db.users.findOne(
  { _id: userId },
  { readPreference: 'primary' }
);

Pros:
✓ Always consistent - no stale reads
✓ Simple solution

Cons:
✗ All read load on primary
✗ Doesn't scale reads

// After write, read from primary for short time
async function updateProfile(userId, updates) {
  await db.users.updateOne(
    { _id: userId },
    { $set: updates },
    { writeConcern: { w: 'majority' } }
  );

  // Set flag: "just wrote, read from primary"
  await redis.setex(`user:${userId}:just-wrote`, 2, '1');
}

async function getProfile(userId) {
  const justWrote = await redis.get(`user:${userId}:just-wrote`);
  
  const readPref = justWrote ? 'primary' : 'secondary';
  
  return db.users.findOne(
    { _id: userId },
    { readPreference: readPref }
  );
}

// User sees their own writes, others can read from secondary

// Read only data acknowledged by majority
db.users.findOne(
  { _id: userId },
  { 
    readConcern: { level: 'majority' },
    readPreference: 'secondary'
  }
);

// Ensures you don't read data that might be rolled back

// 1. Take snapshot of PostgreSQL
pg_dump production_db > snapshot.sql

// 2. Write migration script
const batchSize = 10000;
let offset = 0;

while (true) {
  // Read batch from PostgreSQL
  const rows = await pg.query(`
    SELECT * FROM users 
    ORDER BY id 
    LIMIT ${batchSize} OFFSET ${offset}
  `);

  if (rows.length === 0) break;

  // Transform to MongoDB format
  const docs = rows.map(row => ({
    _id: row.id,
    name: row.name,
    email: row.email,
    // ... transform other fields
    migratedAt: new Date()
  }));

  // Bulk insert to MongoDB
  await mongodb.users.insertMany(docs, { ordered: false });

  offset += batchSize;
  console.log(`Migrated ${offset} records`);
}

// Run during off-peak hours
// Estimated time: 4-8 hours for 100M records

// Update application to write to BOTH databases
class UserRepository {
  async create(userData) {
    // Write to PostgreSQL (primary source of truth)
    const pgResult = await pg.query(
      'INSERT INTO users (...) VALUES (...)',
      userData
    );

    // Async write to MongoDB (fire-and-forget)
    this.writeToMongoDB(userData).catch(err => {
      // Log error but don't fail request
      logger.error('MongoDB write failed', err);
      // Queue for retry
      retryQueue.add(userData);
    });

    return pgResult;
  }

  async writeToMongoDB(userData) {
    return mongodb.users.insertOne(userData);
  }
}

// All writes go to both systems
// PostgreSQL is still primary (reads come from PG)

// Compare data and sync differences
async function catchUpSync() {
  // Get all IDs from PostgreSQL
  const pgIds = await pg.query('SELECT id FROM users');

  for (const {id} of pgIds) {
    // Check if exists in MongoDB
    const mongoDoc = await mongodb.users.findOne({ _id: id });

    if (!mongoDoc) {
      // Missing in MongoDB - copy over
      const pgRow = await pg.query(
        'SELECT * FROM users WHERE id = $1', 
        [id]
      );
      await mongodb.users.insertOne(transformRow(pgRow));
    } else {
      // Exists - verify data matches
      const pgRow = await pg.query(
        'SELECT * FROM users WHERE id = $1', 
        [id]
      );
      if (!dataMatches(pgRow, mongoDoc)) {
        // Data mismatch - overwrite with PG data
        await mongodb.users.replaceOne(
          { _id: id },
          transformRow(pgRow)
        );
      }
    }
  }
}

// Run multiple times until <0.1% mismatches

// Automated validation checks
async function validateMigration() {
  // 1. Count check
  const pgCount = await pg.query('SELECT COUNT(*) FROM users');
  const mongoCount = await mongodb.users.countDocuments();
  assert(pgCount === mongoCount, 'Count mismatch');

  // 2. Sample verification (check 10,000 random records)
  const samples = await pg.query(`
    SELECT * FROM users 
    ORDER BY RANDOM() 
    LIMIT 10000
  `);

  for (const row of samples) {
    const mongoDoc = await mongodb.users.findOne({ _id: row.id });
    assert(dataMatches(row, mongoDoc), `Mismatch for ID ${row.id}`);
  }

  // 3. Checksum verification
  // Compare aggregate checksums of data
}

// Run validation daily until confident

// Gradual read migration with feature flag
class UserRepository {
  async findById(id) {
    const readFromMongo = await featureFlag.isEnabled(
      'read_from_mongodb',
      id  // Can enable per-user or percentage
    );

    if (readFromMongo) {
      return mongodb.users.findOne({ _id: id });
    } else {
      return pg.query('SELECT * FROM users WHERE id = $1', [id]);
    }
  }
}

// Day 1: 1% of traffic to MongoDB
// Day 2: 5% of traffic
// Day 3: 10% of traffic
// ...
// Day 7: 100% of traffic

// Monitor error rates, latency, data consistency

// After reads are stable, switch writes
class UserRepository {
  async create(userData) {
    // Write to MongoDB (now primary)
    const mongoResult = await mongodb.users.insertOne(userData);

    // Keep dual-write to PG for safety (can be async)
    this.writeToPG(userData).catch(err => {
      logger.error('PG write failed', err);
    });

    return mongoResult;
  }
}

// Monitor for 1-2 weeks

// After MongoDB is stable for 2+ weeks:
// 1. Stop dual-writes to PostgreSQL
// 2. Keep PostgreSQL as cold backup for 30 days
// 3. Decommission PostgreSQL database
// 4. Celebrate! 🎉

// Connect to one of the healthy secondaries
mongo mongodb://secondary1:27017

// Check replica set status
rs.status()

// Look for:
// - Which nodes are UP/DOWN
// - Last heartbeat times
// - Current state of each member
// - Any error messages

Expected issue:
- Primary shows as "not reachable"
- Secondaries show as "SECONDARY"
- No "PRIMARY" in the set

// Check replica set configuration
rs.conf()

// Look for:
cfg.members.forEach(m => {
  console.log(`${m.host}: priority=${m.priority}, votes=${m.votes}`);
});

// Check if majority is possible
// Need: (total_voting_members / 2) + 1 members online

// Connect to best secondary (most up-to-date)
mongo mongodb://secondary1:27017

// Check which secondary has latest data
rs.status().members.forEach(m => {
  if (m.state === 2) {  // SECONDARY
    console.log(`${m.name}: optime=${m.optimeDate}`);
  }
});

// Force the most up-to-date secondary to become primary
rs.stepDown(0)  // If there's still a primary somehow

// Method 1: Reconfigure with only healthy members
cfg = rs.conf()
cfg.members = cfg.members.filter(m => 
  m.host !== "failed-primary:27017"
)
cfg.version++
rs.reconfig(cfg, {force: true})

// Method 2: If that doesn't work, reconfigure to single member
cfg = rs.conf()
cfg.members = [
  {
    _id: 0,
    host: "secondary1:27017",
    priority: 1,
    votes: 1
  }
]
cfg.version++
rs.reconfig(cfg, {force: true})

// One secondary should become primary now
rs.status()  // Verify PRIMARY exists

// 1. Verify new primary is accepting writes
db.test.insertOne({ test: "write", timestamp: new Date() })

// 2. Check application can connect
// Update connection string if needed to point to new primary

// 3. Monitor application recovery
// Check logs, metrics, error rates

// 4. Communicate status
// Update incident channel: "Primary restored, monitoring"

// After service is restored and stable:

// 1. Fix or replace failed primary hardware

// 2. Add healthy members back to replica set
cfg = rs.conf()
cfg.members.push({
  _id: 1,
  host: "secondary2:27017",
  priority: 1,
  votes: 1
})
cfg.version++
rs.reconfig(cfg)

// 3. Add repaired primary back (as secondary)
cfg = rs.conf()
cfg.members.push({
  _id: 2,
  host: "repaired-node:27017",
  priority: 1,
  votes: 1
})
cfg.version++
rs.reconfig(cfg)

// 4. Wait for initial sync to complete
rs.status()  // Check stateStr: "SECONDARY" not "RECOVERING"

// 5. Restore original configuration if needed

// Step 1: STOP writes immediately
// Prevent more data changes during recovery
db.fsyncLock()  // On primary

// Step 2: Find the problematic delete operation
db.local.oplog.rs.find({
  op: "d",  // delete operation
  ns: "mydb.customers",
  ts: { 
    $gte: Timestamp(deleteTime - 600, 0),  // 10 min before
    $lte: Timestamp(deleteTime + 600, 0)   // 10 min after
  }
}).pretty()

// Identify exact timestamp of bad delete
const badDeleteTimestamp = Timestamp(1702345678, 1);

// Step 3: Take backup of current state (safety)
mongodump --host localhost --port 27017 --out /backup/current

// Step 4: Create restore environment
// Option A: Use hidden secondary for recovery
cfg = rs.conf()
cfg.members[2].priority = 0
cfg.members[2].hidden = true
rs.reconfig(cfg)

// Step 5: Replay oplog up to bad delete
// On hidden secondary or new instance:

// Start from last backup (24h ago)
mongorestore /backup/daily

// Replay oplog up to (but NOT including) bad delete
db.local.oplog.rs.find({
  ts: { 
    $gt: Timestamp(backupTime),
    $lt: badDeleteTimestamp  // Stop before bad delete!
  }
}).forEach(op => {
  // Apply operation
  applyOp(op);
});

// Step 6: Extract recovered data
mongoexport --db=mydb --collection=customers \
  --out=recovered_customers.json

// Step 7: Import to production
mongoimport --db=mydb --collection=customers \
  --file=recovered_customers.json \
  --mode=upsert

// Step 8: Unlock and resume
db.fsyncUnlock()

// If you have delayed secondary (e.g., 1 hour delay):

// Check if delayed secondary still has the data
// (Delete hasn't replicated yet)

// 1. Stop replication on delayed secondary
db.adminCommand({
  replSetSyncFrom: ""  // Stop syncing
})

// 2. Export data from delayed secondary
mongoexport --host delayed-secondary \
  --db=mydb --collection=customers \
  --query='{"deletedAt": {$exists: false}}' \
  --out=customers.json

// 3. Import back to primary
mongoimport --host primary \
  --db=mydb --collection=customers \
  --file=customers.json \
  --mode=upsert

// Delayed secondaries are EXACTLY for this scenario!

// If oplog has cycled (delete > 48h ago):

// 1. Restore from last backup (24h ago)
mongorestore /backup/daily --db=mydb_restored

// 2. Manually identify missing records
// Compare counts, IDs, checksums

// 3. Merge with current data
// This is complex and may lose 24h of updates

// 4. Consider external sources
// - Application logs
// - Audit trails
// - External backups
// - Partner systems with data sync

// Prevention: Increase oplog size or backup frequency!