Replication Strategies

Answer:

SimpleReplicationStrategy:

• Placement: Simple clockwise walk around token ring
• Awareness: No rack or datacenter awareness
• Configuration: Single RF value for entire cluster
• Use Case: Development/testing only (deprecated)
• Example: 'class': 'SimpleReplicationStrategy', 'replication_factor': 3

NetworkTopologyStrategy:

• Placement: Smart DC/rack-aware algorithm
• Awareness: Full rack and datacenter awareness
• Configuration: Different RF per datacenter
• Use Case: ALL production deployments (recommended)
• Example: 'class': 'NetworkTopologyStrategy', 'US-East': 3, 'US-West': 2

Key Differences:

SimpleReplicationStrategy:
- Places replicas on next N-1 consecutive nodes clockwise
- Could put all replicas in same rack/DC (dangerous!)
- Cannot handle multiple datacenters
- No LOCAL_QUORUM support

NetworkTopologyStrategy:
- Places replicas in different racks within each DC
- Ensures fault tolerance at rack and DC level
- Supports multiple datacenters with different RF
- Full LOCAL_QUORUM support

Real Scenario: With SimpleReplicationStrategy, if Rack 1 fails and all 3 replicas were there → complete data loss! With NetworkTopologyStrategy, replicas are in different racks → survive rack failure.

Recommendation: Always use NetworkTopologyStrategy, even for single DC!

Answer: Even with a single datacenter, NetworkTopologyStrategy provides critical advantages:

1. Rack Awareness (Primary Benefit)

Without Rack Awareness (SimpleReplicationStrategy):
- All 3 replicas could be in same physical rack
- Rack power failure → all replicas lost → DATA LOSS
- Network switch failure → same result

With Rack Awareness (NetworkTopologyStrategy):
- Replicas automatically placed in different racks
- Rack 1 fails → Replicas on Rack 2 & 3 still available
- Survives rack-level failures (power, network, etc.)

2. Future-Proof Architecture

• Adding a second datacenter later is trivial (just ALTER keyspace)
• No migration pain from SimpleReplicationStrategy → NetworkTopologyStrategy
• Planning for growth from day one

3. Cloud Provider Integration

# AWS Example: Map racks to Availability Zones
# cassandra-rackdc.properties on each node:

# Node in us-east-1a:
dc=US-East
rack=us-east-1a

# Node in us-east-1b:
dc=US-East
rack=us-east-1b

# Node in us-east-1c:
dc=US-East
rack=us-east-1c

# Result: Replicas spread across 3 AZs
# Survives entire AZ failure!

4. Zero Performance Penalty

• Same performance as SimpleReplicationStrategy
• No additional latency
• No extra resource usage
• Only benefit, no downside!

5. Industry Standard

• Used by Netflix, Uber, Apple, Instagram
• SimpleReplicationStrategy is deprecated
• Future Cassandra versions may remove SimpleReplicationStrategy

Configuration:

-- Single DC with NetworkTopologyStrategy
CREATE KEYSPACE production WITH replication = {
    'class': 'NetworkTopologyStrategy',
    'datacenter1': 3  -- or your actual DC name like 'US-East'
};

Bottom Line: NetworkTopologyStrategy gives you rack awareness and future flexibility with zero downside. Always use it!

Answer: NetworkTopologyStrategy uses an intelligent algorithm to ensure replicas are well-distributed:

Algorithm Steps:

1. Calculate token: hash(partition_key) = token

2. Find primary node: Node that owns this token

3. For EACH datacenter:
   a. Set RF from keyspace definition (e.g., US-East: 3)
   b. Start with primary node in this DC
   c. Walk clockwise around ring within this DC
   d. For each replica:
      - Find next node in different rack
      - Skip nodes in same rack as existing replicas
      - Add to replica set
   e. Repeat until RF replicas placed in this DC

4. Result: RF replicas per DC, each in different rack

Example Configuration:

CREATE KEYSPACE global WITH replication = {
    'class': 'NetworkTopologyStrategy',
    'US-East': 3,    # 3 replicas in US-East
    'US-West': 3,    # 3 replicas in US-West
    'EU-West': 2     # 2 replicas in EU-West
};

# For partition key user_id=12345:
# Token: -8234719023847192847

# Replicas:
US-East:
  - Node 1 (Rack 1, primary in US-East)
  - Node 2 (Rack 2, different rack)
  - Node 3 (Rack 3, different rack)

US-West:
  - Node 7 (Rack 1, primary in US-West)
  - Node 8 (Rack 2, different rack)
  - Node 9 (Rack 3, different rack)

EU-West:
  - Node 12 (Rack 1, primary in EU-West)
  - Node 13 (Rack 2, different rack)

# Total: 8 replicas across 3 datacenters!

Rack Selection Logic:

# In US-East datacenter with 6 nodes across 3 racks:
Rack 1: Node 1, Node 4
Rack 2: Node 2, Node 5
Rack 3: Node 3, Node 6

# Placing 3 replicas (RF=3):
Replica 1: Node 1 (Rack 1) - primary
Replica 2: Node 2 (Rack 2) - walk clockwise, different rack ✓
Replica 3: Node 3 (Rack 3) - walk clockwise, different rack ✓

# Node 4 skipped (same rack as Node 1)
# Node 5 skipped (same rack as Node 2)
# All 3 replicas in different racks!

Benefits:

• Rack Isolation: Each DC's replicas in different racks
• DC Isolation: Each DC has complete copy (if RF ≥ 1)
• Fault Tolerance: Survive entire DC failure
• Performance: LOCAL_QUORUM queries stay within DC (low latency)

Verification:

$ nodetool getendpoints global users 12345

10.1.0.1   # US-East, Rack 1
10.1.0.2   # US-East, Rack 2
10.1.0.3   # US-East, Rack 3
10.2.0.1   # US-West, Rack 1
10.2.0.2   # US-West, Rack 2
10.2.0.3   # US-West, Rack 3
10.3.0.1   # EU-West, Rack 1
10.3.0.2   # EU-West, Rack 2

# Perfect! Well-distributed across DCs and racks

Answer: Migration requires careful planning but zero downtime:

Step-by-Step Process:

STEP 1: Identify current configuration
--------------------------------
$ cqlsh -e "DESCRIBE KEYSPACE my_keyspace"

CREATE KEYSPACE my_keyspace WITH replication = {
    'class': 'SimpleReplicationStrategy',
    'replication_factor': '3'
};

STEP 2: Find datacenter name
----------------------------
$ nodetool status

Datacenter: datacenter1  ← Your DC name
Status=Up/Down
|/ State=Normal/Leaving/Joining/Moving
--  Address    Rack    Status  State   Load
UN  10.1.0.1   rack1   Up      Normal  245 GB
UN  10.1.0.2   rack2   Up      Normal  238 GB
UN  10.1.0.3   rack3   Up      Normal  251 GB

STEP 3: ALTER keyspace (keep same RF!)
--------------------------------------
ALTER KEYSPACE my_keyspace WITH replication = {
    'class': 'NetworkTopologyStrategy',
    'datacenter1': 3  ← Same RF as before!
};

# Important: Don't change RF during migration!
# Schema change is instant, no data movement yet

STEP 4: Run full repair on ALL nodes
------------------------------------
# This is CRITICAL - redistributes data based on new strategy

# Node 1:
$ nodetool repair -full my_keyspace
[2024-12-26 10:30:00] Starting repair...
[2024-12-26 10:45:00] Repair completed

# Node 2:
$ nodetool repair -full my_keyspace

# Node 3:
$ nodetool repair -full my_keyspace

# Monitor repair:
$ watch nodetool compactionstats

STEP 5: Verify replica placement
--------------------------------
$ nodetool getendpoints my_keyspace users 12345

10.1.0.1  # rack1
10.1.0.2  # rack2
10.1.0.3  # rack3

# Success! Now in different racks

STEP 6: Verify cluster health
-----------------------------
$ nodetool status
# All nodes should be UN (Up Normal)

$ nodetool describering my_keyspace
# Check token ranges look correct

Key Points:

• No Downtime: Clients can keep running throughout
• Keep Same RF: Don't change RF during strategy migration
• Repair is Mandatory: Without repair, data may not be rack-aware
• Repair All Nodes: Not just coordinator or primary nodes
• Off-Peak Recommended: Repair can be resource-intensive

Common Pitfalls to Avoid:

• ❌ Changing RF and strategy at same time
• ❌ Forgetting to run repair (critical step!)
• ❌ Only repairing some nodes
• ❌ Not verifying replica placement after

Timeline:

ALTER keyspace:    Instant (milliseconds)
Full repair:       Hours (depends on data size)
  - 100GB/node:    ~30 minutes
  - 500GB/node:    ~2 hours
  - 1TB+/node:     ~4+ hours

Total migration:   3-6 hours for typical cluster

Answer: Different RF per datacenter allows cost optimization but requires careful consistency level management:

Configuration Example:

CREATE KEYSPACE mixed_rf WITH replication = {
    'class': 'NetworkTopologyStrategy',
    'US-East': 3,    # Primary: full redundancy
    'US-West': 2,    # Secondary: backup
    'EU-West': 1     # Read-only: disaster recovery only
};

# Total: 6 copies of every row across 3 datacenters

How It Works:

• US-East: 3 copies → QUORUM=2, can tolerate 1 failure
• US-West: 2 copies → QUORUM=2, cannot tolerate any failure
• EU-West: 1 copy → QUORUM=1, no redundancy

Consistency Level Implications:

CL	US-East (RF=3)	US-West (RF=2)	EU-West (RF=1)
LOCAL_ONE	✅ Works	✅ Works	✅ Works
LOCAL_QUORUM	✅ Works (need 2)	✅ Works (need 2)	✅ Works (need 1)
QUORUM	✅ Works (need 4 of 6 total)
EACH_QUORUM	✅ Need 2	✅ Need 2	✅ Need 1

Best Practices:

# Recommended CL choices with mixed RF:

# For writes:
INSERT INTO users VALUES (...) 
USING CONSISTENCY LOCAL_QUORUM;
# Fast, available within each DC

# For reads (local users):
SELECT * FROM users WHERE id=?
USING CONSISTENCY LOCAL_QUORUM;
# Fast, consistent within DC

# For critical global consistency:
INSERT INTO financial_transactions VALUES (...)
USING CONSISTENCY EACH_QUORUM;
# Slower, but guaranteed consistency across all DCs

Trade-offs:

• Cost Savings: Lower RF in secondary DCs reduces storage (EU-West: 1 copy vs 3)
• Fault Tolerance: US-East (RF=3) can lose 1 node, EU-West (RF=1) cannot lose any
• Performance: LOCAL_QUORUM always works well regardless of RF
• Availability: Lower RF = lower availability in that DC

Real Scenario:

# Uber's Configuration (approximate):
CREATE KEYSPACE trips WITH replication = {
    'class': 'NetworkTopologyStrategy',
    'US-East': 3,      # Primary: high traffic
    'US-West': 3,      # Primary: high traffic
    'EU-West': 2,      # Secondary: moderate traffic
    'Asia-Pacific': 2  # Secondary: growing market
};

# Write CL: LOCAL_QUORUM
# - In US-East: wait for 2 of 3
# - In EU-West: wait for 2 of 2 (both required!)

# If 1 node fails in US-East: Still works (2 remain for QUORUM)
# If 1 node fails in EU-West: Fails (only 1 remains, need 2!)

# Solution for EU-West: Use CL=ONE for higher availability
# Trade-off: Slightly less consistency, but service stays up