NoSQL Fundamentals

Answer:

NoSQL (Not Only SQL) refers to non-relational databases designed to handle massive scale, flexible schemas, and distributed architectures.

Why it was created:

• Web Scale Requirements: Companies like Google, Facebook, Amazon needed to handle billions of users
• Vertical Scaling Limits: Traditional databases couldn't scale beyond single powerful servers
• Rigid Schemas: Fixed table structures couldn't handle diverse, evolving data types
• ACID Overhead: Strict transactional guarantees too slow for many use cases
• Cost: Expensive enterprise licenses didn't fit internet company budgets

Historical Context: Google's BigTable (2006) and Amazon's Dynamo (2007) papers inspired the NoSQL movement. Cassandra, MongoDB, and other NoSQL databases emerged 2007-2009.

Answer:

The four main types of NoSQL databases are:

• 1. Document Stores:
  • Store data as JSON/BSON documents
  • Examples: MongoDB, CouchDB, Firebase
  • Use case: Content management, catalogs, user profiles
• 2. Key-Value Stores:
  • Simplest model - key maps to value
  • Examples: Redis, DynamoDB, Memcached
  • Use case: Caching, sessions, shopping carts
• 3. Column-Family Stores:
  • Store data in column families (wide rows)
  • Examples: Cassandra, HBase, ScyllaDB
  • Use case: Time-series, analytics, IoT data
• 4. Graph Databases:
  • Store nodes and relationships
  • Examples: Neo4j, Amazon Neptune
  • Use case: Social networks, recommendations, fraud detection

Answer:

Key Differences:

• Data Model:
  • SQL: Tables with fixed rows and columns
  • NoSQL: Flexible (documents, key-value, columns, graphs)
• Schema:
  • SQL: Fixed schema, must define upfront
  • NoSQL: Dynamic schema, can vary per record
• Scaling:
  • SQL: Vertical (bigger servers)
  • NoSQL: Horizontal (more servers)
• Transactions:
  • SQL: ACID (Atomicity, Consistency, Isolation, Durability)
  • NoSQL: BASE (Basically Available, Soft state, Eventually consistent)
• Joins:
  • SQL: Complex joins supported
  • NoSQL: Limited/no joins, data denormalized
• Best For:
  • SQL: Complex queries, transactions, structured data
  • NoSQL: Scale, flexibility, specific use cases

Answer:

BASE is an alternative to ACID transactions, designed for distributed systems:

• B - Basically Available: System guarantees availability, even during partitions. Always responds to requests (might be stale data)
• S - Soft State: System state may change over time without input due to eventual consistency. State isn't instant across all nodes
• E - Eventually Consistent: System will eventually become consistent, but not immediately. Given time without new updates, all replicas converge

Example:

• User updates profile picture on Instagram
• Write succeeds on one node (Basically Available)
• Other users might see old picture for seconds (Soft State)
• After replication completes, everyone sees new picture (Eventually Consistent)

Trade-off: Sacrifices immediate consistency for availability and partition tolerance (CAP theorem).

Answer:

Choose NoSQL when you need:

• Massive Scale: Billions of records, need horizontal scaling
• High Write Volume: Millions of writes/sec (IoT, logs, metrics)
• Flexible Schema: Data structure evolves or varies
• Simple Query Patterns: Mainly key lookups, no complex joins
• High Availability: Must stay online during failures
• Geographic Distribution: Multi-datacenter deployment
• Rapid Development: Schema changes without migrations

Stick with SQL when you need:

• Complex joins across multiple tables
• ACID transactions (banking, e-commerce)
• Strong immediate consistency
• Complex aggregations and analytics
• Well-structured, stable data

Reality: Most applications use BOTH (polyglot persistence)!

Answer:

Eventual Consistency means that if no new updates are made, eventually all replicas will converge to the same data.

How It Works:

• Write accepted on one node immediately
• Asynchronous replication to other nodes
• During replication, different nodes may have different data
• Eventually (typically milliseconds), all nodes consistent

Benefits:

• Higher availability (system always responsive)
• Better performance (no waiting for all replicas)
• Partition tolerance (works during network splits)

When It's Acceptable:

• Social media likes/view counts
• Product recommendations
• DNS records
• Non-critical analytics

When It's NOT Acceptable:

• Bank account balances
• Inventory counts
• Auction bidding
• Booking systems

Answer:

Denormalization is storing duplicate data across multiple records/documents to avoid joins and improve read performance.

SQL Normalized Approach:

Users Table: user_id, name, email
Orders Table: order_id, user_id, product, amount
(Join required to get user with orders)

NoSQL Denormalized Approach:

{
  "order_id": "123",
  "user_name": "John",      // Duplicated data
  "user_email": "john@...",  // Duplicated data
  "product": "Laptop",
  "amount": 999
}

Why NoSQL Uses Denormalization:

• No Joins: NoSQL databases don't support joins well
• Read Performance: All data in one place, single lookup
• Distributed Systems: Joins expensive across multiple servers
• Query-Driven Design: Optimize for how data is accessed

Trade-off: More storage space and update complexity, but much faster reads!

Answer:

Vertical Scaling (Scale Up):

• Add more power to existing server (CPU, RAM, disk)
• Example: Upgrade from 16GB to 64GB RAM
• Pros: Simple, no code changes, strong consistency
• Cons: Hardware limits, expensive, single point of failure
• Used by: Traditional SQL databases

Horizontal Scaling (Scale Out):

• Add more servers to cluster
• Example: Go from 3 servers to 10 servers
• Pros: Unlimited scaling, cost-effective, fault tolerant
• Cons: Complex architecture, eventual consistency
• Used by: NoSQL databases (Cassandra, MongoDB)

Analogy:

• Vertical: Making one restaurant bigger (more tables, kitchen space)
• Horizontal: Opening more restaurant locations across the city

Why NoSQL Prefers Horizontal: Can scale to thousands of servers cheaply, providing unlimited capacity!

Answer:

Polyglot Persistence is using multiple database technologies in a single application, choosing the best database for each specific use case.

Example E-commerce Application:

• PostgreSQL: User accounts, orders, payments (needs ACID transactions)
• Redis: Shopping cart, session storage (needs speed)
• MongoDB: Product catalog (needs flexible schema)
• Elasticsearch: Product search (needs full-text search)
• Cassandra: User activity, analytics (needs scale)
• Neo4j: Product recommendations (needs graph relationships)

Benefits:

• Optimize performance for each use case
• Use right tool for the job
• Avoid one-size-fits-all compromises

Challenges:

• Operational complexity (managing multiple databases)
• Data synchronization between systems
• Requires expertise in multiple technologies

Modern Approach: Most large applications (Netflix, Uber, Amazon) use polyglot persistence!

Answer:

NoSQL databases have several limitations:

• 1. No Joins: Must denormalize data, leading to duplication and update complexity
• 2. Limited ACID Transactions: Most only support single-record atomicity, not multi-record
• 3. Eventual Consistency: Can't guarantee immediate consistency across replicas
• 4. Limited Query Language: Can't do complex SQL-like queries (GROUP BY, subqueries)
• 5. Immature Ecosystem: Fewer tools, less community support than SQL
• 6. Learning Curve: Different paradigm, requires understanding distributed systems
• 7. Data Integrity: No foreign keys or constraints to enforce relationships
• 8. Standardization: Each NoSQL database has different query language and APIs

When These Matter:

• Banking systems need ACID transactions
• Analytics need complex aggregations
• Small teams can't manage distributed complexity

Key Takeaway: NoSQL sacrifices features for scale and performance. Use SQL when you need those features!

Answer:

Key Differences:

• Type:
  • MongoDB: Document Store
  • Cassandra: Column-Family Store
• Architecture:
  • MongoDB: Master-Slave with primary node
  • Cassandra: Peer-to-peer, all nodes equal
• Query Language:
  • MongoDB: Rich query language, ad-hoc queries
  • Cassandra: CQL, requires query planning
• Scalability:
  • MongoDB: Sharding (manual setup)
  • Cassandra: Linear horizontal scaling (automatic)
• Consistency:
  • MongoDB: Strong consistency by default
  • Cassandra: Tunable consistency (eventual by default)
• Write Performance:
  • MongoDB: Good for balanced workloads
  • Cassandra: Exceptional for write-heavy

When to Choose:

• MongoDB: Flexible schema, complex queries, medium scale
• Cassandra: Massive scale, write-heavy, high availability

Answer:

Major Companies Using NoSQL:

• Netflix: Cassandra (2,500+ nodes) for viewing history, recommendations
• Facebook: Created Cassandra, uses for inbox search
• Instagram: Cassandra for user feeds, 400M+ users
• Twitter: Redis for timeline caching, MongoDB for tweets
• Uber: Cassandra for trip data, location tracking
• Apple: Cassandra (100,000+ nodes) for iCloud
• Amazon: DynamoDB (their own NoSQL) for cart, product catalog
• eBay: MongoDB for product catalog (1.4B items)
• LinkedIn: Neo4j for "People You May Know"
• Discord: Cassandra for message storage (150M+ users)

Industry Adoption:

• Gaming: Leaderboards, player profiles (Redis, MongoDB)
• IoT: Sensor data, time-series (Cassandra, InfluxDB)
• E-commerce: Product catalogs, carts (MongoDB, DynamoDB)
• Social Media: Posts, feeds, likes (Cassandra, MongoDB)

Answer:

Schema-less (or Flexible Schema) means you don't need to define the structure of your data before inserting it. Different records can have different fields.

Example in MongoDB:

// User 1 - Basic profile
{
  "name": "Alice",
  "email": "alice@email.com"
}

// User 2 - With additional fields
{
  "name": "Bob",
  "email": "bob@email.com",
  "age": 30,
  "address": "123 Main St",
  "preferences": {
    "theme": "dark",
    "notifications": true
  }
}

Benefits:

• Rapid Development: No migrations when adding fields
• Evolving Requirements: Easy to adapt to changing needs
• Flexible Data: Different entity types in same collection
• No Downtime: Schema changes don't require stopping system

Challenges:

• Data Integrity: No enforcement of required fields
• Application Responsibility: Code must handle varying structures
• Documentation: Need to document expected structure

Reality: Most NoSQL databases support optional schemas for validation!

Answer:

The CAP Theorem states that a distributed database can only guarantee 2 out of 3 properties:

• C - Consistency: All nodes see the same data at the same time
• A - Availability: Every request gets a response (success or failure)
• P - Partition Tolerance: System works despite network failures

Since network partitions WILL happen, you must choose between C and A:

CP Systems (Consistency + Partition Tolerance):

• Prioritize consistency over availability
• System might refuse requests to maintain consistency
• Examples: HBase, MongoDB (with majority read/write)
• Use case: Banking, financial systems

AP Systems (Availability + Partition Tolerance):

• Prioritize availability over immediate consistency
• System always responds (might be stale data)
• Examples: Cassandra, DynamoDB, Riak
• Use case: Social media, analytics, shopping carts

Important: Many NoSQL databases (like Cassandra) let YOU choose the tradeoff per query!

Answer:

Essential Skills for NoSQL Developers:

• 1. Distributed Systems Concepts:
  • CAP theorem understanding
  • Eventual consistency vs strong consistency
  • Partitioning and replication
  • Consensus algorithms (Paxos, Raft)
• 2. Data Modeling:
  • Query-driven design
  • Denormalization techniques
  • Understanding access patterns
  • Avoiding anti-patterns
• 3. Specific Database Knowledge:
  • Query language (CQL, MongoDB query API)
  • Configuration and tuning
  • Operational best practices
• 4. Programming:
  • Database drivers (Python, Java, Node.js)
  • Async programming patterns
  • Connection pooling
• 5. Operations:
  • Cluster management
  • Monitoring and alerting
  • Backup and recovery
  • Performance troubleshooting

Learning Path:

• Week 1-2: NoSQL concepts, CAP theorem, types
• Week 3-4: Choose a database (MongoDB or Cassandra) and learn deeply
• Week 5-6: Data modeling and hands-on projects
• Week 7-8: Operations and production deployment

Career Outlook: NoSQL skills in high demand! Average salary $110K-170K/year!