NoSQL databases

The term NoSQL encompasses a diverse family of non-relational distributed databases engineered to overcome the horizontal scaling limits of traditional single-node relational systems. To deliver predictable, single-digit millisecond latency and linear write scaling across massive datasets, NoSQL databases trade away multi-table SQL joins, arbitrary ad-hoc queries, and multi-record transactions.

In exchange for these constraints, NoSQL systems provide elastic write throughput, continuous global availability, and flexible data models that evolve without disruptive schema migrations. Rather than viewing NoSQL as a wholesale replacement for relational databases, experienced systems architects treat it as a specialized set of engines tailored for specific data access patterns.

Architectural models

Cloud providers offer several distinct NoSQL architectures, each optimized for different data shapes and query patterns:

• A is the simplest and most performant distributed database shape. It maps opaque data payloads to unique alphanumeric keys via internal hash rings. Because lookups bypass complex query parsing and relational joins, key-value stores deliver ultra-low-latency point lookups regardless of dataset size. This architecture is the standard foundation for user session management, shopping cart state, distributed caching, and feature flags.
• A manages semi-structured data as hierarchical, schema-free JSON or BSON documents. Unlike pure key-value systems, document stores index fields within the documents, enabling expressive filtering, nested attribute queries, and range evaluations within individual collections. Document stores excel at e-commerce product catalogs with heterogeneous attributes, dynamic content management, and user profiles that gain fields over time without formal database migrations.
• A (descended from Google Bigtable and Apache Cassandra) organizes data into rows with dynamic, sparse column sets ordered sequentially on disk by partition and clustering keys. This sorted layout enables rapid sequential range scans across ordered keys. Wide-column stores are built for high-throughput write ingestion, making them ideal for time-series telemetry, IoT sensor events, and financial transaction logs.
• An in-memory cache is a volatile, RAM-based key-value store that sits in front of persistent storage tiers to serve frequent reads with sub-millisecond latency, preventing read-heavy traffic from overwhelming primary databases.

Architecture	Primary lookup key	Query & filtering capabilities	Common operational use cases
Key-value store	Single primary partition key	Direct key lookups only; no secondary query capabilities	Session management, shopping carts, feature toggles, distributed state
Document store	Document identifier or any indexed attribute	Rich queries, range scans, and secondary indexes within collections	E-commerce product catalogs, user profiles, content management
Wide-column store	Partition key followed by clustered sorting key	Sequential range scans across sorted rows within a partition	High-throughput telemetry, IoT time series, financial ticker logs
In-memory cache	Single primary key	In-memory key lookups; complex data structures (sets, sorted sets)	Low-latency caching layers, rate limiting, leaderboards

Why distributed stores scale

Traditional relational databases maintain foreign keys and transaction logs on a single primary node, making vertical CPU and memory limits an unavoidable architectural ceiling. Distributed NoSQL databases achieve horizontal scalability through (also called sharding), which divides datasets across a cluster of independent storage nodes.

Each node owns a subset of the total keyspace. Because write and read operations against different keys execute on separate physical machines without inter-node locks or shared memory contention, adding more nodes yields linear increases in total cluster throughput.

The trade-off is that queries spanning multiple partitions require coordinating across multiple nodes. A query that does not specify a partition key must execute as a distributed scan across every machine in the cluster, resulting in high latency and heavy resource consumption. Consequently, NoSQL schema design must always begin by identifying the exact queries the application needs to run, selecting keys that align directly with those access patterns.

The partition key

The is the specific attribute passed to the database's internal hash function to determine which physical storage node holds a given record. Selecting the partition key is the single most critical decision in NoSQL architecture.

An effective partition key exhibits high cardinality and distributes writes and reads uniformly across all cluster partitions. Conversely, a poorly chosen key directs disproportionate traffic to a single node, creating a "hot partition." A hot partition quickly exhausts its provisioned throughput, throttling queries and degrading application performance even when the overall cluster is running well below capacity.

Common partition key anti-patterns include:

• Sequential timestamps: Directing all current writes for a given second or minute to the same physical partition.
• Low-cardinality values: Partitioning by status flags, categories, or country codes that group millions of records under a handful of keys.
• Skewed tenant identifiers: In multi-tenant systems where a single enterprise client generates 80% of all traffic, using tenant ID as the sole partition key creates severe partition skew. High-entropy keys (such as user UUIDs or composite keys combining tenant IDs with entity hashes) provide much more balanced data distribution.

Choosing a partition key

Derive partition key design directly from your application's primary query access patterns.

1. Document every specific query the application will execute, noting the exact fields known at query time.
2. Select a partition key that is present in the majority of queries and provides high cardinality across distinct values.
3. Verify that write and read volumes will distribute uniformly across partitions, avoiding sequential timestamps or skewed tenant IDs.
4. If secondary queries require searching on non-partition attributes, create targeted global secondary indexes rather than performing full table scans.

Consistency

Relational databases guarantee that every read receives the most recent committed write. In a distributed NoSQL store, data is replicated across multiple nodes and zones. When a write is accepted by the primary partition replica, changes propagate asynchronously to secondary replicas.

A read that queries a replica before the update has propagated may return stale data; this behavior is known as . Eventual consistency provides maximum read throughput and resilience against node outages, making it appropriate for product reviews, social feeds, and recommendation engines. However, for critical workflows where reading stale state causes financial or business errors (such as verifying account balances or validating user authorizations), applications must explicitly request strongly consistent reads. Strongly consistent reads query a quorum of replicas to guarantee data freshness, which incurs slightly higher latency and reduced availability during network partitions.

Billing and cost management

Managed NoSQL databases decouple billing from traditional server provisioning, charging based on consumed read and write throughput alongside stored gigabytes. Providers offer both provisioned capacity (reserving fixed reads and writes per second) and on-demand pricing (billing per individual operation).

On-demand billing is ideal for unpredictable or bursty workloads with intermittent traffic. However, unindexed queries that perform full-table scans read every record in the table, consuming immense capacity units and producing shocking billing spikes. Enforcing strict query patterns that leverage partition keys protects both system latency and operational budgets.

When NoSQL is the right answer

NoSQL databases are the right architectural choice when write throughput exceeds the physical capacity of a single relational primary, when records are inherently semi-structured and dynamic, or when systems require predictable, microsecond-to-millisecond point-lookup latencies at global scale.

If your data model is deeply relational, requires multi-table joins, or demands strict transactional atomicity across multiple entities, a relational database remains the superior tool. Attempting to implement manual joins across multiple NoSQL tables within application code is inefficient, prone to concurrency bugs, and essentially rebuilds an unoptimized relational database engine.

Terms introduced

• Key-value store: a database that returns a value for a key, at any scale, and does nothing else.
• Document store: a database that holds JSON-like documents and indexes fields inside them.
• Wide-column store: a database of keyed, ordered rows with sparse columns, read by range.
• Partitioning: splitting data by key across machines so that writes to different keys never contend.
• Partition key: the field that decides which machine holds a record, and therefore which queries are fast.
• Eventually consistent: a read that may return a value written a moment ago rather than the latest one.

---

How providers do it

Distributed NoSQL offerings reflect differing provider philosophies: AWS maintains distinct, specialized products for each data shape; Azure consolidates multiple NoSQL data models under Azure Cosmos DB; and Google Cloud provides two purpose-built engines in Cloud Firestore and Cloud Bigtable.

Concept	AWS	Azure	Google Cloud
Key-value store	Amazon DynamoDB	Azure Table Storage, Cosmos DB Table API	Cloud Firestore (or Cloud Bigtable for high throughput)
Document store	Amazon DynamoDB; Amazon DocumentDB (MongoDB API)	Azure Cosmos DB (NoSQL or MongoDB API)	Cloud Firestore
Wide-column store	Amazon Keyspaces (Apache Cassandra compatible)	Azure Cosmos DB (Cassandra API)	Cloud Bigtable
In-memory caching layer	Amazon ElastiCache, DynamoDB Accelerator (DAX)	Azure Cache for Redis, Azure Managed Redis	Google Cloud Memorystore
Partitioning identifier	Partition key (with optional sort key)	Partition key (defined at container creation)	Row key (Cloud Bigtable); automatic document indexing (Firestore)
Throughput capacity metric	Read/Write Capacity Units (RCU/WCU), or On-Demand	Request Units per second (RU/s)	Per-operation pricing (Firestore); node provisioning (Bigtable)
Change data capture stream	DynamoDB Streams	Cosmos DB Change Feed	Firestore triggers, Bigtable change streams
Multi-region active writes	DynamoDB Global Tables (last-writer-wins)	Cosmos DB multi-region writes	Multi-region Spanner (Firestore and Bigtable are regional/dual-region)

Every product name and technical mapping above is confirmed against provider documentation. Specific feature compatibility limits for DocumentDB and operational tiers for Azure Managed Redis are marked unconfirmed in the provider tabs below.

A crucial behavioral difference involves default read consistency:

• Amazon DynamoDB defaults to eventually consistent reads, requiring an explicit SDK parameter (ConsistentRead = true) to enforce strong consistency.
• Google Cloud Firestore provides strong consistency by default for document and collection queries. Migrating application logic between these platforms without reviewing consistency configurations can introduce subtle read-after-write bugs.

AWS

What this maps to: Amazon DynamoDB is the key-value and document store. It is also what most people mean when they say "NoSQL on AWS". The cache is Amazon ElastiCache. If you arrive with Cassandra or MongoDB code, Amazon Keyspaces and Amazon DocumentDB are the compatibility products for running it.

Concept	On AWS	Status
Key-value and document store	DynamoDB. Items are keyed by a partition key and an optional sort key	confirmed
Billing	On-demand per request, or provisioned read and write capacity units with autoscaling	confirmed
Consistency	Reads are eventually consistent by default; a strongly consistent read costs twice as much and is not available on global secondary indexes	confirmed
Queries without the key	Global secondary index (any key) or local secondary index (same partition key, different sort)	confirmed
Hot partition	DynamoDB spreads throughput across partitions; a single key taking a large share is throttled. Adaptive capacity softens this	confirmed
Cache	ElastiCache for Redis OSS, Valkey, or Memcached. DAX is a cache built specifically for DynamoDB	confirmed
Wide-column	Keyspaces, Cassandra-compatible	confirmed
Document, MongoDB API	DocumentDB, MongoDB-compatible, with the API version it supports lagging upstream	unconfirmed; check current compatibility
Change feed	DynamoDB Streams, which can trigger a Lambda function on every write	confirmed
Multi-region	Global tables, replicating a table across regions with last-writer-wins conflict resolution	confirmed

Their vocabulary

Standard term	Their term
Record	Item
Partition key	Partition key (hash key in older docs)
Range within a partition	Sort key (range key in older docs)
Throughput	Capacity units

Where to look

Consumed capacity and throttled requests are on the table's Metrics tab in the DynamoDB console. To see which keys are hot, turn on Contributor Insights.

Last verified: never.

Azure

What this maps to: Azure Cosmos DB is one product that covers every shape on the core page, through several APIs. For caches you have Azure Cache for Redis and Azure Managed Redis. If you only need a cheap, simple key-value store, Table Storage lives inside a storage account.

Concept	On Azure	Status
Document store	Cosmos DB with the NoSQL API (its native API) or the MongoDB API	confirmed
Key-value and wide-column	Cosmos DB with the Table API or the Cassandra API; Gremlin API for graphs	confirmed
Partition key	Chosen per container at creation and cannot be changed afterwards. Logical partitions have a size cap, so a key with one very large value hits it	confirmed
Billing	Request units (RUs) per second, either provisioned (with autoscale) or serverless per request, plus storage. Every read and write has an RU cost that the response reports	confirmed
Consistency	Five levels, from strong through bounded staleness, session, consistent prefix, to eventual, set per account with a per-request override. Session is the default	confirmed
Cache	Azure Cache for Redis, and Azure Managed Redis on newer Redis versions	unconfirmed; check current positioning of the two
Change feed	Cosmos DB change feed, consumable from Functions	confirmed
Multi-region	Any account can add read regions. Multi-region writes are a setting	confirmed
Simple key-value	Table Storage, inside a storage account, with the same API as the Cosmos DB Table API and a fraction of the cost	confirmed

Their vocabulary

Standard term	Their term
Record	Item (NoSQL API), document (MongoDB API), entity (Table)
Table	Container
Throughput	Request units per second
Hot partition	Hot partition, visible in the Insights blade

Where to look

The Cosmos DB account's Insights blade shows RU consumption, throttled requests (HTTP 429), and the hottest partition key ranges.

Last verified: never.

Google Cloud

What this maps to: Firestore is the document store and Bigtable the wide-column store. Memorystore is the cache. Google Cloud has no separate key-value product. Firestore or Bigtable covers it, depending on which side you come at it from.

Concept	On Google Cloud	Status
Document store	Firestore, with a native mode and a Datastore mode for the older API. Documents live in collections. Every query needs an index, though single-field indexes are created for you automatically	confirmed
Consistency	Firestore reads are strongly consistent by default	confirmed
Billing	Firestore bills per document read, write, and delete, plus storage. No provisioned capacity to size	confirmed
Wide-column	Bigtable, with rows ordered by a row key and read by range. Sized by nodes; billed per node-hour plus storage	confirmed
Hot partition	Bigtable calls it hotspotting. Row keys that share a prefix, such as a timestamp, concentrate load on one node. Key Visualizer shows it	confirmed
Cache	Memorystore for Redis, Valkey, and Memcached	confirmed
Partition key	Bigtable: the row key. Firestore: none to choose; the service partitions collections itself	confirmed
Change feed	Firestore triggers through Eventarc; Bigtable change streams	confirmed
Multi-region	Firestore multi-region locations; Bigtable replication across clusters in different regions	confirmed
Real-time listeners	Firestore pushes document changes to connected clients, which the mobile and web SDKs use	confirmed

Their vocabulary

Standard term	Their term
Record	Document (Firestore), row (Bigtable)
Partition key	Row key (Bigtable)
Table	Collection (Firestore), table (Bigtable)

Where to look

Firestore's Usage tab shows reads, writes, and deletes per day. For Bigtable, the monitoring page has CPU per node and Key Visualizer has heat by key range.

Last verified: never.

---

Check your understanding

0 of 4 answered

1. Why can a partitioned NoSQL store take more writes than a relational database?

   Data is split by key across machines, so writes to different keys go to different machines and never contend.It skips the disk and writes only to memory.It does not check the data before writing it.It compresses each write.
2. A time-series table uses the timestamp as its partition key. What goes wrong?

   Every write in a given second lands on one partition, which becomes hot while the rest sit idle.The timestamps collide and overwrite each other.Queries by time range become impossible.Nothing, because timestamps have many distinct values.
3. An application keeps orders, customers, and products in three separate key-value stores and joins them in code. What has it built?

   A relational database, badly. The data is relational and belongs in one.A correct microservice design.A cache layer.An event-driven system.
4. Which read must not be eventually consistent?

   A balance check immediately before taking a payment.The number of likes shown on a post.A product description on a catalogue page.A user's display name in a comment thread.