ON THIS PAGE
## Architecture
The biggest difference among cloud data warehouses are whether they separate storage and compute, how much they isolate data and compute, and what clouds they can run on.
| Feature | Redshift | Druid |
|---|---|---|
| Separation of storage and compute | RA3 instances enable separation of compute and storage, but limited workload isolation compared to other platforms | No |
| Supported cloud infrastructure | AWS only | Can be installed anywhere |
| Isolated tenancy – option for dedicated resources | • Isolated tenant & resources • Runs in your VPC | Single tenant |
| Control vs abstraction of compute | • Configurable cluster size • Configurable compute types | • Complex configuration of compute tier with multiple role-specific nodes • Configurable node count • Configurable compute types (virtual machines or kubernetes) |
| Self-hosted and hybrid deployment options | Limited hybrid options with Redshift Serverless | Self-managed deployment required |
| ACID Compliance and Transactions | ACID compliant at table level with some limitations on concurrent operations | Limited ACID support with eventual consistency |
Redshift has the oldest architecture, being the first Cloud DW in the group. Its architecture wasn't designed to separate storage & compute. While it now has RA3 nodes which allow you to scale compute and only cache the data you need locally, all compute still operates together. You cannot separate and isolate different workloads over the same data, which puts it behind other decoupled storage/compute architectures. Redshift runs as an isolated tenant per customer, and unlike other cloud data warehouses, it is deployed in your VPC. Redshift offers a serverless option which is based on an abstracted unit called Redshift Processing Unit (RPU) ranging from 8 to 512 in increments of 8. Each RPU provides 2 vCPU and 16GB RAM. Thus, 8 RPU is equivalent to 16 vCPU / 128GB RAM. The minimum RPU is 8.
Druid is an OLAP engine designed to provide fast real time analytics. Druid adopts a clustered architecture with servers that host various role specific processes. These processes address real time and batch ingestion, indexing, querying of historical and real time data. Apache Druid can be deployed as a virtual machine or a Kubernetes based cluster. Druid does not support a decoupled compute & storage architecture. Deep storage in the form of object storage is used to replicate data to.
## Scalability
There are three big differences among data warehouses and query engines that limit scalability: decoupled storage and compute, dedicated resources, and continuous ingestion.
| Feature | Redshift | Druid |
|---|---|---|
| Elasticity – Scaling for larger data volumes and faster queries | Available via Elastic Resize – slow and limited, downtime required | Scale-up of nodes requires careful planning and downtime. Addition of new nodes for scale-out is possible |
| Elasticity – Scaling for higher concurrency | • 5 concurrent queries per WLM queue by default (up to 8 queues) • Concurrency Scaling enables thousands of concurrent queries | Supports 100s to 100,000s queries per second (1000+ QPS) with proper configuration and scaling |
Redshift is limited in scale because even with RA3, it cannot distribute different workloads across clusters. While it can scale to up to 10 clusters automatically to support query concurrency, it can only handle a maximum of 50 queued queries across all clusters by default.
Druid provides the ability to handle fast ingest and high concurrency. Custom sizing and cluster tuning are required to balance the compute, memory, storage needs of each process within Druid and to provide high concurrency. Druid clusters can be grown by adding nodes with automatic rebalancing of storage segments assigned to nodes. Self hosted Druid on Kubernetes is an option that users leverage to simplify scaling. Additionally, Cloud based managed Druid offerings are being rolled out. However, these managed offerings are limited in scale and scaling is not granular.
## Performance
Performance is the biggest challenge with most data warehouses today. While decoupled storage and compute architectures improved scalability and simplified administration, for most data warehouses it introduced two bottlenecks; storage, and compute. Most modern cloud data warehouses fetch entire partitions over the network instead of just fetching the specific data needed for each query. While many invest in caching, most do not invest heavily in query optimization. Most vendors also have not improved continuous ingestion or semi-structured data analytics performance, both of which are needed for operational and customer-facing use cases.
| Feature | Redshift | Druid |
|---|---|---|
| Indexes | None | Compressed bitmap indexes for data access and roll-ups to manage aggregations |
| Compute tuning | Choice over number of nodes and their type | On-premises, self-managed hardware. Druid requires infrastructure management and leverages commonly available instance types |
| Storage format | Columnar & compressed storage (RA3 nodes) | Columnar storage format with time-based sorting |
| Table-level partition & pruning techniques | • No table partitions • User-defined distribution & sort keys are used to optimize for speed | Restrictive time-based partitioning. Can partition based on other secondary columns |
| Result cache | Yes | Ability to support caching on broker (set to off by default) |
| Warm cache (SSD) | Only with RA3 nodes at partition-level granularity | Yes, at much larger segment level granularity |
| Support for semi-structured data & JSON functions within SQL | Limited | Recommend flattening JSON or translate to array prior to loading. No support for JSON parsing at query runtime |
| Vector Search and AI Capabilities | Limited AI capabilities – primarily through integrations | No native AI or vector search capabilities |
| Query Optimizations | • Basic query optimizer • Materialized views • Result caching • ANALYZE for table statistics • Workload management (WLM) • Automated materialized views (AutoMV) • AI-driven scaling (Serverless preview) | • Compressed bitmap indexes • Roll-up aggregations • Time-based optimization • Query optimization requires manual tuning |
Redshift does provide a result cache for accelerating repetitive query workloads and also has more tuning options than some others. But it does not deliver much faster compute performance than other cloud data warehouses in benchmarks. Sort keys can be used to optimize performance, but their contribution is limited. There is no support for indexes, and low-latency analytics at large data volumes is hard to achieve. Because Redshift decoupling of storage & compute is limited compared to other cloud data warehouses, it doesn't support isolating workloads, which means performance can degrade under pressure and competition for resources.
Druid provides high performance through columnar storage format, parallel processing, bitmap indexes and roll-ups. Druid, however, recommends a denormalized data model for performance needs. Join operations in Druid are a relatively new feature with various limitations, especially if there is a need to join large datasets.
## Use cases
There are a host of different analytics use cases that can be supported by a data warehouse. Look at your legacy technologies and their workloads, as well as the new possible use cases, and figure out which ones you will need to support in the next few years.
| Feature | Redshift | Druid |
|---|---|---|
| Low-latency dashboards | • Seconds to tens of seconds load times at 100s of GB scale • Can achieve faster performance with Concurrency Scaling and proper tuning | • Sub-second load times optimized for time-series and real-time analytics • Built for high-concurrency interactive dashboards • Requires denormalized data model |
| Enterprise BI | • Mature and comprehensive Enterprise DW feature set • Extensive integrations with Enterprise BI ecosystem • Strong AWS ecosystem integration | • Limited integrations with traditional Enterprise BI tools • Strong for real-time operational dashboards • Requires specialized visualization tools |
| Data Apps and AI Applications (Customer-facing low-latency high concurrency) | • 5 concurrent queries per WLM queue by default (up to 8 queues) • Concurrency Scaling enables thousands of concurrent queries • Seconds-level response times typical • Automatic scaling for burst workloads • Limited AI application support | • Built for high concurrency (1000+ QPS) with distributed architecture • Sub-second response times for time-series data • Optimized for real-time operational applications • No AI capabilities |
| Ad hoc | • Performance dependent on predefined distribution & sort keys • Elastic Resize enables adding compute resources • Typically subset of data loaded for ad-hoc analysis | • Not optimized for ad-hoc queries • Requires predefined roll-ups and data modeling • Limited flexibility for exploratory analysis |
Redshift was originally designed to support traditional internal BI reporting and dashboard use cases for analysts. As such, it is typically used as a general-purpose Enterprise data warehouse. With deep integrations into the AWS ecosystem, it can also leverage AWS ML service, making it also useful for ML projects. However, given the coupling of storage & compute, and the difficulty in delivering low-latency analytics at scale, it is less suited for operational use cases and customer-facing use cases like Data Apps. The coupling of storage and compute, together with the need to predefine sort & dist keys for optimal performance, make it challenging to use for Ad-Hoc analytics.
Druid is designed as an OLAP engine to provide fast access to aggregations that are run against large volumes of data. Druid is typically used for customer facing analytics and streaming data processing. Druid is used as an add-on with other data warehousing products that are efficient at scaling, joining, and filtering large volumes of data. It is not a suitable option for data warehouse replacement.