analytics resource efficiency
June 22, 2022
June 22, 2022

A new level of efficiency in analytics

Listen to this article

Listen to this article

Analyze more. Use less compute resources.

The cloud has given us easier than ever elasticity and scale capabilities for our analytics. But too often our analytics projects have a huge appetite for compute resources, which drives costs up and leaves many projects hanging. In order to unlock value from TB++ scale data, we need technology that can analyze large amounts of data while needing less compute resources to get the job done. This is what Firebolt was built to do. 

Are you spending more than you planned on your Data Warehouse?

Welcome to the club. But you don’t have to compromise on your data & analytics vision to meet your budget. Firebolt is built on a modern and highly efficient engine that can analyze larger volumes of data, at faster speeds, while relying on less compute resources. This means you can deliver richer, more granular and faster analytics experiences, without exceeding your budget.

The same queries, a fraction of the data scanned

One of the biggest performance bottlenecks and compute hoarders in cloud analytics is scanning large files to find the little pieces our queries are interested in. Only after finding the right parts of the data, aggregations and calculations start. 

To solve this, Firebolt approaches how data is stored and pulled differently. Using a proprietary optimized storage format with indexing built into it, the data our queries are looking for can be found with dramatically less data scans, reducing query runtimes significantly.

New data available at index-speed, without cost overhead

A commonly used technique for accelerating queries is to pre-aggregate data, or to rely on materialized views which help with simplifying pre-aggregation. However, this often results in high and unplanned for compute costs, because as new data keeps flowing in, the files containing the aggregated data have to be rewritten. Those rewrites are compute intensive and expensive. 

Firebolt uses a different approach for pre-aggregating data, called “aggregating indexes”. These allow for newly arriving data to be added immediately to the index and available for sub-second querying, without requiring expensive file rewrites. 

High concurrency without expensive scale-out

 A common challenge with cloud data warehouses is supporting high concurrency workloads. One cluster typically starts choking up already at less than a few dozen concurrent queries. Scaling out to a second compute cluster can help with supporting higher concurrencies, but that also immediately doubles the cost of the compute resources. 

Firebolt’s ability to analyze TB++ scale data at sub-second, while keeping hunger for CPU low, means that a single cluster can easily handle hundreds of concurrent queries. This means you can deliver your analytics without adding more expensive clusters as usage climbs. 

Granular control over price-performance with decoupled storage & compute

Firebolt natively decouples storage and compute, which means that you can easily scale and isolate different workloads over the same copy of the database. Compute resources can be spun up and down quickly and programmatically, each sized for its purpose, so you don’t have to spend money on unutilized resources. Firebolt gives users granular control over compute resources, so that you can scale in small linear increments, and not just in doubles. 

Deliver great analytics experiences, on budget - Contact us!

Read all the posts

Intrigued? Want to read some more?