Firebolt uses advanced query processing techniques such as granular range-level data pruning with sparse indexes, incrementally updated aggregating indexes, vectorized multi-threaded execution, and tiered caching, including sub-plan result caching.These techniques both minimize data being scanned and reduce CPU time by reusing precomputed, enabling query processing times in tens of milliseconds latency on hundreds of TBs of data.
Yes. By default, creating an engine would result in the creation of the underlying engine clusters and start the engine. This would enable the engine to be in a running state where it is ready to start serving the queries. However, you have the option to defer the creation of the underlying clusters for an engine by setting the property “INITIALLY STOPPED” to True while calling CREATE ENGINE. You can start the engine at a later point, when you are ready to start running queries on the engine. Note that you cannot modify this property after an engine has been created.
CREATE ENGINE IF NOT EXISTS MyEngine WITH
TYPE = “S” NODES = 2 CLUSTERS =1 START_IMMEDIATELY = FALSE;
We use AWS Shield, WAF, and other logical layers to protect against DDoS. Additionally, we leverage auto-scaling to maintain availability during attacks by dynamically adjusting resources like EC2 instances, ELBs, and other global services capacity. (Though some scenarios may require manual intervention).
FBU consumption is reported in real time and can be used to calculate costs by multiplying the consumed FBU by the price listed on the pricing page or a custom deal rate. However, the Billing and Consumption page updates daily, and AWS storage costs have a ~48-hour delay. For more information, check our documentation.
To start an engine:
sql
START ENGINE MyEngine;
To stop an engine:
vbnet
STOP ENGINE MyEngine;
For more information, please refer to the Work with Engines Using DDL article in the Firebolt Documentation.
Firebolt's Python SDK provides detailed error message handling for SQL queries. When an error occurs, the SDK generates helpful error messages, allowing users to quickly diagnose and fix issues such as syntax problems or missing credentials. The SDK also offers robust logging and debugging capabilities, making it easier for developers to troubleshoot errors in their applications. For more information, refer to the Firebolt Python SDK documentation or visit the GitHub repository for examples.
Firebolt employs a comprehensive security strategy that includes network security policies, encryption practices, tenant isolation, and governance controls. We are committed to safeguarding your data through state-of-the-art security systems, policies, and practices.
You can use the AUTO_STOP feature available in Firebolt engines to make sure that your engines are automatically stopped after a certain amount of idle time. Engines in stopped state will not be charged, hence do not incur any costs. As with other engine operations, this can be done via SQL or the UI. For example, while creating an engine, you can specify the idle time, using AUTO_STOP, as below:
CREATE ENGINE IF NOT EXISTS MyEngine WITH
TYPE = “S” NODES = 2 CLUSTERS =1 AUTO_STOP = 15;The above command will ensure that MyEngine will be automatically stopped if it has been idle for 15 minutes continuously. Alternatively, you can achieve the same after an engine has been created.
ALTER ENGINE MyEngine SET AUTO_STOP = 15;For mor information, please see the Engine Consumption Documentation.
Multistage distributed execution allows complex ELT queries to utilize all cluster resources by splitting stages across nodes. This parallelization optimizes resource utilization, speeding up ELT processes.
When using service accounts in Firebolt, access errors can occur due to incorrect credentials, missing permissions, or expired tokens. To troubleshoot these errors, follow these steps:
- Check your credentials: Ensure that the service account credentials (client ID and secret) are correct and active.
- Verify permissions: Make sure the service account has the appropriate role or permissions to access the resources you are trying to interact with, such as S3 buckets or Firebolt tables.
- Refresh authentication tokens: If you're using tokens, ensure they have not expired. Tokens generated for service accounts typically have a limited lifespan.
- Logs and error details: Review the error message logs for more specific information about the access failure.
- Review documentation: Follow the Firebolt documentation on service accounts for proper setup, permissions, and token management.
If the engine has the AUTO_START option set to True, an engine in a stopped state will be automatically started when it receives a query. By default, this option is set to True. If this option is set to False, you must explicitly start the engine using the START ENGINE command. For more information, please refer to the Work with Engines Using DDL article in the Firebolt Documentation.
To work with authentication tokens in Firebolt:
Generate a token via Firebolt’s authentication endpoint using your client ID and secret.
Example:
curl -X POST https://id.app.firebolt.io/oauth/token \
--header 'Content-Type: application/x-www-form-urlencoded' \
--data-urlencode 'grant_type=client_credentials' \
--data-urlencode 'client_id=YOUR_CLIENT_ID' \
--data-urlencode 'client_secret=YOUR_CLIENT_SECRET'Use the token in API requests by including it in the authorization header:
--header 'Authorization: Bearer YOUR_ACCESS_TOKEN'Refresh tokens regularly, as they expire. Keep tokens secure using environment variables or secret managers.
For more details, refer to the Firebolt API documentation.
Firebolt provides three different observability views that provide insight into the performance of your engine.
1/ engine_running_queries - This view provides Information about currently running queries. This includes whether a query is running or in the queue. For queries that are currently running, this view also provides information on how long it has been running.
2/ engine_query_history - This view provides historical information about past queries - for each query in history, this includes the execution time of the query, amount of CPU and Memory consumed and amount of time the query spent in queue, among other details.
3/ engine_metrics_history - This view provides information about the utilization of CPU, RAM and Storage for each of the engine clusters. You can use these views to understand whether your engine resources are being utilized optimally, whether your query performance is meeting your needs, what percentage of queries are waiting in the queue and for how long. Based on these insights, you can resize your engine accordingly.
For more information, please refer to the Sizing Engines article in our Documentation.
Use the engine_metering_history information schema view to track FBU consumption for each engine.
Yes, ELT processes can be automated using: Firebolt Python SDK for programmatic database operations. Apache Airflow for scheduling and automating complex workflows. dbt (Data Build Tool) for managing data transformations in a version-controlled environment.
Yes. The engine_running_querie and engine_query_history tables provide insights into current workloads. For more information see our Information Schema Documenation.
Firebolt enables running ELT jobs on a separate engine isolated from the customer-facing engine. This prevents disruptions and allows scaling ELT engines dynamically with auto-start and auto-stop features to reduce costs.
Monitoring the status of your Firebolt engine using the REST API is a key step to ensure smooth operations. Firebolt provides a way to programmatically check engine status by querying the system engine. This article explains how to authenticate, retrieve the system engine URL, and query the engine status using Firebolt's REST API.To begin, ensure that you have an active service account with the necessary permissions. You will need the service account credentials to generate an access token for API authentication.
After obtaining an access token, use the following request to retrieve the system engine URL:
curl https://api.app.firebolt.io/web/v3/account/<account_name>/engineUrl \
-H 'Accept: application/json' \
-H 'Authorization: Bearer <access_token>'Once you have the system engine URL, you can query it to check the engine's status with a simple SQL query, as shown below:
curl --location 'https://<system_engine_URL>/query' \
--header 'Authorization: Bearer <access_token>' \
--data "select status from information_schema.engines where engine_name = '<your_engine_name>'This will return the current status of your engine, helping you monitor its activity and health.
For more information please refer to the Using the API Documentation.
No, Firebolt doesn't support adding new columns without rebuilding the table. However, you can create a new table with the updated schema or use views to simulate schema changes.
This ensures that the schema remains optimized for performance, which is critical in high-performance analytical databases like Firebolt.
Alternatively, Firebolt offers a flexible approach where you can create views to simulate changes like renaming or restructuring tables without needing to rebuild or re-ingest data. For instance, you can create a view that selects all columns from the original table, effectively simulating the addition of new columns:
Example Usage: To simulate renaming a table or altering its structure, create a view:
CREATE VIEW IF NOT EXISTS new_games ASSELECT * FROM games;This approach allows you to redirect queries to the new view (new_games), making it function like a table with updated schema without altering the original table.
Firebolt uses Firebolt Units (FBU) to track engine consumption. For example, for an engine with Type "S", 2 nodes, and 1 cluster running for 30 minutes, it would consume 8 FBUs:
FBU per Hour = 8 * 2 * 1 = 16 FBUs
Consumption = (16 / 3600) * 1800 seconds = 8 FBUs
Firebolt database itself inherently reduces the risk of SQL injection by minimizing the use of certain vulnerable constructs. Customers are still encouraged to implement additional controls at their application level such as:
- Ensure all user inputs are strictly validated before being processed.
- Escape potentially dangerous characters that could be used in unexpected ways.
- Include SQL injection tests in your regular security testing and code review processes.
Use the AUTO_STOP feature to automatically stop engines after a certain amount of idle time. Example:
:ALTER ENGINE MyEngine SET AUTO_STOP = 15;For more information, read more about Engine Consumption in our Documentation.
CSV file ingestion into an external table may fail with errors such as:
Cannot parse input: expected '|' but found '<CARRIAGE RETURN>' instead.This usually means the file delimiter in the CSV doesn't match the table definition or the number of columns differs.
To troubleshoot check the delimiter: Ensure FIELD_DELIMITER matches the CSV file's delimiter. Also, compare the file to the table definition column-by-column. Finally, if the file is large, create a temporary external table to view the entire row as a single string:
CREATE EXTERNAL TABLE ext_tmp (blob text) URL = 's3://some_bucket/somefolder/' TYPE = (CSV FIELD_DELIMITER=' ');Example Query:
SELECT * FROM ext_tmp LIMIT 2;This helps inspect rows and verify column consistency.
You can use the engine_metering_history information schema view for detailed tracking of FBU consumption.
Use the NULLIF function to convert empty strings to NULL, which can then be cast to the appropriate data type.
When casting columns to data types like DATE or NUMERIC in Firebolt, empty strings in the source data can cause errors. This occurs because empty strings cannot be directly cast to other data types.
Use the NULLIF function to convert empty strings to NULL, which can then be cast to the appropriate data type without causing errors.
Example:
INSERT INTO tournaments_nullif_example_fact
SELECT
NULLIF(dt, '')::date
FROM tournaments_nullif_example;In this example, NULLIF(dt, '') converts empty strings in the dt column to NULL, allowing the data to be safely cast to a DATE type. This method ensures smooth casting of columns with empty strings in Firebolt.
Firebolt provides Role-based Access Control (RBAC) to help customers control which users can perform what operations on a given engine. For example, you can provide users with only the ability to use or operate existing engines but not allow them to create new engines. In addition, you can also prevent users from starting or stopping engines, allowing them to only run queries on engines that are already running. These fine-grained controls help ensure that customers do not end up with runaway costs resulting from multiple users in an organization creating and running new engines.
Customer data is stored in S3 buckets with high availability and durability. Our recovery objectives are:
- RTO (Recovery Time Objective): 12 hours
- RPO (Recovery Point Objective): 1 hour
- SLA (Service Level Agreement): 99.9%
To investigate query timeouts or delays, you can start by using Firebolt’s Query History and Query Profile tools, which provide detailed insights into query performance, including execution time, memory usage, and any potential bottlenecks. You can also check engine logs and metrics using Firebolt’s Engine Metrics History to identify issues like memory limitations, network latency, or resource constraints.
For troubleshooting steps, check the Firebolt documentation on query analysis.
Spilling happens when a query requires more memory than allocated, causing intermediate query results to be stored on disk (SSD) instead of in-memory. While this ensures the query completes, it may affect performance.
For more information, check the Firebolt Documentation on Engine Metrics History.
Set up SSO authentication: https://docs.firebolt.io/godocs/Guides/security/sso/sso.html
Configure your IdP: https://docs.firebolt.io/godocs/Guides/security/sso/configuring-idp-for-sso.html#custom.
Stopping an engine in Firebolt results in the eviction of the local cache. This leads to a "cold start" upon restarting the engine, as queries initially must fetch data directly from storage, slowing down performance until the cache is replenished with frequently accessed data. To minimize performance degradation, consider pre-warming the engine with essential queries or data after it is restarted. For more information please check our documentation article Work with Engines using DDL.
What is the system engine, and how is it used for metadata-related queries?The system engine in Firebolt is a lightweight, always-available engine specifically designed for metadata-related queries and administrative tasks. It supports various commands:
Access Control Commands: Manage roles, permissions, and users.
Metadata Commands: Execute queries on information schema views, such as information_schema.tables and information_schema.engines.
Non-Data Queries: Perform operations like SELECT CURRENT_TIMESTAMP() that do not involve table data.
Typical Use Cases:
Retrieve information about databases, tables, indexes, and engines.
Manage system configurations or user permissions.
Execute DDL operations like creating tables and views, and managing all engine-related operations (start, stop, drop, alter).
Yes, Firebolt allows for dynamic resizing of engines during operation. You can adjust the number of nodes or the node type without stopping the engine, which lets workloads continue with minimal disruption. Use the ALTER ENGINE command to resize an engine. Newly started clusters post-resize will initially perform slower until they are warmed up.
When an engine is resized dynamically, queries in execution will continue under the engine's original configurations until completion or until a timeout of 24 hours, after which they will be dropped if still running. The changes in engine size or type will only affect new queries submitted post-resize. Please check our documentation for more information.
You can query the INFORMATION_SCHEMA.ENGINES table to check the last known status of an engine:
SELECT engine_name, last_started, last_stopped
FROM INFORMATION_SCHEMA.ENGINES;
This will show when each engine was last started and stopped.
Scaling with More Clusters:
This approach is ideal when you need to improve query concurrency—i.e., the ability to handle multiple queries simultaneously without significant performance degradation.
Scaling with a Higher Number of Nodes:
This is suitable when you find that the CPU utilization is consistently high, and queries are CPU-intensive. Adding more nodes spreads the workload across more computing units, thus alleviating CPU bottlenecks.
Scaling with Bigger Nodes:
This method is effective when the workload requires more memory or higher disk I/O capacity than what is currently available.
If it's a new account, you'll need to set up a new user within that account, although this user can be linked to the existing service account.
If it's a new organization, you'll need to establish both a new service account and a new user within that organization and any associated accounts.
Firebolt’s AI-related features, such as vector search, are included within our standard pricing model. While these capabilities do utilize compute resources, there are no separate licensing fees or AI-specific upcharges. You only pay for the compute and storage you use, ensuring cost efficiency without hidden AI-related costs.
For a detailed breakdown of how AI workloads impact pricing, reach out to our team for a tailored estimate based on your use case.
Firebolt is purpose-built for AI applications that require low-latency analytics. Unlike traditional cloud data warehouses, Firebolt delivers sub-second query performance for AI-driven workloads while supporting vector search and AI-driven optimizations. It enables faster and more efficient AI-powered analytics without the high costs and performance bottlenecks of legacy solutions.
Firebolt is optimized for low-latency, high-performance queries, but it is not a real-time processing platform. It excels in fast analytics on large-scale data but is not designed for event-driven streaming workloads. If your AI use case requires sub-second query execution, Firebolt is a great fit.
Firebolt supports vector search but does not generate embeddings. Unlike dedicated vector databases, which specialize in unstructured data, Firebolt integrates vector search within a high-performance analytical data warehouse. This allows you to run hybrid queries (structured + unstructured) efficiently without managing separate systems.
If you already have embeddings generated from models like OpenAI, Hugging Face, or your own ML pipeline, Firebolt can store and query them at high speed and low latency, enabling AI-powered search and recommendations within your existing analytics environment.
Firebolt is built on advanced indexing, vectorized query execution, and efficient storage optimizations, ensuring sub-second query performance even on large datasets. While it is not a real-time platform, Firebolt is ideal for AI-driven analytics, interactive dashboards, and personalized AI applications that demand ultra-fast queries on your data.
Users can verify the correct setup by ensuring they have obtained the API token, created the service account, and set up the appropriate automation steps. They can test ingestion by running queries to check if data has been successfully loaded into Firebolt.
Firebolt users should define primary indexes based on frequently filtered columns, such as event dates and brand identifiers. By including relevant dimensions in the index, query performance can be significantly improved, as seen in the session where adding "brand" as a primary index reduced query execution time from minutes to seconds.
Firebolt does not natively save queries across different user accounts. Users need to manually copy queries and store them externally, such as in Slack, Google Docs, or a shared repository, to ensure accessibility across their team.
Users should avoid scanning large datasets unnecessarily by leveraging filtering on indexed columns. For example, filtering on both "event date" and "brand" significantly improves performance. Additionally, aggregating indexes can be used to precompute and store frequently used aggregations, reducing query execution time.
Aggregating indexes in Firebolt store precomputed aggregations for faster query performance. They update automatically upon new data ingestion, reducing query execution time significantly. The trade-off is a slightly increased ingestion time since the indexes must be maintained.
Yes, Firebolt provides a query execution plan. Users can view a visual representation after query execution or generate a text-based plan using the EXPLAIN command. This helps users compare performance against other engines like Athena.
In the discussion, the Firebolt team recommended creating new, pre-joined (or otherwise streamlined) tables rather than performing large, multi-table joins at query time. This approach, sometimes called "join elimination," can significantly reduce query overhead. In addition, the Firebolt team highlighted the importance of setting appropriate primary indexes on these new tables to further optimize performance.
Firebolt does not strictly require more tables; however, to achieve high-performance queries on large datasets, many teams choose to create specialized tables with carefully designed primary indexes. Although Firebolt can perform joins directly in SQL, the discussion emphasized that pre-joining or restructuring certain data tables often yields better performance. This approach leverages Firebolt’s indexing and reduces the run-time cost of large joins.
Firebolt advises creating a brand-new organization under the desired domain or AWS account via the standard sign-up process. Once the new organization is set up, copy over any needed configurations, tables, or data from the old organization. Because a new organization starts as a new trial, you also receive fresh Firebolt usage credits. After verifying that everything works correctly in the new organization, the old one can be retired or deleted.
Batch inserts are generally recommended for Firebolt. Inserting rows one at a time creates excessive overhead on the engine, leading to performance issues (especially on smaller engines). By sending records in small to moderate batches (e.g., once per second or at some reasonable time interval), the engine processes data more efficiently without overloading resources.
Use derived date columns (e.g., day-level granularity) to lower cardinality. For instance, store closed_at_day by truncating a TIMESTAMP to DATE. Incorporate the derived column (e.g., closed_at_day) into the primary index alongside other frequently used filters (e.g., tenant_id), allowing Firebolt to skip irrelevant data segments. Because raw timestamp columns can be extremely granular, indexing them directly often leads to poor selectivity. Restructuring the schema to include day- or hour-level columns can significantly improve performance. Leverage Firebolt’s caching features (result cache, sub-result cache) for repeated queries.
Firebolt has introduced an auto-vacuum feature that runs on the write engine. It triggers after a set number of transactions and reclaims space without blocking ingestion. Manual vacuuming is largely unnecessary in most use cases now. A dedicated engine is not typically required; auto-vacuum operates seamlessly on the existing write engine.
Query performance primarily depends on the amount of data scanned. If queries remain selective (e.g., filtering by tenant ID and a truncated date range), Firebolt only scans relevant slices, keeping query times stable as data grows. Broad queries (e.g., SELECT * over wide date ranges) will naturally slow as more data must be scanned. Best practices include indexing on commonly used filters, leveraging caching, and avoiding unbounded queries to maintain good performance over time.
Firebolt offers multiple engine sizes (S, M, L, etc.). Smaller engines can handle many queries per minute if those queries are well-optimized. Heavier workloads or larger datasets may require a bigger engine or multiple concurrent engines. Firebolt charges by actual runtime (hourly or per second). Costs can be reduced by auto-stopping engines when not in use.
A viable workaround is to export query results to a file in S3, then read and process that file in smaller chunks. While it adds complexity (you must manage file paths, permissions, and cleanup), it avoids buffering the entire result set in application memory until real streaming is available in the Firebolt Node.js SDK.
Firebolt can ingest data at terabytes-per-hour scale, supported by internal benchmarks (e.g., half a terabyte in ~800 seconds on four S-sized engines). Actual throughput depends on factors such as file format, table schema, partitioning, and engine size. Organizations can scale up (larger engines or more engines) to accelerate big batch loads and scale down for smaller, more frequent delta loads.
Order columns by frequency of use in queries. For example, if tenant_id and closed_at_day appear in most filters, list them first. Within equally common columns, order from lowest cardinality to highest cardinality (fewest unique values to most). This approach ensures that Firebolt’s indexing effectively prunes unnecessary data scans for highly repetitive or frequently queried columns.
Firebolt’s Customer Success and Support teams provide complementary query optimization guidance—including index design, join performance tuning, and ingestion configuration—at no extra cost. Users can reach out via Slack or support tickets for best practices and troubleshooting. In many cases, optimization is a collaborative, ongoing process. If you notice a slower query, you can share it with Support; sometimes the solution is a schema or indexing change, other times a product fix may be required.
In most cases, no. The sub-result cache benefits queries that overlap in the underlying data scanned or join results. If the tenant ID changes and there is little or no data overlap, the previous sub-results become irrelevant, so the cache will not offer a speed-up.
Queries with highly selective filters (e.g., smaller date ranges or high-selectivity columns) scan less data and often run in sub-second time. Queries that must scan large portions of the dataset (e.g., SELECT * over a broad date range) naturally take longer, especially on first (cold) runs when data must be read from storage. Firebolt’s sub-result caching reduces execution time for repeated or similar queries by caching portions of join results and aggregations. Proper indexing on commonly used filter columns can also significantly reduce the amount of data scanned, improving performance.
Firebolt is designed for a “decoupled compute” architecture where you can spin up separate engines for different workloads. A dedicated write engine handles ingestion, while one or more read engines handle queries. This ensures that write operations do not slow down queries and vice versa. You can also configure auto-start/auto-stop so that engines only run (and incur costs) when needed.
Firebolt’s ingestion engine can be turned off when not actively loading data. Many teams schedule ingestion windows (e.g., hourly or daily) and then auto-stop the engine to save on costs. Billing is based on actual runtime, so you are not charged for idle ingestion clusters.
Separate Accounts: Each account cleanly isolates its data and can map to separate AWS buckets or IAM roles. Single Account with Multiple Databases: Environments share the same account, so you must carefully permission each database. Most teams that maintain separate AWS resources (e.g., dev vs. staging vs. production buckets and roles) find it more straightforward to mirror that approach with separate Firebolt accounts.
Before auto vacuum, you would typically schedule vacuum after a certain number of inserts or on a time-based schedule (e.g., nightly). Firebolt’s auto vacuum feature (released around early 2025) automatically triggers a non-blocking vacuum every few hundred transactions in the background, substantially reducing or eliminating the need for manual vacuum scheduling. This occurs with minimal overhead and typically does not require an engine size increase.
Firebolt support engineers have the ability to access customer accounts for troubleshooting via Okta—only if they have the specific permissions. While it is generally recommended to keep support access open for fast incident resolution, you can request to block or limit their access if you have strict security requirements.
One approach is to restructure the table by setting the primary index on event_time to better leverage Firebolt’s indexing capabilities. Additionally, an aggregating index on event_time can be beneficial. However, if queries still take longer than expected (e.g., 15 seconds for 30 days of data), it may help to review: - The structure of the primary index and ensure it aligns with the query’s filtering. - Whether unnecessary dimensions are included in the dataset, increasing granularity unnecessarily. - If joins or aggregations can be optimized, possibly through pre-aggregated tables. Firebolt’s architecture is designed to improve query efficiency by avoiding costly full scans and optimizing indexing structures.
Yes, Firebolt can support a single table design that includes multiple reporting dimensions, such as unique counts, event times, and injected data. This consolidation can improve performance by reducing the need for complex joins and maintaining a single source of truth for analytics. However, when merging different data use cases into a single table, it is important to: - Optimize indexing to balance performance across different query patterns. - Consider partitioning or using aggregating indexes to precompute frequent aggregations. - Evaluate whether all reporting needs can be met within a single table without sacrificing efficiency.
The “1.8 TB” figure refers to the SSD cache associated with a particular engine, not the total limit on data Firebolt can handle. Firebolt stores your full data in S3 for effectively unlimited capacity. Only the segments (tablets) relevant to a query are pulled into the SSD cache for faster processing. If your dataset exceeds 1.8 TB, Firebolt will still process it by cycling portions of data into and out of the SSD cache (first-in, first-out).
By subscribing through AWS Marketplace, you can consolidate Firebolt billing under your existing AWS billing arrangements. You will be directed to complete a few additional steps (“more clicks”) to finalize the purchase. Once completed, charges for your Firebolt usage appear in your AWS bill, simplifying vendor management if you prefer a single billing channel.
Firebolt has “GenAI” initiatives on its product roadmap. While exact capabilities may evolve, the published information highlights plans for: AI-Assisted Querying (e.g., query recommendations, natural language querying), Auto-Tuning & Optimization powered by machine learning, and Improved Developer Experience leveraging AI-based insights. For a deeper discussion of upcoming features, Firebolt can arrange a roadmap review session with its product team.
It is not clearly documented whether you can rename an existing organization URL. The typical workaround is to contact Firebolt support to see if they can rename it. If that is not feasible, you might need to recreate the organization under a new domain (e.g., using an email address at “velo”) and then migrate data or user setups.
Concurrency & Overlapping Updates: If two CDC operations try to update the same row simultaneously, one transaction may fail. Implement a retry mechanism if you anticipate this scenario. Vacuum Operations: Frequent small inserts create multiple “tablets.” Vacuum consolidates and optimizes these for better query performance. Firebolt’s new “auto vacuum” (rolling out in early 2025) will greatly reduce the need for manual vacuum scheduling by automatically running a non-blocking vacuum in the background after a set number of transactions.
Organization Level (Authentication): You manage logins (email addresses) at the organization/workspace level. Account Level (Authorization): Each account defines its own users, roles, and permissions. A single login can exist in multiple accounts with different roles. Support Access: Firebolt support engineers can access accounts via Okta (with appropriate permissions), but you can opt to block this access if desired (not recommended).
Yes, Firebolt provides monitoring capabilities through its information schema and metadata. Users are encouraged to implement custom monitoring and alerting processes on their side, although Firebolt also monitors performance and proactively alerts users to critical issues.
For users, it’s mainly about governance and logical isolation. Separate databases allow for different owners and permissions. Since custom schemas aren’t available yet, databases are the main way to group tables and views (this will change once schemas are supported).On the backend, metadata caching happens per database, so a single large database could add slight load. However, this is unlikely to have a practical impact unless in very large or complex cases.
Aggregating indexes in Firebolt pre-compute aggregated values to significantly speed up aggregation queries. They perform best when aggregations occur on a single fact table. They are less effective or infeasible when aggregation queries require multiple table joins because an aggregating index must be built on a single table only.
Firebolt recommends an incremental ingestion approach using S3 as a staging area. Data from PostgreSQL can be segmented (e.g., by ID range or time interval), pushed to S3, and loaded into Firebolt using the Firebolt SDK. This method ensures manageable load times and easy scaling by controlling the volume of data incrementally loaded.
You need to ensure that use_token_cache is disabled so that it won't write to the home/sbx_user.
Yes, we can view query history prior to the last engine restart. The support team is able to retrieve the query history for the customer if they are able to provide the type of query it was (e.g., SELECT, INSERT, etc.), the approximate time it was executed, and which engine they used to execute it.
At this time, COPY FROM does not support direct manipulation of S3 bucket data, however starting with 4.18 you can filter and alter data when reading using READ table-valued functions using full glob pattern capabilities (https://en.wikipedia.org/wiki/Glob_(programming)):
- Insert into an existing table using INSERT INTO + READ_PARQUET or READ_CSV
- Create a new table with CREATE TABLE AS + READ_PARQUET or READ_CSV
Firebolt is available as a connector directly from within Tableau. At this time, when you select the Firebolt connector from within Tableau, we will install a Firebolt V1 integration. To connect to V2 you will need to download the new connector and place it on the appropriate directories locally or on your server. You will also need a version of JDBC compatibile with Tableau. Full instructions can be found at https://docs.firebolt.io/Guides/integrations/tableau.html#integrate-with-tableau.
If you created your database containing upper-case letters without quotation marks, the saved name of your database will be in all lowercase letters. Confirm the name of your database from the expolorer, information_schema.catalogs, or show catalogs. From other systems, such as an SDK, use the always use the 'official' name of your database. Within the application, you can still access your all-lowercase database name using upper case letters, without quotation marks, since that is transformed into a lower-case name behind the scenes. If you wish your object names to be case sensitive, always wrap definitions in double quotes. Please note that definitions in information_schema are constructed and will not match exactly what was executed on creation, including use of quotes.
For high concurrency, use multiple clusters within your engine. Clusters help handle more simultaneous queries by distributing the load. Keep in mind that cache is shared across nodes in a cluster, but not between clusters, so the right balance depends on your workload. You can also consider using auto-scaling to dynamically adjust resources based on demand.
First, on your S3 account, confirgure the permission policy found in the help center article, https://docs.firebolt.io/Guides/loading-data/configuring-aws-role-to-access-amazon-s3.html#use-aws-iam-roles-to-access-amazon-s3. While still in your AWS Identity and Access Management (IAM) Console, start the process to upload data through the plus sign icon in the develop space. After selecting an ingestion engine, you can select 'IAM Role' as your authetnication method and you can create an IAM role in the application. Copy the trust policy here and follow the rest of the instructions in the article to apply to your AWS account. Note that you don't actually have to upload anything to create the IAM role.
You can label a query by setting the query_label system setting before running it:
cursor.execute("set query_label = '<label>';")
cursor.execute("your_query_here")
Here’s a full example using the Firebolt Python SDK:
id = '****'
secret = '****'
connection = connect(
database="<db_name>",
account_name="<account_name>",
auth=ClientCredentials(id, secret)
)
cursor = connection.cursor()
cursor.execute("start engine <engine_name>")
cursor.execute("use engine <engine_name>")
cursor.execute("use database <database_name>")
cursor.execute("set query_label = '123';")
cursor.execute("select 1;")
print(cursor.fetchone())
connection.close()Yes, semi-joins (implemented via WHERE IN clauses) can be more performant than explicit joins, as Firebolt has built-in optimizations that leverage semi-joins for better data pruning. Using semi-joins helps reduce intermediate row counts earlier in query execution, especially beneficial for high-cardinality datasets.
In Firebolt's query profiling, CPU time refers to the actual processing time on CPU cores, while thread time represents the total wall-clock time across all threads and nodes. When thread time is significantly higher than CPU time, it typically indicates waits due to data loading from storage (like S3) or node concurrency constraints. This distinction helps diagnose bottlenecks related to IO-bound or compute-bound workloads.
Firebolt proatively maintains a status page at https://firebolt.statuspage.io/ where we keep you notified about any active incidents that may cause interruption to your access or services. From this page, you can also hit the 'subscribe' button to stay informed by phone, RSS, email, or Slack.
Query performance in high-cardinality joins is significantly impacted by data cardinality, joins resulting in large intermediate row outputs, and data shuffles across nodes. Firebolt users should leverage the EXPLAIN ANALYZE functionality to identify expensive operations such as table scans, joins, and shuffles. Reducing data volume before joins through effective indexing, semi-joins, or aggregation indexes can mitigate these impacts.
You can find "Format Script" when you click on the three dots on the SQL Tab.
For users, it’s mainly about governance and logical isolation. Separate databases allow for different owners and permissions. Since custom schemas aren’t available yet, databases are the main way to group tables and views (this will change once schemas are supported).On the backend, metadata caching happens per database, so a single large database could add slight load. However, this is unlikely to have a practical impact unless in very large or complex cases.
You can add more users to your Firebolt account by either adding them through the web application under or with SQL commands. First create a login, using the email address of your invitee as the login_id. Next, associate the login to a user and assign them the appropriate permissions. Your invitee wiill automatically receive an email invitation to join your account. For more information visit our documentation.
Yes, primary indexes significantly impact query performance in Firebolt. Ensuring correct and optimized indexes is crucial, especially during migration. Indexes should be carefully reviewed and implemented based on query patterns and use cases.
Yes, Firebolt provides monitoring capabilities through its information schema and metadata. Users are encouraged to implement custom monitoring and alerting processes on their side, although Firebolt also monitors performance and proactively alerts users to critical issues.
Switching production workloads to Firebolt typically involves updating configuration to point to Firebolt endpoints. If all validation is complete and data is already present, this process is straightforward.
Firebolt recommends using aggregating indexes where possible for regularly queried granularities (e.g., daily or weekly), and employing pre-joined or pre-aggregated tables to simplify and speed up dashboard queries. Ensure indexes align closely with filter criteria to optimize query performance across various granularities.
Firebolt recommends an incremental ingestion approach using S3 as a staging area. Data from PostgreSQL can be segmented (e.g., by ID range or time interval), pushed to S3, and loaded into Firebolt using the Firebolt SDK. This method ensures manageable load times and easy scaling by controlling the volume of data incrementally loaded.
Yes. Firebolt recommends using separate engines for ingestion and query processing. Separating these concerns ensures ingestion tasks do not degrade query performance, leading to predictable and stable user experiences.
When a tenant comprises a large percentage of data (e.g., 20-25% of all data), avoid subqueries or joins that initially select large volumes of data and subsequently discard most rows. Instead, optimize queries and table structures to filter data as early and narrowly as possible, potentially using aggregated or pre-joined tables.
Primary indexes should include the most frequently used filters, such as tenant_id and date/time columns if queries consistently filter data by tenant and date ranges. A well-chosen primary index ensures queries access only relevant data partitions, maintaining fast performance even as data volumes scale significantly.
Aggregating indexes in Firebolt pre-compute aggregated values to significantly speed up aggregation queries. They perform best when aggregations occur on a single fact table. They are less effective or infeasible when aggregation queries require multiple table joins because an aggregating index must be built on a single table only.
Firebolt supports both using views and pre-joined tables. However, if most of the query execution time is spent on joins rather than aggregations, pre-joining tables (i.e., creating wider, denormalized tables during data ingestion) is often more performant. Views are effective for reusable SQL but may become slower with complex joins at scale. Aggregating indexes, which can pre-materialize aggregation results for fast query responses, work best on single tables without cross-table joins.
Setting up Apache Superset with Firebolt involves: - Installing Superset locally or on a server. - Configuring the Firebolt connector with appropriate credentials and connection parameters. - Testing queries in Superset to ensure Firebolt’s indexing structure is leveraged efficiently. - Optimizing queries for dashboard performance by using Firebolt’s indexing features to minimize latency. In this case, there were some challenges with reinstalling Superset, but Firebolt’s team is available to assist with setup and troubleshooting.
Yes, Firebolt can support a single table design that includes multiple reporting dimensions, such as unique counts, event times, and injected data. This consolidation can improve performance by reducing the need for complex joins and maintaining a single source of truth for analytics. However, when merging different data use cases into a single table, it is important to: - Optimize indexing to balance performance across different query patterns. - Consider partitioning or using aggregating indexes to precompute frequent aggregations. - Evaluate whether all reporting needs can be met within a single table without sacrificing efficiency.
One approach is to restructure the table by setting the primary index on event_time to better leverage Firebolt’s indexing capabilities. Additionally, an aggregating index on event_time can be beneficial. However, if queries still take longer than expected (e.g., 15 seconds for 30 days of data), it may help to review: - The structure of the primary index and ensure it aligns with the query’s filtering. - Whether unnecessary dimensions are included in the dataset, increasing granularity unnecessarily. - If joins or aggregations can be optimized, possibly through pre-aggregated tables. Firebolt’s architecture is designed to improve query efficiency by avoiding costly full scans and optimizing indexing structures.
Engine consumption data is available in the information_schema.engine_metering_history table, which provides hourly usage details at the account and engine level, including resource consumption and cost metrics. This data can also be retrieved via an API request.
Firebolt support engineers have the ability to access customer accounts for troubleshooting via Okta—only if they have the specific permissions. While it is generally recommended to keep support access open for fast incident resolution, you can request to block or limit their access if you have strict security requirements.
It is not clearly documented whether you can rename an existing organization URL. The typical workaround is to contact Firebolt support to see if they can rename it. If that is not feasible, you might need to recreate the organization under a new domain (e.g., using an email address at “velo”) and then migrate data or user setups.
Firebolt has “GenAI” initiatives on its product roadmap. While exact capabilities may evolve, the published information highlights plans for: AI-Assisted Querying (e.g., query recommendations, natural language querying), Auto-Tuning & Optimization powered by machine learning, and Improved Developer Experience leveraging AI-based insights. For a deeper discussion of upcoming features, Firebolt can arrange a roadmap review session with its product team.
Organization Level (Authentication): You manage logins (email addresses) at the organization/workspace level. Account Level (Authorization): Each account defines its own users, roles, and permissions. A single login can exist in multiple accounts with different roles. Support Access: Firebolt support engineers can access accounts via Okta (with appropriate permissions), but you can opt to block this access if desired (not recommended).
By subscribing through AWS Marketplace, you can consolidate Firebolt billing under your existing AWS billing arrangements. You will be directed to complete a few additional steps (“more clicks”) to finalize the purchase. Once completed, charges for your Firebolt usage appear in your AWS bill, simplifying vendor management if you prefer a single billing channel.
Before auto vacuum, you would typically schedule vacuum after a certain number of inserts or on a time-based schedule (e.g., nightly). Firebolt’s auto vacuum feature (released around early 2025) automatically triggers a non-blocking vacuum every few hundred transactions in the background, substantially reducing or eliminating the need for manual vacuum scheduling. This occurs with minimal overhead and typically does not require an engine size increase.
Separate Accounts: Each account cleanly isolates its data and can map to separate AWS buckets or IAM roles. Single Account with Multiple Databases: Environments share the same account, so you must carefully permission each database. Most teams that maintain separate AWS resources (e.g., dev vs. staging vs. production buckets and roles) find it more straightforward to mirror that approach with separate Firebolt accounts.
Concurrency & Overlapping Updates: If two CDC operations try to update the same row simultaneously, one transaction may fail. Implement a retry mechanism if you anticipate this scenario. Vacuum Operations: Frequent small inserts create multiple “tablets.” Vacuum consolidates and optimizes these for better query performance. Firebolt’s new “auto vacuum” (rolling out in early 2025) will greatly reduce the need for manual vacuum scheduling by automatically running a non-blocking vacuum in the background after a set number of transactions.
The “1.8 TB” figure refers to the SSD cache associated with a particular engine, not the total limit on data Firebolt can handle. Firebolt stores your full data in S3 for effectively unlimited capacity. Only the segments (tablets) relevant to a query are pulled into the SSD cache for faster processing. If your dataset exceeds 1.8 TB, Firebolt will still process it by cycling portions of data into and out of the SSD cache (first-in, first-out).
In most cases, no. The sub-result cache benefits queries that overlap in the underlying data scanned or join results. If the tenant ID changes and there is little or no data overlap, the previous sub-results become irrelevant, so the cache will not offer a speed-up.
Query performance primarily depends on the amount of data scanned. If queries remain selective (e.g., filtering by tenant ID and a truncated date range), Firebolt only scans relevant slices, keeping query times stable as data grows. Broad queries (e.g., SELECT * over wide date ranges) will naturally slow as more data must be scanned. Best practices include indexing on commonly used filters, leveraging caching, and avoiding unbounded queries to maintain good performance over time.
Order columns by frequency of use in queries. For example, if tenant_id and closed_at_day appear in most filters, list them first. Within equally common columns, order from lowest cardinality to highest cardinality (fewest unique values to most). This approach ensures that Firebolt’s indexing effectively prunes unnecessary data scans for highly repetitive or frequently queried columns.
Firebolt’s ingestion engine can be turned off when not actively loading data. Many teams schedule ingestion windows (e.g., hourly or daily) and then auto-stop the engine to save on costs. Billing is based on actual runtime, so you are not charged for idle ingestion clusters.
Firebolt can ingest data at terabytes-per-hour scale, supported by internal benchmarks (e.g., half a terabyte in ~800 seconds on four S-sized engines). Actual throughput depends on factors such as file format, table schema, partitioning, and engine size. Organizations can scale up (larger engines or more engines) to accelerate big batch loads and scale down for smaller, more frequent delta loads.
Firebolt offers multiple engine sizes (S, M, L, etc.). Smaller engines can handle many queries per minute if those queries are well-optimized. Heavier workloads or larger datasets may require a bigger engine or multiple concurrent engines. Firebolt charges by actual runtime (hourly or per second). Costs can be reduced by auto-stopping engines when not in use.
Queries with highly selective filters (e.g., smaller date ranges or high-selectivity columns) scan less data and often run in sub-second time. Queries that must scan large portions of the dataset (e.g., SELECT * over a broad date range) naturally take longer, especially on first (cold) runs when data must be read from storage. Firebolt’s sub-result caching reduces execution time for repeated or similar queries by caching portions of join results and aggregations. Proper indexing on commonly used filter columns can also significantly reduce the amount of data scanned, improving performance.
A viable workaround is to export query results to a file in S3, then read and process that file in smaller chunks. While it adds complexity (you must manage file paths, permissions, and cleanup), it avoids buffering the entire result set in application memory until real streaming is available in the Firebolt Node.js SDK.
Firebolt has introduced an auto-vacuum feature that runs on the write engine. It triggers after a set number of transactions and reclaims space without blocking ingestion. Manual vacuuming is largely unnecessary in most use cases now. A dedicated engine is not typically required; auto-vacuum operates seamlessly on the existing write engine.
Use derived date columns (e.g., day-level granularity) to lower cardinality. For instance, store closed_at_day by truncating a TIMESTAMP to DATE. Incorporate the derived column (e.g., closed_at_day) into the primary index alongside other frequently used filters (e.g., tenant_id), allowing Firebolt to skip irrelevant data segments. Because raw timestamp columns can be extremely granular, indexing them directly often leads to poor selectivity. Restructuring the schema to include day- or hour-level columns can significantly improve performance. Leverage Firebolt’s caching features (result cache, sub-result cache) for repeated queries.
Firebolt is designed for a “decoupled compute” architecture where you can spin up separate engines for different workloads. A dedicated write engine handles ingestion, while one or more read engines handle queries. This ensures that write operations do not slow down queries and vice versa. You can also configure auto-start/auto-stop so that engines only run (and incur costs) when needed.
Firebolt’s Customer Success and Support teams provide complementary query optimization guidance—including index design, join performance tuning, and ingestion configuration—at no extra cost. Users can reach out via Slack or support tickets for best practices and troubleshooting. In many cases, optimization is a collaborative, ongoing process. If you notice a slower query, you can share it with Support; sometimes the solution is a schema or indexing change, other times a product fix may be required.
Batch inserts are generally recommended for Firebolt. Inserting rows one at a time creates excessive overhead on the engine, leading to performance issues (especially on smaller engines). By sending records in small to moderate batches (e.g., once per second or at some reasonable time interval), the engine processes data more efficiently without overloading resources.
Firebolt advises creating a brand-new organization under the desired domain or AWS account via the standard sign-up process. Once the new organization is set up, copy over any needed configurations, tables, or data from the old organization. Because a new organization starts as a new trial, you also receive fresh Firebolt usage credits. After verifying that everything works correctly in the new organization, the old one can be retired or deleted.
Firebolt does not strictly require more tables; however, to achieve high-performance queries on large datasets, many teams choose to create specialized tables with carefully designed primary indexes. Although Firebolt can perform joins directly in SQL, the discussion emphasized that pre-joining or restructuring certain data tables often yields better performance. This approach leverages Firebolt’s indexing and reduces the run-time cost of large joins.
In the discussion, the Firebolt team recommended creating new, pre-joined (or otherwise streamlined) tables rather than performing large, multi-table joins at query time. This approach, sometimes called "join elimination," can significantly reduce query overhead. In addition, the Firebolt team highlighted the importance of setting appropriate primary indexes on these new tables to further optimize performance.
Firebolt provides documentation on connecting with Apache Superset. Their internal analytics team actively uses Superset, making it easier to provide support for any setup or troubleshooting questions.
Yes, Firebolt provides a query execution plan. Users can view a visual representation after query execution or generate a text-based plan using the EXPLAIN command. This helps users compare performance against other engines like Athena.
Aggregating indexes in Firebolt store precomputed aggregations for faster query performance. They update automatically upon new data ingestion, reducing query execution time significantly. The trade-off is a slightly increased ingestion time since the indexes must be maintained.
Users should avoid scanning large datasets unnecessarily by leveraging filtering on indexed columns. For example, filtering on both "event date" and "brand" significantly improves performance. Additionally, aggregating indexes can be used to precompute and store frequently used aggregations, reducing query execution time.
Firebolt does not natively save queries across different user accounts. Users need to manually copy queries and store them externally, such as in Slack, Google Docs, or a shared repository, to ensure accessibility across their team.
Firebolt users should define primary indexes based on frequently filtered columns, such as event dates and brand identifiers. By including relevant dimensions in the index, query performance can be significantly improved, as seen in the session where adding "brand" as a primary index reduced query execution time from minutes to seconds.
Users can verify the correct setup by ensuring they have obtained the API token, created the service account, and set up the appropriate automation steps. They can test ingestion by running queries to check if data has been successfully loaded into Firebolt.
Firebolt is built on advanced indexing, vectorized query execution, and efficient storage optimizations, ensuring sub-second query performance even on large datasets. While it is not a real-time platform, Firebolt is ideal for AI-driven analytics, interactive dashboards, and personalized AI applications that demand ultra-fast queries on your data.
Firebolt supports vector search but does not generate embeddings. Unlike dedicated vector databases, which specialize in unstructured data, Firebolt integrates vector search within a high-performance analytical data warehouse. This allows you to run hybrid queries (structured + unstructured) efficiently without managing separate systems.
If you already have embeddings generated from models like OpenAI, Hugging Face, or your own ML pipeline, Firebolt can store and query them at high speed and low latency, enabling AI-powered search and recommendations within your existing analytics environment.
Firebolt is optimized for low-latency, high-performance queries, but it is not a real-time processing platform. It excels in fast analytics on large-scale data but is not designed for event-driven streaming workloads. If your AI use case requires sub-second query execution, Firebolt is a great fit.
Firebolt is purpose-built for AI applications that require low-latency analytics. Unlike traditional cloud data warehouses, Firebolt delivers sub-second query performance for AI-driven workloads while supporting vector search and AI-driven optimizations. It enables faster and more efficient AI-powered analytics without the high costs and performance bottlenecks of legacy solutions.
Firebolt’s AI-related features, such as vector search, are included within our standard pricing model. While these capabilities do utilize compute resources, there are no separate licensing fees or AI-specific upcharges. You only pay for the compute and storage you use, ensuring cost efficiency without hidden AI-related costs.
For a detailed breakdown of how AI workloads impact pricing, reach out to our team for a tailored estimate based on your use case.
If it's a new account, you'll need to set up a new user within that account, although this user can be linked to the existing service account.
If it's a new organization, you'll need to establish both a new service account and a new user within that organization and any associated accounts.
Scaling with More Clusters:
This approach is ideal when you need to improve query concurrency—i.e., the ability to handle multiple queries simultaneously without significant performance degradation.
Scaling with a Higher Number of Nodes:
This is suitable when you find that the CPU utilization is consistently high, and queries are CPU-intensive. Adding more nodes spreads the workload across more computing units, thus alleviating CPU bottlenecks.
Scaling with Bigger Nodes:
This method is effective when the workload requires more memory or higher disk I/O capacity than what is currently available.
Set up SSO authentication: https://docs.firebolt.io/godocs/Guides/security/sso/sso.html
Configure your IdP: https://docs.firebolt.io/godocs/Guides/security/sso/configuring-idp-for-sso.html#custom.
You can query the INFORMATION_SCHEMA.ENGINES table to check the last known status of an engine:
SELECT engine_name, last_started, last_stopped
FROM INFORMATION_SCHEMA.ENGINES;
This will show when each engine was last started and stopped.
Firebolt values transparency and customer feedback when shaping its AI roadmap. To explore upcoming features or see open feature requests, reach out to Firebolt’s support or your customer success manager. Firebolt actively incorporates user feedback into its AI development efforts, with regular updates shared through newsletters, user forums, and product announcements. Stay connected to gain insights into innovations like query optimization for AI apps, text-to-SQL capabilities, and other enhancements for AI Apps tailored to your needs.
Queries are saved in information_schema.engine_query_history for as long as the engine remains active. However, the engine_query_history view is cleared upon an engine restart unless you have enabled the Persistent Query History feature.
FBU consumption is reported in real time and can be used to calculate costs by multiplying the consumed FBU by the price listed on the pricing page or a custom deal rate. However, the Billing and Consumption page updates daily, and AWS storage costs have a ~48-hour delay. For more information, check our documentation.
1. Use a WITH clause to define a Common Table Expression (CTE) and query its result.
2. Create a VIEW based on the result set and query the VIEW.
There is no limit to the number of regex expressions you can use with REGEXP_LIKE_ANY.
Use the source_file_name virtual column to filter rows based on the Parquet file name. For example:
SELECT $source_file_name, *
FROM external_table
WHERE $source_file_name ILIKE '%filename.parquet%';
This query retrieves rows where the source_file_name contains the specified file name. %filename.parquet% can be replaced with any pattern to match your file name.
For more information, see Using metadata virtual columns docs
Use ARRAY_SORT to sort one array and apply the same order to the other. For example, if you have array1 and array2:
array1: [4, 1, 3, 2]
array2: [Z, X, Y, R]
SELECT
ARRAY_SORT(x, y -> y, ARRAY_AGG(array2), ARRAY_AGG(array1)) AS sorted_array2
FROM your_table;This ensures the order in array1 is applied to both arrays, maintaining their alignment.
Output:
sorted_array1: [1, 2, 3, 4]
sorted_array2: [X, R, Y, Z]
For more information, check our documentation.
The behavior of quote escaping is controlled by the setting standard_conforming_strings. When this setting is enabled (the default behavior), backslashes are treated literally, and strings are parsed without escaping. This ensures consistent handling of literal strings and avoids unexpected transformations. If standard_conforming_strings is disabled, backslashes can be used as escape characters, altering how strings are interpreted. For more information, check our documentation.
Yes, an AWS account is linked to an organization. However, it is not possible to link accounts within an organization to different AWS accounts, billing is on the Organization level. For more information, check our documentation.
When an engine is resized dynamically, queries in execution will continue under the engine's original configurations until completion or until a timeout of 24 hours, after which they will be dropped if still running. The changes in engine size or type will only affect new queries submitted post-resize. Please check our documentation for more information.
Yes, Firebolt allows for dynamic resizing of engines during operation. You can adjust the number of nodes or the node type without stopping the engine, which lets workloads continue with minimal disruption. Use the ALTER ENGINE command to resize an engine. Newly started clusters post-resize will initially perform slower until they are warmed up.
What is the system engine, and how is it used for metadata-related queries?The system engine in Firebolt is a lightweight, always-available engine specifically designed for metadata-related queries and administrative tasks. It supports various commands:
Access Control Commands: Manage roles, permissions, and users.
Metadata Commands: Execute queries on information schema views, such as information_schema.tables and information_schema.engines.
Non-Data Queries: Perform operations like SELECT CURRENT_TIMESTAMP() that do not involve table data.
Typical Use Cases:
Retrieve information about databases, tables, indexes, and engines.
Manage system configurations or user permissions.
Execute DDL operations like creating tables and views, and managing all engine-related operations (start, stop, drop, alter).
Stopping an engine in Firebolt results in the eviction of the local cache. This leads to a "cold start" upon restarting the engine, as queries initially must fetch data directly from storage, slowing down performance until the cache is replenished with frequently accessed data. To minimize performance degradation, consider pre-warming the engine with essential queries or data after it is restarted. For more information please check our documentation article Work with Engines using DDL.
Firebolt values transparency and customer feedback when planning its roadmap. To view the current roadmap or see open feature requests, reach out to Firebolt’s support or your customer success manager. Additionally, Firebolt’s team actively gathers feedback from users and considers feature requests as part of ongoing development efforts. Regular updates are communicated through newsletters and user forums. Stay connected to get insights into upcoming releases and features tailored to your needs.
Firebolt stores unsaved scripts in your browser’s local storage, which has a limit of around 5 MB. If multiple websites use local storage, it can get full, causing unsaved scripts in the Firebolt SQL editor to be erased.
To avoid this:
Save your scripts regularly.
Clear your browser cache/cookies to free up local storage and prevent data loss.
Remember, clearing your cache will also remove other saved data, so use this solution carefully.
Yes, our insurance includes:
- Commercial General Liability
- Workers' Compensation and Employers' Liability
- Crime Insurance
- Professional & Technology Errors and Omissions
- Cyber Security Liability
Customer data is stored in S3 buckets with high availability and durability. Our recovery objectives are:
- RTO (Recovery Time Objective): 12 hours
- RPO (Recovery Point Objective): 1 hour
- SLA (Service Level Agreement): 99.9%
Researchers can report vulnerabilities by contacting security@firebolt.io.
Customers own their data and can delete it via commands like DROP DATABASE. Regardless, and upon contract termination, all customer data is deleted within 30 days.
Besides our runtime binary hardening, Firebolt leverages a runtime protection tool that provides deep visibility and protection at the process level.
Firebolt database itself inherently reduces the risk of SQL injection by minimizing the use of certain vulnerable constructs. Customers are still encouraged to implement additional controls at their application level such as:
- Ensure all user inputs are strictly validated before being processed.
- Escape potentially dangerous characters that could be used in unexpected ways.
- Include SQL injection tests in your regular security testing and code review processes.
Yes, we support encryption for data at rest and in motion. More on our technical best practices can be found in our Security blog: “Building Customer Trust: A CISO's Perspective on Security and Privacy at Firebolt”
Yes, customers can choose the region in which they run the service to meet data sovereignty requirements. More on our Available Region page.
Yes, this feature is supported. More details on our Identity Management page.
Yes, both IP Allow/Deny-listing is supported. More details on our Network Policy page.
Firebolt employs a comprehensive security strategy that includes network security policies, encryption practices, tenant isolation, and governance controls. We are committed to safeguarding your data through state-of-the-art security systems, policies, and practices.
We use AWS Shield, WAF, and other logical layers to protect against DDoS. Additionally, we leverage auto-scaling to maintain availability during attacks by dynamically adjusting resources like EC2 instances, ELBs, and other global services capacity. (Though some scenarios may require manual intervention).
%20logo.png){kind=icon}
