Microsoft Fabric Updates Blog

Understanding Fabric KQL DB Capacity

With the free preview usage of Fabric workloads (other than Power BI) coming to an end on October 1, 2023 it’s important for everyone to understand how a KQL Database is billed in the SaaS world of Microsoft Fabric.

In Microsoft Fabric, you can purchase a Fabric SKU which comes with a set of Capacity Units (CU). Back in June 2023 we announced the availability of Fabric SKUs. Workspaces are assigned to capacities and everything inside a workspace such as KQL Database, Warehouse, Lakehouse, Dataset, etc., consume capacity units based on consumption.

The storage costs for data in OneLake will be billed separately from the Fabric capacity usage at a pay-as-you-go rate. OneLake storage pricing is comparable to Azure ADLS (Azure Data Lake Storage) pricing.

KQL Database and Capacity Usage

A KQL Database will utilize capacity via Operations that can be monitored using the Microsoft Fabric Capacity Metrics.

KQL Database Consumption

This is the number of seconds that your KQL database is active in relation to the number of virtual cores used by your database. For example, if my database uses 4 virtual cores and is active for 10 minutes then you will utilize 2,400 seconds of Capacity Units. An auto-scale mechanism is utilized to determine the size of your KQL database. This ensures the most cost-optimized and best performance based on your usage pattern. 

OneLake Transactions

Based on the number of reads and write transactions that your KQL Database performs, capacity units will be utilized from your Capacity. This will utilize the OneLake Read and Write meters which are equivalent to read and write operations on Azure ADLS Storage Accounts.

KQL Database and OneLake Storage

Data ingested into a KQL database is compressed and indexed. This compressed data is stored in two tiers of storage. This is your premium cache capacity which will be charged against OneLake Cache Storage and your standard tier of storage that will be charged against One Lake Standard Storage.

OneLake Standard Storage: This is the persistent storage, so if my retention is set to 365 days then all 365 days will be kept in OneLake Standard Storage. Mirroring a database or table allows you to have one logical copy of your data in OneLake. In cases where you mirror data to OneLake you will not get charged for OneLake Storage in both the KQL Database and OneLake.

OneLake Cache Storage: This is your high performance tier that provides the best query performance. The caching policy controls the amount of data that is stored in your OneLake Cache.

 

Operation Details

Operations Name

Type

Release

KustoUpTime

Capacity

October 1

KustoOneLakeRead

Capacity

GA

KustoOneLakeWrite

Capacity

GA

KustoStandardStorage

Storage

Post October 1/Pre GA

KustoCacheStorage

Storage

GA

 

Monitoring Fabric Capacity

The Microsoft Fabric Capacity Metrics app provides visibility into capacity usage for all Fabric workspaces tied to a capacity. It is used by capacity administrators to monitor the performance of workloads and their usage, compared to purchased capacity. Consider the following use cases:

  • Identify CU usage trends across warehouse items within a capacity.
  • Monitor overload information, and drill through specific timepoints to identify users/operations causing spikes.

A screenshot of a computer

Description automatically generated

KQL Database operation

The operation categories seen in this view are:

Here you can see that the Demo22 database that is in the RTA Field Demos workspace was active for 60 seconds and utilized 255 CU (s). From this information this particular KQL Database was utilizing 4.25 virtual cores during this time (255/60).

Timepoint – Explore graph

A graph of a number of red and blue lines

Description automatically generated with medium confidence

This graph in the Microsoft Fabric Capacity Metrics app shows utilization of resources compared to capacity purchased. 100% of utilization represents the full throughput of a capacity SKU and is shared by all Fabric experiences. This is represented by the yellow dotted line. The Logarithmic scale option enables the Explore button, which opens a detailed drill through page.

In general, operations are classified either as interactive or background, and denoted by color. All operations in KQL Database category are reported as interactive.

Timepoint – Drill through graph

This table in the Microsoft Fabric Capacity Metrics app provides a detailed view of utilization at specific timepoints. The amount of capacity provided by the given SKU per 30-second period is shown, along with the breakdown of interactive and background operations.

A screenshot of a computer

Description automatically generated

For more information on Fabric warehouse usage reporting, please check out our documentation Billing and utilization reporting.

Monitoring OneLake Storage

In the same Microsoft Fabric Capacity Metrics app that was utilized to monitor your capacity usage you can also monitor your OneLake Storage usage.

A screenshot of a computer

Description automatically generated

As storage is billed for the GB stored in OneLake, this tab gives you details on the GB stored at a Microsoft Fabric Workspace level. You are able to see the overall storage along with the accumulation of storage overtime.

**NOTE: The current OneLake tab only shows OneLake Standard storage as this is the only storage currently being reported. As we introduce OneLake Cache storage this app will be updated accordingly**

Gerelateerde blogberichten

Understanding Fabric KQL DB Capacity

oktober 7, 2024 door Alex Lin

Introducing Managed VNet Support for Fabric Eventstream! By creating a Fabric’s Managed Private Endpoint, you can now securely connect Eventstream to your Azure services, such as Azure Event Hubs or IoT Hub, within a private network or behind a firewall. This integration ensures your data is securely transmitted over a private network, enabling you to … Continue reading “Secure Data Streaming with Managed Private Endpoints in Eventstream (Preview)”

oktober 2, 2024 door Ruixin Xu

Coauthor: Misha Desai We’re always exploring new ways to help you more easily unlock the full potential of your data. Today, we are excited to announce upcoming pricing and billing updates to make Copilot and AI features in Fabric more accessible and cost-effective. Introducing the “Copilot and AI” billing meter As we continue to expand … Continue reading “Important billing updates coming to Copilot and AI in Fabric “