Microsoft Fabric Updates Blog

Data Warehouse SKU Guardrails for Burstable Capacity

We are excited to announce that SKU guardrails for burstable capacity is now active for Data Warehouse and SQL Endpoint on Microsoft Fabric.

Burstable Capacity

A Fabric capacity is a distinct pool of resources that’s size (or SKU) determines the amount of computational power available. Warehouse and SQL Endpoint provide burstable capacity that allows workloads to use additional resources to achieve better performance.

For example, a typical customer workload may require more capacity while running nightly loads than at any other time in a 24-hour period. There are also high demand reporting periods that span just a few hours and sparse usage in the late evening. We understand that customer workload patterns are not uniform, so we’ve designed a solution that caters to those variable demands with bursting.

Bursting is automatic and it happens when the workloads being executed demand more resources to run optimally. Instead of running a large nightly load job on 16 capacity units (CU) and completing it in 60 minutes, bursting can run it on 48 CU and complete it in 20 minutes. It is the same amount of work, just completed faster.

Smoothing

Burstable capacity is made possible by allowing the over-usage of a capacity to simply be averaged out over time. In the Fabric capacities blog post earlier this month, it was shown how smoothing helps streamline your capacity management by allowing you to plan for average usage instead of peak.

For SQL Endpoint and Warehouse in Microsoft Fabric, we classify all activity as background so that users get the maximum smoothing period possible which is 24 hours. All of your query activity gets the same 24-hour smoothing period as your ETL jobs. This allows for large ad-hoc queries to “burst” for a short period of time and not be penalized by interactive throttling, therefore providing optimal performance.

SKU Guardrails

With the introduction of throttling earlier this month, burstable capacity should be held within reasonable constraints as related to the baseline capacity units of the SKU size purchased.

Bursting is great, but without guardrails in place, it can lead to overconsumption in a short period of time when your Fabric capacity is undersized for your workload.

For an illustration, let’s review the below scenarios where a workload consumes 88,000 capacity units and is running on a capacity with a SKU of F8.

SKU ScenarioWorkload runtimeWorkload CUCU/sBurstable Scale FactorHours before Overconsumption
F8 (No Bursting)~3 hours88,00080N/A
F8 (Bursting with no Guardrails)5 minutes88,00029436x< 1 hour
F8 (Bursting with Guardrails)30 minutes88,000496x~4 hours

With no bursting, an F8 would take 3 hours to run this workload. It can only consume 8 capacity units (CU) a second as that is the compute capacity of the SKU. However, the capacity never enters a throttling state if continuously running this workload due to overconsumption.

With bursting, but no guardrails, the workload is blazing fast and runs in 5 minutes. However, it greatly overconsumes resources at the rate of 294 CU a second and therefore hits overconsumption in less than an hour if running this workload continuously.

With guardrails in place, the workload still completes in a reasonable amount of time, and overconsumption can easily be smoothed by the natural peaks and valleys of daily usage.

Note that some current SQL Endpoint and Warehouse workloads currently in public preview may experience a performance degradation if the workspace they are running on is assigned to a capacity that is undersized.

For more information on SKU guardrails for burstable capacity, please check out our documentation https://learn.microsoft.com/fabric/data-warehouse/burstable-capacity

To learn more about Fabric Capacity, see the blog Fabric Capacities – Everything you need to know about what’s new and what’s coming | Microsoft Fabric Blog | Microsoft Fabric

Related blog posts

Data Warehouse SKU Guardrails for Burstable Capacity

April 9, 2024 by Idris Motiwala

In the era of digital transformation, advanced analytics and an AI driven world, data has emerged as the new oil, powering businesses, and driving decision-making. But what good is this oil if it’s not refined and ready for use when needed? Moreover, managing, and ingesting data into a central platform for analytics and AI is … Continue reading “Announcing Mirroring Azure SQL Database in Fabric for Public Preview”

March 26, 2024 by Charles Webb

Mirroring is now available to customers in Public Preview. You can now seamlessly bring your databases into OneLake in Microsoft Fabric, enabling seamless zero-ETL, near real-time insights on your data – and unlocking warehousing, BI, AI, and more.