Microsoft Fabric Updates Blog

Data Warehouse SKU Guardrails for Burstable Capacity

We are excited to announce that SKU guardrails for burstable capacity is now active for Data Warehouse and SQL Endpoint on Microsoft Fabric.

Burstable Capacity

A Fabric capacity is a distinct pool of resources that’s size (or SKU) determines the amount of computational power available. Warehouse and SQL Endpoint provide burstable capacity that allows workloads to use additional resources to achieve better performance.

For example, a typical customer workload may require more capacity while running nightly loads than at any other time in a 24-hour period. There are also high demand reporting periods that span just a few hours and sparse usage in the late evening. We understand that customer workload patterns are not uniform, so we’ve designed a solution that caters to those variable demands with bursting.

Bursting is automatic and it happens when the workloads being executed demand more resources to run optimally. Instead of running a large nightly load job on 16 capacity units (CU) and completing it in 60 minutes, bursting can run it on 48 CU and complete it in 20 minutes. It is the same amount of work, just completed faster.

Smoothing

Burstable capacity is made possible by allowing the over-usage of a capacity to simply be averaged out over time. In the Fabric capacities blog post earlier this month, it was shown how smoothing helps streamline your capacity management by allowing you to plan for average usage instead of peak.

For SQL Endpoint and Warehouse in Microsoft Fabric, we classify all activity as background so that users get the maximum smoothing period possible which is 24 hours. All of your query activity gets the same 24-hour smoothing period as your ETL jobs. This allows for large ad-hoc queries to “burst” for a short period of time and not be penalized by interactive throttling, therefore providing optimal performance.

SKU Guardrails

With the introduction of throttling earlier this month, burstable capacity should be held within reasonable constraints as related to the baseline capacity units of the SKU size purchased.

Bursting is great, but without guardrails in place, it can lead to overconsumption in a short period of time when your Fabric capacity is undersized for your workload.

For an illustration, let’s review the below scenarios where a workload consumes 88,000 capacity units and is running on a capacity with a SKU of F8.

SKU ScenarioWorkload runtimeWorkload CUCU/sBurstable Scale FactorHours before Overconsumption
F8 (No Bursting)~3 hours88,00080N/A
F8 (Bursting with no Guardrails)5 minutes88,00029436x< 1 hour
F8 (Bursting with Guardrails)30 minutes88,000496x~4 hours

With no bursting, an F8 would take 3 hours to run this workload. It can only consume 8 capacity units (CU) a second as that is the compute capacity of the SKU. However, the capacity never enters a throttling state if continuously running this workload due to overconsumption.

With bursting, but no guardrails, the workload is blazing fast and runs in 5 minutes. However, it greatly overconsumes resources at the rate of 294 CU a second and therefore hits overconsumption in less than an hour if running this workload continuously.

With guardrails in place, the workload still completes in a reasonable amount of time, and overconsumption can easily be smoothed by the natural peaks and valleys of daily usage.

Note that some current SQL Endpoint and Warehouse workloads currently in public preview may experience a performance degradation if the workspace they are running on is assigned to a capacity that is undersized.

For more information on SKU guardrails for burstable capacity, please check out our documentation https://learn.microsoft.com/fabric/data-warehouse/burstable-capacity

To learn more about Fabric Capacity, see the blog Fabric Capacities – Everything you need to know about what’s new and what’s coming | Microsoft Fabric Blog | Microsoft Fabric

منشورات المدونات ذات الصلة

Data Warehouse SKU Guardrails for Burstable Capacity

أكتوبر 21, 2024 بواسطة Ambika Jagadish

In the era of Generative Artificial Intelligence, data warehouses play a pivotal role for organizations to store, manage, and analyze massive amounts of data. They serve as the foundation for business intelligence and help make informed decisions. However, like any technology, data warehouses are not immune to failures – whether from hardware malfunctions, software bugs, … Continue reading “Ensuring Data Continuity in Fabric Warehouse: Best Practices for Every Scenario”

أكتوبر 4, 2024 بواسطة Jason Himmelstein

We had an incredible time in our host city of Stockholm for FabCon Europe! 3,300 attendees joined us from our international community, and it was wonderful to meet so many of you in person. Throughout the week of FabCon Europe, our teams published a wealth of valuable content, and we want to ensure you have … Continue reading “Fabric Community Conference Europe Recap”