Mastering SKU Estimations with the Microsoft Fabric SKU Estimator
In today’s ever-changing analytics landscape it can be difficult to plan out your next project or your enterprise analytics roadmap.
Designed to optimize data infrastructure planning, the Microsoft Fabric SKU Estimator helps customers and partners to accurately estimate capacity requirements and select the most suitable SKU for their workloads, protecting users from under-provisioning and overcommitment.
Below we explore a different scenario in more detail
This section of the blog is based on three scenarios that build on top of each other to showcase the tool’s versatility when planning out complex analytics projects, the three scenarios are:
- AI-Ready Analytics for Manufacturing
- Migrating Synapse Analytics Data Science Platform
- Data APIs, SQL Analytics and Power BI Embedded
Overview: Contoso Manufacturing
Contoso Manufacturing, a fictitious company, is a globally recognized leader in industrial and consumer manufacturing, renowned for its innovative solutions and commitment to sustainability. The company specializes in producing high-quality goods across diverse industries, including automotive, electronics, and healthcare. With a strong emphasis on research and development, Contoso continuously pushes the boundaries of technology to deliver cutting-edge products and services that cater to evolving market demands. Headquartered in the United Kingdom, Contoso Manufacturing operates a network of facilities and offices worldwide, ensuring seamless global operations and customer support.
Scenario 1: AI-Ready Analytics for Manufacturing
Contoso Manufacturing has contracted a leading systems integrator to help them modernize their existing enterprise analytics platform and corporate reporting solution, the aim is to create an AI Ready analytics platform that will provide data driven insights to support its operations, maximize efficiency, and achieve unparalleled productivity.
The current solution that needs to be modernized has the following high-level metrics:
- 50 source systems batch processed twice per day
- 1500 entities and tables are part of the ETL pipelines
- 50 TB of data is estimated to be stored in the platform over the next five years
- The plat operations contain 4000 IoT sensors providing telemetry every 10 seconds with an average telemetry message size of 600 bytes
- There are 10000 plant workers with daily clock in events for shift start/end and on-break/off-break generating 8 datapoints per worker per day
- An estimated 1300 Power BI dashboard and report users per day
- 150 Power BI Report authors
- An estimated maximum Semantic model size of 25GB
After a few discoveries and design workshop the systems integrator has come up with the following high-level architecture:
Part of the systems integrators deliverable after the discovery workshops is to present the high-level architecture with a budgetary estimate for the cost of the solution.
To simplify this task the Solutions Architect decides to use the Microsoft Fabric SKU Estimator.
To get started he 1st need to extract the required high level inputs and work load specific inputs based on the high-level metrics above.
He calculates that the total of the data in the architecture will be approximately 8533 GB based on an estimated 6:1 compression ratio,
He also deduces that the platform will have 2 batch cycles processing 1500 tables and data sets across all 50 source systems, he enter this information into the SKU Estimator as shown below:
Next he needs to select all workloads that needs to be included in the estimate, six workloads are identified:
Data Factory, Spark, Eventstream, RealTime Intelligence, Power BI and Data Activator.
These workloads are now selected in the Fabric SKU Estimator.
Now he needs to complete the workload specific inputs, in the Data Factory section he inputs a 0 as the solution does not include Dataflow Gen2:
In the Power BI section, he inputs the numbers as per the high-level metrics:
For Eventstream he needs to extrapolate the telemetry numbers into a daily ingestion size, he does this by using the following calculation:
( Sensors * events per day ) * event message size, this gives him a total of ~19GB
He then inputs the number of eventstreams as 1 and Total number of destinations as 5, this number represents the number of topics that the sensor data consist of.
For Eventstream source connectors he enters 0 as all the events are collected from an Azure EventHub and does not need to be counted.
In the Eventhouse section he estimates that the Daily telemetry data number will be 75% of the event stream data so he estimates that at 14GB, the Hot data retention is set to 30 days and the total Retention is set to 90 days, since the plats operate 24×7 he enters a 24h duty cycle.
When it comes to Data Activator he enters the estimated 80000 events and enters 5 for the number of alert rules as there are 5 operational notifications that the platforms need to notify about IE: excessive shift lengths, low operator capacity, excessive workforce capacity and other proactive notifications.
Once this is completed, he then clicks the “Calculate” button and he is presented with an estimated viable SKU with workload break down, he also gets an estimation for the storage consumption and the amount of Power BI Pro licenses that may be required.
He is now able to get a commercial estimate from the Azure Price Calculator that he can include in the proposal to Contoso.
Scenario 2: Migrating Data Science Platform
The data science team of Contoso is wanting to move its workloads from Azure Synapse Analytics to Fabric.
They currently have a Dedicated SQL Pool operating at DWU2500c SKU with 2 TB of data and 300 daily source tables.
This platform has 20 Data scientists operating the environment with 15 predictive models, they currently run three model training jobs per day.
The Solution Architect has been tasked to add this to the estimate and ensure that the data science team minimizes the amount of risk to the business as there is a decade of business logic tied up in the Synapse Dedicated SQL Pool.
Since there is a requirement to preserve existing business logic and minimize risk in outcomes he opts for a Lift-and-Shift approach to Microsoft Fabric Data Warehouse and to migrate the Synapse Data Science models to the Fabric Data Science he selects the workloads in the Microsoft Fabric SKU Estimator.
He updates the number of sources to reflect the additional 300 source tables
and adds the 2TB of data to the estimate.
He finishes up by completing the Data Warehouse section using the “Migrate to Fabric” experience and the Data Science section as shown below.
He now has enough information to amend his previous estimate and can clearly see the changes in the Estimated Viable SKU section.
Scenario 3: Data APIs, SQL Analytics and Power BI Embedded
Contoso Manufacturing has once again asked the systems integrator to also include their customer and partner data APIs in the platform with an estimated 500 daily sessions, as well as to cater for 200 daily Ad-Hoc SQL Analytics users and 800 daily embedded Power BI dashboard sessions.
The solutions architect now adds Ad-Hoc SQL Analytics and Power BI Embedded to the Fabric SKU Estimator.
He then combines the 500 API sessions with the 200 Daily Ad-Hoc SQL Analytics sessions and enters this into the Ad-Hoc SQL Analytics section, he also enters the 800 daily Embedded sessions into the Power BI Embedded section.
He has now completed the full and final estimate and is now ready to submit his final proposal.
Summary
This example illustrates how the Microsoft Fabric SKU Estimator simplifies a task that would have taken days into minutes providing a standardised framework built on best practices, real world telemetry and decades of experience.
To get started using the Microsoft Fabric SKU estimator follow the link below:
https://aka.ms/FabricSKUEstimator
