Table of Content

1. Summary

2. What are the Standard Fabric SKUs & CUs

3. Track & Analyze your usage patterns

4. Identiying Peak & Off- Peak Hours

5. Determining the Appropriate SKU to Manage Peak Hours Effectively

Summary

Determining the cost of Microsoft Fabric across different use cases presents a complex business problem due to its diverse pricing structure and consumption-based model. Microsoft Fabric, an integrated data platform, offers services like data engineering, data integration, real-time analytics, machine learning, and business intelligence, all of which have variable costs based on usage patterns, data volumes, and specific resource allocation.

Key challenges include:

1. Usage Variability: Different use cases, such as ETL processes, real-time data streaming, or advanced analytics, require different resources (e.g., compute, storage, or memory), leading to fluctuating costs.
2. Scaling and Sizing: Predicting how a solution will scale based on future data growth or user interaction can be difficult. Misestimating this can either lead to over-provisioning (higher costs) or under-provisioning (performance issues).
3. Resource Optimization: Identifying the most cost-efficient combination of services (e.g., choosing the right service tiers or workloads) while balancing performance and reliability is critical.
4. Lack of Cost Visibility: For organizations operating in complex environments with multiple teams and projects, keeping track of resource consumption and attributing costs to specific departments or use cases can be a challenge.

These challenges highlight the need for robust cost management strategies to forecast, track, and optimize Microsoft Fabric expenditures across different business scenarios.

What are Standard Fabric F SKUs and CUs

Capacities are split into Stock Keeping Units (SKUs). Each SKU provides a set of Fabric resources for your organization. Your organization can have as many capacities as needed.

Fabric Capacity, measured in Capacity Units (CUs), defines the speed and performance of your Fabric workloads. You can choose lower or higher SKUs (Stock Keeping Units) for Microsoft Fabric capacity license, based on your workload size. There are two main categories of SKUs: Azure (F SKUs) and Microsoft 365 (P SKUs). Azure SKUs range from F2 to F2048, with each F SKU providing 2 Capacity Units (CU) of compute power. For example, F2 has 2 CUs, F4 has 4 CUs, and so on. With larger SKU sizes, more total capacity units are accessible, which enables more efficient and concurrent running of complex workloads.

The capacity and SKUs table lists the Microsoft Fabric SKUs. Capacity Units/Compute Units (CU) are used to measure the compute power available for each SKU. For the benefit of customers who are familiar with Power BI, the table also includes Power BI Premium per capacity P SKUs and v-cores. Power BI Premium P SKUs support Microsoft Fabric. 

A and EM SKUs only support Power BI items.

^*SKUs that are smaller than F64 require a Pro or Premium Per User (PPU) license, or a Power BI individual trial to consume Power BI content.

Track and analyze your usage patterns

You should monitor utilization to get the most out of your capacities. Foremost, it’s important to understand that Fabric operations are either interactive or background.

Each experience within Microsoft Fabric supports unique operations. An operation’s consumption rate is what converts the usage of the experience’s raw metrics into Compute Units (CU).
The Microsoft Fabric Capacity Metrics app’s compute page provides an overview of your capacity’s performance and lists Fabric operations that consume compute resources.

Monitoring can reveal to you that throttling is taking place. Throttling can happen when there are numerous or long-running interactive operations. Typically, background operations related to SQL and Spark experiences are smoothed, meaning they’re spread out over 24 hours.

The Fabric Capacity Metrics App is the best way to monitor and visualize recent utilization. The app breaks down to item type (semantic model, notebook, pipeline, and others), and helps you to identify items or operations that use high levels of compute (so that they can be optimized).

Throttling occurs when a tenant’s capacity consumes more capacity resources than it has purchased. Too much throttling can result in a degraded end-user experience.

Identifying Peak and Off-Peak Hours

To effectively manage and optimize your capacity utilization within Microsoft Fabric, it’s crucial to identify the peak and off-peak hours of operation.

These are the times when your resources are most and least consumed, respectively. Understanding these patterns allows for better resource allocation and helps in minimizing the chances of throttling, particularly during high-demand periods.

1. Monitoring Usage Trends

Utilize the Microsoft Fabric Capacity Metrics app to monitor usage patterns across different times of the day. By analyzing the data over some time, you can pinpoint when your interactive and background operations are at their highest and lowest levels. The app’s compute page provides detailed insights into which operations are consuming the most compute units (CUs) and at what times these spikes occur.

2. Determining Peak Hours

Peak hours are generally the times when the majority of your organization’s users are actively engaging with Fabric experiences, leading to a higher number of interactive operations. These can include actions like querying large datasets, running complex reports, or executing intensive computations. Typically, peak hours align with your organization’s working hours, but this can vary depending on your specific workload and operational needs.

The Microsoft Fabric Capacity Metrics app allows you to identify these peak periods by tracking the volume and intensity of compute resource consumption. During these times, it’s common to see a higher rate of throttling if the operations exceed the available compute capacity.

3. Identifying Off-Peak Hours

Off-peak hours, on the other hand, are the times when there is minimal interaction with the system. These are often outside of regular business hours, such as late at night or early in the morning, depending on your organization’s operational schedule. During these times, background operations, such as those related to SQL and Spark experiences, are more evenly distributed over the 24-hour period, resulting in smoother operation and lower risk of throttling.

Identifying these off-peak periods is essential for planning maintenance activities, scheduling resource-intensive background operations, and performing system optimizations without impacting the performance of interactive operations.

4. Balancing Workloads

Once peak and off-peak hours are identified, you can strategize to balance workloads more effectively. For instance, by scheduling non-urgent background operations during off-peak hours, you can reduce the strain on your resources during peak times. This not only helps in avoiding throttling but also ensures that critical operations receive the necessary compute resources when demand is high.

Determining the Appropriate SKU to Manage Peak Hours Effectively

Choosing the right SKU (Stock Keeping Unit) for your Microsoft Fabric capacity is a critical step in ensuring that your resources can handle the demands during peak hours. The SKU determines the amount of compute power, memory, and other resources available to your operations, directly influencing how well your system can manage high-demand periods.

1. Assessing Peak Hour Demands

The first step in determining the appropriate SKU is to thoroughly assess the demands on your system during peak hours. By using the Microsoft Fabric Capacity Metrics app, you can gather detailed insights into the types and volumes of operations that are consuming the most compute resources during these periods. This includes identifying which Fabric experiences (such as Power BI reports, SQL queries, or Spark jobs) are most active and resource-intensive during peak times.

2. Understanding SKU Options

Microsoft Fabric offers a range of SKUs, each designed to support different levels of performance and capacity. These SKUs vary in the amount of compute units (CUs), memory, and other resources they provide. For example, a higher-tier SKU will offer more CUs, allowing for better handling of concurrent operations and more complex computations during peak hours. Lower-tier SKUs, while more cost-effective, may not provide sufficient resources to meet the demands of peak usage.

3. Mapping Resource Needs to SKU Capabilities

Once you have a clear understanding of your peak hour demands, the next step is to map these needs to the capabilities of the available SKUs. Consider the following factors:

• Compute Units (CUs): Ensure that the selected SKU provides enough CUs to handle the volume of operations during peak hours. If your peak usage involves many simultaneous interactive operations, you may need a higher-tier SKU with more CUs.
• Memory: Determine if your peak operations require significant memory resources, such as those needed for large datasets or complex analytical queries. SKUs with higher memory capacity can prevent slowdowns and ensure smooth performance during peak times.
• Scalability: Consider whether the SKU allows for easy scaling. If your usage patterns fluctuate, selecting an SKU that supports autoscaling can help manage sudden increases in demand without manual intervention.

4. Testing and Optimization

Before finalizing your SKU choice, it’s advisable to conduct testing during peak hours with the SKU you are considering. This allows you to observe how well the system handles the actual workload and whether there are any performance bottlenecks or instances of throttling. If necessary, you can adjust your SKU selection based on these test results.

Additionally, continuous monitoring and optimization are key. As your organization grows or your usage patterns change, you may need to revisit your SKU choice to ensure it remains aligned with your peak-hour demands.

5. Cost-Effectiveness Considerations

While it’s important to choose an SKU that can handle peak-hour demands, cost-effectiveness should also be considered. Higher-tier SKUs come with increased costs, so it’s crucial to balance performance needs with budget constraints. By identifying and shifting non-critical operations to off-peak hours, you may be able to manage peak loads more efficiently and reduce the need for the highest-tier SKU.

6. Automation

Introduce automation to effectively manage Fabric capacities. There are various ways in which automation can be used to manage the Fabric capacity SKU

i) Smart Data Pipelines: A recommended strategy is to scale the capacity to a higher-tier SKU only when necessary, and subsequently reduce it to the minimum SKU required to maintain general operational functionality. This ensures efficient resource utilization while avoiding unnecessary costs.

ii) Schedule-Based Scaling: Capacity can be configured to scale up or down at predetermined times during the day. This approach is beneficial when the workload schedule is well understood, allowing for proactive resource management.

iii) Data-Driven Scaling: In this approach, Fabric’s metrics are used to determine when capacity should be increased or decreased. This method offers the most efficient and precise way to manage Fabric capacity, as it responds dynamically to actual usage patterns.

iv) Seasonality-Based Scaling: For businesses with seasonal fluctuations, capacity can be adjusted based on the expected demand during peak seasons. For example, during the holiday season, capacity can be scaled up to accommodate the increased workload, ensuring that the system can handle the additional demand.

Blog Author