Fabric October 2024 Monthly Update
Welcome to the October 2024 Update!
Here are a few, select highlights of the many we have for Fabric this month. API for GraphQL support for Service Principal Names (SPNs). Introducing a powerful new feature in Lakehouses: Sorting, Filtering, and Searching capabilities. An addition to KQL Queryset that will revolutionize the way you interact with your data.
There is more to explore, please continue to read on.
Get certified in Microsoft Fabric—for free!
Get ready to fast-track your career by earning your Microsoft Certified: Fabric Analytics Engineer Associate certification. For a limited time, the Microsoft Fabric Community team is offering 5,000 free DP-600 exam vouchers to eligible Fabric Community members. Complete your exam by the end of the year and join the ranks of certified experts. Don’t miss this opportunity to get certified.
A new Fabric certification for data engineers
We are excited to announce a brand-new certification for data engineer. The new Microsoft Certified: Fabric Data Engineer Associate certification will help demonstrate your skills with data ingestion, transformation, administration, monitoring, and performance optimization in Fabric. To earn this Certification, pass Exam DP-700: Implementing Data Engineering Solutions Using Microsoft Fabric, currently in beta.
Microsoft Fabric and AI Learning Hackathon
Are you ready to unleash your creativity? Register for the hackathon as an opportunity to explore how Microsoft Fabric powers innovation, and of course for an opportunity to win cool prizes! Don’t forget to check out our livestreamed events here to get answers to your questions.
Content
- Copilot and AI
- Reporting
- Modeling
- Data connectivity
- Mobile
- Visualizations
- Developers
- Data Warehouse
- Data Engineering
- API for GraphQL support for Service Principal Names (SPNs)
- Automatic code generation in API for GraphQL
- Notebook Git integration & deployment pipeline GA
- Notebook in Org APP
- Notebook onboarding tour
- Notebook mode switcher
- Free selection support on display() table view
- Filter, sort and search your Lakehouse objects
- Real-Time Intelligence
Copilot and AI
Enhanced Power BI Report Creation with Copilot (Public Preview)
Improvements have been made to the create page capability in Power BI with Copilot! These updates are designed to make your experience clearer and more transparent.
Increased Clarity and Contextual Awareness
To help you initially build more valuable reports, Copilot can now engage with users to gather more details before creating a page. This ensures that Copilot has a comprehensive understanding of your needs right from the start to create a more relevant page for you. Copilot can also offer recommendations on fields and measures to use in a report.
Page Outline and Increasing Transparency
After creating a page with Copilot, you’ll now see an outline in the Copilot chat pane. This allows you to review the content and ensure that the page meets your requirements. The outline also helps add transparency so users can see what data fields copilot is using to build out the report.
Please visit our bigger blog post and documentation to learn more about creating reports in the Power BI service with Copilot.
Quick measure suggestions with Copilot to be replaced with Microsoft Fabric Copilot
Quick measure suggestions with Copilot feature are no longer available in public preview.
To continue to use natural language to write DAX formulas for measures, Microsoft Fabric Copilot in DAX query view can help you write DAX queries, which can include query-scoped measures that can be added to your model.
Quick measures will continue to let you choose a calculation from a list and create a measure using a drag-and-drop template, generating the DAX formula upon clicking Add. You can see and modify these measures in the DAX formula bar. Quick measure suggestions public preview introduced quick queries as a pane, allowing you to drag-and-drop from the Data pane instead of within a dialog. Quick measures will continue to be in the Quick measures pane, and the dialog experience will no longer be available.
Starting in October 2024, the tenant setting for this feature will also no longer be available and users who have older versions of Power BI Desktop and the public preview switch enabled will see the feature as disabled.
Learn more with these resources:
- Quick measures documentation
- DAX query view documentation
- Copilot to write and explain DAX queries documentation
- Microsoft Copilot for Power BI documentation
Reporting
Visual calculations update (Preview)
Combo charts are now supported
You can now use visual calculations in combo charts, such as the line and clustered column chart, just like you could in the other chart types. Here’s an example of a visual calculation returning the moving average over three quarters:
The visual calculation used here on the Line y-axis is:
ThreeQuarterMovingAverage = MOVINGAVERAGE([Sales Amount], 3)
Field parameters are now supported
This month we have enabled the use of visual calculations with field parameters, you can add a visual calculation to a visual that contains a field parameter and vice versa.
Field parameters can be used to quickly switch around what’s shown in a visual. For example, you can create a field parameter to enable your users to decide which attributes of a dimension to show. In this example a field parameter called Product Attribute can be used to determine what the Percent of grand total visual calculation returns:
The Percent of grand total visual calculation is defined using the template as:
Percent of grand total = DIVIDE([Sales Amount], COLLAPSEALL([Sales Amount], ROWS))
Since the Percent of grand total visual calculation used here refers to ROWS as its axis, it will update and reflect the correct values when another product attribute is picked:
Try out combining field parameters and visual calculations and let us know what would make the combined experience better for you by commenting below or at feedback.
Faster way to add a templated visual calculation
You can now add a templated visual calculation with fewer clicks by clicking on the bottom part of the New visual calculation ribbon button to see a menu that includes the templates. Clicking on a template will open the visual calculation mode where you can fill in the template and add your visual calculation.
If you want to create a new visual calculation without using a template, either select the top part of the New visual calculation button or choose Custom from the visual calculation template menu shown above.
Learn more about visual calculations in our documentation.
Azure Map Update – Data Bound Reference Layers
This month, the Azure Maps visual brings a powerful enhancement to its functionality with data bound reference layers.
In previous releases, the Azure Maps reference layer was limited to static shapes without the ability to conditionally format or bind geometries to customers’ business data. Additionally, the static nature of the reference layers limited user interaction, preventing actions such as selecting, filtering, clicking, or accessing tooltips for polygons and points, unlike other visual components.
With the data bound reference layer, this limitation is addressed by allowing seamless integration between the reference layers and customer business data. Reference layers can now be dynamically bound with the spatial fields used, empowering customers to visualize their business data in context with geographic or spatial elements. Users can now update their visuals in real time, interact with their data through Power BI’s standard features such as filtering, cross-highlighting, and tooltips—greatly enhancing the flexibility and interactivity of the Azure Map visual.
Making your reference layers data bound is easy to do. We’ll automatically map the shapes in your reference layer to values of the field in the Location bucket in the Build pane based on the name property you provide in your reference layer file.
This update also allows you to customize the colors of your shapes as well, using features like conditional formatting or through tying their color to a legend color.
Shapes that aren’t tied to a value in your model are considered unmapped. You can format them to use custom colors or hide them completely from your map. As cross-highlighting is a temporary filter on the map, the treatment you apply here is also what will happen to unselected shapes when cross-highlighting from another visual.
Marker enhancements
Revamping the rendering of columns, bars, ribbons, lines, area charts, and markers is a top priority. These elements form the foundation of our core visuals and will eventually impact other areas. By providing more control, our report creators can enhance their storytelling and help users easily interpret data.
In the October 2024 update, markers for line charts, scatter charts, and anomalies are improved with this revamp. This update introduces new options that offer greater customization and flexibility, explores these new options and maximizes their potential.
Markers for line and scatter charts can be customized in two ways:
- Categories: When the chart has no series, the dropdown menu displays categories. You can customize each data point’s marker based on the selected x-axis category.
- Series: When the chart displays a legend, the dropdown menu displays series. You can customize the markers for the complete set of data points within the selected series.
You can hide or show markers for a specific data point category by toggling the ‘Show for this category’ option. Please note that the ‘Markers’ toggle has been moved under ‘Show for all series.’
New format settings have been added to markers for line charts, scatter charts, and anomalies, including:
- Shape: Shape markers continue to offer control over their type and size. Additionally, rotation is now available for all shape types, except for the circle shape. Rotating shapes enhance the variety of shape types at your disposal, which is particularly convenient when multiple lines require unique shapes.
- Color: Changing the color of markers has always been a convenient control. Now, you can also modify the transparency of markers for a specific category, series, or all markers.
- Border (New): Borders for markers have been introduced, allowing you to add borders to a specific marker category, series, or all markers. Additionally, you can fully customize the marker borders by adjusting their transparency and width.
New List Slicer (Preview)
In this update, we would like to introduce the new List Slicer. To try it, navigate to Options and settings > Options > Preview features > List slicer visual. Major enhancements are coming, including image support, labels, additional conditional formatting options, and improved default styles specifically designed for hierarchical layouts.
Please note, this new visual is in its early development stage, we don’t recommend using it in production currently. However, this is an excellent opportunity to experience the capabilities of this new slicer and provide us with feedback for future improvements.
The new List Slicer can become hierarchical when more than one field is dragged into the field data well. This action will activate additional format settings specific to hierarchical data.
Another advantage of the new slicer is the level of customization it offers, including:
- Selection: Customize how items are selected within the slicer.
- Shape: Adjust the shape of the slicer to fit your design needs.
- Layout: Modify the layout to better organize the slicer elements.
- Overflow: Manage how the slicer handles overflow content (e.g., Continuous scroll, paginated).
- State styles: Define styles for different states (e.g., selected, unselected, on hover).
- Selection icon: Choose an icon to represent selected items. Available for the ‘Tile slicer’ too.
- Expand/Collapse icon: Select icons for expanding or collapsing hierarchical data.
- Button styles: Customize the appearance of buttons within the slicer.
Power BI in Teams – Announcing the retirement of the ‘Teams activity analytics’ report
‘Teams activity analytics’ report is an out of the box report users can create in their own workspace, which tracks their Teams usage data. The option to create this report is from Power BI application in Teams, Outlook and M365.
We will be retiring this feature starting from December 31, 2024. Starting January 1st, 2025, users will not be able to create that report, and reports that were already created will not be updated.
Why are we retiring the ‘Teams activity analytics’ report?
Teams provide native analytics view for teams and channels, which gives users insight into usage patterns and activity on their teams. Users can see data such as the number of active users, posts, replies, and more.
This is part of a more complete set of reporting available in Teams which provides a wider view of Teams usage. For example, you can see how many users communicate through channel and chat messages and the kinds of devices they use to connect to Teams. For more information, please refer to this documentation: View analytics for your teams in Microsoft Teams – Microsoft Support
How this will affect you
If you’ve already created your ‘Teams activity analytics’ report, the data in this report will no longer be updated.
The option to create a new ‘Teams activity analytics’ report will be removed from the Create section in Power BI app inside Teams/Outlook. Therefore, you will no longer be able to create this report.
What you need to do to prepare
For users who need to understand how Teams is being used in their organization we advise you to use Teams ‘Teams Analytics’ instead.
For more information, refer to our documentation: View analytics for your teams in Microsoft Teams – Microsoft Support
Modeling
Dynamic format strings for measures (Generally Available)
Dynamic format strings for measures are now generally available, giving you ultimate flexibility in how measures are displayed in visuals. These format strings can be conditionally applied using Data Analysis Expressions (DAX) based on the measures value, filters applied, and/or to add additional information, such as units.
If you haven’t already, check out the Guy in a Cube video, the deep dive blog, and feature documentation to learn more and get ready-to-use examples of how to apply them today. If you know of a community post or video about these, please share it in the comments!
Introducing INFO.VIEW Data Analysis Expressions (DAX)
DAX query view introduced new DAX functions to get metadata about your semantic model with the INFO DAX functions, and now four of these functions are also available as INFO.VIEW DAX functions, which convert IDs to friendly names, and can be used in calculated tables of the semantic model in addition to being able to run them in DAX query view.
Adding them as calculated tables ensures your model self-documents and stays current with all your latest changes automatically. Not only can these be used in calculated tables, but they will show the name or value of a column previously only displaying an ID.
INFO.VIEW.TABLES() shows information about the tables in your model, including what storage mode each table is in. You can also quickly identify tables marked as a date table by the Data Category of Time.
INFO.VIEW.RELATIONSHIPS() shows information about all the relationships in your model, including a relationship column giving a quick summary of the to and from columns with direction and cardinality.
INFO.VIEW.MEASURES() shows information about the model measures, including if it’s in a valid or error state.
INFO.VIEW.COLUMNS() shows information about the columns, including data category and data type.
Value filter behavior (Preview)
DAX has an automatic filtering mechanism that occurs when multiple columns from the same table are filtered. This behavior is informally called ‘auto-exist’. DAX understands that not all combinations of values across these columns are valid and as a result it automatically excludes invalid combinations. The DAX Engine generated a coalesced value filter that not only returns valid combinations but also affects measured calculations.
This month we are giving you more control over whether you want this behavior in your semantic model. You can decide whether you want to turn off coalesced values filters and turn on independent value filters instead. Turning on independent value filters by setting the ‘Value filter behavior’ setting to Independent (see below) results in multiple filters on the same table being kept separate instead of the DAX engine combining these into one.
Understanding current value filter behavior
When you are filtering multiple columns on the same table, the current default value filter behavior takes these filters and combines them into one, considering only the combinations that exist. Consider the following two columns on the same table:
- Year, which contains values like ‘2023’.
- Month, which contains values like ‘January 2024’.
If you filter on both Year and Month, since these columns are on the same table, the value filter behavior combines the filters into one, but only the combinations that exist are considered. Of course, the combination of the month January 2024 with year 2023 does not exist and would not be included in the filter. There are, however, situations in which the results are surprising.
Let’s look at an example, where we have a catalog showing availability of colors for products by year. The manufacturer of these products has experimented with making products in various colors throughout the years:
We have three products that have been available in various colors over the years. Notice how there are no red products offered in 2024, that is going to be important later.
Now, let’s count the number of products by adding the following measure:
Number of Products = COUNTROWS( 'Catalog' )
The following matrix shows the number of products that are available in various colors per year:
Now, let’s add another measure to calculate the total number of products for all years:
Number of Products All Years = CALCULATE ( [Number of Products], ALL ( 'Catalog'[Year] ) )
Let’s put these measures side-by-side and filter to year 2023 and just the blue and red colors (no black). As you can see the number of products is 4 and the number of products across all years for these two colors is 6:
If we switch the Year to 2024, we expect the ‘Number of Products’ measure to return 2, as there are just two products that are blue in 2024 and there are no red products in that year.
On top of that, we would expect that the number of products for all years will not change, because, after all, it is supposed to be calculated across all years. However, the ‘Number of Products for All Years’ changes from 6 to 5:
The number of products across all years should still be 6, not 5. What we are seeing here is the value filter behavior in action: it is combining filters on the same table, removing combinations that did not exist. The filters are Year = 2024 and Color = Blue or Red. Since these two filters are on the same table, these filters are combined into one filter that only filters for the combinations that exist. Since there are no red products in 2024, the applied filter is Year = 2024 and Color = Blue.
Therefore, the number of products for all years now counts just the number of blue products, not the blue or red products. This returns 5, as you can confirm in the table.
Influencing the value filter behavior
This month we are giving you control over whether you want this behavior in your semantic model, by using the ‘Value filter behavior’ setting on your semantic model in the properties pane in the model view:
Three options are available:
- Automatic – This is the default setting and currently turns on the Coalesced behavior. When we wrap up this preview, new models set to ‘Automatic’ will use Independent, there will be an announcement at that time.
- Independent – This forces filters on the same table to be kept separate. After setting the ‘Value filter behavior’ setting to ‘Independent’, the total number of products for all years returns 6 as expected (see below).
- Coalesced – This forces the value filter behavior to be enabled for the semantic model and will result in combining the filters on the same table into one. The number of products for all years in our example will continue to return to 5.
The table below shows the impact of this setting to our example:
| Value filter behavior setting | Filters applied in the example | Result of example measure |
| Automatic | Year = 2024, Color = Blue |
5 |
| Independent | Year = 2024, Color = Blue or Red |
6 |
| Coalesced | Year = 2024, Color = Blue |
5 |
Note that setting the ‘Value filter behavior’ to Automatic, means it is equal to Coalesced for now, but will be switched to Independent for new semantic models in the future.
If you set the ‘Value filter behavior’ to Independent, the number of products for all returns 6, as expected, since the filters are Year = 2024 and Color = Blue or Red and are no longer combined:
Learn more
Refer to our documentation to read more about visual filter behavior.
Data connectivity
Snowflake connector updates
- The driver used by Snowflake connector is updated to the latest version for incremental improvements.
- The Snowflake connector has improved performance by reducing metadata queries when not necessary.
Mobile
Power BI Mobile apps will no longer connect to Report Server using OAuth and AD FS 2016
As of March 1st, 2025, the Power BI Mobile app will no longer be able to connect to Report Server using the OAuth protocol through AD FS configured on Windows Server 2016.
Today, the Power BI Mobile apps use two authentication libraries: MSAL when connecting to Power BI service and non-AD FS based Report Server scenarios. And ADAL when connecting to AD FS using OAuth protocol.
Since ADAL library reaches end-of-life, it is time for us to migrate all authentication scenarios to use MSAL. Given that MSAL requires AD FS 2019 or newer, the Power BI Mobile apps will no longer be able to connect to AD FS 2016 once this migration is completed.
If your organization is using AD FS 2016 with their Report Server, you will have to upgrade to Windows Server 2019 or later, or use Microsoft Entra application proxy, to be able to connect from Power BI mobile apps to their Report Server, by March 1st, 2025.
Visualizations
New in AppSource
Sales Velocity Compass by Office Solution
Date Picker by Powerviz
The Ultimate Date Slicer for Power BI.
The ‘All language support’ and ‘Highlight Invalid dates using DAX’ options were added in the recent version update.
The Date Picker visual offers a modern calendar view, Presets, Pop-up mode, Default Selection, Themes, and more, making it a must-have date slicer for Power BI reports. Its rich formatting options help with brand consistency and a seamless UI experience.
Key Features:
- Display Mode: Choose between Pop-up and Canvas modes.
- Presets: Many commonly used presets like Today, Last Week, YTD, MTD, or create your preset using field.
- Default Selection: Control the date period selected when the user refreshes or reopens the report.
- Filter Type: Choose between Range and Start/End types.
- Month Style: Select single- or double-month date slicer.
- Multiple Date Ranges: Flexibility to select multiple date ranges
- Themes: 15+ pre-built themes with full customization.
- Holidays and Weekends: Customize holidays/weekends representation.
- Import/Export JSON: Build templates and share your designs.
- Invalid Dates: Customize invalid dates in your data or the ability to mark certain dates invalid via a DAX.
Many more features and customizable options
🔗 Try Date Picker for FREE from AppSource
📊 Check out all features of the visual: Demo file
📃 Step-by-step instructions: Documentation
💡 YouTube Video: Video Link
📍 Learn more about visuals: https://powerviz.ai/
✅ Follow Powerviz: https://lnkd.in/gN_9Sa6U
Cycle Plot by Nova Silva
A cycle plot is a powerful tool for visualizing and understanding seasonal patterns in time series data. Unlike traditional line graphs, which may obscure cyclical trends, cycle plots separate data into cycles (e.g., months, days, or quarters) and plot them individually for each period within a cycle.
This method highlights within-cycle variations and trends across cycles. For example, a cycle plot of monthly sales over several years would display each month’s trend over time, making it easier to identify whether certain months consistently perform better or worse. It also reveals how overall trends (like yearly growth) affect individual periods.
Cycle plots are particularly useful for businesses and analysts who need to identify seasonality in sales, website traffic, or other metrics. By visualizing data this way, decision-makers can better plan for seasonal peaks and troughs, optimize inventory, and tailor marketing efforts.
In summary, cycle plots offer a clearer and more detailed perspective on time series data, making them an essential tool for uncovering and leveraging seasonal patterns.
Try the Cycle Plot for FREE now on your own data by downloading it from the AppSource.
Questions or remarks? Visit us at: https://visuals.novasilva.com/.
Drill Down Network PRO by ZoomCharts
Create network charts from your categorical data easily with Network PRO! Simply add category and value fields, and the visual will automatically detect relationships and visualize them as nodes in an interactive and easy-to-explore chart with a force-directed layout. With more than 100 customization settings for nodes, links, legends, labels, and other aspects, you can create the perfect chart for your report.
Network PRO excels at visualizing hierarchies, making it easier to understand relationships and faster to find answers. Whereas the Pin version of Network PRO is designed for dashboards, the Filter version will seamlessly integrate in your reports to make them more interactive and insightful. With cross-chart filtering enabled, selecting one or multiple nodes will dynamically filter data in other visuals and vice versa.
Main Features:
- Automatic relationship detection
- Up to nine levels of hierarchy
- Dynamic node scaling based on value
- Node, link, and label customization
- Force-directed layout
- Touch support
- Cross-chart filtering
🌐 Get Drill Down Network PRO on AppSource
Product Page | Documentation | LinkedIn | Report Examples
Decomposition Tree – All Expanding
Custom visual helps to break down a metric or a key performance indicator (KPI) into its contributing factors. It can be used to identify the underlying causes behind a particular metric’s value.
The following features make it unique compared to native decomposition tree visual.
- It allows users to expand all nodes simultaneously.
2. Users can add a target measure as well.
3. It allows 2 color patterns; the first pattern allows separate colors for each level. The second pattern allows a node and its descendants to have different colors from other nodes.
4. It allows users to select whether to display children only or all descendants on node click.
5. It allows users to adjust tree height, tree width, bar height.
6. It allows different label positioning such as under the bar, start and end.
Watch a demo of these features in a short video.
Download this visual from APPSOURCE
Download demo file from APPSOURCE
For more information visit excelnaccess DecompositionTree or contact zubair@excelnaccess.com
EDITable for reference/master data management with approval workflow, audit & governance
EDITable provides a self-service platform that helps you manage reference data, master data & meta data for your Power BI reports and applications.
It complements your Master Data Management (MDM) solution and offers a lightweight alternative for business users that supports both reference data (such as Customer Region. And Product master) and flat tables (such as Price list, Contracts, Projects, and Discounts).
HIGHLIGHTS
- Edit dimensions & measures in flat tables
- Designed for business users requires minimal help and setup from IT
- Supports LIVE sync with data residing in Azure SQL, Databricks, Snowflake, Microsoft Fabric, RedShift, and Postgres
FEATURES
- Bulk Insert/Edit values
- Custom approval workflows (table-level, dimension-level, and value-based)
- Supports Slowly Changing Dimensions (SCD) Type II
- Change log for enabling audit
- Conditional Formatting
- Commenting & Collaboration
- Data integration with webhooks
- Granular user activity permissions (add/delete/edit rows)
- Columnar permissions (ACL)
Use Cases
Master Data, Reference Data, Meta Data, Flat table editor
Additional information
- Check out our 2-Min Editable Introductory Video.
- Visit our website to learn more.
- Replay a webinar and experience a LIVE DEMO.
- Read the documentation.
- Follow EDITable on LinkedIn and YouTube.
- Have a question? Post it in our community.
- Schedule a demo with our editable team.
ADWISE Advanced Column v1.0
Advanced Column is easy to use column chart for comparing one or more values with clearly interpretable difference lines. It is best suited for comparison of values between time periods, categories or quantities. As usual with our visuals, you can apply multiple formatting options to render the visual the way you want, to be easily readable for users and to correspond with your corporate identity. You can still use all the standard features of column chart such as Axis X, Y formatting, gridlines, values + totals, legend and so on, but with the ability to compare any columns with ‘drag & drop’ simplicity.
This brand-new visual brings to you these features:
- Difference lines – Configurable lines to clearly show compared columns.
- Values – Show difference in percentage or absolute values or both.
- Shapes – You can choose from rectangle, circle or ellipse with configurable positioning.
- Positive/negative difference – Distinguish differences with colors and symbols to be readily identifiable by users.
- Series labels – Use changing labels for Axis X, e.g. when you use calendar months for column names.
- Series formatting – Format data and total labels per category (legend).
- And other formatting – Thousand and decimal separator, interaction with other visuals, empty data screen.
See more info (including ChangeLog) on our website: ADWISE Advanced Column
This visual offers in-app purchases and comes with a 30-day free trial version. We want you to have full experience of our visual during the trial period, so all features are accessible in the trial.
You can use basic functionality without watermark. Licensing and pricing plans can be found in our FAQs: ADWISE Advanced Column FAQs
Don’t forget to try our other successful visual ADWISE RoadMap / Gantt.
Developers
Power BI enhanced report format (PBIR) update (Preview)
The Power BI enhanced report format (PBIR), along with Power BI Project (PBIP) files, provides a great source-control and co-development experience due to its folder representation of the report definition in a public JSON format.
During the PBIR launch a few months ago, we acknowledged important service limitations and commitment to addressing them in the upcoming months.
The following features are now available for reports using PBIR format:
- Publishing a PBIR report from Power BI Desktop.
- Downloading a PBIX file using PBIR format from a workspace.
- Uploading a PBIX file using PBIR format to a workspace.
In the coming weeks, PBIR reports will be supported within Power BI Apps.
For further information regarding PBIR format, please refer to the documentation.
Data Warehouse
CI Collation (Warehouse only)
We’re excited to announce Case Insensitivity (CI) Collation support—Latin1_General_100_CI_AS_KS_WS_SC_UTF8 in Fabric DW! This new feature lets you configure your warehouse with CI Collation via RESTful API. If no collation is specified, the default will be Case Sensitivity (CS) with ‘Latin1_General_100_BIN2_UTF8’.
CI Collation simplifies queries, improves user experience, and enhances integration with other systems. It also ensures consistency when migrating data from different case sensitivity settings and boosts performance by simplifying comparisons.
We’re dedicated to offering flexible collation options to suit your needs. Note that COLLATE T-SQL clause support is coming soon. This will enable you to utilize the COLLATE command with CREATE or ALTER TABLE to directly specify the collation for your VARCHAR fields. Stay tuned for updates!
Concurrency Performance Improvements
We have recently optimized our task scheduling algorithm in our distributed query processing engine (DQP) to reduce contention when the workspace is under moderate to heavy concurrency. In testing we have observed that this optimization makes significant performance improvements in querying workloads. Customers who run complex queries with multiple users should benefit from these improvements.
For more information about DQP and autonomous workload management in Fabric warehouse, see our documentation: documentation
VARCHAR(MAX)/VARBINARY(MAX) types (Warehouse only) public preview
You now have the capability to store a large amount of text and binary data in VARCHAR(MAX) and VARBINARY(MAX) columns in the Datawarehouse tables. Fabric Datawarehouse allows you to use VARCHAR(MAX) and VARBINARY(MAX) in the CREATE TABLE statement and define the columns that contain large text or binary data. These data types allow you to store up to 1MB of data per cell, enabling you to store free text or JSON data without the risk of potential data truncation.
Additionally, we are implementing several performance enhancements for operations involving VARCHAR(MAX) and VARBINARY(MAX) columns, including string filters, text searches (LIKE operator), and more efficient processing of large datasets in the GROUP BY, ORDER BY, and JOIN operations. These improvements ensure that you can utilize VARCHAR(MAX) and VARBINARY(MAX) types without the concern of significant performance degradation.
JSON support enhancements
JSON functionalities in Datawarehouse and SQL endpoints for Lakehouse and mirrored databases have been improved!
The new features that you can use are:
- The FOR JSON query clause that formats T-SQL query output as JSON text, addressing top feedback from the Fabric ideas site.
- New functions JSON_OBJECT and JSON_ARRAY enable you to format cells as JSON objects or arrays. The JSON_PATH_EXISTS function can be used to check if the path exists in a JSON document.
- The existing OPENJSON function can now be used with CROSS APPLY and OUTER APPLY operators – another requested feature from the Fabric ideas site.
These new JSON capabilities allow you to query JSON text in SQL Server, Azure SQL database, and Fabric seamlessly.
Data Engineering
API for GraphQL support for Service Principal Names (SPNs)
We have made a significant leap in enhancing enterprise-grade security and authentication by introducing Service Principal Names (SPN) support for API for GraphQL. This feature offers organizations looking to integrate their apps with API for GraphQL in Microsoft Fabric tie seamlessly with their enterprise identity and access management systems.
By leveraging SPNs, businesses can now implement robust application-to-GraphQL authentication without relying on user credentials. This facilitates automated processes and streamlines the management of complex applications and microservices architectures. It’s a perfect fit for the zero-trust security model, enabling the fine-grained access controls and auditing capabilities that are essential in today’s regulatory landscape.
From a security standpoint, the benefits are clear. SPNs provide a secure method to authenticate service accounts, effectively reducing the risks associated with shared user accounts. This feature empowers administrators to apply the principle of least privilege, assigning only necessary permissions to each service principal. Moreover, it enhances auditing and monitoring capabilities, offering valuable insights into data access patterns and improving overall security posture.
Using SPNs with API for GraphQL is very simple: enable the use of Service Principals in your Fabric tenant then create an App Registration in Entra with a client secret. Next, simply grant the App access to your GraphQL item in Fabric and to your data sources exposed by the API, and you’re all set.
Automatic code generation in API for GraphQL
Fabric API for GraphQL now adds the ability to automatically generate Python and Node.js code based on GraphQL queries tested in the API Explorer. This new feature simplifies the workflow for developers by providing a seamless transition from query testing to code implementation. By allowing users to test their GraphQL queries within the API Explorer and then instantly generate the corresponding code snippets, this feature saves valuable time and reduces potential errors. The result is a more efficient development process, enabling developers to focus on higher-order tasks without getting bogged down in repetitive coding.
This new functionality is particularly beneficial for those who want to quickly test the access to the API locally with a simple boilerplate code. The integration with the API Explorer makes the process intuitive and accessible, further lowering the barrier to entry for developers looking to leverage GraphQL in their projects. This feature exemplifies Microsoft’s commitment to creating user-friendly tools that empower developers to build robust and scalable applications with greater ease.
Notebook Git integration & deployment pipeline GA
The notebook git integration and deployment pipeline are now generally available (GA)! This milestone marks a significant enhancement, enabling seamless integration and deployment of notebooks with Git.
Notebook Git integration now supports persisting the mapping relationship of the attached Environment when syncing to new workspace, which means when you commit the notebook and attached environment together to the Git repo, and sync it to another workspace, the newly generated notebook and environment will be bound together.
Notebook in Deployment pipeline now supports auto-binding experience that will bind the default lakehouse and attached environment within the same workspace when deploying to next stage. The change will have impacts on existing notebooks in deployment pipeline. The default lakehouse and attached environment (when all dependent items are in the same workspace) will be replaced by newly generated items in target workspace, the notebook metadata change will be highlighted in the diff view in next round of deployment
Impacts:
Users who are already utilizing the notebook git integration and deployment pipeline experience will notice changes in behavior due to the new auto-binding dependencies feature. The changes will be reflected in the ‘diff view’ of notebook Git and deployment pipeline experiences.
We encourage everyone to explore these new capabilities and take advantage of the improved workflows.
For more details, please refer to the updated documentation and reach out to us with any questions.
Notebook in Org APP
The Notebook feature is now supported in OrgAPP! This new integration is designed to enhance your productivity by using Notebook as a rich content carrier and streamline your workflow by providing a seamless experience within OrgAPP, making it easier than ever to interact with data insights!
You can easily embed the Notebook code&markdown cells, visuals, tables, charts, and widgets in OrgAPP, it can be a practical storytelling tool like dashboards.
Notebook onboarding tour
Fabric Notebook Onboarding Tour is now available to the public! This tour is designed to help you get started with the essential Notebook features and learn the new capabilities.
Key Highlights
Introduction to Notebook: Learn about the new core features of the Notebook.
Getting Started: Step-by-step instructions on key features and how to navigate the Notebook features’ interface.
What’s new: Highlight the new features that are introduced in Notebook, learn the basic functionalities and workflow. You can always find the recent new features on What’s new in the ‘View’ tab.
Notebook mode switcher
We are excited to announce the launch of the Notebook Mode Switcher! This new feature provides flexible access modes for your notebooks, which can help you easily manage the permissions to the notebook and the corresponding view.
When sharing a notebook with team members, you can grant the appropriate permissions based on their roles and needs. The recipients will see the best available notebook mode according to their permissions and will be able to switch between the modes they have access to.
Develop Mode: This mode allows users to have full access to the notebook, including reading, writing, and executing permissions. It is ideal for users who need to develop and modify notebooks extensively.
Run Only Mode: In this mode, users have read and execute permissions, enabling them to run the notebook without making any changes1.
Edit Mode: This mode provides users with read and write permissions, allowing them to edit the notebook without executing it1.
View Mode: Users in this mode have read-only access, making it perfect for those who need to view the notebook without making any modifications.
We encourage you to explore the Mode Switcher and leverage its capabilities to enhance your collaboration tasks!
Free selection support on display() table view
The free selection function on the rich dataframe preview in the notebook can improve the data analysis experience by providing flexible and intuitive selection capabilities. This feature allows users to interact with dataframes more efficiently and gain deeper insights with ease.
Multi-Column Selection: Users can select multiple columns simultaneously, enabling a more comprehensive analysis of the data columns.
Row selection: Easily select each row and know the column count of the dataframe.
Cell Content Preview: Preview the content of individual cells, providing a quick and detailed look at the data without the need to write additional code.
Column Summary: Get a summary of each column, including data distribution and key statistics, to quickly understand the characteristics of the data.
Free area selection: Select any continuous segment, get a basic understanding of the total selected cells and the numeric values in the selected area.
Filter, sort and search your Lakehouse objects
Introducing a powerful new feature in Lakehouses: Sorting, Filtering, and Searching capabilities. This improvement is designed to make data exploration and analysis more efficient by allowing you to quickly retrieve the information you need based on specific criteria, right within the Lakehouse environment. Whether you’re organizing schemas, tables, files, or folders, this feature will significantly boost productivity and streamline collaboration.
In this first release, these capabilities are available both in the OE and the main-view area. Also, both schema-enabled and no-schema Lakehouses will have access to:
- Sorting by name or created date for:
- Schemas, tables, files, and folders.
- Please note that only immediate children under the selected Folder will be sorted and not the grandchildren.
- Filtering by type, loading status, and created date for:
- Schemas and tables in the OE.
- Files and folders in the main view area.
- Searching by name for:
- Schemas and tables in OE.
- Files and folders in the main view area.
This feature will make your data more accessible, enabling faster decision-making and smoother workflows. We’re confident that enhanced sorting, filtering, and searching capabilities will take your Lakehouse experience to the next level, helping you focus on what truly matters, gaining insights and driving results.
We hope you enjoy using this new feature and look forward to hearing how it transforms the way you work with your Lakehouse data!
Real-Time Intelligence
Real-Time Dashboards Integration with GitHub
Who hasn’t faced the frustration of discovering that a dashboard has changed without them knowing? Whether it’s accidental edits or unintentional overwrites, these surprises can disrupt your work at the most inconvenient times.
Fabric’s Git integration is now available for Real-Time Dashboards and solves exactly that problem, giving you the version control and confidence you need to manage collaborative updates and avoid last-minute scrambles.
Git integration in Microsoft Fabric enables developers to integrate their development processes, tools, and best practices straight into the Fabric platform. It allows developers who are developing in Fabric to:
- Backup and versions of their work.
- Revert to previous stages as needed.
- Collaborate with others or work alone using Git branches.
- Apply the capabilities of familiar source control tools to manage Fabric items.
Learn more Overview of Fabric Git integration – Microsoft Fabric | Microsoft Learn
Quickly visualize query results in KQL Queryset
We are thrilled to announce an addition to KQL Queryset that will revolutionize the way you interact with your data. Say goodbye to the time-consuming process of re-running queries to view your data in different visualizations and formats. With this new feature, you can now graphically visualize query results instantly and effortlessly and control the formatting without the need for re-run queries – all using a familiar UI.
Save a query to a dashboard (Pin to Dashboard)
This feature allows users to save the outcome of any query written in KQL Queryset directly to a new or existing Real-Time Dashboard, making it easier to persist results. It accelerates productivity and streamlines work, enabling users to effortlessly share insights and data visualizations with their teams.
