Please enter your information to subscribe to the Microsoft Fabric Blog.

Thanks for subscribing to the Microsoft Fabric Blog. You’ll receive email notifications when we publish new posts.

Microsoft Fabric Updates Blog

  1. Updates
  2. Apache Spark

Optimizing Spark Compute for Medallion Architectures in Microsoft Fabric

Guidance to Maximizing Productivity and Efficiency for your Data Engineering Workloads Data engineering teams often grapple with the complexities of planning and configuring compute resources for their data platforms. This is especially true when working with large-scale, complex datasets and demanding downstream SLAs. A one-size-fits-all approach is rarely effective, as different data layers and datasets … Continue reading “Optimizing Spark Compute for Medallion Architectures in Microsoft Fabric”

Enhancing Open Source: Fabric’s Contributions to FLAML for Scalable AutoML

At Fabric, we’re passionate about contributing to the open-source community, particularly in areas that advance the usability and scalability of machine learning tools. One of our recent endeavors has been making substantial contributions back to the FLAML (Fast and Lightweight AutoML) project, a robust library designed to automate the tedious and complex process of machine … Continue reading “Enhancing Open Source: Fabric’s Contributions to FLAML for Scalable AutoML”

Introducing High Concurrency Mode for Notebooks in Pipelines for Fabric Spark

We’re excited to introduce high concurrency mode for notebooks in pipelines, bringing session sharing to one of the most popular orchestration mechanisms for enterprise data ingestion and transformation. Notebooks will now automatically be packed into an active high concurrency session without compromising performance or security, while paying for a single session. Key Benefits: Why Use … Continue reading “Introducing High Concurrency Mode for Notebooks in Pipelines for Fabric Spark”

Announcing the Fabric Apache Spark Diagnostic Emitter: Collect Logs and Metrics

Fabric Apache Spark Diagnostic Emitter for Logs and Metrics is now in public preview. This new feature allows Apache Spark users to collect Spark logs, job events, and metrics from their Spark applications and send them to various destinations, including Azure Event Hubs, Azure Storage, and Azure Log Analytics. It provides robust support for monitoring … Continue reading “Announcing the Fabric Apache Spark Diagnostic Emitter: Collect Logs and Metrics”

Organizing your tables with lakehouse schemas and more (Public Preview)

We’re thrilled to introduce a new feature in Fabric: Lakehouse schemas. This feature lets users arrange their lakehouse tables into a folder-like structure, enhancing data discovery and more. Many users will be familiar with schemas in Fabric using Data Warehouse, and we are bringing aligned capabilities into Lakehouse. Schemas created in your lakehouses will also … Continue reading “Organizing your tables with lakehouse schemas and more (Public Preview)”