We are thrilled to announce the public preview of Run Series Analysis, in conjunction with our recent announcement of the Autotune feature at the Fab conference. These two features are designed to help you gain insights into Spark application executions across your recurring runs of your Notebook and Spark job definitions, facilitating performance tuning and … Continue reading “Fabric Spark Autotune and Run Series Job Analysis”
Exciting news! The environment has officially become a generally available feature within Microsoft Fabric. What is the environment in Microsoft Fabric? The environment serves as a comprehensive container for both your hardware and software settings within Spark. Within this unified interface, you have the ability to select the desired Spark runtime, install libraries, and configure … Continue reading “Environment is now generally available”
Problem Statement: You are a data engineer developing Spark notebooks using Microsoft Fabric. You are having performance issues and you want to know if your spark code is running efficiently. You also want to know if increasing the resources would improve its performance. Discussion: In this blog, you will learn how to leverage Sparklens, an … Continue reading “Profiling Microsoft Fabric Spark Notebooks with Sparklens”
At the recent Fabric Conference, we announced that both code-first automated machine learning (AutoML) and hyperparameter tuning are now in Public Preview, a key step in making machine learning more complete and widely accessible in the Fabric Data Science. Our system seamlessly integrates the open-source Fast Library for Automated Machine Learning & Tuning (FLAML), offering … Continue reading “Introducing Code-First AutoML and Hyperparameter Tuning: Now in Public Preview for Fabric Data Science”
We are excited to announce a new feature which has been a long ask from Synapse Spark customers, Optimistic Job Admission for Spark in Microsoft Fabric.This feature brings in more flexibility to optimize for concurrency usage (in some cases ~12X increase) and prevents job starvation. This job admission approach aims to reduce the frequency of … Continue reading “Introducing Optimistic Job Admission for Fabric Spark”
