Microsoft Fabric Updates Blog

Microsoft Fabric Lifecycle Management – Getting started with Git Integration and Deployment Pipelines

Lifecycle Management is a topic that is crucial for any organization to establish, and with Microsoft Fabric, it has never been easier to embark on this journey. In this article, we will delve into the essentials of enabling it through a demo scenario, but before we dive in, let’s take a moment to explore what Lifecycle Management is, and what it means in Fabric.

For those unfamiliar with this topic, Lifecycle Management utilizes tools by facilitating a standard process of communication and collaboration between developers and their teams. Within Microsoft Fabric, this is done through continuous releases, using effective delivery methods through an ongoing flow of new features or bug fixes. There are two components of doing this in Fabric and those are Git and Deployment Pipelines.

A screenshot of a computer

Description automatically generated

Utilizing Git provides an effective process for development teams to reliably and frequently update supported items in a workspace. The current items include:

  • Lakehouse
  • Notebooks
  • Paginated reports
  • Reports (except reports connected to semantic models hosted in Azure Analysis Services, SQL Server Analysis Services or reports exported by Power BI Desktop that depend on semantic models hosted in MyWorkspace)
  • Semantic models (except push datasets, live connections, model v1, and semantic models created from the Data warehouse/lakehouse.)

However, Fabric can still work with the unsupported items even if the workspace or Git directory has them. They appear within the source control pane, but you cannot modify them. More items will be supported in the future so keep an eye out for future updates!

There are a few limitations to be aware of when configuring Git with Fabric. The current limitations are:

  • Currently, only Git in Azure Repos with the same tenant as the Fabric tenant is supported.
  • If the workspace and Git repo are in two different geographical regions, the tenant admin must enable cross-geo exports.
  • Azure DevOps on-prem isn’t supported.
  • Sovereign clouds aren’t supported.

Let’s quickly pivot to the next component of Lifecycle Management – Deployment Pipelines. These are pipelines that allow content within one stage, to be cloned to the next. Within Fabric today, there are between 2 and 10 stages that are supported. Typically, a pattern followed for stages are DEV, TEST, UAT, and PROD. The configuration and stage setup of your deployment pipelines will depend on your environment. Just like Git, there are supported items to be aware of within Deployment Pipelines. The current items include:

  • Dataflows Gen1
  • Datamarts
  • Lakehouse
  • Notebooks
  • Paginated reports
  • Reports (based on supported semantic models)
  • Semantic models (except for Direct Lake semantic models)
  • Warehouses

Now that we have a basic understanding of what Lifecycle Management is and the components which make it up, let’s see how we can enable Git and configure a Deployment Pipeline between a DEV and TEST workspace.

First, I added Fabric items within my FabricDEV workspace which I would like to add the Lifecycle Management components to. These include a Semantic model, Report, Lakehouse, Default Lakehouse Semantic model and a Notebook. This is the first step you’ll want to take in configuring this setup by adding existing items to a workspace:

A screenshot of a computer

Description automatically generated

Next, navigate to the Azure DevOps service (https://dev.azure.com/) and create a new Project. In this demo, we will be creating a private project:

A screenshot of a computer

Description automatically generated

Within the Fabric Git Integration project, create a new repository for where you would like to store the version control of the supported Fabric items:

A screenshot of a computer

Description automatically generated

Now that the repository is created, let’s hook it up to the FabricDEV workspace in Fabric. This is done by going to the Workspace settings, selecting Git integration and from the drop-down menu, selecting all that apply to connect to the Azure DevOps account:

A screenshot of a computer

Description automatically generated

Once the connect and sync option is selected, the items within the Fabric workspace will begin to sync to the Git repo:

A white background with black text

Description automatically generated with medium confidence

This will take a short time for the Git repo to sync. Once the sync has completed, you’ll also notice the Fabric items which are not supported. Additionally, at the bottom of the workspace, there will be information displaying the branch you are currently working on and last time it was synchronized. In the top right corner of the workspace, you’ll also be able to keep track of all the changes that are pending:

A screenshot of a computer

Description automatically generated

Navigating back to the Azure DevOps Repo we created, let’s look at the files now to see what that looks like:

A screenshot of a computer

Description automatically generated

As you can see, the Fabric items from the FabricDEV workspace have been synchronized and committed to the Git repository. This is the exciting part – we can now leverage Deployment Pipelines and synchronize the supported Fabric items to other workspaces through the stage and deployment process! Let’s look at how we can quickly configure it.

Navigate to the Deployment Pipelines hub and select new pipeline, enter a pipeline name, and click next:

A screenshot of a computer

Description automatically generated

For this demo, we are going to create two stages, but up to 10 are supported within Deployment Pipelines. I’ve added the stage names and will now create them:

A screenshot of a computer

Description automatically generated

Now that the pipeline stages are created, we can assign a workspace to each stage. Within Fabric, we will use the existing FabricDEV workspace in which our Git repository was set up and add a FabricTEST workspace as a destination for our Fabric items. Once the workspaces have been selected from the drop down, click on assign workspace for each deployment stage:

A screenshot of a computer

Description automatically generated

Within the deployment pipelines, you’ll now be able to see the items that are within the current stage (FabricDEV workspace) and compare the items against the target stage (FabricTEST workspace):

A screenshot of a computer

Description automatically generated

Let’s click deploy and copy these items to our target stage. We will get a message to confirm our deployment, the items which will be deployed, the ability to add a note to the deployment and an override option to continue it in case one or more items fail. Everything looks good, so we can go ahead and select Deploy:

A screenshot of a computer

Description automatically generated

Let’s check the FabricTEST workspace and as you can see, all our Fabric items have synced:

A screenshot of a computer

Description automatically generated

That’s it! We’ve now completed moving Fabric items from a DEV to a TEST workspace utilizing Deployment pipelines.

For further information and understanding all aspects of the deployment process between stages, please refer to this documentation: The Microsoft Fabric deployment pipelines process – Microsoft Fabric | Microsoft Learn. Also, to setup take the Fabric Lifecycle Management a step further, see this documentation on configuring release pipelines for complete automation: Automate your deployment pipeline by using APIs and Azure DevOps – Microsoft Fabric | Microsoft Learn

To summarize this overview, we learned what Lifecycle Management in Microsoft Fabric is and the components which make it up. We walked through how to get started with setting up a Git repository in Azure DevOps, how to sync it to a Fabric workspace, and enable Fabric Deployment Pipelines for workspace isolation in the development process. I hope you find this overview helpful and relevant to getting started using these components of Lifecycle Management in Fabric!

References:

Introduction to the CI/CD process as part of the ALM cycle in Microsoft Fabric – Microsoft Fabric | Microsoft Learn

Overview of Fabric Git integration – Microsoft Fabric | Microsoft Learn

Overview of Fabric deployment pipelines – Microsoft Fabric | Microsoft Learn

The Microsoft Fabric deployment pipelines process – Microsoft Fabric | Microsoft Learn

Related blog posts

Microsoft Fabric Lifecycle Management – Getting started with Git Integration and Deployment Pipelines

October 15, 2024 by Someleze Diko

This session is part of the Microsoft Fabric and AI Learning Hackathon which focuses on how you can leverage Copilot in Microsoft Fabric. It will guide you through the various capabilities that Copilot offers for you to use Microsoft Fabric, empowering you to enhance productivity and streamline your workflows. We will dive deep into practical … Continue reading “Microsoft Fabric and AI Learning Hackathon: Copilot in Fabric”

October 9, 2024 by Misha Desai

At Fabric, we’re passionate about contributing to the open-source community, particularly in areas that advance the usability and scalability of machine learning tools. One of our recent endeavors has been making substantial contributions back to the FLAML (Fast and Lightweight AutoML) project, a robust library designed to automate the tedious and complex process of machine … Continue reading “Enhancing Open Source: Fabric’s Contributions to FLAML for Scalable AutoML”