Microsoft Fabric Updates Blog

Introducing Mirroring in Microsoft Fabric

As mentioned in Arun’s blog post, we are thrilled to announce that Mirroring in Microsoft Fabric is coming soon.

In today’s AI driven world, analytics platforms are only as good as their data. With the ever-increasing amount of data being collected in various applications, databases, and data warehouses in an enterprise, managing and ingesting data into a central platform for purposes of analytics and AI is a cumbersome and costly process. Databases and data warehouses use proprietary storage formats making the ability to create shortcuts to their data impossible. Data needs to be extracted, transformed, normalized, and made available in a central place for analytics. Even when this is implemented, data is not real-time making any insights stale pretty quicky resulting in users having to query the data in the source.

Mirroring provides a modern way of accessing and ingesting data continuously and seamlessly from any database or data warehouse into the Data Warehousing experience in Microsoft Fabric. This is all in near real-time thus giving users immediate access to changes in the source!

Let’s dive deeper into its capabilities.

Access and manage any database or data warehouse

Any database can be accessed and managed centrally from within Fabric without having to switch database clients. By just providing connection details, your database is instantly available in Fabric as a Mirrored database. Familiar database editors provide the ability to manage the database. Here is an example of the Azure Cosmos DB editor available within the Mirrored database in Fabric. Any edits go directly against the Azure Cosmos DB source.

A screenshot of a computer

Description automatically generated

Real-time data replication

There is no complex setup or ETL for data replication. With the same connection details, data is replicated in a reliable way in real-time. An initial snapshot is created after which data is kept in sync in near real-time with every transaction whether a new table is created, or new data is inserted/updated/deleted. Replication uses the database’s Change Data Capture (CDC ) technology, transforms it into appropriate Delta tables and lands it in OneLake. Intelligent logic determines when the source has changed, ensuring compute isn’t used unnecessarily before replicating the data. Granular controls enable configuring what is mirrored into Fabric. Detailed monitoring is also available to gain insights into mirroring operations and when the replica in Fabric OneLake was last refreshed. From here on, the data is ready for consumption immediately in any Fabric workload.

Data warehousing experiences simplified!

Every Mirrored database comes with default data warehousing experiences via a SQL Analytics Endpoint which houses the metadata of the Delta tables and points to the data in OneLake. Whether a SQL developer or citizen developer, one can query using the T-SQL editor which comes with full Intellisense or the visual query editor.

A screenshot of a computer

Description automatically generated

Cross-joining Mirrored databases, Warehouses, Lakehouses

As all data in Fabric, whether added as a shortcut to a lake in a Lakehouse or is data ingested through Data Factory, Warehouse or Spark is already in Delta format, data can be cross joined. Now with Mirroring, data across any database can be cross joined as well enabling querying across any database, warehouse or lakehouse whether that be data in Azure Cosmos DB, Azure SQL DB, Snowflake, MongoDB etc.

A screenshot of a computer

Description automatically generated

Data Science experiences unlocked!

Any data scientist can create a Lakehouse with a shortcut to the Mirrored database and use Notebooks to analyze and create models with the data.

A screenshot of a computer

Description automatically generated

Power BI Direct Lake mode

Power BI reports and semantic models can be built in Direct Lake mode getting the blazing fast performance of import mode but without duplicating the data. As Direct Lake mode also supports reading Delta tables right from OneLake, the Mirrored database is Power BI ready.

A screenshot of a computer

Description automatically generated

Mirroring Availability

Azure Cosmos DB, Azure SQL DB and Snowflake customers will be able to use Mirroring to mirror their data in OneLake and unlock all the capabilities of Fabric Warehouse, Direct Lake Mode, Notebooks and much more. SQL Server, Azure PostgreSQL, Azure MySQL, MongoDB and other databases and data warehouses will be coming in CY24. For participation in our early adopter program, submit your application here.

In the meantime, to try out Fabric, sign up for a free trial.

Related blog posts

Introducing Mirroring in Microsoft Fabric

February 9, 2024 by Ruixin Xu

During January 2024, we announced the worldwide availability for public preview of Copilot in Microsoft Fabric. This preview includes Copilot for Power BI, Data Factory and Data Science & Data Engineering. With the Copilot in preview, Microsoft Fabric brings an improved way to transform, enrich and analyze data, and shortens the time to insights.  Today, we announce that Copilot … Continue reading “Announcing Fabric Copilot pricing “

February 1, 2024 by Kimberly Williams

We are excited to announce that Microsoft Fabric, our all-in-one analytics solution for enterprises, has achieved new certifications for HIPAA and ISO 27017, ISO 27018, ISO 27001, ISO 27701. These certifications demonstrate our commitment to providing the highest level of security and privacy for our customers’ data. What are these certifications and why do they matter? HIPAA (Health Insurance Portability … Continue reading “Microsoft Fabric is now HIPAA compliant”