Microsoft Fabric Updates Blog

Ingest, transform, and route real-time events with Microsoft Fabric event streams

See Arun Ulagaratchagan’s blog post to read the full Microsoft Fabric preview announcement.
 
You can now ingest, capture, transform and route real-time events to various destinations in Microsoft Fabric with a no-code experience using Microsoft Fabric event streams. It enables customers to ingest real-time event data from external event sources into the data stores in Fabric. The events could be transformed into the native formats required for target destination. For example, Eventstream could transform the events into Delta Lake format for the Lakehouse, into SQL columns based on the table schema, or filter events so that homogenous data can be sent to a KQL table.
 
To create a new instance of Microsoft Fabric event streams, navigate to Real-time Analytics in your Fabric workspace where you will find Eventstream.

After creating a new Eventstream item, you will land into a no code & easy to use experience where you can configure your eventstream from source to destination.
  

What is Fabric event streams

  1. Centralized place for real-time events: It provides the capability to capture, modify, and direct your streaming data in real-time using a fully managed and scalable infrastructure.
  2. Multiple source connectors: It enables you to ingest your real-time streaming data from 3 source types today: Azure Event Hubs, Sample data or Custom application.
  3. Multiple destinations: It enables you to transform/capture/route real-time streaming data to 3 destination types today: KQL database, Lakehouse, or Custom application.
  4. No code experience: It provides an intuitive and easy to use drag & drop experience with end-to-end data visibility and monitoring.


 
Let’s dive into each point stated above in detail:

1. Centralized place for real-time events 

Microsoft Fabric event streams utilize a scalable infrastructure that efficiently manages scaling and resourcing automatically. This enables you to effortlessly capture your real-time events, facilitating streaming ETL (Extract, Transform, Load) operations, as well as transforming and routing the events to your desired destination.
 

2. Multiple source connectors

Microsoft Fabric event streams enable you to ingest your real-time event data from 3 source types today:

    • Azure Event Hubs: You can seamlessly ingest events from an Azure event hub configured using Fabric Data connections.
    • Sample data: You can easily configure your eventstream to continuously ingest sample events from either Yellow Taxi data or Stock Market data.
    • Custom application: With this source, a streaming endpoint is created that allows customers to have their custom streaming application communicate directly with Fabric with either a Kafka client or an AMQP client.

 

3. Multiple destinations

Microsoft Fabric event streams enable you to transform and/or route your real-time event data to 3 destination types today: 

    • Fabric KQL Database: This destination helps with direct ingestion of your real-time events into KQL database. Once the data lands in KQL database, you can then perform additional queries/analysis to gain deeper insights or build Power BI reports.
    • Fabric Lakehouse: With this destination, you can transform (manipulate, filter, aggregate etc.) your real-time events before routing into a Lakehouse table. Real-time events are converted into Delta Lake format and then stored in the designated Lakehouse tables.
    • Custom application: With this destination, you can configure your custom applications to use their Kafka or AMQP client application to pull the events directly from Fabric.

Evenstream sources and destination

 

4. No code experience

There is a super simple & easy to use drag & drop experience with end-to-end data visibility. You can view the shape of your data at every step from source to transformation to destination. You can also monitor each step by looking at meaningful insights.

Drag & drop no-code experience. Transformation logic and data preview.

End-to-end data visibility

 
To see detailed documentation, please see link.  If you have any questions or feedback, please reach out to us at askeventstreams@microsoft.com 
  

Get started with Microsoft Fabric

Microsoft Fabric is currently in preview. Try out everything Fabric has to offer by signing up for the free trial—no credit card information required. Everyone who signs up gets a fixed Fabric trial capacity, which may be used for any feature or capability from integrating data to creating machine learning models. Existing Power BI Premium customers can simply turn on Fabric through the Power BI admin portal. After July 1, 2023, Fabric will be enabled for all Power BI tenants.
 
Sign up for the free trial. For more information read the Fabric trial docs.
  

Other resources

If you want to learn more about Microsoft Fabric, consider:

Related blog posts

Ingest, transform, and route real-time events with Microsoft Fabric event streams

October 7, 2024 by Alex Lin

Introducing Managed VNet Support for Fabric Eventstream! By creating a Fabric’s Managed Private Endpoint, you can now securely connect Eventstream to your Azure services, such as Azure Event Hubs or IoT Hub, within a private network or behind a firewall. This integration ensures your data is securely transmitted over a private network, enabling you to … Continue reading “Secure Data Streaming with Managed Private Endpoints in Eventstream (Preview)”

October 4, 2024 by Jason Himmelstein

We had an incredible time in our host city of Stockholm for FabCon Europe! 3,300 attendees joined us from our international community, and it was wonderful to meet so many of you in person. Throughout the week of FabCon Europe, our teams published a wealth of valuable content, and we want to ensure you have … Continue reading “Fabric Community Conference Europe Recap”