Fast Copy boosts the speed and cost-effectiveness of your Dataflows Gen2. We loaded a 6 GB CSV file to a Lakehouse table in Microsoft Fabric with 8x faster and 3x cheaper result. See our last post for details. Today, we’re excited to announce that Fast Copy in Dataflow Gen2 now supports high-performance data transfers from … Continue reading “Fast Copy with On-premises Data Gateway Support in Dataflow Gen2”
Author: Kyungchul (Kevin) Lee | LinkedIn Introduction Welcome back to the blog series on Advanced Scheduling techniques in Microsoft Fabric Data pipelines. In our first blog we covered how to replicate event driven scheduling by polling a storage location (Fabric Data Pipelines – Advanced Scheduling Techniques (Part 1) | Microsoft Fabric Blog | Microsoft Fabric). … Continue reading “Fabric Data Pipelines – Advanced Scheduling Techniques (Part 2: Run a Pipeline on a Specific Day)”
We are excited to share that the new modern get data experience of data pipeline now supports copying to Lakehouse and Datawarehouse across different workspaces with an extremely intuitive experience. When you are building a medallion architecture, you can easily leverage Data Pipeline to copy your data into Bronze Lakehouse/Warehouse across different workspaces. This feature … Continue reading “Easily Move Your Data Across Workspaces Using Modern Get Data of Fabric Data Pipeline”
When we talk about Microsoft Fabric workspace collaboration, a common scenario is developers and their teams using a shared workspace environment, which means they have access to “live items”. A change made directly within a workspace would override and affect all other developers or users utilizing that workspace. This is where git becomes increasingly important … Continue reading “Microsoft Fabric Lifecycle Management: Getting started with development in isolation using a Private Workspace”
Developing ETLs/ELTs can be a complex process when you add in business logic, large amounts of data, and the high volume of table data that needs to be moved from source to target. This is especially true in analytical workloads involving relational data when there is a need to either fully reload a table or incrementally update a table. Traditionally this is easily completed in a flavor of SQL (or name your favorite relational database). But a question is, how can we execute a mature, dynamic, and scalable ETL/ELT utilizing T-SQL with Microsoft Fabric? The answer is with Fabric Pipelines and Data Warehouse.
