Unfortunately, there isn’t a migration tool to convert your Power BI dataflow (gen1) to Microsoft Fabric dataflow (gen2). If you have Fabric capacity licenses, it just makes sense to do that migration because Dataflow Gen2 gives you data destinations into four destinations, which we don’t have in Dataflow Gen1. However, converting Gen1 to Gen2 isn’t that complicated. The process is explained in this blog and video.
Video
Dataflow Gen1 (Power BI or Power Platform Dataflows)
Dataflow in a Power Query process that runs online and loads data into a destination. That destination, however, When you use Power BI dataflow is ADLS Gen2 (Azure Data Lake Storage Gen2), and when you use Power Platform Dataflows (those that you create in the Power Apps portal, for example), is Dataverse.
Read more about Gen1 dataflows here:
Dataflow Gen2 (Microsoft Fabric Dataflows)
Microsoft Fabric introduced a new generation of Dataflows, Gen2. Gen2 Dataflows have all the capabilities of Gen1 plus extra features. At the moment, the most important extra feature is having the choice of four destinations for each of the tables: Azure SQL Database, Kusto database, Lakehouse, or Warehouse.
This capability makes the Dataflow Gen2 an important part of the ETL process designed in Microsoft Fabric.
read more about Dataflow Gen2 here:
But since there isn’t a migration tool, how can you migrate it? It is simple. Here it is;
Migrate this way: Power Query Template
Using the Power Query Template, you can export all the metadata (tables, parameters, functions, folders, etc.) in a Dataflow solution. Power Query Template is a file with an extension of PQT. This file then can be used to generate new dataflow. And that is how you can create Gen2 dataflows from Gen1 templates.
Learn more about Power Query Template here:
Let’s go through an example.
Step 1: Generate Power Query Template from Dataflow Gen1
Open the Dataflow Gen1 in Edit mode,
open Editor (by clicking on Edit tables), and select export template.
This will download a PQT file (Power Query Template) file.
Step 2: Create a Dataflow Gen2 using the Power Query Template
Start creating a Dataflow Gen2 (In a Fabric-enabled workspace);
Choose to Import Power Query Template
Then select the PQT file from the previous step here.
This would load all the queries, parameters, functions, folders, etc, in the new Dataflow. Remember to set a name for your Dataflow, otherwise it would be called Dataflow 1, 2, …
Step 3: Set the connection properties
The next step is to set the connection properties. Since connection properties are not part of the PQT file, you have to set them again, which means clicking on “configure connection” and setting all the properties, such as usernames, passwords, API keys, etc.
You just need to do that once per connection, then all queries using that connection will automatically get fixed in the validation process.
Step 4: Set the destinations
The last step is to set the data destination for your queries. This is one of the main reasons you may have migrated from Dataflow Gen1 to Gen2: to have destinations such as Lakehouse or Warehouse. Click on a query and set the destination for it. Follow the wizard to select which Lakehouse or Warehouse the destination is going to be, the table, the mapping, etc.
Remember that this process has to be done for all the queries you want to be loaded into the destination. To learn more about Dataflow Gen2, read this article;
That’s it you have a Gen2 dataflow version now. You can schedule it to run or you can add it as an activity inside a data pipeline.
Summary
Although migrating from Dataflow Gen1 to Gen2 isn’t possible at the moment, the process isn’t complex. You can do that using a Power Query Template. Once you have Dataflow Gen2, you can use features such as setting the data destination to be a Lakehouse or Warehouse or adding the Dataflow as an activity inside a Data Pipeline to streamline the ETL process.
Here are links related to this topic that you can use to study more;
Trainer, Consultant, Mentor
Reza Rad is a Microsoft Regional Director, an Author, Trainer, Speaker and Consultant. He has a BSc in Computer engineering; he has more than 20 years’ experience in data analysis, BI, databases, programming, and development mostly on Microsoft technologies. He is a Microsoft Data Platform MVP for 12 continuous years (from 2011 till now) for his dedication in Microsoft BI. Reza is an active blogger and co-founder of RADACAD. Reza is also co-founder and co-organizer of Difinity conference in New Zealand, Power BI Summit, and Data Insight Summit.
Reza is author of more than 14 books on Microsoft Business Intelligence, most of these books are published under Power BI category. Among these are books such as Power BI DAX Simplified, Pro Power BI Architecture, Power BI from Rookie to Rock Star, Power Query books series, Row-Level Security in Power BI and etc.
He is an International Speaker in Microsoft Ignite, Microsoft Business Applications Summit, Data Insight Summit, PASS Summit, SQL Saturday and SQL user groups. And He is a Microsoft Certified Trainer.
Reza’s passion is to help you find the best data solution, he is Data enthusiast.
His articles on different aspects of technologies, especially on MS BI, can be found on his blog: https://radacad.com/blog.
Be the first to comment