Stop Using the Power BI Folder Connector
I know that it’s tempting to use the Folder connector to combine multiple files with the same schema and load them in Power BI Desktop with a few clicks – an idea promoted by Microsoft’s own Power BI Dashboard in a Day (DIAD) training.
But what happens if you want to automate the dataset refresh after uploading your model to Power BI Service? Suddenly, things don’t work so well anymore. The only option is to install a personal gateway on your laptop with all the issues surrounding it.
Instead, I recommend you upload files to Azure Data Lake Storage and use the Power BI Azure Data Lake Storage Gen2 connector. Setting this up will require some assistance from your friendly IT department to set up ADLS and grant you access, but they shouldn’t complain if they are serious about data governance.
Like the Folder connector, you can combine multiple files from an ADLS folder. Moreover, this deployment options offers several benefits over keeping files locally:
- Eliminates gateways.
- Allows files to be centralized and organized nicely in folders.
- Enables integration scenarios, such as a third-party system pushing files on a schedule.
- Provides secured access and supervision by IT.
- Supports CDM folders in case you ever want to go there, such as to load files from Power BI dataflows or Dynamics CRM.
- It’s a highly available, performant, and scalable.
