• Replicating BigQuery to Fabric

    October 3, 2025 / No Comments »

    A recent engagement required replicating some DW tables from Google BigQuery to a Fabric Lakehouse. We considered the Fabric mirroring feature (back then in private preview, now publicly available) and learned some lessons along the way: 1. 400 Error during replication configuration – Caused by attempting to use a read-only GBQ dataset that is linked to another GBQ dataset but the link was broken. 2. Internal System Error – Again caused by GBQ linked datasets which are read-only. Fabric mirroring requires GBQ change history to be enabled on tables so that it can track changes and only mirror incremental changes after first initial load. 3 (Showstopper) The two permissions that raised security red flags are bigquery.datasets.create and bigquery.jobs.create. To grant those permissions, you must assign one of these BigQuery roles: • BigQuery Admin • BigQuery Data Editor • BigQuery Data Owner • BigQuery Studio Admin • BigQuery User All these...

  • Atlanta Microsoft BI Group Meeting on October 9th (Everything You Want to Know About SQL Databases in Fabric)

    October 3, 2025 / No Comments »

    Atlanta BI fans, please join us in person for our next meeting on Thursday, October 9th (note that we are meeting on Thursday for this meeting) at 18:30 ET. Sukhwant (Senior Product Manager, Microsoft) will explain why you should consider Fabric SQL databases. And your humble correspondent will walk you through some of the latest Power BI and Fabric enhancements. For more details and sign up, visit our group page. Delivery: In-person Level: Intermediate Food: Pizza and drinks will be provided Agenda: 18:15-18:30 Registration and networking 18:30-19:00 Organizer and sponsor time (news, Power BI latest, sponsor marketing) 19:00-20:15 Main presentation 20:15-20:30 Q&A Overview: Microsoft Fabric is an all-in-one analytics platform, right? Wrong! With the introduction of SQL databases last year, we now have an all-in-one data platform. During this session you will hear directly from the product team about why we added SQL databases to Fabric, who should be using them, how...

  • Atlanta Microsoft BI Group Meeting on September 8th (End-to-End Azure DevOps for Data Engineering in Microsoft Fabric)

    September 5, 2025 / No Comments »

    Atlanta BI fans, please join us in person for our next meeting on Monday, September 8th at 18:30 ET. Jeff Levy (Data Architect @ Protiviti) will show us how to implement Azure DevOps for data engineering projects in Microsoft Fabric. And your humble correspondent will walk you through some of the latest Power BI and Fabric enhancements. For more details and sign up, visit our group page. Delivery: In-person Level: Intermediate Food: Pizza and drinks will be provided Agenda: 18:15-18:30 Registration and networking 18:30-19:00 Organizer and sponsor time (news, Power BI latest, sponsor marketing) 19:00-20:15 Main presentation 20:15-20:30 Q&A Overview: This session explores how to implement Azure DevOps for data engineering projects in Microsoft Fabric. You'll learn the following: Version Control Lakehouse assets (Pipelines / Notebooks / SQL Objects) Manage environments with reusable YAML templates Apply CI/CD Practices via the DevOps Build and Release Pipelines The session is ideal for data engineers and...

  • Migrating Fabric Import Semantic Models to Direct Lake (Part 2)

    August 24, 2025 / No Comments »

    I’ve previously shared my experience with migrating a Fabric imported semantic model to Direct Lake. This blog follows up with additional observations about performance. The following screenshot is taken from the Fabric Capacity Metrics app and it shows the maximum metrics over 14 days. The two enclosed items of interest are the original imported semantic model (the first item on the list) and its DL counterpart (the seventh item on the list). Memory utilization As I explained in the first part, the whole reason for taking this epic journey was to solve the out-ot-memory blowouts and constant pressure to climb the Fabric capacity ladder. With 1/5 of the user audience testing the dataset in production environment, that dataset grew to a maximum of 25 GB memory utilization which is in line with the imported model. It could have been interesting to downgrade the capacity, such as to F64, and observe...

  • From Prompt to Insight: My Daily Dance with AI

    August 10, 2025 / No Comments »

    “I go checking out the reports, digging up the dirt You get to meet all sorts in this line of work And when I find the reason, I still can't get used to it And what have you got at the end of the day? What have you got to take away?” Private investigations, Dire Straits Here we go Me: “Write code to do this and that.” LLM: “I’m glad to help. Here is the code.” Me: “It doesn’t work because of this error <nasty error message follows>” LLM: “You get this error because…Here is the correct code.” Me: “Doesn’t work again because of this new error <nastier error message follows>.” LLM: “You get this error because…Here is the correction of the corrected code.” After N iterations and mutual blame, we either get eventually to working code or give up and start cursing each other. LLM usually quits first, claiming...

  • Atlanta Microsoft BI Group Meeting on August 4th (Power BI Built-in Gems: Time-Saving Features You Should Be Using)

    July 30, 2025 / No Comments »

    Atlanta BI fans, please join us in person for our next meeting on Monday, August 4th at 18:30 ET, which marks the 15th anniversary of the Atlanta Microsoft BI Group! Lakshmi Ponnurasan (a Microsoft Data Platform MVP and a Certified Power BI specialist) will show us how to apply Power BI time-saving built-in features to create stunning and impactful reports in less time. And your humble correspondent will walk you through some of the latest Power BI and Fabric enhancements. For more details and sign up, visit our group page. Delivery: In-person Level: Intermediate Food: Pizza and drinks will be provided Agenda: 18:15-18:30 Registration and networking 18:30-19:00 Organizer and sponsor time (news, Power BI latest, sponsor marketing) 19:00-20:15 Main presentation 20:15-20:30 Q&A Overview: Power BI is like a treasure chest packed with powerful built-in features- except these gems are often hidden in plain sight, waiting to be discovered. These time-saving built-in...

  • Migrating Fabric Import Semantic Models to Direct Lake

    July 20, 2025 / No Comments »

    I’ve recently written about strategies for addressing memory pressures with Fabric large semantic models and I mentioned that one of them was switching to Direct Lake. This blog captures my experience of migrating a real-life import semantic model to Direct Lake. About the project In this case, the client had a 40 GB semantic model with 250 million rows spread across two fact tables. The semantic model imported data from a Google BigQuery (GBQ) data warehouse. The client applied every trick in the book to optimize the model, but they’ve found themselves forced to upgrade from a Power BI P1 to P2 to P3 capacity. I’ve written in the past about my frustration with Power BI/Fabric capacity resource limits. While the 25 GB RAM grant of a P1/F64 capacity for each dataset is generous for smaller semantic models, such as for self-service BI, it’s inadequate for large organizational semantic models....

  • Atlanta Microsoft BI Group Meeting on July 7nd (Getting Started with Copilot in Microsoft Fabric)

    July 1, 2025 / No Comments »

    Atlanta BI fans, please join us in person for our next meeting on Monday, July 7nd at 18:30 ET. Shabnam Watson (BI consultant and Microsoft MVP) will introduce you to Microsoft Fabric copilots. And your humble correspondent will walk you through some of the latest Power BI and Fabric enhancements. For more details and sign up, visit our group page. Delivery: In-person Level: Intermediate Food: Pizza and drinks will be provided Agenda: 18:15-18:30 Registration and networking 18:30-19:00 Organizer and sponsor time (news, Power BI latest, sponsor marketing) 19:00-20:15 Main presentation 20:15-20:30 Q&A Overview: Microsoft Fabric is revolutionizing data analytics, and Copilot is here to make it even smarter! Now available across all Fabric capacities, Copilot is more accessible than ever, allowing users at all levels to take advantage of AI-driven efficiencies. This session is a beginner-friendly introduction to Copilot within Microsoft Fabric, showcasing how AI can streamline data workflows, enhance insights, and...

  • SaaS Losers and Winners: Workday and NetSuite

    June 29, 2025 / No Comments »

    “Well, I'll be damned Here comes your ghost again But that's not unusual…” “Diamonds and Rust”, Joan Baez I’ve said in the past that I consider it a travesty when a SaaS provider doesn’t give you direct access to the data in its native storage, such as by ODBC and OLE DB providers, and forcing you to use file extracts or APIs (often horrible). This greatly inhibits data integration scenarios, such as extracting data for data warehousing. I wrote on this subject many times, including here, here and here. Let’s look at how two popular vendors have approached data access. SaaS Loser: Workday Workday is a popular HR cloud platform. Unfortunately, the only way to get your data out is via their horrible APIs, as a large law firm has discovered. If you use Workday, I recommend you explore the Workday Query Language (WQL) so you don’t have to parse...

  • Live Monitoring of Fabric Mirroring

    June 10, 2025 / No Comments »

    A current project called for mirroring a Google BigQuery dataset to Fabric. This feature is currently in private preview so don’t try to find it. However, the tips I share here should be applicable to other available mirroring scenarios, such as mirroring from Azure SQL Database. One of the GBQ tables was a transaction fact table with some 130 million rows. The issue was that the mirroring window would show this table as normally replicating table with Running green status, but we waited and waited and nothing was happening... First, the “Rows Replicated” counter is not continuously updating (doesn’t report live mirroring info). Instead, it’s updated when the table is fully replicated. That’s because the Delta table operations are atomic in nature. So, how do we do live monitoring of the initial synchronization then? We can use the Azure Storage Explorer to access directly the delta files of the mirrored database...

Training

We offer onsite and online Business Intelligence classes! Contact us about in-person training for groups of five or more students.
For more information or to register click here!

Books

Learn Power BI at your own pace with our latest book Applied Microsoft Power BI. Targeting information workers, data analysts, pros, and developers, It is designed as an easy-to-follow guide for learning how to implement BI solutions spanning the entire personal-team-organizational BI spectrum.

Syndication