Blog – Page 6 – Prologika

Fabric Data Warehouse: The Good, The Bad, and the Ugly

July 3, 2023 / No Comments »

"Patience my tinsel angel, patience my perfumed child One day they'll really love you, you'll charm them with that smile But for now it's just another Chelsea [TL: BI] Monday" Marrillion Continuing our Power BI Fabric journey, let's look at another of its engines that I personally care about – Fabric Warehouse (aka as Synapse Data Warehouse). Most of my real-life projects require integrating data from multiple data sources into a centralized repository (commonly referred to as a data warehouse) that centralizes trusted data and serves it as a source to Power BI and Analysis Services semantic models. Due to the venerable history of relational databases and other benefits, I've been relying on relational databases powered by SQL Server to host the data warehouse. This usually entails a compromise between scalability and budget. Therefore, Azure-based projects with low data volumes (up to a few million rows) typically host the warehouse...
Fabric Lakehouse: The Good, The Bad, and the Ugly

June 25, 2023 / No Comments »

"Hey, I got this lake house, you should see it It's only down the road a couple miles I bet you'd feel like you're in Texas I bet you'll wanna stay a while" Brad Cox Continuing our Fabric purview, Lakehouse is now on the menu. Let's start with a definition. According to Databricks which are credited with this term, a data lakehouse is "a new, open data management architecture that combines the flexibility, cost-efficiency, and scale of data lakes with the data management and ACID transactions of data warehouses, enabling business intelligence (BI) and machine learning (ML) on all data." Further, their whitepaper "argues that the data warehouse architecture as we know it today will wither in the coming years and be replaced by a new architectural pattern, the Lakehouse, which will (i) be based on open direct-access data formats…" In other words, give us all of your data in...
Fabric OneLake: The Good, the Bad, and the Ugly

June 3, 2023 / No Comments »

To know a tool is to know its limitations – Teo's proverb In a previous post, I shared my overall impression of Fabric. In this post, I'll continue exploring Fabric, this time sharing my thoughts on OneLake. If you need a quick intro to Fabric OneLake, the Josh Caplan's "Build 2023: Eliminate data silos with OneLake, the OneDrive for Data" presentation provides a great overview of OneLake, its capabilities, and the vision behind it from a Microsoft perspective. If you prefer a shorter narrative, you can find it in the "Microsoft OneLake in Fabric, the OneDrive for data" post. As always, we are all learning and constructive criticism would be appreciated if I missed or misinterpreted something. What's Fabric OneLake? In a nutshell, OneLake is a Microsoft-provisioned storage where the ingested data and the data from the analytical (compute) engines are stored (see the screenshot below). Currently, PBI datasets (Analysis...
Atlanta Microsoft BI Group Meeting on June 5th (How to Fix an Inherited Power BI Report)

May 29, 2023 / No Comments »

Please join us for the next meeting on Monday, June 5th, at 6:30 PM ET. Kristyna Hughes (Senior data & analytics consultant at 3Cloud will join us remotely and share best practices for dissecting a complicated and broken Power BI report and a checklist of how to fix it. Improving will sponsor the meeting. Your humble correspondent will demonstrate how Power BI can integrate better with SharePoint and OneDrive. For more details and sign up, visit our group page. WE NOW MEET BOTH IN-PERSON AND ONLINE. WE STRONGLY ENCOURAGE YOU TO ATTEND THE EVENT IN PERSON FOR BEST EXPERIENCE AND BECAUSE AN EMPTY AUDIENCE IS DISCOURAGING TO SPEAKERS AND SPONSORS. ALTERNATIVELY, YOU CAN JOIN OUR MEETINGS ONLINE VIA MS TEAMS. WHEN POSSIBLE, WE WILL RECORD THE MEETINGS AND MAKE RECORDINGS AVAILABLE HERE. PLEASE RSVP ONLY IF COMING TO OUR IN-PERSON MEETING AND PLAN TO EAT Presentation: How to Fix an...
A First Look at Microsoft Fabric: The Good, the Bad, and the Ugly

May 28, 2023 / No Comments »

"May you live in interesting times" – Chinese proverb Microsoft Fabric is upon us with a grand fanfare. You can get a good overview of its vision and capabilities by watching the Microsoft Fabric Launch Digital Event (Day 1) and Microsoft Fabric Launch Digital Event (Day 2) recordings. Consultants and experts are extolling its virtues and busy fully aligning with Microsoft. There is a lot of stuff going on in Fabric and I'm planning to cover the technologies I work with and care about in more detail in future posts as Microsoft reveals more what's under the kimono. This post is about my overall impression on Fabric, in an attempt to cut through the dopamine and adrenaline-infused marketing hype. As always, please feel free to disagree and provide constructive criticism. The Good Let's just say that after 30 years working with Microsoft technologies, I'm very, very skeptical when I hear...
Atlanta MS BI and Power BI Group Meeting on May 1st (Getting started with Power BI Deployment Pipelines)

April 19, 2023 / No Comments »

Please join us for the next meeting on Monday, May 1st, at 6:30 PM ET. Akshata Revankar (Data Engineer, Specialist at McKinsey & Company) will show you how to leverage Power BI deployment pipelines to promote content changes between environments, such as DEV and PROD. For more details and sign up, visit our group page. WE NOW MEET BOTH IN-PERSON AND ONLINE. WE STRONGLY ENCOURAGE YOU TO ATTEND THE EVENT IN PERSON FOR BEST EXPERIENCE AND BECAUSE AN EMPTY AUDIENCE IS DISCOURAGING TO SPEAKERS AND SPONSORS. ALTERNATIVELY, YOU CAN JOIN OUR MEETINGS ONLINE VIA MS TEAMS. WHEN POSSIBLE, WE WILL RECORD THE MEETINGS AND MAKE RECORDINGS AVAILABLE HERE. PLEASE RSVP ONLY IF COMING TO OUR IN-PERSON MEETING AND PLAN TO EAT Presentation: Getting started with Power BI Deployment Pipelines Date: May 1st Time: 18:30 – 20:30 ET Place: Onsite and online Level: Intermediate Food: Food and drinks will be available for this meeting Agenda:...
Fixing SSIS Crashes

April 2, 2023 / No Comments »

I've spent hours on this so someone else might find the solution useful. I've developed an SSIS package that uses a ForEach Loop container. Then, I closed Visual Studio and reopen it. The SSIS designer opens the package, thinks for a few seconds if it likes it or not, and then it crashes Visual Studio. I've noticed that the VS status bar shows a message that it validates the ForEach Loop container, which was an important clue. How do we fix this horrible issue? Initially, I was thinking that it was interference from Visual Studio 2022 that someone else has recently installed. So, I upgraded, uninstalled, repaired, tried VS 2022, etc. to no avail. Finally, I open the package code in text editor and added "DTS:DelayValidation="True" to the container task to disable the upfront validation on package open. This fixed the issue although I had no idea what caused the...
Atlanta MS BI and Power BI Group Meeting on April 3rd (Power BI Dashboard in an Hour)

March 28, 2023 / No Comments »

Please join us for the next meeting on Monday, April 3rd, at 6:30 PM ET. Your humble correspondent will revisit Power BI important fundamentals in a demo-packed session. For more details and sign up, visit our group page. PLEASE NOTE THAT OUR IN-PERSON MEETING LOCATION HAS CHANGED! WE STRONGLY ENCOURAGE YOU TO ATTEND THE EVENT IN PERSON FOR BEST EXPERIENCE. ALTERNATIVELY, YOU CAN JOIN OUR MEETINGS ONLINE VIA MS TEAMS. WHEN POSSIBLE, WE WILL RECORD THE MEETINGS AND MAKE RECORDINGS AVAILABLE AT HTTPS://BIT.LY/ATLANTABIRECS. PLEASE RSVP ONLY IF COMING TO OUR IN-PERSON MEETING. Presentation: Power BI Dashboard in an Hour (DIAH) Date: April 3rd Time: 18:30 – 20:30 ET Place: Onsite and online Level: Beginner Food: Food and drinks will be available for this meeting Agenda: 18:15-18:30 Registration and networking 18:30-19:00 Organizer and sponsor time (events, Power BI latest, sponsor marketing) 19:00-20:15 Main presentation 20:15-20:30 Q&A ONSITE (RECOMMENDED) Improving Office 11675...
Working with Large Tables in SQL Server

March 24, 2023 / No Comments »

Warning: This blog contains old tricks of an old dog. Scenario: Suppose you have a large table in SQL Server, e.g. hundreds of millions or even a billion rows. DML operations (SELECT, INSERT, UPDATE, DELETE) take long time. How do you speed them up? Do you split the large table into multiple tables? Or, do you ask for better hardware? Or, do you start looking for a new job with less data? Solution: It's nothing new but I see clients struggle with this all the time because they don't know any better. The solution is to partition the table and use partition switching that SQL Server has supported since time immemorial. Cathrine Wilhelmsen has a great step-by-step blog covering different scenarios, but the process goes like this: Configure page compression for the large table (see benefits here). Partition the large table, such as by month. Create a not-partitioned staging table...
Atlanta MS BI and Power BI Group Meeting on March 6th (The Semantic Lakehouse: Power BI and Databricks)

March 3, 2023 / No Comments »

Please join us for the next meeting on Monday, March 6th, at 6:30 PM ET. Leo Furlong (Senior Solutions Architect at Databricks) will share their point of view on why “the best data warehouse is a lakehouse.” For more details and sign up, visit our group page. PLEASE NOTE THAT OUR IN-PERSON MEETING LOCATION HAS CHANGED! WE STRONGLY ENCOURAGE YOU TO ATTEND THE EVENT IN PERSON FOR BEST EXPERIENCE. ALTERNATIVELY, YOU CAN JOIN OUR MEETINGS ONLINE VIA MS TEAMS. WHEN POSSIBLE, WE WILL RECORD THE MEETINGS AND MAKE RECORDINGS AVAILABLE AT HTTPS://BIT.LY/ATLANTABIRECS. PLEASE RSVP ONLY IF COMING TO OUR IN-PERSON MEETING. Presentation: The Semantic Lakehouse: Power BI and Databricks Date: March 6th Time: 18:30 – 20:30 ET Place: Onsite and online Level: Intermediate Food: Food and drinks will be available for this meeting Agenda: 18:15-18:30 Registration and networking 18:30-19:00 Organizer and sponsor time (events, Power BI latest, sponsor marketing) 19:00-20:15...