Blog – Page 2 – Prologika

LLM Adventures: Text2SQL

July 7, 2024 / No Comments »

As inspired by Kevin Jordain, who clued me about the increased demand for using natural questions to analyze client’s data, John Kerski, who did a great presentation to our Atlanta BI Group on integrating Power Query with Azure OpenID, and Kyle Hale, who believes (wrongly 😊) that the Databricks Genie will make Power BI and semantic modeling obsolete, I set on a quest to find how effective Large Language Models (LLM) are in generating SQL from natural questions, also known as Text2SQL. What a better way to spend some free time around the 4th of July holiday, right? I have to say that I was impressed with LLM. I used the excellent Ric Zhou’s Text2SQL sample as a starting point inside Visual Studio Code. The sample uses the Python streamlit framework to create a web app that submits natural questions to Azure OpenAI. My humble contributions were: I switched to...
Atlanta Microsoft BI Group Meeting on July 1st (Commenting Power Query with Azure OpenAI)

June 25, 2024 / No Comments »

Atlanta BI fans, please join us in person for the next meeting on Monday, July 1st at 6:30 PM ET. John Kerski (Microsoft MVP) will shows us how to integrate ChatGPT with Power BI. Your humble correspondent will help you catch up on Microsoft BI latest. CloudStaff.ai will sponsor the event. For more details and sign up, visit our group page. Details Presentation: Commenting Power Query with Azure OpenAI Delivery: In-person Date: July 1, 2024 Time: 18:30 – 20:30 ET Level: Intermediate Food: Pizza and drinks will be provided Agenda: 18:15-18:30 Registration and networking 18:30-19:00 Organizer and sponsor time (events, Power BI latest, sponsor marketing) 19:00-20:15 Main presentation 20:15-20:30 Q&A Venue Improving Office 11675 Rainwater Dr Suite #100 Alpharetta, GA 30009 Overview: Large Language Models (such as ChatGPT) can greatly enhance the way you develop and deliver Power BI solutions. In this session I will show you how to integrate Azure Open AI into...
Fabric Analytics Engineer Certification Snagged

June 7, 2024 / No Comments »

Just passed DP-600 and therefore I'm officially a Fabric Analytics Engineer. Besides renewals, I intend this to be my last Microsoft certification. After more than 30 years of certifying (12 certifications and 30 exams) and helping other certify with training and books, I think it's time to close this chapter. Call me certifiable...
Modern Data Warehouse (MDM) Reloaded

June 1, 2024 / No Comments »

"Where are the prophets, where are the visionaries, where are the poets To breach the dawn of the sentimental mercenary" "Fugazi", Marillion I’ve written in the past about the dangers of blindly following “modern” data architectures (see my posts “Are you modern yet?” and “Data Lakehouse: The Good, the Bad, and the Ugly”) but here we go again. Once upon a time, a large company hired a large Microsoft partner to solve a common and pervasive challenge. Left on their own, departments have set up their own data servers, thus creating data silos leading to data duplication and inconsistent results. How did the large vendor attempt to solve these horrible issues? Modern Data Warehouse (MDM) architecture of course. Nothing wrong with it except that EDW and organizational semantic model(s) are missing and that most of the effort went into implementing the data lake medallion architecture where all the incoming data...
Atlanta Microsoft BI Group Meeting on June 3rd (Power BI Direct Lake storage mode)

May 28, 2024 / No Comments »

Atlanta BI fans, please join us in person for the next meeting on Monday, June 3rd at 6:30 PM ET. Shabnam Watson (Consultant and Owner of ABI Cube) will discuss the benefits of using the Direct Lake storage mode in Microsoft Fabric. Your humble correspondent will help you catch up on Microsoft BI latest. CloudStaff.ai will sponsor the event. For more details and sign up, visit our group page. Presentation: Power BI Direct Lake storage mode: How to achieve blazing fast performance without importing data Delivery: In-person Time: 18:30 – 20:30 ET Level: Beginner/Intermediate Food: Pizza and drinks will be provided Agenda: 18:15-18:30 Registration and networking 18:30-19:00 Organizer and sponsor time (events, Power BI latest, sponsor marketing) 19:00-20:15 Main presentation 20:15-20:30 Q&A Venue Improving Office 11675 Rainwater Dr Suite #100 Alpharetta, GA 30009 Overview: Power BI engine in Microsoft Fabric has been significantly revamped to work directly with Delta files in OneLake. This brand-new...
Tips for Configuring Azure Data Factory Alerts

April 15, 2024 / No Comments »

Alerting is an important monitoring task for any ETL process. Azure Data Factory can integrate with a generic Azure event framework (Azure Monitor) which makes it somewhat unintuitive for ETL monitoring. You can set up and change the alerts using the ADF Monitoring hub. You might find the following tips useful for setting up an ADF alert: Dimension – I like to nest pipelines where a master pipeline starts dimension and fact master pipelines, which in turn execute unit pipelines, such as a loading a dimension table. This allows me to restart ETL at any level. In the Dimension dropdown (nothing to do with DW dimensions), I select all such master pipelines because if an error is triggered downstream, it will bubble up to the master. Failure Type – I select all failure types. Condition – I configure the alert to be generated as soon as one failure occurs. Period...
Atlanta Microsoft BI Group Meeting on April 1st (Real-Time Analytics with Microsoft Fabric: Unlocking the Power of Streaming Data)

March 26, 2024 / No Comments »

Atlanta BI fans, please join us in person for the next meeting on Monday, April 1st at 6:30 PM ET. Aravinth Krishnasamy (Principal Architect at Ecolab) will provide an end-to-end overview of Microsoft Fabric real-time analytics capabilities. Your humble correspondent will help you catch up on Microsoft BI latest. CloudStaff.ai will sponsor the event. For more details and sign up, visit our group page. Presentation: Real-Time Analytics with Microsoft Fabric: Unlocking the Power of Streaming Data Delivery: In-person Date: April 1, 2024 Time: 18:30 – 20:30 ET Level: Beginner/Intermediate Food: Pizza and drinks Agenda: 18:15-18:30 Registration and networking 18:30-19:00 Organizer and sponsor time (events, Microsoft BI latest, sponsor marketing) 19:00-20:15 Main presentation 20:15-20:30 Q&A Overview: This session will provide an end-to-end overview of Microsoft Fabric Real-Time Analytics capabilities. We will go over the following topics: 1. Introduction to Real-Time Analytics: Overview of the platform and its capabilities 2. Data...
What Can Fabric Do For My Lake?

March 13, 2024 / No Comments »

Previously, I discussed the pros and cons of Microsoft Fabric OneLake and Lakehouse. But what if you have a data lake already? Will Fabric add any value, especially if your organization is on Power BI Premium and you get Fabric features for free (that is, assuming you are not overloading your capacity resources)? Well, it depends. Managed Area A Fabric lakehouse defines two areas: managed and unmanaged. The managed area (Tables folder) is exclusively for Delta/Parquet tables. If you have your own data lake with Delta/Parquet files, such as Databricks delta lake, you can create shortcuts to these files or folders located in ADLS Gen 2 or Amazon S3. Consequently, the Fabric lakehouse would automatically register these shortcuts as tables. Life is good in the managed area. Shortcuts to Delta/Parquet tables open interesting possibilities for data virtualization, such as: Your users can use the Lakehouse SQL Analytics endpoint to join...
Make a Pledge and Trouble is at Hand: Microsoft AI and Fabric Copilots

March 11, 2024 / No Comments »

Rounding up the wisdom from the inscriptions of the Apollo Temple (see the Nothing in Excess and Know Thyself posts), the last known famous inscription was "Make a pledge and trouble is at hand." Although the original intention was lost in time, it's believed that this inscription instructs us to understand our limits so we don't overpromise. Fast forward to modern data analytics, I remember how Microsoft Power BI started less than 10 years ago with the grand promise to democratize BI to the masses. One would believe that AI would be an essential ingredient to fulfill that promise. To its credit, Power BI includes various built-in and useful ML features, such as Analyze Increase/Decrease (my favorite), Decomposition Tree, Key Influencers, Get Insights, and Q&A. It's surprising to me though that Fabric copilots are an ultra premium feature as they require at least P1 or its Fabric equivalent of F64...
A First Look at DAX Visual Calculations: the Good, the Bad, and the Ugly

February 28, 2024 / No Comments »

The February 2024 release of Power BI Desktop includes a preview of visual calculations. As its name suggests, a visual calculation is a visual-scoped DAX measure that works at the aggregate (visual) level. The Good Visual calculations make previously difficult tasks much easier. Consider the following matrix: Suppose you need a measure that calculates the difference between the product categories in the order they were sorted in the visual. Implementing this as a regular DAX measure is a challenge. Yet, if we had a way to work with the cells in the visual, we can easily find a way to get this to work. Ideally, this would work similar in Excel, but DAX doesn’t know about relative references. However, visual calculations do (kind of). Let’s right-click on the visual and select “New calculation”. In the visual-level DAX formula bar, enter the following formula: Category Diff = [Sum of SalesAmount] -...