Types of Power BI Real-time Datasets

/2 Comments/in /by

Everyone wants real-time BI, even when it doesn’t have to be really “real time”. Today Microsoft announced General Availability of Power BI Real-Time Streaming Datasets. There are actually three types of Power BI real-time datasets, as mentioned in the documentation.

  • Push – Power BI permanently stores the data, enabling historic analysis, and reports creation atop the dataset. Behind the scenes, Power BI provisions an Azure SQL instance when the dataset is created. New data is pushed into SQL. Power BI then connects to that dataset via DirectQuery. Query Refresh (sending new queries to Azure SQL to update dashboard visuals) occurs whenever data is pushed in. When you create the dataset programmatically, you can specify a retention policy (defaultRetentionPolicy setting). When defaultRetentionPolicy is set to None, the dataset accumulates data to the maximum allowed Power BI limit (currently 1 GB). When set to basicFIFO, the dataset holds up to 200,000 rows and after that older rows are pushed out when the new ones come in.
  • Streaming — Power BI stores the data only in a transient cache – this means report creation, historic analysis is disabled, but in return there is consistently lower latency between when the data is pushed in and when the visuals update. The data flows into a Redis cache, and the dashboard visuals directly pull data from that Redis cache. Therefore, consider streaming datasets when you want the lowest latency (we are talking about milliseconds here) but you are limited to a few pre-defined visualizations supported by the Power BI dashboard real-time tiles. You can’t create custom reports.
  • Hybrid — Hybrid datasets send data to both the “push” and “streaming” endpoints, thereby getting the benefits of both at the expense of duplicate storage.

Unless you use StreamInsight (currently, it supports only push datasets) or PubNub (supports streaming datasets), you must create the dataset programmatically using the Power BI REST APIs. Currently, you can’t use Power BI Desktop to create real-time datasets.

Atlanta MS BI Group Meeting on January 30th

/0 Comments/in , /by

MS BI fans, join me for the next Atlanta MS BI and Power BI Group meeting on Monday, January 30th at 6:30 PM. One of our most experienced consultants, Neal Waterstreet, will share his real-life experience in master data management with SQL Server MDS. I’ll update you on the latest with Power BI and SQL Server 2017. Prologika will sponsor the meeting.

Rate this meetinghttp://aka.ms/PUGSurvey, PUG ID: 104
Presentation:Master Data Management with SQL Server 2016 MDS
Level: Intermediate
Date:January 30th, 2017
Time6:30 – 8:30 PM ET
Place:South Terraces Building (Auditorium Room)

115 Perimeter Center Place

Atlanta, GA 30346

Overview:In this presentation we’ll first discuss the position Master Data Management plays in an organization’s overall data strategy. We’ll review the key concepts, different roles and responsibilities that members of the team typically play, potential risks, and best practices to help you get your organization moving forward with MDM. We’ll then take a look at some of the features of MDS 2016 such as the different ways of processing data, security improvements, Changesets and Entity Sync that make it an excellent tool for MDM.
Speaker:Neal Waterstreet is a BI Architect/Consultant with Prologika. He has more than 20 years of industry experience. Neal is skilled in the entire BI spectrum, including dimensional modeling, ETL design and development using Integration Services (SSIS), designing and developing multidimensional cubes and Tabular models using Analysis Services (SSAS) and Master Data Management using Microsoft Data Services (MDS). He’s also involved with the database community and is the co-founder and co-leader of the PASS Healthcare Virtual Chapter and the Atlanta Modern Excel User Group.
Sponsor:Prologika is one of the most trusted names in Data Analytics. Our clients, from small businesses to Fortune 100 enterprises, derive tremendous value from our services. Our mission is to help organizations make sense of data by applying the latest technologies for descriptive and predictive analytics and get actionable insights. Your organization will spend less time mining for information and be better equipped to make sound business decisions.
Prototypes with Pizza“Power BI subscriptions” and “Update on deploying Power BI reports to SSRS” by Teo Lachev

Unblocking the On-premises Data Gateway

/0 Comments/in /by

Scenario: You have configured the Power BI on-premises data gateway for centralized data access and verified that its data sources test just fine. Direct query connections work. However, when you go to Power BI Service and attempt to schedule a data refresh for a dataset, you might find that the data gateway is disabled.

Solution: The most common reasons for Power BI to disable the on-premises data gateway for refresh are:

  1. Unlike the personal gateway, the on-premises data gateway requires you to register data sources. You must go to the gateway properties and create data sources for all data sources used in your Power BI Desktop file. Unfortunately, as it stands Power BI doesn’t allow you to select which data sources in the Power BI Desktop file will be refreshed and which ones don’t require a refresh. It’s all or nothing proposition. So, if one data source is not compatible or can’t be refreshed, the gateway will be disabled.
  2. The connection strings in data sources in the Power BI Desktop file might differ from the settings of the data sources you registered in the on-premises gateway. For example, in Power BI Desktop you might have imported data from a local Excel file. Then, you might have moved the file to a network share and established a gateway data source to point to the network share. Because the connection strings differ, Power BI Service won’t find an on-premises gateway to serve the Excel file and it will disable the gateway for refresh. So, triple-verify the that data sources match.
  3. You might have manually added a table to your model and entered some data using the Power BI Desktop “Enter Data” feature. Because custom tables can’t refresh, Power BI disables the gateway.

012817_0130_Unblockingt1.png

Customer Success Case – ZynBit

/0 Comments/in , /by

One of our customers, ZynBit, made the Power BI blog today! Initially, ZynBit was considering Tableau but abandoned it in favor of Power BI because of the Power BI superior data modeling capabilities and the cost effective licensing model of Power BI Embedded. Prologika helped ZynBit to transition their solution to Power BI, including designing the data model and integrating reports with Power BI Embedded. Read our case study here.

Monitoring Progress of UPDATE

/0 Comments/in /by

How to monitor the progress of an UPDATE statement sent to SQL Server? Unfortunately, SQL Server currently doesn’t support an option to monitor the progress of DML operations. In the case of UPDATE against large tables, it might be much faster to recreate the table, e.g. with SELECT … INTO. But suppose that INSERT could take a very long time too and you prefer to update the data instead. Here is how to “monitor” the progress while the UPDATE statement is doing its job.

Suppose you are updating the entire table and it has 138,145,625 rows (consider doing the update in batches to avoid running out of log space). Let’s say the UPDATE statement changes the RowStartColumn column to the first day of the month:

UPDATE bi.FactAccountSnapshot

SET RowStartDate = DATEADD(MONTH, DATEDIFF(MONTH, 0, RowStartDate), 0);

Use these statements to monitor the remaining work by using a reverse WHERE clause. Make sure to execute both statements (SET TRAN and SELECT together) so that the SELECT statement can read the uncommitted changes.

SET TRAN ISOLATION LEVEL READ UNCOMMITTED;

SELECT CAST(1 – CAST(COUNT(*) AS DECIMAL) / 138145625 AS DECIMAL(5, 2)) AS PercentComplete ,COUNT(*) AS RowsRemaining

FROM bi.FactAccountSnapshot

WHERE RowStartDate <> DATEADD(MONTH, DATEDIFF(MONTH, 0, RowStartDate), 0);

What about INSERT? If you know how many rows you are inserting, you can simply check the current count of the inserted rows by using the sp_spaceused stored procedure:
sp_spaceused ‘tableName’.

Make BI Great Again!

/0 Comments/in , /by

…with the second edition of my “Applied Microsoft Power BI” book. After seven books and starting from scratch every time, I finally got to write a revision! Thoroughly revised to reflect the current state of Power BI, it added more than 20% new content and probably that much content was rewritten to keep up with the ever changing world of Power BI. Because I had to draw a line somewhere, Applied Microsoft Power BI (2nd Edition) covers all features that were that were released by early January 2017 (including subscriptions). As with my previous books, I’m committed to help my readers with book-related questions and welcome all feedback on the book discussion forum on the book page. While you are there, feel free to check out the book resources (sample chapter, front matter, and more). Consider also following my blog at https://prologika.com/blog and subscribing to my newsletter at https://prologika.com to stay on the Power BI latest.

  • Buy the paper copy from Amazon
  • Buy the Kindle ebook from Amazon
  • Other popular channels in 2-3 weeks

Power BI Subscriptions

/0 Comments/in /by

Today Microsoft released a highly anticipated Power BI feature – subscribed report delivery. Similar to SSRS individual subscriptions, users can go to a Power BI report and subscribe to one or more of its pages to receive a snapshot of the page on a scheduled basis. The following scenarios are possible depending on the report data source:

  • Imported datasets – the subscription follows the dataset refresh schedule. You’ll get an email every time the scheduled refresh happens, so long as you haven’t gotten an email in the last 24 hours.
  • DirectQuery datasets – Power BI checks the data source every 15 minutes. You’ll get an email as soon as the next check happens, provided that you haven’t gotten an email in the last 24 hours (if Daily is selected), or in the last seven days (if Weekly is selected).
  • Live connection to SSAS – Power BI checks the data source every 15 minutes and it’s capable of detecting if the data has changed. You’ll get an email only if the data has changed if you haven’t gotten an email in the last 24 hours
  • Connected Excel reports – Power BI checks the data source every hour. You’ll get an email only if the data has changed if you haven’t gotten an email in the last 24 hours.

011617_1307_PowerBISubs1.png

Power BI subscriptions have these limitations:

  • The only export option is screenshot. You can’t receive the page exported to PowerPoint, for example.
  • Users can create individual subscriptions only. You can’t subscribe other users as you can do with Reporting Services data-driven subscriptions.
  • The Power BI admin can’t see or manage subscriptions across the tenant.

APPLIED MICROSOFT POWER BI
 (BRING YOUR DATA TO LIFE!)

/0 Comments/in /by

book1Bring your data to life today and learn how Power BI changes the way everyone gains insights from data.

    • Publication date: 1/1/2016
    • Size: 314 pages, 7.5″ x 9.25″
    • Price: $44.95
    • ISBN 10: 0-9766353-6-4
    • ISBN 13: 978-0-9766353-6-9

Introduces information workers, data analysts, IT pros, and developers to Microsoft Power BI — a cloud-hosted, business intelligence and analytics platform that democratizes and opens BI to everyone, making it free to get started!

There is a newer edition of this book

Power BI changes the way you gain insights from data; it brings you a cloud-hosted, business intelligence and analytics platform that democratizes and opens BI to everyone. It does so under a simple promise: “five seconds to sign up, five minutes to wow!”

“That is why resources like this fantastic book will become instrumental for you!”
Jen Underwood
Principal Program Manager
Microsoft Business Intelligence

Synopsis

An insightful tour that provides an authoritative yet independent view of this exciting technology, this guide introduces Microsoft Power BI—a cloud-hosted, business intelligence and analytics platform that democratizes and opens BI to everyone, making it free to get started!

Information Workers will learn how to connect to popular cloud services to derive instant insights, create interactive reports and dashboards, and view them in the browser and on the go! Data Analysts will discover how to integrate and transform data from virtually everywhere and then implement sophisticated self-service models. The book also teaches BI and IT Pros how to establish a trustworthy environment that promotes collaboration, and they’ll implement Power BI-centric solutions for descriptive, real-time, and predictive analytics. Developers will find out how to integrate custom applications with Power BI, to embed reports, and to implement custom visuals to effectively present any data.

Ideal for both experienced BI practitioners and beginners, this book doesn’t assume you have any prior data analytics experience. It’s designed as an easy-to-follow guide that introduces new concepts with step-by-step instructions and hands-on exercises.

What’s inside

  • Get insights from popular cloud services on any device!
  • Implement sophisticated personal BI models!
  • Enable team BI and implement descriptive, predictive, and real-time BI solutions!
  • Extend Power BI with custom visuals and report-enable custom apps!
    …and much more!

Resources

Front matterSample chapter (Chapter 1)Errata
IndexSource codeForum
Back cover

Reviews

“The true power in Power BI cannot be appreciated without understanding what the offering can do and how to best use it. That is why resources like this fantastic book will become instrumental for you. This book starts by providing an overview of the foundational components of Power BI. It introduces Power BI Desktop, data modeling concepts, building reports, publishing and designing dashboards. Readers will be up and running in no time. It then moves on to bring you up to speed on deeper dive topics such as data gateways, data re-fresh, streaming analytics, embedding and the Power BI data visualization API. Not only is Teo one of the first people in the world to learn and write about Power BI 2.0, he also brings a wealth of knowledge from deploying the first real-world implementations. Much like Teo’s previous books on Analysis Services and Reporting Services, this Power BI book will be a must read for serious Microsoft professionals. It will also empower data analysts and enthusiasts everywhere.”

Jen Underwood
Principal Program Manager, Microsoft Business Intelligence

“I’m impressed about the breadth of the topics covered by Teo Lachev in this book, I’ve just took a quick look at every chapter, and Teo covered all the topics at least at the point where you can start doing something (and in some chapter also more than just an intro). Considering the speed of Power BI releases and the effort required in writing a book, I know the huge effort behind this. My kudos to this book!”

Macro Russo
Consultant, SQLBI

[scrapeazon asin=”0976635364″ width=”” height=”400″ truncate=”1000″ summary=”true” border=”false”]

How to purchase

Buy the paper copy from Amazon
Buy the Kindle ebook from Amazon