Cumulative Update 3 for SQL Server 2008 Service Pack 1

Microsoft released today a Cumulative Update 3 for SQL Server 2008 Service Pack 1. Among other things it fixes the Report Builder 2.0 ClickOnce deployment issue on x64 which I reported here.

Excel Page Numbering Issue

An issue was reported about page numbering and exporting reports to Excel with Reporting Services 2008. Specifically, if the page header or footer uses a placeholder for the page number, when exported and previewed in Excel, the report will show Page 1 on all pages.

Page [&PageNumber] of [&TotalPages]

While waiting for Microsoft to fix this, use an expression that concatenates the PageNumber member of the Globals collection.

= “Page” & Globals!PageNumber & ” of “ & Globals!TotalPages

To enter the expression, select the textbox item, right-click and choose Expression.

Google – The Best Thing that Ever Happened to Microsoft

On a different subject, you’ve probably heard the news: Google will release an operating system called Google Chrome OS which will challenge Windows and instill fear into the Microsoft camp. To the contrary, I think Google is the best thing that ever happened to Microsoft. How come?

I remember reading somewhere that after the perestroika, Mikhail Gorbachev had supposedly told Ronald Reagan “we’ll now do the worst thing to you (USA); we’ll leave you without enemy”. Perhaps, too extreme but paraphrased to business, completion is a good thing. When challenged, good companies become better, products improved and consumers benefit. So, I hope Google will continue expanding their ambitions. Similarly, I hope Microsoft gives Google a run for its money by competing relentlessly to increase their share of the search market. I’d personally love to see business intelligence features added to the search results. Why not, the first two letter s of bing are BI, right? How come I can’t even sort the search results by date to see the most recent matches on top? A chart showing the popularity of the search item over time? People who searched for X searched also for Y?

It will be interesting to see how this mega competition will evolve in time. As Chinese say “may you live in interesting times”.

How to Test SSRS MDX Queries in SQL Server Management Studio

I often need to capture an MDX query that a report sends to Analysis Services and execute it in SQL Server Management Studio (SSMS). I usually do this to understand what parameters the report passes to the query and to troubleshoot the query itself. Unfortunately, SSMS doesn’t support parameterized MDX queries so you have to resort to the following technique:

Capturing MDX Queries

The easiest way to capture an MDX query is to use the SQL Server Profiler.

  1. Start SQL Server Profiler from the Microsoft SQL Server <version>-> Performance Tools program group.
  2. Click File -> New Trace or press Ctrl+N.
  3. In the Connect To Server dialog box, choose Analysis Services, enter your server name, and click Connect.
  4. In the Trace Properties dialog box, click the Event Selection tab. Check the Show All Events and Show All Columns checkboxes.
  5. Optionally, if you want to see only your events, that is events triggered by your actions only, click the Column Filters button. In the Edit Filter dialog box, select NTUserName. Expand the Like tree node on the right and enter your Windows login surrounded by %, e.g. %t_lachev%.
  6. Back to the Trace Properties dialog box, click Run to start the trace.
  7. Now run the report whose queries you want to capture and watch the profiler output for a Query Begin event.

Typically, a report would execute several MDX queries because there will be queries for the report parameters. You can use the profiler find function (Ctrl+F) to search for the query text you need in the TextData column in order to find the report main query (that one that supplies the report with data).

  1. Selec t the Query Begin event of the query you want. In the lower pane of the profiler, you will see the query MDX statement followed by Parameters and PropertyList nodes, such as:

    WITH MEMBER MyKPI as StrToMember( @KPISelector)

    SELECT NON EMPTY {MyKPI} ON COLUMNS,

    NON EMPTY StrToSet(@Dataset)

    * StrToMember(@StartPeriod) : StrToMember(@EndPeriod) ON ROWS

    FROM Viking

    WHERE (StrToMember(@Organization) )

    CELL PROPERTIES VALUE, BACK_COLOR, FORE_COLOR, FORMATTED_VALUE, FORMAT_STRING, FONT_NAME, FONT_SIZE, FONT_FLAGS

    <Parameters xmlns:xsi=”http://www.w3.org/2001/XMLSchema-instance” xmlns:xsd=”http://www.w3.org/2001/XMLSchema” xmlns=”urn:schemas-microsoft-com:xml-analysis”>

    <Parameter>

    <Name>KPISelector</Name>

    <Value xsi:type=”xsd:string”>[Measures].[Return On Average Assets]</Value>

    </Parameter>

    <Parameter>

    <Name>StartPeriod</Name>

    <Value xsi:type=”xsd:string”>[Date].[Month].&amp;[20060101]</Value>

    </Parameter>

    <Parameter>

    <Name>EndPeriod</Name>

    <Value xsi:type=”xsd:string”>[Date].[Month].&amp;[20061201]</Value>

    </Parameter>

    <Parameter>

    <Name>Dataset</Name>

    <Value xsi:type=”xsd:string”>{ [Dataset].[Dataset].&amp;[-2],[Dataset].[Dataset].&amp;[-1] }</Value>

    </Parameter>

    <Parameter>

    <Name>Organization</Name>

    <Value xsi:type=”xsd:string”>[Organization].[Organization Hierarchy].&amp;[-1]</Value>

    </Parameter>

    </Parameters>

    <PropertyList xmlns=”urn:schemas-microsoft-com:xml-analysis”>

    <Catalog>VANTAGE_CUBE_M_BASELINE DEV</Catalog>

    <LocaleIdentifier>1033</LocaleIdentifier>

    <Format>Tabular</Format>

    <Content>SchemaData</Content>

    <Timeout>0</Timeout>

    <ReturnCellProperties>true</ReturnCellProperties>

    <DbpropMsmdFlattened2>true</DbpropMsmdFlattened2>

    </PropertyList>

  2. Click anywhere in the lower pane and press Ctrl+A to select all text, then Ctrl+C to copy it to the clipboard.

Testing MDX Queries

Now that you have the query, you are ready to test it in SSMS.

  1. Open SSMS and connect to the Analysis Services server.
  2. Right-click on the SSAS database and click New Query -> MDX to open and create a new MDX query. Press Ctrl+V to paste the query text that you captured in the profiler.
  3. Since the profiler escapes certain characters, first you need to unescape them. For example, replace (Ctrl+H) all &amp; with &.
  4. One by one, go through the parameters listed in Parameters node and replace the parameter placeholder in the MDX query with the actual value. For example, press Ctrl+H to search for
    @KPISelector (note that the parameters are prefixed with @ in the MDX query) and replace all occurrences (Replace All option) with the actual parameter value. Since typically SSRS MDX queries use StrToMember or StrToSet functions, you need to surround the parameter values with quotes, e.g. “[Measures].[Return On Average Assets]”.
  5. Once you replace all parameters, comment the Parameters and PropertyLists nodes by selecting all text in these nodes and clicking the Comment Out the Selected Lines toolbar button. The only text that it not commented should be the MDX query text.

At this point, you can execute the query by pressing Ctrl+E or clicking the Exclamation toolbar button.

UPDATE

When things get tedious, the developer writes a program. As Greg Galloway pointed out, Darren Gosbell has already written a small but incredibly useful utility, SSAS Query Capture, that automates the process of capturing the queries and replacing the parameters. Moving forward to Visual Studio 2008 and SQL Server 2008, you need to:

  1. Upgrade the source code to Visual Studio. To do so, just open the C# source project in Visual Studio 2008 and accept the defaults.
  2. Delete the Microsoft.AnalysisServices and reference Microsoft.AnalysisServices.dll which you can find in the \Program Files\Microsoft SQL Server\100\SDK\Assemblies\ folder.
  3. Build the project and run QueryCapture.exe in the bin\debug folder.

A cautionary note: The utility always encloses the parameter values in quotes because it assumes that they will be inside the StrToMember and StrToSet functions which SSRS MDX Query Designer generates when you add parameters. If you reference parameters literally, as I demonstrated in this blog, remove the quotes surrounding these parameter values. Otherwise, the chances are that the query will return no data.

I couldn’t imagine how much time Darren’s Query Capture utility could have saved me if I only knew about it! I hope he adds it to the BIDS Helper project.

Maintaining State in Reporting Services 2008

Sometimes, more advanced reporting needs may require maintaining state using custom code. Recently I had to implement a rather involved report consisting of two sections: a detail section that used a recursive sum, Sum(Fields!SomeField.Value, , True), and a summary section with additional calculations which would reference some of the aggregated values in the detail section. Since Reporting Services is not Excel, you cannot reference arbitrary fields on a report. So, I had to cache the values from the detail section in a hashtable using custom code so the summary section could obtain these values when needed. The following embedded code gets the job done:

Friend Shared _hashtable As New System.Collections.Hashtable()

Function GetKey(ByVal id as Long, ByVal column as String) as String

return Report.User!UserID & id & column

End Function

Function AddValue(ByVal id as Long, ByVal column as String, ByVal value As Object) As Object

Dim key as String = GetKey(id, column)

If _hashtable.ContainsKey(key)=False Then

            _hashtable.Add(key, value)

End If

Return value

End Function

Function GetValue(ByVal id as Long, ByVal column as String) As Object

Dim key as String = GetKey(id, column)

If _hashtable.ContainsKey(key) Then

    Return _hashtable(key)

End If

Return Nothing

End Function

Protected Overrides Sub OnInit()

Dim keys As String() = New String(_hashtable.Keys.Count – 1) {}

_hashtable.Keys.CopyTo(keys, 0)

For Each key as String In keys

         If key.StartsWith(Report.User!UserID) Then

         _hashtable.Remove(key)

        End if

Next key

End Sub

In my scenario, tablix cells in the detail section would call the AddValue function to cache the dataset field values. Then, cells in the summary section would call GetValue to obtain these cached values. This technique is not new and you’ve probably used it to solve similar challenges.

What’s new is that moving to Reporting Services 2008 you may need to revisit your custom code and make changes. First, in prior version of Reporting Services, you probably declared the hashtable variable as an instance variable and that worked just fine. However, Reporting Services 2008 introduced a new on-demand processing engine where each page is processed separately. Consequently, you may find that the report “loses” values. Specifically, if the report fits on one page, everything will work as expected. But if the report spills to more pages, the hashtable collection will lose the values loaded on the first page. The solution is to declare the hashtable object as a static (Shared in Visual Basic) object so it survives paging.

Friend Shared _hashtable As New System.Collections.Hashtable()

But because a static object is shared by all users running the report, you need to make the hashtable key user-specific, such as by using the User!UserID variable which returns the user identity.

Function GetKey(ByVal id as Long, ByVal column as String) as String

return Report.User!UserID & id & column

End Function

Are we done? Well, there is a nasty gotcha that you need to be aware of. If the report takes parameters, you will surprised to find out that the custom code returns the cached values from the first report run and changing the parameters (report data) doesn’t change the values in these cells that call the GetValue function. To make the things even more confusing, testing the report in the BIDS Report Designer or Report Builder 2.0 will work just fine. But you will get the above issue after you deploy and run the server report, such as when you request the report in Report Manager or SharePoint.

What’s going on? As it turns out, the hashtable object survives report requests as a result of parameter changes. Consequently, the code that checks if the hashtable key exists will find the key when the report is reposted and it will not add the new value.

If _hashtable.ContainsKey(key)=False Then

_hashtable.Add(key, value)

End If

The solution is to clear the user-specific hashtable items each time the report is run. This takes place in the OnInit method. The OnInit method is a special method which gets executed before the report is processed. You can use this method to do initialization tasks or, in this case, clear state held in global variables.

Are we there yet? Almost. As the reader samsonfr pointed out in a blog comment, we’ll need to make this code thread-safe because the chances are that multiple users may be running the report at the same time so concurrent threads may need to write and read to/from the static hashtable variable at the same time. As the Hashtable documentation explains “Hashtable is thread safe for use by multiple reader threads and a single writing thread. It is thread safe for multi-thread use when only one of the threads perform write (update) operations, which allows for lock-free reads provided that the writers are serialized to the Hashtable. To support multiple writers all operations on the Hashtable must be done through the wrapper returned by the Synchronized method, provided that there are no threads reading the Hashtable object. Enumerating through a collection is intrinsically not a thread safe procedure. Even when a collection is synchronized, other threads can still modify the collection, which causes the enumerator to throw an exception. To guarantee thread safety during enumeration, you can either lock the collection during the entire enumeration or catch the exceptions resulting from changes made by other threads.”

Although the code adds and removes user-specific values only (the collection key uses User!UserID), Robert Bruckner from the SSRS dev team clarified

“Everytime you have code like this in a multi-threaded environment, there is potential for a race-condition:

if (!_hashtable.ContainsKey(“abc”)) _hashtable.Add(…)

In this case you have to take a lock to ensure ContainsKey and Add are run as an atomic piece of code.”

Keeping this in mind, my proposal for making the code thread-safe code follows (concurrent experts, please comment if you find anything substantially wrong with my attempt to make a hashtable access thread-safe).

Friend Shared _hashtable As New System.Collections.Hashtable()

Dim _sht as System.Collections.Hashtable = System.Collections.Hashtable.Synchronized(_hashtable)

Function GetKey(ByVal id as Long, ByVal column as String) as String

return Report.User!UserID & id & column

End Function

Function AddValue(ByVal id as Long, ByVal column as String, ByVal value As Object) As Object

If id = -1 or id = -2 or id=-3 Then

Dim key as String = GetKey(id, column)

If _sht.ContainsKey(key)=False Then

    _sht.Add(key, value)

End If

End If

Return value

End Function

Function GetValue(ByVal id as Long, ByVal column as String) As Object

Dim key as String = GetKey(id, column)

If _sht.ContainsKey(key) Then

    Return _sht(key)

End If

Return Nothing

End Function

Protected Overrides Sub OnInit()

SyncLock _hashtable.SyncRoot

Dim keys As String() = New String(_hashtable.Keys.Count – 1) {}

_hashtable.Keys.CopyTo(keys, 0)

For Each key as String In keys

     If key.StartsWith(Report.User!UserID) Then

         _hashtable.Remove(key)

    End if

Next key

End SyncLock

End Sub

Suppressing Auto-generation of MDX Parameter Datasets

There is an unfortunate issue with the MDX Query Designer in SSRS 2008 Report Designer and Report Builder 2.0 where changing the main dataset overwrites the parameter datasets. This is a rather annoying issue because often you need to make manual changes to the parameter datasets. However, when you make a change to the main dataset, the MDX Query Designer wipes out the manual changes and auto-generates the parameter databases from scratch. According to the feedback I got from Microsoft, the issue will be fixed in SQL Server 2008 R2.

Meanwhile, there is a simple workaround which requires manually changing the dataset definition in RDL. Basically, you need to add a SuppressAutoUpdate designer switch to the parameter dataset, as follows:

<Query>

<DataSourceName>Viking</DataSourceName>

<CommandText> …

<rd:SuppressAutoUpdate>true</rd:SuppressAutoUpdate>

<rd:Hidden>false</rd:Hidden>

</Query>

TechEd 2009 North America BIN304 Slides and Code Uploaded

I’ve uploaded the slides and code from my Reporting Services 2008 Tips, Tricks, Now and Beyond breakout presentation delivered on May 13th 2009 at TechEd 2009 North America.

TechEd 2009 BI Power Hour Demos

The TechEd 2009 BI Power Hour demos, which I blogged about before, are posted on the Microsoft BI Blog. Robert Bruckner has also posted the SSRS demo from the TechED USA 2008 Power Hour, which he renamed to Sea Battle.

Transmissions from TechEd USA 2009 (Day 4)

I got the final evaluation results from my session. Out of some 15 Business Intelligence sessions, mine was the fourth most attended session with 169 people attending. Based on 35 evaluations submitted, it was ranked as the third most useful session with an average satisfaction score of 3.46 on the scale from 1 to 4. For someone who does presentations occasionally, I’m personally happy with the results. Thanks for all who attended and liked the session!

I took it easy today. I attended the Scott Ruble’s Microsoft Office Excel 2007 Charting and Advanced Visualizations session in the morning. This was purely Excel-based session with no BI included. It demonstrated different ways to present information effectively in Excel, such as with conditional formatting, bar charts, sparklines, etc. Next, I was in the Learning Center until lunch.

In the afternoon, I decided to do some sightseeing and take a tour since I’ve never been to LA. I saw Marina Del Ray, Santa Monica, Venice Beach, Beverly Hills, Sunset Strip, Hollywood (and the famous sign of course), Mann’s Chinese Theatre, and Farmer’s Market. Coming from Atlanta and knowing the Atlanta traffic, I have to admit that the LA traffic is no better. In Atlanta, the traffic is bad during peak hours. In LA, it seems it’s bad all the time. Movies and premieres make the situation even worse. There was a huge movie premiere at 7 PM in the Chinese Theater with celebrities arriving with limos. This jammed the entire area. There were at least two movies being shot in different parts of the cities. But the rest of tour was fun. LA is one of the few cities in the world where almost every building has a famous story behind it.

This concludes my TechEd USA 2009 chronicles. Tomorrow, I’ll have time only for a breakfast and packing my luggage. I’m catching an early flight as it would take me four hours to fly to Atlanta. With three hours time difference, I’ll be hopefully in Atlanta by 9 PM and at home by midnight.

Transmissions from TechEd USA 2009 (Day 3)

Today was a long day. I started by attending the Richard Tkachuk’s A First Look at Large-Scale Data Warehousing in Microsoft SQL Server Code Name “Madison”. Those of you familiar with Analysis Services would probably recognize the presenter’s name since Richard came from the Analysis Services team and maintains the www.sqlservernanalysisservices.com website. He recently moved to the Madison team. Madison is a new product and it’s based on a product by DATAllegro which Microsoft acquired sometime ago. As the session name suggests, it’s all about large scale databases, such as those exceeding 1 terabyte of data. Now, this is enormous amount of data that not many companies will ever amass. I’ve been fortunate (or unfortunate) that I never had to deal with such data volumes. If you do, then you may want to take a look at Madison. It’s designed to maximize sequential querying of data by employing a shared-nothing architecture where each processor core is given dedicated resources, such as a table partition. A controller node orchestrates the query execution. For example, if a query spans several tables, the controller node parses the query to understand where the data is located. Then, it forwards the query to each computing node that handles the required resources. The computing nodes are clustered in a SQL Server 2008 fail-over cluster which runs on Windows Server 2008. The tool provides a management dashboard where the administrator can see the utilization of each computing node.

Next, I attended the Fifth Annual Power Hour session. As its name suggests, TechEd has been carrying out this session for the past five years. The session format was originally introduced by Bill Baker who’s not longer with Microsoft. If you ever attended one of these sessions, you know the format. Product managers from all BI teams (Ofice, SharePoint, PerformancePoint, and SQL Server) show bizarre demos and throw t-shirt and toys to everything that moves (OK, sits). The Office team showed an Excel Services demo where an Excel spreadsheet ranked popular comics characters. Not to be outdone, the PerformancePoint team showed a pixel-based image on Mona Lisa. Not sure what PerformancePoint capabilities this demonstrated since I don’t know PerformancePoint that well but it looked great.

The Reporting Services team showed a cool demo where the WinForms ReportViewer control would render a clickable map (the map control will debut in SQL Server 2008 R2) that re-assigns the number of Microsoft sales employees around the US states. For me, the coolest part of this demo was that there was no visible refresh when the map image is clicked although there was probably round tripping between the control and the server. Thierry D’Hers later on clued me in that there is some kind of buffering going on which I have to learn more about. This map control looks cool! Once I get my hands on it with some tweaking maybe I’ll be able to configure it as a heat map that is not geospatial.

Finally, Donald Farmer showed another Gemini demo which helped learn more about Gemini. I realized that 20 mil+ rows were compressed to 200 MB Excel file. However, the level of compression really depends on the data loaded in Excel. Specifically, it depends on the redundant values in each column. I learned that the in-memory model that is constructed in Excel is implemented as in-process DLL whose code was derived from the Analysis Services code base. The speed of the in-memory model is phenomenal! 20 mil rows sorted within a second on the Donald’s notebook (not even laptop, mind you). At this point Microsoft hasn’t decided yet how Gemini will be licensed and priced.

As usual, after lunch I decided to hang around in the BI learning center and help with questions. Then, it was a show time for my presentation! I don’t why but every TechEd I get one of these rooms that I feel intimidated just to look at them. How come Microsoft presenters who demo cooler stuff than mine, such as features in the next version, get smaller rooms and I get those monstrous rooms? It must be intentional; I have to ask the TechEd organizers. The room I got was next to the keynote hall and could easily accommodate 500-600 people, if not more. Two years ago, I actually had a record of 500+ people attending my session which was scheduled right after the keynote.

This year, the attendance was more modest. I don’t have the final count yet, but I think about 150+ folks attended my session so there was plenty of room to scale up. I think the presentation well very well. The preliminary evaluation reports confirm this. I demoed report authoring, management, and delivery tips sprinkled with real-life examples. We had some good time and I think everyone enjoyed the show.

It’s always good to know that your work is done. I look forward to enjoying the rest of TechED and LA.