SQL Server 2008 November CTP Is Out

/0 Comments/in /by

The November CTP (CTP5) of SQL Server 2008 is out. Here are the most significant BI-related highlights that caught my attention.

Reporting Services

The standalone Report Designer has undergone a major facelift. It now sponsors a shiny Office 2007 ribbon interface. It will surely charm end users who are planning to use the standalone Report Designer for ad hoc reporting. The bad news is that there is still much integration work ahead. For example, the Analysis Services and custom data processing extensions are not integrated yet. Further, only the Dundas chart component has made the CTP5 cut. Nevertheless, I am very excited about the standalone Report Designer and its long-term potential to unite professional and end-user reporting needs.

111907_1814_SQLServer201

Analysis Services

The block computation enhancements debut in CTP5. Based on my experiments, they result in 50-60 percent and more improvement in query response times. For example, a highly inefficient Report Builder query that filters on a measure would take hours with SSSAS 2005. With CTP5 of SSAS 2008, the same query finishes under a minute! Put in highly technical terms, the nastier the query is, the faster it will with SSAS 2008. There are also optimizations in MOLAP write-back that remove the need to query ROLAP partitions although I haven’t given them a try.

Integration Services

CTP5 brings lookup performance enhancements. Are the SSIS guys lazy or what? J

Relational Engine

One BI-related enhancement with a great potential is Change Data Capture (CDC). When enabled, this features tracks changes to data, including inserts, deletes, and updates. This means that ETL process can quickly detect what changes have been made to a given table by just querying the CDC internal tables. For example, you query CDC to return only the new records that have been added to a table after a given data. As you can imagine, this baby will be a welcome enhancement with large data warehouses and can help you optimize the ETL processes.

Parameterized Parameter Prompts

/0 Comments/in /by

One new SSRS 2008 feature that international users will undoubtedly appreciate is parameterized parameter prompts.

101807_2114_Parameteriz1

Previously, the parameter prompts were static. In SSRS 2008, you will notice the familiar function symbol (fx) next to the parameter prompt which means that the prompts can be expression-based. This allows the report author to change the parameter prompt caption at runtime. For example, you can retrieve the localized version of the parameter prompt from an external resource assembly based on the user language.

UPDATE 03/27/2009 This feature didn’t make it to the release version of SSRS 2008. SSRS 2008 doesn’t support expression-based parameter prompts. The feature has been slated to a future release.

NEB Revenge in Katmai

/0 Comments/in /by

An advanced SSAS performance-related feature is Non Empty Behavior (NEB) which has been discussed in many places including this Mosha’s blog. The issue with NEB in SSSAS 2005 is that you can very easily get it wrong. The server will trust you (this could be an optimistic assumption) and accept a NEB even if it’s not correct. Example:

CREATE MEMBER CURRENTCUBE.[Measures].[Profit Gap To Prior Period]

AS [Measures].[Profit]-([Date].[Calendar].PrevMember, [Measures].[Profit]),

FORMAT_STRING = “#,#;(#,#)”,

NON_EMPTY_BEHAVIOR = { [Measures].[Profit]},

VISIBLE = 1;

Here, the intention is to default the expression for the Profit Gap To PriorPeriod calculated member to NULL if Profit is NULL. But the expression may be not be NULL even if Profit is NULL. This can yield incorrect results in 2005. In 2008, optimizations were made that don’t use the NEB expression. The net effect is that some calculated members that use wrong NEBs can produce different, albeit correct, results in 2008. If I am not wrong, in this webcast, Donald Farmer mentioned in this webcast that NEBs will be de-emphasized in SQL Server 2008. Meanwhile, if you are to use NEBs be sure that they are right to save you many hours of testing and head-scratching.

Protect UDM with Dimension Data Security Reloaded

/0 Comments/in /by

SQL Server Magazine published the second part of my Protect UDM with Dimension Data Security article in the October issue of the magazine. In this article, I explain how to use MDX expressions to implement dynamic dimension security. I discuss two practical approaches for meeting advanced requirements for securing UDM data. First, I present a solution in which the security policies are stored in a factless fact table. Next, I explain how to leverage SSAS stored procedures to integrate UDM with an external security service.

The article is available for public access. Enjoy!

Scale-Out Querying with Analysis Services Using SAN Snapshots

/0 Comments/in /by

If your organization can afford SAN, here is another best practices article by Carl Rabeler from the SSAS team. The Scale-Out Querying with Analysis Services Using SAN Snapshots article “describes the use of virtual copy SAN snapshots in a load-balanced scalable querying environment for Microsoft SQL Server 2005 Analysis Services to increase the number of concurrent queries to Analysis Services servers with a rapid response time per query.”

MCTS Self-Paced Training Kit (Exam 70-445): Microsoft SQL Server 2005 Business Intelligence—Implementation and Maintenance

/2 Comments/in /by

Those of you preparing for Microsoft exam 70-445 Microsoft SQL Server 2005 Business Intelligence – Implementation and Maintenance may be interested to know that Microsoft Press has just published a training kit with the same title which I co-authored with Erik Veerman, Dejan Sarka, and Javier Loria from Solid Quality Learning. I wrote four of the SSAS chapters. I also took the actual exam prior to writing the content so I hope the book will help you to prepare and pass 70-445.

SSRS 2008 Variables

/0 Comments/in /by

As I mentioned in my Under the Hood post, the SSRS 2008 engine has been redesigned to perform on-demand report processing for better scalability. As a result, textbox values are calculated on-demand every time the containing page is rendered. Irrespective of the on-demand behavior, values of dataset-bound textboxes won’t change because the underlying dataset is cached in the report database. Therefore, the end result for dataset-bound textboxes is the same as with SSRS 2005.

What about expression-based values? For example, you may have a textbox that references some custom method. Unlike SSRS 2005, the new engine will execute that method each time the page is rendered. This may be or may not be what you want. If you need up-to-date information, the new behavior will be welcome. However, there will be cases, when you may need the old behavior where the custom code is executed once. For example, imagine that the custom code takes very long time to execute and you want to cache the result. Or, you don’t want the report results to change when you navigate pages, such as when you need to deal with currency conversion rates. Enter variables – a new feature in SSRS 2008.

In SSRS 2008, variables provide a mechanism to guarantee an at most once evaluation semantics. Just like programming variables, SSRS variables store values. Variables can be scoped at report or group levels (note that the CTP 4 build of the Report Designer Preview exposes only report variables). A report-level variable is evaluated once for the lifetime of the report. A group-level variable is evaluated each time the group changes.

Report variables

They are accessible from within the report body and page sections.

<Report …>

…

<Variables>

<Variable Name=”VExchangeRate”>

<Value>

=MyCustomAssembly.GetExchangeRate()

</Value>

</Variable>

</Variables>

Group variables

Their value is only defined within the current group instance and the inner group hierarchy.

…

<TablixMember>

<Group Name=”table1_Group1″>

<GroupExpressions>

<GroupExpression>=Fields!SupplierID.Value</GroupExpression>

</GroupExpressions>

<Variables>

<Variable Name=”VProductCountPerSupplier”>

<Value>

=Count(Fields!ProductID.Value)

</Value>

</Variable>

</Variables>

</Group>

Here is a report that demonstrates a group variable.

090907_1419_SSRS2008Var1

The Direct Code column has an expression-based textbox that references the custom method directly =Code.Util.GetValue(). The GetValue method simply generates a random value. When I navigate from and back to a page, the Direct Code column values change as a result of the on-demand behavior. In contrast, the Var column uses the following expression: =Variables!Var2.Value. The Var2 variable is declared as a group-level variable inside the tablix group:

<TablixMember>

<Group Name=”Tablix1_ProductCategory”>

<GroupExpressions>

<GroupExpression>=Fields!ProductCategory.Value</GroupExpression>

</GroupExpressions>

<Variables>

<Variable Name=”Var2″>

<Value>

=Code.Util.GetValue()

</Value>

</Variable>

</Variables>

</Group>

Now, the result is completely different. Because a variable stores the result, the column values don’t change within the same group. More importantly, the values are constant between page refreshes. At this point, you are probably thinking about other useful scenarios for variables. However, you cannot use variables to maintain state.  They are evaluated at most once and are read-only.

Under the Hood

/0 Comments/in /by

One of the major themes in SSRS 2008 is scalability. A significant effort will be made to ensure that SSRS scales well with large reports. Why is this needed? Recall that in SSRS 2005 (and 2000), report processing is instance-based. This means that the report engine would process the entire report as a snapshot, including textbox values, styles, etc. As a result, SSRS 2005 reports are memory-bound. In general, the memory consumed by SSRS 2005 is proportional to the size of the report. The bigger the report, the more memory it would take. Therefore, large reports won’t scale well.

In contrast, the SSRS 2008 processing engine processes the reports on-demand. It pre-computes and saves only certain invariants, such as grouping, sorting, filtering, and aggregates. However, textbox values are calculated on-demand every time the containing page is rendered. To test this, drop a textbox on the report and set its value to =Now(). In SSRS 2005, when you page back and forth, the textbox value will not change. In contrast, in SSRS 2008, the textbox value would change to the most current date each time the containing page is rendered.

As a result of the on-demand processing model, the in-memory presentation of the report is much smaller. If you want to learn more about the engine improvements in Katmai, I highly recommend the Chris Baldwin’s excellent Enterprise Scale Report Engine webcast. At this point, you may wonder how much memory the new engine could save. I tested print preview for an existing 1,270 report with the old and new engine and here are the results.

 SQL Server 2005

Time (s)

Memory SQL (MB)

Memory RS (MB)
TFP (time to first page)

262

130

240
TLP (time to last page)

610

207

312
SQL Server 2008
TFP

218

56

95
TLP

430

44

95
Improvement (TLP)

30%

79%

70%

As you can see, in this case the new engine takes 70% less memory which is a pretty remarkable improvement. This is not to encourage you to author such monster reports of course. However, if you have to, for audit, regulatory, or whatever reasons, the new engine should be able to help.

Identifying and Resolving MDX Query Performance Bottlenecks Whitepaper

/1 Comment/in /by

Performance is the most common incentive for using Analysis Services. Yet, despite the advances in the SSAS management tools, optimizing the MDX query performance is still considered as “black magic” by many. Graphical optimization tools similar to the SQL query optimizer and query showplan would definitely help but they are not expected anytime soon. Meanwhile, Microsoft has just published a 60-page whitepaper Identifying and Resolving MDX Query Performance Bottlenecks in SQL Server 2005 Analysis Services by Carl Rabeler and Eric Jacobsen. At first sight this resource looks like a must-read for anyone looking for tips to make that query execute a few seconds faster. From the article summary:

“To improve the performance of an individual MDX query that is performing poorly, you must first identify the source(s) of the performance bottlenecks in the execution of the query. This requires that you understand how to use Windows and SQL Server 2005 Analysis Services troubleshooting tools. This best practices article provides information about available troubleshooting tools and demonstrates how to use the most common of these tools to identify and resolve MDX query performance bottlenecks.”

Microsoft Launches Tafiti

/0 Comments/in /by

Microsoft launches a preview of a cool search portal dubbed Tafiti to bring the web search to a new level and give Google a run for its money. From FAQ “Tafiti, which means ‘do research’ in Swahili, is an experimental search front-end from Microsoft, designed to help people use the Web for research projects that span multiple search queries and sessions by helping visualize, store, and share research results. Tafiti uses both Microsoft Silverlight and Live Search to explore the intersection of richer experiences on the Web and the increasing specialization of search.”

The Silverlight-based UI looks pretty sleek.