-
MS BI Guy Does Hadoop (Part 5 – The Big Picture)
July 1, 2012 / No Comments »
Let's recap my thoughts on Hadoop. I've found the case studies presented in the chapter 16 of the "Hadoop: The Definitive Guide" book very useful to understand how organizations are currently using Hadoop. In general, the following deployment scenario emerges as a common pattern: An organization accumulates Big Data from various sources, such as web logs, sensors, website traffic, and so on. The organization decides to store the raw data in the Hadoop fault-tolerant file system (HDFS). As far as cost, RainStor published an interesting study about the cost of running Hadoop. It estimated that you need an investment of $375 K to store 300 TB which translates to about $1,250 per terabyte before compression. I reached about the same conclusion from the price of single PowerEdge C2100 database server, which Dell recommends for Hadoop deployments. Note I favor the term "raw data" as opposed to unstructured data. In my...
-
MS Guy Does Hadoop (Part 4 – Analyzing Data)
June 30, 2012 / No Comments »
In my previous blog, I talked about Hive. Hive provides a SQL-like layer on top of Hadoop so you don't have write tons of MapReduce code to query Hadoop and to aggregate and join data. To facilitate working with Hive, Microsoft introduced a Hive ODBC driver (as of this writing, the driver is only available to Hadoop on Azure CTP subscribers). You can use this driver to connect to Hive running on Microsoft Azure or your local Hadoop server. Danny Lee has provided detailed instructions of how to do the former. I'll show you how to use it to connect to your local Hive server. Start the Hive Server If you use the Cloudera VM, the Hive server is not running by default. This service allows external clients to connect to Hive. To start it: Configure your Cloudera VM to obtain an IP address on your network. To do so...
-
Report Timeout Quest Reloaded
June 25, 2012 / No Comments »
A long time ago, I wrote a blog about possible reasons for Reporting Services reports to time out. This issue raised its head again; this time in SharePoint 2010 environment where long running reports would time out after about 2 minutes. To resolve: On each Web Front End (WFE) server, edit the web.config file of the SharePoint web application. For the default web application, the web.config file is located in C:\inetpub\wwwroot\wss\VirtualDirectories\80. Find the httpRuntime element and change the executionTimeout setting. The default value is 110 seconds. <httpRuntime maxRequestLength="51200" executionTimeout="1000"/> Save the web.config file. No need for iisreset since the ASP.NET process will apply the settings on change. As a side note, you shouldn't have such report hogs but sometimes you can't avoid them. In this case, the end users requested a report that includes pretty much all measures in the cube sliced by the dimension with the highest cardinality so...
-
MS Guy Does Hadoop (Part 3 – Hive)
June 24, 2012 / No Comments »
Writing MapReduce Java jobs might be OK for simple analytical needs or distributing processing jobs but it might be challenging for more involved scenarios, such as joining two datasets. This is where Hive comes in. Hive was originally developed by the Facebook data warehousing team after they concluded that "… developers ended up spending hours (if not days) to write programs for even simple analyses". Instead, Hive offers a SQL–like language that is capable of auto-generating the MapReduce code. The Hive Shell Hive introduces the notion of a "table" on top of data. It has its own shell which can be invoked by typing "hive" in the command window. The following command shows the Hive tables. I have defined two tables: records and records_ex. [cloudera@localhost book]$ hive hive> show tables; OK records records_ex Time taken: 4.602 seconds hive> Creating a Managed Table Suppose you have a file with the...
-
MS BI Guy Does Hadoop (Part 2 – Taking Hadoop for a Spin)
June 10, 2012 / No Comments »
In part 1 of my Hadoop adventures, I walked you through the steps of setting the Cloudera virtual machine, which comes with CentOS and Hadoop preinstalled. Now, I'll go through the steps to run a small Hadoop program for analyzing weather data. The program and the code samples can be downloaded from the source code that accompanies the book Hadoop: The Definitive Guide (3rd Edition) by Tom White. Again, the point of this exercise is to benefit Windows users who aren't familiar with Unix but are willing to evaluate Hadoop in Unix environment. Downloading the Source Code Let's start by downloading the book source and the sample dataset: Start the Cloudera VM, log in, and open the File Manager and create a folder downloads as a subfolder of the cloudera folder (this is your home folder because you log in to CentOS as user cloudera). Then, create a folder book...
-
MS BI Guy Does Hadoop (Part 1 – Getting Started)
June 4, 2012 / No Comments »
With Big Data and Hadoop getting a lot of attention nowadays, I've decided it's time to take a look so I am starting a log of my Hadoop adventures. I hope it'll benefit Windows users especially BI pros. If not, at least I'll keep a track of my experience, so I can recreate it if needed. Before I start, for an excellent introduction to Hadoop from a Microsoft perspective, watch the talk on Big Data - What is the Big Deal? by David Dewitt. Previously, I've experimented and I got my feed wet with Apache Hadoop-based Services for Windows Azure, which is the Microsoft implementation for Hadoop in the cloud, but I was thirsty for more and wanted to dig deeper. Microsoft is currently working on CTP of Hadoop-based Services For Windows, which will provide a supported environment for installing and running Hadoop on Windows. While waiting, O'Reilly was king enough to...
-
Is Excel Calculation Services Needed on a PowerPivot for SharePoint Application Server?
June 2, 2012 / No Comments »
Scenario: A SharePoint 2010 scale-out farm with two application servers: APP1 – Runs Excel Calculation Services (ECS) APP2 – Runs PowerPivot for SharePoint (R2 or 2010) Do you need to start ECS on APP2 in order for PowerPivot for SharePoint to work? Answer: You don't but if you don't have the updated Analysis Services OLE DB Provider on APP01, you'll get the dreaded "Failed to create new connection…" error when you browse the PowerPivot workbook. SharePoint ships with the SQL 2008 ADO MD, MSOLAP, etc. stack; not the R2 or later components. Therefore, you must upgrade the OLE DB provider (R2 or 2010 depending on the PowerPivot version) as explained in the How to: Install the Analysis Services OLE DB Provider on an Excel Services Computer topic in Books Online. Thanks to Dave Wickert for clarifying this.
-
Organizational BI vs. Personal BI
May 27, 2012 / No Comments »
This is a blog post that I've been planning for a while. With the ever-increasing power of laptops and desktop computers and declining hardware prices, personal BI is on the rise. Typically, a new technology usually has a self-propelled upward spiral to a point – vendors are talking about it to clients, executives are talking about it on golf courses, consultancies are talking about it, and are rushing to fill in the void. There is a lot of money to be made with a lot of misinformation and sometimes outright lies. I'll be quick to point out that personal BI alone is not going to fix your BI and data challenges. However, it can complement organizational BI well and open possibilities when organizational BI alone is not enough. You might find the following information interesting when you're contemplating which way to go. Organizational BI Organizational BI is a set of...
-
Why an Analytical Layer?
May 26, 2012 / No Comments »
I had a presentation on the BI Semantic Layer and Tabular modeling for the Atlanta BI Group on Monday. Midway during the presentation, a DBA asked why we need an analytical layer on top of data. I'm sure that those of you who are familiar with traditional reporting and haven't discovered yet Analysis Services might have the same question so let's clarify. Semantic layer In general, semantics relates to discovering the meaning of the message behind the words. In the context of data and BI, semantics represents the user's perspective of data: how the end user views the data to derive knowledge from it. As a modeler, your job is to translate the machine-friendly database structures and terminology into a user-friendly semantic layer that describes the business problems to be solved. To address this need, you create a semantic layer. In the world of Microsoft BI, this is the Business...
-
Presenting at Atlanta BI Group
May 19, 2012 / No Comments »
I'll be presenting "The Analysis Services 2012 Tabular Model" for the Atlanta BI Group on Monday, May 21st. And, Darren Herbold will share some great insights harvested from a real-life project of how to use SSIS to integrate with Salesforce.com. I hope you can make the event, which will be sponsored by Prologika. "SQL Server 2012 introduces the BI Semantic Model that gives BI pros two paths for implementing analytical layers: Multidimensional and Tabular. The Tabular model builds upon the xVelocity engine that was introduced in PowerPivot. Although not as feature-rich as Multidimensional, Tabular promotes rapid professional development and great out-of-box performance. This session introduces you to Tabular development and shares lessons learned. I'll also discuss how Tabular and Multidimensional compare."
Training
We offer onsite and online Business Intelligence classes! Contact us about in-person training for groups of five or more students.For more information or to register click here!
Books
Learn Power BI at your own pace with our latest book Applied Microsoft Power BI. Targeting information workers, data analysts, pros, and developers, It is designed as an easy-to-follow guide for learning how to implement BI solutions spanning the entire personal-team-organizational BI spectrum.
