Monthly Archives: March 2010

My Job

Posted on by
1

This is from a recent assignment I submitted. I thought it was relevant and decided to share. Enjoy!

~~

Business Intelligence Developer Position

My position is a Business Intelligence Developer, which means that I build analytical database systems, which allows business people to make better decisions based on the improved visibility into the measures of their business. I first started this career as an analyst building reports in Microsoft Excel. My curious mind and ambitious nature lead me to quicker and more efficient ways of delivering these analyses. Ten years later I have built many relationships, done consulting, presented at many events, and am currently working on my first book. With that context provided, here are my interpretations of how my position measures up against Hackman and Oldham’s Job Characteristics model.

Skill Variety

Business Intelligence is a highly specialized field. This field still contains however, much variety. From a pure technical perspective, being a Business Intelligence Developer can require Relational Database Management Systems (RDBMS) knowledge, Multi-Dimensional Database Design (OLAP), Extract Transform and Load practice (ETL), and presentation layer / report design. Doing the job of a Business Intelligence developer well also requires a solid understanding of the business. This is where Business Intelligence is unique in its attempt to bridge the gap between the business side and the technology side.

Task Identity

In small BI shops, the BI developer may be responsible for designing, building, and delivering the entire solution. In larger organizations, the one I work in, tasks are a bit more segregated however goals are still defined at a team level which encourages the team to work together and gain an understanding of the entire process.

Task Significance

The goal of Business Intelligence is to improve decision making in the company that eventually leads to increased revenue, reduced costs and increased profits. The correlation between BI efforts and these eventual outcomes however, is not always so clear.

Autonomy

Similar to most Business Technology Services (BTS) projects, Business Intelligence traditionally is managed from a program and project approach. Programs define the strategy and overall goal of the BI program while projects make up the individual deliverables all working towards the same goal. Depending on the project management method being used (eg. waterfall or agile) it can be difficult to have much autonomy over organizing your tasks. In my position we are lucky to use an Agile project management methodology that allows the core team, including myself, to have a great say in what tasks are required to do my job well.

Feedback

The most common way feedback is received by the BI team is when something goes wrong with the nightly ETL processing that updates the data in the presentation layer. This feedback of course is always negative however, positive feedback is also received when the data is on time and accurate. There is some gray area here where no feedback is given and the data being presented is assumed to be correct when in fact it is not. This is why we have placed a laser like focus on the quality of our data being presented.

Where to improve

Since I have had a lot of input into the processes and procedures, we have in place currently it is difficult for me to say where we can improve much.  I will note however that the challenges we face with relating task significance and feedback are common among all Business Intelligence programs and can always use some more attention.

Short response to Sanju Bansal about BI and Performance

Posted on by
Reply

“The truth is, there’s a good way to model data in a database… but the way business users want to see the data is far different than that.” (Bansal, 2010)

I disagree. This is the exact reason why Data Warehouses need to be modeled using dimensional modeling techniques that are designed around how the business wants to see the data and are setup for fast query performance. With the data already transformed into the format users are expecting to see it with natural hierarchies and relationships built in, queries are simple and much faster.

There is also the potential to pre-calculate all or most of the aggregations into a multi-dimensional database structure. I have done this countless times with Microsoft’s SQL Server Analysis Services and routinely am able to query fact tables with 20M+ rows in milliseconds.

In my opinion BI interfaces trying to make queries faster is a valiant effort however, without a solid dimensional data structure modeled after the way users want to see the data there is only so much they will be able to accomplish.

Centralized Business Intelligence

Posted on by
Reply

My approach as a BI professional has always been to organize the data into conformed structures, present that to end users for consumption in their ad-hoc analysis, and only build formal reports and dashboards as necessary. I think the conformed structures provide great consistency for BI across the enterprise no matter who is building it and allow users to get answers to questions faster.

Of course, there is no way IT can keep up with the onslaught of report requests so it makes sense to provide users with the tools to do these themselves. Typically, I would offer up a connection to a cube through Excel and now with the addition of PowerPivot I believe this type of process is going to become even more prevalent.

Some I have talked with have expressed concern of decentralization of BI by tools like PowerPivot however; I see these tools as giving users a sort of proving ground for their BI applications. If an information worker successfully creates a BI application in PowerPivot that becomes widely used across the enterprise IT does not have to guess that formalizing it into a more formal BI application will have value.

Other methods exist that I think are worth for determining the value of a BI application (eg. BI Pathway Method) however I believe this is for more formal BI versus the ad-hoc type of analysis that users clamor for on a day-to-day basis.