How Do I Approach Requirements for a Business Intelligence Project?

Reader Question:

The organization I work for is expanding its Business Intelligence capabilities, introducing a new Enterprise Data Warehouse for management reporting. This is based on the concept of OLAP Data Cubes – permitting greater flexibility for user reporting.

My question is – what’s the best approach for eliciting and documenting requirements for BI/Data Cubes? There’s less emphasis on the structure of reports due to the flexibility of the cube, and more emphasis on the definition of the data and the ways in which it can be manipulated. To what degree should I be focusing on the detail of the data? Should I learn about ‘star schemas’ and how to structure the data in ‘dimensions’? Or should I focus more on the business need – on what metrics are required, what the data/reports are to be used for, by who, how frequently etc.

I know that there is never a cookie cutter approach to eliciting and documenting requirements of any kind, but I’d be eager to know of any specific areas I should be covering to ensure that the specification I handover is comprehensive.

Finally, if there are any recommended books on Business Intelligence & BI requirements, I’d be grateful for any suggestions.

Anup’s Answer:

You are asking all the right questions. This reminded me of my first BI project.

My First Business Intelligence Project

The request from the client consisted of an excel sheet with some existing reports that needed some changes. They also wanted some new reports. Since these reports were manually created, they were always concerned about the accuracy of the data. Also the reports were delivered late due to the time taken to generate the reports.

(Click here to learn more about how to collect reporting requirements)

We jumped on it and built the cubes (explained later) and delivered the reports. Everyone was very happy. However, after 3 months; the management wanted new reports and enhancements to existing reports. Though this required substantial changes to the solution, we were able to meet the requirements. However, by the time we delivered these, the management wanted more reports and changes.

This did get us thinking and we got together to figure out a better way of handling the project. By this time, we had understood that requirements for BI solutions had to be handled in a different way. Most of us are used to doing requirements for Transactional systems; i.e. applications that help in day to day operations of the business which are very different from Business Intelligence applications. BI applications leverage the data from the Transactional systems to help management make informed decisions and take calculated risks.

This was a big turning point and led to defining an overall approach for handling BI projects in general. We broke the team into 3 groups which were responsible for OLAP Database, Cubes and Reporting Dashboard. A fourth group was added later to take care of the Data Mining area.

Business Intelligence

Business Intelligence solutions help the management in making informing decisions and taking calculated risks.

OLAP Team Created the Data Archive

The focus of this group was to create the OLAP (Online Analytical Processing) database from the current OLTP (Online Transaction Processing) database. The key difference between the two is that while OLTP databases are designed to support the day to day functions (transactions); OLAP databases are designed to support analysis of the data. Data is pushed to OLAP databases from OLTP database through the process known as ETL (Extraction, Transformation and Loading). The key responsibility of the team was to define an overall archiving strategy to move data from OLTP DB to OLAP DB so that this data can be used for analysis.

One of the key activities of this team was to identify data elements that can be used for analysis and discard other unnecessary details. For example: We just retained the zip codes of shipping and billing addresses instead of the whole address. Another interesting addition was to have flags to indicate whether these addresses were same or different.

Other activities included performance tuning of this database. Performance tuning of OLAP database is different from OLTP. OLAP databases are generally tuned for bulk inserts and retrieval; while the OLTP databases are tuned for smaller but frequent inserts and updates.

From Business Analysis function perspective, we had data analysts in this team who worked with the management and development team to create the OLAP database. They defined data extraction, standardization and transformation rules to facilitate this process.

Cubes Team Focused on Analyzing Data to Support Reports

In general databases are two dimensional, i.e. rows and columns. Cubes are multi-dimensional databases and store the data with more than two dimensions. Cubes facilitate faster processing of large amount of data. This team focused on developing the cubes that would be used for reporting.

The team started with building dimensions and fact tables. Their key focus was to identify hierarchy for each dimension and how different data elements can be used. The team consumed the OLAP database, and providing feedback to the OLAP team on the changes required. The idea of adding flags for different addresses was one of the inputs from this team. The data analysts on this team defined the dimensions, fact tables and KPIs (Key Performance Indicators) as per the requirements of the management keeping the big picture in mind.

Reporting Dashboard Team Defined Reports and Security

This team worked with the executive team to understand their needs and define requirements. In a way this part of the process was closest to the requirements gathering processes followed for any project. The Business Analysis team here consisted of functional analysts. They defined the reports in detail including report formats and filters. They also detailed the requirements around the delivery and distribution of the reports including security. The team also identified the need for Statistical analysis. This lead to the creation of Data Mining group.

Data Mining Team Leveraged the Data for Business Insights

Data Mining team was added later to the project. This consisted of members focused on applying statistical tools to identify trends and data patterns to provide valuable business insights.

Together, we Defined Supporting Processes to Ensure Success

In addition to identifying the groups, we laid down some other processes. These processes were very instrumental in success of our project.

  • Each group worked independently. It was very critical that the OLAP and the Cubes team worked independently and were focused on the bigger picture. Initially they were guiding by the reporting needs which skewed the overall framework.
  • Security was a key concern. We were dealing with sensitive and confidential business information that had major repercussions if it landed in wrong hands. The need to understand reports and security together led to the development of the reporting dashboard team which streamlined the delivery of the reports and took care of the security aspects as well.
  • One of the key changes was to reset the expectations of the management and other teams. Before, when we received a change request, the teams scrambled to deliver it through the cubes which resulted in a ripple effect of changes across the systems. Now, the reports were delivered from the OLAP database or even OLTP database if they could not be supported from the cubes. Of course, the reports took longer to generate but ensured the overall schema was not compromised. The underlying queries were expected to change as soon as cubes and OLAP databases supported it. To address the performance issue, a feature was developed to deliver the reports through email to the respective managers.

This post lays out about a 20,000 feet view of the entire process of building a BI solution. Depending on your industry and domain, these processes may vary.

>>Learn More About What Questions to Ask

Interested in receiving a comprehensive set of questions you can ask in almost any project context? Want to feel more confident asking questions in a new domain? The Requirements Discovery Checklist Pack includes over 700 questions, categorized and cross-referenced so you can prepare for your next elicitation session with a sense of ease and confidence.

Click here to learn more about the Requirements Discovery Checklist Pack

Free Training - Quick Start to Success

(Stop the frustration and earn the respect
you deserve as a business analyst.)

Click here to learn more

By signing up, you agree to our Privacy Policy.

Comments

  1. Hi Anup,

    I would like to seek your help for my current project. This is my first project as a Business Analyst and I am really very confused with requirements elicitation –
    Where to start?
    What questions to ask?
    What kind of technical questions I need to dig into?
    Project – it’s a new report generation project from an application for compliance purpose. The application runs on AS400, after getting involved with the business users I can envision the details and elements users are expecting in the new report to serve their purpose of audit.
    Now I am struggling how to initiate my requirements sessions with SME of that application, as I have no knowledge of back end or technical side of this application. I have been told to skip the BRD and start the FRD, so starting your first project straight with functional specs is giving me hard time.
    I just can’t figure out what questions I should ask to SME’s

  2. What I consider one of the most critical (if not the most critical) steps to a successful BI project is to severely limit the scope of any project. Work with one department or area of the organization, ideally one with some level of clarity of existing reports (Finance can be a good place to start). Work to develop a “Finance Mart” (a data mart of their info). Build the infrastructure and security you need for them, build 1-2 reports and deploy them.

    Learn.

    Expand and enhance!

    You should certainly give the overall Finance Mart needs a decent look to ensure you do not waste any major time… but remember, in most cases YAGNI (you ain’t gonna need it). So time box this effort to limit wasted time and effort.

    I have watched warehouse projects that went on for years as the data was analyzed and mapped… nothing was ever delivered – until the scope was reduced to something that could be achieved in a few weeks.

    • Anup Mahansaria says

      Jake, thanks for sharing your thoughts. Can’t agree with you more.

      Business Intelligence implementations are not projects but an ongoing process. Iterative/ agile methodology is an excellent way of handling BI implementations. It helps deliver small pieces quickly and get feedback from the business on a regular basis which is so important. As the business starts leveraging the benefits of the solution they get more involved and committed to the process.

      Most of the time, people struggle to justify the ROI of BI projects. It becomes more difficult when it is treated as one big project. Breaking it down also helps in getting sponsorship more easily.

      Of course, while the implementation needs to be more agile, the thought process should never miss the bigger picture. A balance of the two would help in making these projects extremely successful and rewarding for everyone involved in the process.

  3. This is a great read Anup. I was involved on a project where my team was charged with implementing a BI Solution many years ago. The company made a big push to empower executives to be able to pull high level adhoc reports on the fly before heading out to meetings to help them analyze their business and make better decisions.

    The role that I played was more like that of your Reporting Dashboard Team. I was charged with gathering requirements for reports and turning those requirements into a set of canned reports that were to be run daily, weekly, monthly and yearly. Of course there were times when, due to my knowledge of the system and data, I would get adhoc request from various people throughout the department that I supported. I was also responsible for the security, distribution, and maintenance of this system as well training the end users.

    My biggest challenge was learning the data and the system up front. This was my first job right out of college and even though I knew how to use a computer I had no formal training in programming languages or processes. Luckily I was good with people and had the determination to learn as my company invested in my training of the technical aspects of the job. Fast forward to today… I’m working for another company that is implementing a BI tool and my role is being defined as I go. Though I am technically in a better place than where I started years ago, the anxiety still exist as I’m sitting in the mist of actual developers and programmers that probably know tons more than I do about how systems work. My question to you is what reading, training or course of action would you recommend to someone in my situation… (8 years of BA experience and starting a new job after being out of the field for 3 years)? I’m still a people person and great a great communicator but somehow I guess my confidence is a little shaky right now.

    • Anup Mahansaria says

      Joe, glad that you liked the post. Thanks for the appreciation.
      I understand the anxiety one feels in your situation. Rest assured, you will do great! Your past experience and skills will sail you through. Here are some of my thoughts.

      Read as much about the business domain as possible. This will help you get more comfortable with the terminology and in understanding the application.

      Leverage your people skills to create good rapport with colleagues, who will help you whenever you are stuck.

      Don’t feel shy to ask questions. Others appreciate it more than we think. The message it sends across is that you are making effort to understand the system.

      Spend some time daily to understand the database schema for the first few months. This will make you very comfortable with the data and help in finding required information quickly.

      Participate actively in team meetings. Pay attention and feel free to make recommendations when appropriate. The biggest contribution you can make to an existing team is by bringing fresh ideas to the table. As a team keeps working on a project, they have more project knowledge. However, it does affect their ability to think out of the box. You can bring in new perspective to the team which will be appreciated by everyone.

      Hoping, some of these tips turn out helpful. Good Luck and keep us posted.

      Thanks
      Anup

Comment

*

Before you go, would you like to receive our absolutely FREE workshop?

(No formal experience required.)

21689
21690

Quick Start to Success
as a Business Analyst

By signing up, you agree to our Privacy Policy.