Where is the BW Technical Debt?

A reporting solution will involve a minimum of three (3) main areas that data will flow through:

  • Extractor;
  • Data Warehouse;
  • Report.

A great degree of flexibility occurs due to SAP BW Data Modeling concepts involving philosophies, strategies and technique to design and implement an Enterprise Data Warehouse, successfully. In a lot of cases there is no right or wrong, just a choice of which best practices will be implemented based upon where the effort will be focused and what that will cost.

Check

The best practices will sometimes conflict with each other and you will have to choose which philosophy or strategy is more important. This choice should always be guided by the understanding that the reports ultimately exist to serve the business, like a helpful assistant. Knowing this, a DataModeler will start with Consumption, as it defines the purpose and overarching guidelines to be met by the Extraction and Transformation.

Following this approach, along with other core philosophies of Data Warehousing, you will inevitably find yourself faced with the decision of ‘Where will the Technical Debt live?’. This is fundamentally a choice of where will the complexity of the business rules be implemented:

  • Additional DataStores and complex transformations that simplify query design; or
  • Complicated query definitions based upon simple cubes of data.

The art of balancing these design choices is one of the capabilities of a DataModeler and Solution Architect. There are several main factors that impact these choices:

  • Time line of when the result is expected;
  • Budget to do the work;
  • Availability of other people;
  • Required skills for maintenance;
  • Existing environment.

The Power User community who are generally Subject Matter Experts (SME) will have a good base of business knowledge and should have the confidence to build their queries. This enables them to be self-sufficient in bridging the gap between the business rules and the technology used to implement it.

Most environments will have a number of Power Users who can also train others. This is usually the main reason why a lot of environments will begin their Business Intelligence journey by investing the technical debt into the reporting queries; the Consumption layer.

This is a great way to get data into the hands of the business, sooner rather than later. It is also a way to stimulate business interest because they can see and interact with the prototype. This in turn helps to resolve a common project failing where the business users are not 100% on-board with the, often frustrating, aspect of gathering business requirements.

“Does your Data Modeling consider
‘Build Time’ vs ‘Run Time’?”

The business requirements are a critical first step in most projects as they define the metrics for successful delivery. Did your last blueprint include a maximum query run-time metric? How can you ever hope to meet it when the complexity of the business rules are not fully defined? The DataModeler or Solution Architect needs this base information to be able to choose an appropriate strategy and technique. This will usually start with the complex query approach.

As more information becomes available the strategies and techniques will shift. The technical debt will migrate from the Consumption layer to the Transformation layer; simplifying the queries, adding DataStores and cubes with more business rules implemented in the transformations.

At some point along this path of choices, it will be time to lock in the proposed solution to be built for this phase of the project. There will be a mixture of locations where the technical debt has been implemented. This in turn will directly identify the required skills to maintain the solution post go-live.

It is important that this is communicated to all parties involved, preferably before the build phase of the project begins. Unfortunately a lot of projects are not this clean. There will be an overlap of competing stages:

  • New meetings to gather or confirm business requirements;
  • Solution Discussions to choose where the technical debt will live given all the project constraints;
  • Any existing environment problems; including network, hardware, software and other business initiatives;
  • Business process and data investigations;
  • Gaps in the existing Data Warehouse. Missing documentation, prior half-built solutions, security, performance, etc.

With all of the above in mind, have you allowed time to consider the technical debt throughout the project? It can be the difference between success and failure. Have you aligned the reporting end user expectation with the known query performance and on-going ability to implement new business rules in an appropriate time frame? Perhaps you need to adjust the strategy or technique now, not later.