The ‘WEEKNIGHT in Formula Builder’ article highlighted a common technical gap in executing nightly process chains. The failure to deliver the original solution with absolute alignment to business requirements causes a Technical Debt to be introduced into the Enterprise Data Warehouse (EDW).
The BW Development Team who builds the solution is able to save a little time and money (from their deadline and Capex budget, respectively) and keeps their project manager happy [Huummm, maybe only slightly happier] by using the standard WEEKDAY() Formula in the nightly load windows Decision Process Variant. When building the process chain it takes only 1 minute to use the existing WEEKDAY() Formula and then move onto the next part of creating the process chain.
The alternative is to pause the process chain build, implement the SYST-UZEIT System Field and WEEKNIGHT() Formula solutions. Depending upon the available skill levels within the BW Development team, this can take anywhere from 2 hours to 2 days by the time it is; Implemented, unit tested, transported, verified, CAB approved and made available in the BW Production system.
At this point the choice to pause delivering the current solution in order to add additional features to the BW Formula Builder Engine will seem like a step backwards. From the projects point of view it is less desirable but if you look at the choice from the Enterprise point of view then there is much more at stake.
The choice is simple: A single BW Development team takes the time to implement the WEEKNIGHT() Formula solution once or everyone continues to use the WEEKDAY() Formula. If the choice always lands on ‘Use the WEEKDAY() Formula’ then the ongoing accumulation of …
“the Technical Debt will turn into
Compounded Support Debt payments”
Each time a BW Development team chooses to implement another instance of the WEEKDAY() Formula which is not strictly aligned to the business requirement of ‘once a week’, they are increasing the Technical Debt of the EDW. What makes this worse is that the Technical Debt is usually triggered only when the BW system is mis-behaving and there are a whole lot of other issues occurring at the same time.
This is because the other issues introduced a processing time delay which then pushed the Decision Process Variant beyond midnight which in turn then decided to not execute that branch of the process chain for that week. This timing of when the Technical Debt occurs is exactly why it turns into Support Debt.
Support Debt turns into Support Avalanches
Leveraging the BW Support personnel time is a false economy for the Enterprise.
If one BW Development team takes the time to implement the SYST-UZEIT System Field and WEEKNIGHT() Formula solutions then it is available to all future BW Development projects. They will benefit from being able to just use the WEEKNIGHT() Formula (instead of WEEKDAY) and it will only take them less than 1 minute to create their new Decision Process Variant.
The real Enterprise savings can then be realised:
- When the BW Support team;
- Has less posted transaction data to delete and re-load;
- Can spend their time investigating real data quality issues;
- Can spend their time improving the DataModel;
- Can spend their time improving the system monitoring KPI’s;
- Spends less time explaining yet another reporting delay;
- Does not have to justify why no one noticed the weekly load was not being executed;
- When future development teams can just use the right formula;
- Nightly load window has less data quality failures;
- Less of the Opex Budget is spent paying the Support Debt;
- More of the Opex Budget is spent on true business benefits.
When we take a step back and look at the bigger picture (the Enterprise point of view) we can see that the initial BW Development teams decision to not implement the solution in alignment with the real business requirements was the beginning of a Support Avalanche. They may have saved a little time up front but this initial Technical Debt turns into Compounded Support Debt payments the longer it is there.
Reduce the Data Quality Risks
It is tempting to consider the SYST-UZEIT System Field and WEEKNIGHT() Formula solutions as inconsequential and not worth the time and effort to Build, Test and move into the Production system. Don’t be fooled by that initial judgement because the true cost of supporting the integrity of the data quality will cost the Enterprise so much more in the long term.
When a weekly job is not executed (because it missed the technical deadline of midnight imbedded in the WEEKDAY() Formula) the BW Support team has to manually repair all the dependent Transaction Data loads that needed the more accurate number from that week. This is a Support Debt Payment. This is real world cash spent through personnel wages / contracts and more importantly a waste of their time repairing data loads that should have never been broken in the first place.
Now compound that cost with the BW Support teams time that was not able to be spent on other business data investigations or general system improvements. There are even real world examples where the weekly job has not executed for the last seven (7) weeks, which in turn meant that business decisions were being made upon stale statistics. No one in the business questioned the results because they wanted to see the numbers go in that general direction.
When the issue was eventually identified and resolved, it was too late, the damage had already been done. The business personnel had to explain to management why the prior months reported numbers had to be adjusted and show a business trend much worse than desired. This in-turn damaged the IT Departments reputation within the Enterprise … unnecessarily.
These three actions will ensure the BW Enterprise Data Warehouse will have a better chance of delivering quality data, on time and less Support Debt Payments. A process chain in a nightly load window that uses a Decision Process Variant centered around midday (not midnight) will happily accommodate a delayed execution trigger (after midnight) when the BW system is mis-behaving due to other unrelated issues. The weekly load will still get executed, even when the process chain starts during business hours the next morning.
Action 1: Implement the ‘Application Server Time (SYST-UZEIT) System Field in BW Formula Builder’ solution.
Action 2: Implement the ‘WEEKNIGHT() Function in BW Formula Builder’ solution.
Action 3: Find and enhance the existing Decision Process Variants in the Enterprises BW Development system. Where appropriate, use the WEEKNIGHT() Formula instead of the WEEKDAY() Formula.
How did you go? Do you have a question or success story to share then comment below.