Your BI Platform Is Not the Product. The Decision Is.

The success of a business intelligence project is typically measured against the wrong set of criteria:

• How many dashboards have been created?
• How many data sources have been connected?
• How fast can the report be refreshed?
• How many users have access to self-service analytics?

Such criteria are relevant but insufficient, because there is a key question left unanswered: has the platform helped someone to make a better decision?

In more than 18 years of my professional experience building enterprise analytics solutions, I have seen a lot of efforts being invested into cloud data warehouses, stream processing pipelines, semantic layers, and visualizations with no positive effect on decision-making in the organization – the decision-makers still doubt numbers. The dashboards may look fancy. The infrastructure may be scalable and fast.

But the executives still extract data into spreadsheets. The finance team still reconciles conflicting figures before meetings. The sales department still disagrees on what revenue KPI means. The operations team still relies on analysts to find explanations for changes.

And the problem is rarely that visualizations are not good enough. The problem is that the organization has built a reporting system, rather than a decision system.

The Dashboard Is Only the Final Interface

Every dashboard is just the top surface of a complex architecture. Behind every chart lies a set of decisions concerning:

1. How the data is ingested
2. How the records are matched
3. How the business rules are applied
4. How the historical changes are stored
5. How the metrics are calculated
6. How the exceptions are processed
7. How the access is secured
8. How quickly the data becomes available

The dashboard can only be as trusted as the system underneath.

Let us take a simple example of a metric like "outstanding receivables." It seems like an obvious calculation: the sum of unpaid invoice values, grouped by customer, geography, or any other dimension. However, the list of questions behind becomes quite long:

• Which customer record does the invoice refer to, if different systems use different naming conventions for customers?
• Should the disputed invoices be taken into account?
• How should we treat the partial payments of the invoices?
• What happens if the ownership of the customer record changes over time?
• How should the currency conversions be performed?
• What is the determining timestamp for the invoice: the invoice date, payment date, or when the record has arrived into the data warehouse?
• When does an overdue invoice become significant for operations?

The visualization is the easy part here. The challenge is to engineer a consistent solution.

Fast Data Is Not Necessarily Intelligent Data

Real-time analytics has become one of the common enterprise goals. Modern analytics platforms can easily ingest events in real-time, perform continuous transformations and refresh the analytical models within minutes or seconds.

It is a great feature, but just having fast metrics is not enough to build intelligence. A metric that refreshes every few seconds does not have to be understood and interpreted by anyone. Sometimes the real-time data processing can even harm, exposing all inconsistencies, duplications, lateness of events and incompleteness of mappings, and thus giving false sense of precision without the foundation of accuracy.

Before choosing real-time metrics, I ask myself three questions.

1. How quickly do we need to take a decision?

A fraud detection system should process an alert within milliseconds. A disruption in supply chain may require immediate action. A collections team may need a daily prioritization of accounts. A board-level metric may be calculated once per reporting period.

Not every decision requires instant response.

2. What is the price of acting upon the partially received data?

If the executive dashboard shows revenue before all adjustments, cancellations or reconciliations have arrived, the result will be fast but deceptive.

3. Is the organization ready to act instantly?

It makes little sense to have the metric updated every minute if we only act once per week anyway.

The purpose of the real-time analytics is not to update the metrics as quickly as possible. The purpose is to reduce the gap between the event and the response.

The Semantic Layer Is Where Trust Is Built

Many analytical teams consider the semantic layer as just a convenient technical tool. They miss its real value.

The semantic layer is where raw records turn into business concepts. Here we define the business meanings for entities like customer, active user, revenue, backlog, renewal, churn, risk, profit margin, etc.

Without a governed semantic layer, every dashboard creator will interpret the data in his or her own way. One team will filter out the cancelled transactions, while another will include them. One report will use invoice dates, another will apply payment dates, and the third will use dates when the data arrived into the warehouse. All three reports will be technically correct, but useless together.

Standardization of the KPIs is not just about documenting the formulas. This is about the architectural decisions. A proper semantic layer will provide:

• Definitions of the business entities
• Common algorithms to calculate shared KPIs
• Consistent treatment of the time and currency
• Ownership of the metric definition
• Handling of the late arriving or corrected data
• Tracking of the historical changes if the business context changes
• Traceability of the executive metrics down to the source data

Having such a layer, we can switch from one visualization tool to another, without breaking anything. Without such a layer, the fancy dashboards will remain a source of discussion.

Data Modeling Is a Business Decision

When talking about data modeling, we usually imply purely technical choices. Build a star schema. Define the facts and dimensions. Apply slowly changing dimensions. Optimize join queries. Create aggregate tables.

It is all correct. However, we are making business decisions every time we are choosing data model.

For example, let us look at the customer analytics case. To build the Customer 360 model, we may need to bring information from sales, billing, support, product usage, marketing systems, and third-party services. The same customer may be represented differently in several legal entities, different accounts or subsidiaries, different email domains or other IDs.

Simple exact match will not help. Here comes the need of the fuzzy logic matching and entity resolution algorithms.

However, this is not only a technical problem. It is a governance problem as well:

• At what confidence level the match will be considered as a definitive match and processed automatically?
• When should we route the record for human approval?
• How to split the merged records if the match turns out to be false?
• Whose records are more authoritative?

Entity matching algorithm can detect two similar records. The business decides if they are the same.

History Is More Important Than Most Dashboards Think

In many analytical systems the old value is over-written with the new one. Customer changes its region, and the region field in the record gets updated. The customer is assigned a new sales rep, and the record ownership changes. The product is moved to the different category, the previous one becomes unknown.

This is OK for operational applications. Not for analytics.

Let us imagine that the leadership needs to evaluate the previous year sales performance by region. However, customer regions could change since then. By using the current regions, we will be evaluating past sales using today's structure.

Slowly Changing Dimensions of type 2 exists precisely to handle this issue by storing the previous state of the record and time intervals during which this particular state was true. It is an old concept widely known by data engineers, however, the business importance of it is frequently underestimated.

The historical accuracy is important for:

• Performance evaluation of the sales territory
• Customer segmentation
• Product profit margins
• Ownership
• Regulatory reporting
• Contract analytics
• Forecasts evaluation
• Executive compensation

A data platform that is unable to tell what the business looked like at the specific point of time is not an analytical one. It is a static snapshot viewer.

Self-Service Analytics Does Not Equal Uncontrolled Exploration

Self-service analytics is normally sold as a way to reduce the reliance on centralized BI teams. It is a good goal. Business users should not go through tickets for every single trend, segment or comparative analysis that they want to explore.

However, the unstructured self-service analytics is a different problem. Instead of one reporting bottleneck, the organization will get hundreds of inconsistent reports.

To get real self-service capabilities, the organization needs controlled freedom. Users should be able to explore trusted data without rebuilding the business logic from scratch every time. A mature self-service environment usually divides the responsibilities into three layers.

The data engineering layer

This layer is responsible for ingestion, transformations, quality checks, orchestration, and reliability.

The governed analytics layer

This layer holds the definitions of shared entities, metric certification, security rules, and analytical models.

The exploration layer

This layer allows to filter, compare and visualize the data.

The mistake is in letting users to access the raw tables directly and call it the self-service. It is not the self-service. It is distributed data engineering without standards.

Executive Reporting Has a Different Requirement of Accuracy

Operational dashboards can be a bit tolerant of exploration. Executive reporting cannot.

The metrics used in the communications with investors, in earnings, in board meetings and strategic plans require much more control. The numbers must be reproducible. The logic of the metrics must be traced. The adjustments must be explained and documented. The refresh of the data must be predictable. The access must be controlled. The same metric must be calculated the same way by any person opening the report.

This is why the executive reports should never rely on manual spreadsheets, filters, extracts, and other ad-hoc methods.

The process of executive reporting should answer:

• Where does this number come from?
• Which source records have been used for its calculation?
• Which business rules have been applied?
• When the data was last refreshed?
• Is there any change in the logic since the previous reporting period?
• Whose approval does the metric definition have?
• How can the result be reproduced?

If a platform cannot answer such questions, the dashboard may support the meeting but not the accountability.

Automation Is About Reducing Preparation, Not Eliminating Judgment

Often the results of the analytics automation are measured in hours saved. It is good, however, even better, we should see how automation influences people's attention.

Analysts should not spend most of their time:

• Downloading the files
• Reformating the columns
• Copying the values between the systems
• Reconciling the repeated totals
• Updating the presentation slides
• Fixing the broken formulas in the spreadsheets
• Explaining why two dashboards show different figures

This routine takes people's attention away from the analysis of the business. A properly built platform should automate the repetitive tasks to leave the attention for analysis:

• Why receivables increased?
• Which customer segments are causing the change?
• Is this temporary or structural issue?
• Which accounts need intervention?
• Which actions would bring the maximum impact?

Automation is moving people from the preparation to the decision analysis. It should not eliminate the human judgment from the decisions requiring business context.

Anomalies Need Context Too

Predictive analytics and anomaly detection have become common features for enterprise BI platforms. By using the model, the system can detect unusual payment patterns, sudden changes in the usage, unexpected regional performance or deviations from historical trends.

However, anomaly detection is not the same as explanation. The anomaly can indicate the fraud attempt, technical malfunction, seasonality effects, renewal of contracts, pricing changes or a legitimate business growth.

Technical model detects the anomaly. Business decision system should help to understand why this happened.

Contextual anomaly detection should include:

• Expected interval
• Observed value
• Deviation magnitude
• Historical period of comparison
• Related business dimensions
• Possible data quality issues
• Similar events in the past
• Contributing records

The anomaly alert without context is more work to do. The alert with context is the starting point of the investigation.

Governance Must Be Embedded Into the Platform Architecture

Data governance is often perceived as a separate initiative, involving committees, policies and documentation. It is important, however, data governance must live inside the platform too.

A governed platform should enforce the following:

• Role-based access
• Row-level and column-level security
• Data lineage
• Metric certification
• Validation of data quality
• Retention rules
• Audit logging
• Separation of environments
• Controlled deployments
• Ownership metadata

Governance should not depend on people's memory of the policy. The system should make the compliant way the easiest way.

It is especially important as organizations start adopting cloud-native data architecture and connecting more operational systems to the centralized analytics platform. The easier it becomes to move and analyze data, the more important it becomes to govern the data definitions, access and usage.

Design Backward From the Decision

One of the most efficient ways to design an analytics platform is to start with the decision and work backward.

Do not start with the dashboard. Start with the person who needs to make the decision and ask him:

1. What decision he is going to make?
2. How frequently does he make this decision?
3. What information impacts the decision?
4. How current does the information have to be?
5. What is the required accuracy?
6. How exceptions should be explained?
7. What actions need to be taken after the insight is gained?
8. How should the result be measured?

Then the team can start to design the metrics, semantic layer, data transformations, ingestion and visualization. Only such approach will prevent the platform from becoming a set of impressive but disconnected components.

The right BI platform is not a set of the best dashboards. It is a platform reducing the uncertainty at the moment of the decision.

Conclusion

Enterprise data industry talks a lot about platforms. Snowflake or some other cloud warehouse. Power BI or Tableau. Batch ingestion or stream processing. ETL or ELT. Centralized models or decentralized ownership.

These are important choices, but they are not the ultimate goals. The goal is a platform that helps to gain consistency, timeliness, and explainability of the information.

This is achievable not just by connecting the data sources and building the dashboards. It is achieved by implementing reliable pipelines, smart data modeling, governed metrics, accurate history, contextual alerts and proper understanding of how the information influences the decision.

The dashboard is not the product. The warehouse is not the product. The pipeline is not the product.

The product is the decision that becomes quicker, easier, and more defensible because the whole system works seamlessly.