For AI to be truly intelligent, it needs to understand, evaluate and apply knowledge in completely novel contexts.
Michael Weidman, Director of Pre-Sales Consulting, causaLens
Why causal AI matters for all companies of all sizes
The causal AI revolution is here, with the importance of considering causation in machine learning increasingly clear. Inflexion’s portfolio heard about the significance of causal statistics in addressing the limitations of correlation-based approaches at the recent Data Exchange.
Data is growing exponentially, as is data gathering and computational power to analyse it.
However, despite these advancements, many attempts to apply machine learning in practice have failed (for example 87% of machine learning projects in the industry never make it beyond an experimental phase, according to Forbes). One of the main challenges stems from the fact that most statistical approaches are based on correlations rather than causations – and while they can help in making predictions, they don’t always cut it when it comes to understanding the underlying reasons or making changes based on cause-and-effect relationships.
This is the premise of Michael Weidman, Director of Pre-Sales Consulting at causaLens, a specialist software company. He reminds delegates that correlation does not imply causation – yet all traditional machine learning only relies on correlations to learn from the past. “It means they purely see patterns from history and blindly apply them to the future. This is a problem,” he says.
He illustrates this with two examples: predicting shark attacks based on ice cream sales and modelling churn based on employee salaries. In both cases, correlations exist, but they are confounded by other factors that lead to counterintuitive outcomes. For instance, higher salaries may be correlated with increased churn due to the tendency to be the best employees likely to be poached by competitors.
Confounding confuses correlation
These confounding relationships can mislead traditional correlation-based models. “As long as we only rely on the model for prediction, correlation based models are good enough. But if we ask more questions, you see the model isn’t fit for taking action and reducing attacks even though it is excellent at predicting them. There is nothing in the maths to tell us the causes of shark attacks or how to try and decrease the rate of attacks. If you have ‘why’ or ‘what if’ questions, you need to be careful with the models you’re using.”
Likewise, with the salary model, the data could inadvertently show that lower salaries could actually help retention. “This highlights the imperfection of the model owing to confounding relationships – the best employees will get the largest salaries and are also the most poachable. It turns intuition on its head and so will turn any correlation model on its head.”
Causal AI can address these limitations by focusing on causation rather than mere correlations. It provides tools and methodologies to identify and analyse cause-and-effect relationships, enabling more accurate decision-making. Causal analysis can uncover the underlying factors driving certain outcomes and help answer more questions, leading to better strategies and actions.
Jan Beitner, Assistant Director for Data & AI at Inflexion adds “Even if you do not use AI, your descriptive analysis in BI dashboards is often essentially a correlation analysis. It is crucial consider if the results are supported by causality.“
Why Causal AI matters
Reasoning is crucial for intelligent decision making: Current AI techniques match historical patterns without contextual understanding, and decision-making based on historical patterns can’t be fully trusted.
But correlation ML cannot reason: Cause-and-effect relationships (especially with confounding causes) resolve and explain these examples, so building a cause-and-effect model is the answer for these questions.
Causal AI is the only AI that uses reason: It is much easier to explain through intuitive causal diagrams than many correlation-based AI models that come with complex data science metrics. Unlike conventional ML, causal AI generates models that are both accurate and inherently explainable.
All Inflexion portfolio companies, regardless of size or ownership stake, have full access to our value acceleration resources covering digital enhancement (including data, AI, technology, cybersecurity and digital marketing), international expansion, M&A, ESG, commercial strategy and talent management.
Contact
