As a result of the AI hype, expectations do not map delivery. The increasing rate of AI project failures is easy to blame the technology, but this failure can be due to methodology gaps, rather than technical shortcomings. As in so many cases, there is the need for methodologies to facilitate the business-technical divide. In this post, we will look at differences between CRISP-DM, TDSP, IBM’s ASUM-DM, and OSEMN.
The implications are profound for business leadership as well as data scientists. Legacy approaches, like their partner in crime legacy software, often become quagmires in our age of agile delivery and real-time decision-making. Ultimately, they can fail in their goal of serving data scientists and the business, which can have differing objectives. Data scientists are trapped in documentation loops while business leaders grow impatient with “black box” projects that fail to connect to P&L statements.
The data revolution, as well as the focus on AI, has entered its second act. Organisations grapple with implementing technology to support petabyte-scale datasets. Boardroom demands for AI-driven ROI intensify the ‘business versus technology’ pressure as expectations and results do not match up to reality. While people often focus on the technology, a silent crisis unfolds: our industry’s once-reliable methodologies are not supporting the balanced implementation of AI that meets business goals successfully that reconciles the different facets of delivering AI.
CRISP-DM, the 25-year-old framework that defined a generation of analytics, now resembles a vintage roadmap in an era of hyperloop transit – structurally sound yet dangerously misaligned with modern terrain. In this post, let’s take a look at some of these frameworks so we can understand the modern context for businesses. The selection of framework will be the right one for your business. The key is to discern when the follow the principles and when to tweak them for business purposes. Here is a diagrammatic overview of some of the frameworks and their focus over time. Click on the picture so that it pops out.
The evolution of data science methodologies reflects shifting industry demands, from early academic frameworks to agile, business-aligned processes. For context, please find below a comparative analysis of CRISP-DM against modern alternatives, examining their historical context, structural differences, and real-world applicability. There are key differentiators between the various frameworks, and these are noted below.
CRISPDM
The enduring relevance but increasing limitations of CRISP-DM in the face of modern AI and data science needs, including the necessity for agile, collaborative, and continuously improving frameworks (Reference).
Microsoft’s Team Data Science Process (TDSP) / Cloud Adoption Framework
TDSP excels in enterprise environments with its Azure integration and defined DevOps practices. It appears to have been subsumed by the Cloud Adoption Framework and it now includes an ethics component. (Reference) The importance of agile, iterative, and team-based processes is exemplified by Microsoft’s Team Data Science Process (TDSP). It is a symptom of the industry’s shift towards frameworks that enable faster deployment and collaboration. It seems to be merged into the Cloud Adoption Framework as Microsoft links now point to an AI Plan (Reference).
IBM’s ASUM-DM
ASUM-DM strengthens model operationalisation but retains CRISP-DM’s documentation overhead. Developers are not always known to have a fondness for documentation. In my experience, this issue is often due to time constraints. If you want your team to document, then you need to give them time and space to do so. IBM’s ASUM-DM is described as an evolution of CRISP-DM, focusing on automation, efficiency, and integration with modern platforms, making it suitable for organisations with large data volumes and established practices (Reference).
OSEMN
OSEMN enables rapid prototyping but needs more business alignment mechanisms, which is important when managing business expectations. The OSEMN framework is highlighted as a modern, technically focused alternative that streamlines data handling from acquisition to actionable insight, supporting rapid prototyping and iterative work(Reference).
The key differences are summarised below.
| Feature | CRISP-DM | TDSP | ASUM-DM | OSEMN |
|---|---|---|---|---|
| Phases | Business Understanding → Deployment | Includes Customer Acceptance | Enhanced deployment | Technical focus |
| Team Structure | Role-agnostic | Defined roles (Data Engineer, etc) | Process Master role | Developer-centric |
| Big Data Support | Limited | Native cloud integration | Partial | API/script focused |
| Tech Adaptation | Tool-neutral | Azure ecosystem | SPSS integration | Python/R emphasis |
| Iteration Model | Phase-looping | Agile sprints | Waterfall-agile mix | Linear flow |
From a business perspective, nothing calms down people more than a Gantt chart! Unfortunately, AI and data specialists can be solists, and they do not hold the same perspective.
People like to plan and organise, and data and AI projects can often be difficult to put together. It is not the same scenario as running a technology upgrade, for example, where it is easier to understand requirements – and possibly also because business people ask less questions of the technologists.
If we assume that business people love frameworks and project management – and data specialists do not – then where does that leave us in terms of meeting mismanaged expectations?
This issue is more than a technical squabble between data professionals. It is a strategic inflection point. Despite being largely unseen, it has a direct impact on business competitiveness. Consider these industry research findings:
As businesses increasingly focus on their data, yesterday’s frameworks will be difficult to apply to modern projects. Businesses need frameworks so that they can plan for what is coming next. Humans like predictability and control but our data and AI systems do not promise or deliver either. This is why we need to consider frameworks and methodology as well as AI technology.
Three main shifts are driving this transformation, and they are explored in detail here.
The Death of the Waterfall Mindset: Businesses now demand continuous value delivery, not perfect models. Lone-wolf data scientists do not scale, so organisations have had to move to cross-functional teams. Organisations are peeling away from the traditional waterfall model of project development. The waterfall model was not working at the speed of the business, leading to lengthy timelines and inflexible processes. Instead, there’s a growing emphasis on delivering continual value through incremental improvements. This agile approach enables businesses to adapt and innovate quickly to customer feedback and changing market demands.
The Rise of Data Products: Data is most useful when it is put with other data to enrich it. Data products help businesses to informed decisions quickly and efficiently. This integration transforms data from a standalone asset to enhancing productivity and driving positive outcomes.
Fostering Cross-Functional Collaboration: The emergence of collaborative, cross-functional teams is changing the role of data scientists from lone-wolf to team player. For some people, this change is met with resistance and relucance, as they are no longer permitted to work in isolation. Instead, professionals from various disciplines—such as data engineering, analytics, and business strategy—come together to solve problems and drive initiatives forward to meet business goals.
Businesses get hyped about AI for many reasons, and the technology can be genuinely empowering if it is devised and implemented wisely. Without frameworks, implementing AI can be challenging. The ensuing revolution could result in poor teamwork, leading to incomplete projects.
Businesses need to harness the silent revolution to combat failure. In any business initiative, it is necessary collaboration to succeed. However, it is not always easily won. Organisations need to be very intentional about teamwork and methodology. It supports and enriches the decision-making process but also facilitates the sharing of diverse insights, leading to more comprehensive and effective solutions to complex business challenges. There can be a temptation to see some parts of the process, such as data engineering, as less glamorous or even menial. However, teamwork, collaboration is a decisive factor in success.
While CRISP-DM remains popular, it can be perceived as brittle in the software sense. Necessarily, as frameworks implement rigidity and structure as part of their purpose. However, this can mean a lack of support for exploratory data analysis and iterative data science projects. As the world moves towards data as a product, initiatives need to be product-oriented. As a result, CRISP-DM becomes less fit for contemporary needs.
Unclear on the best approach to facilitate your AI projects? Or need collaborative support to help you to make your AI work for your next project? Get in touch to book an introductory call with Jennifer Stirrup here, or sign up to Jennifer’s newsletter. It goes without saying that we will never share or sell your data.
Jennifer has a postgraduate degree in AI, and her MBA thesis focused on delivery of AI projects. Starting her career in AI before AI was trendy, Jennifer is well-placed to help support, coach and mentor your organisation through the process.
Let’s work together to make your AI work for your business and your customers.