Using Kaizen, Kaikaku to deliver Analysis Services 2008

I have been interested recently in applying different methodologies to delivering Business Intelligence projects, particularly in Analysis Services 2008. In particular, Agile and Kaizen seem to be ‘hot topics’ right now, particularly given the credit crunch. It wasn’t clear to me that this approach would work well in Business Intelligence, so I’m currently exploring this idea as I progress on this journey!

Kaizen is derived from Japanese: kai, meaning change, and zen, meaning good. Kaizen deals with micro-improvements, which gradually increment to make larger changes. These changes are focused on eliminating waste from the process. By keeping progress under constant review, it means that problems are exposed much earlier on in the process. In terms of customer satisfaction and project delivery, Kaizen might work well since it focuses on constantly trying to make things better, and for the customer, that can only be a good thing. In particular, it focuses on looking at the facts, rather than a ‘hunch’; looking at data, rather than following a soundbite, should be the way forward in BI in any case. However, a process of constant change can have an effect on individuals on the team, since they are working under scenarios of constant change. Generally speaking, in my opinion, people are generally opposed to change in principle, so this might not suit everyone.

Related to Kaizen is the concept of Kaikaku, which can be translated as transformation, which is usually rapid in form, and it is also aimed at eliminating waste to create greater value. Kaizen tends to be associated with small, incremental changes which are more gradual in nature. Kaikaku is a deeper, more rapid change; this might mean something more drastic, such as making a decision on the inclusion or exclusion of a data source or Analysis Services 2008 cube. In order to eliminate waste, a Kaikaku way might be to introduce this radical change without excessive documentation to achieve it; instead, a lighter documentation effort, to note what is required and agreed before moving forward, may be more appropriate. This might differentiate these approaches from more traditional Business Intelligence projects, where a great deal of documentation may be produced (and perhaps not read!)

So, when implementing projects, I see it as a process of Kaikaku and Kaizen; a radical improvement might be required first (Kaikaku), which is then followed by a process of incremental improvement (Kaizen). This could be translated into the activities of managing a Microsoft Business Intelligence project; a radical improvement could be made by the inclusion of an Analysis Services 2008 cube, for example, followed by iterative amendments to the Analysis Services 2008 cube itself. Kaizen changes might involve something small, such as renaming calculated members so that the users can navigate their way around a cube more easily: thus reducing motion and wasted value. Thus, used together, these approaches are not inconsistent with the process of producing cubes in Analysis Services, for example. This traditionally requires a lot of ‘behind the scenes’ work in cleansing the data in a format that is comfortable to use in a cube, creating and developing the cube, and so on. These activities could be seen in the Kaikaku perspective rather than a Kaizen perspective: as radical changes rather than iterative changes, but equally aimed at producing value.

It is important to note that Kaizen may offer something more to BI rather than simply a gimmick. For example, the ‘headline’ had been that Scrum meetings are normally taken standing up, and should take no more than twenty minutes. At first, this sounds a bit gimmicky but then, the reason for this tactic comes clear; for many developers, meetings are often a ‘waste’. Given that Kaikaku and Kaizen are about eliminating waste, then it becomes apparent that meetings need to be kept to a minimum in order to eliminate waste as far as possible.

So it seems to me that more traditional approaches of Business Intelligence projects could be translated to work within a Kaizen environment. By detecting problems earlier upstream, this can reduce costs in the longer term since issues are fixed earlier, and have less ramifications. On the other hand, the customer has visibility of problems early on in the process. This can mean that the customer’s initial experiences of the project are not as positive as the perceptions that might be gained through producing a lot of project and requirements documentation in a more traditional Business Intelligence project, for example. One way which this might be tackled is by allowing the users to have early visibility of the data; for example, using data visualisation such as Tableau to see the results of Analysis Services cubes or even just SQL Server views. Thus, although the customer can see issues early on, they also see results earlier as well; this might mitigate the risk around problem perception early on in the process.

To summarise, the various management methodologies around project delivery and software development are useful to know, and might be additional tools to help BI projects go more smoothly. I’m all for that!