Putting the Business back into Business Intelligence: reduce Data Debt by reconciling IT Department with Finance Professionals

It’s ironic that there are so many technical tools ways to communicate, there are still problems in achieving effective internal communication in companies. This is particularly evidence in relationships between Finance and IT. In this topic, the issues will be explored further  The issue with having so many technical tools is that the quantity of communication may increase, so this can give the appearance that communication is going well. However, the quantity of communication does not always reflect the quality of communication. IT and Finance concepts are difficult for non-specialists to understand, and messages may not be received as they were intended when working across different teams. It is hard for each department to see through the lens of the other department, and this issue can be further exacerbated if there is a culture of blame in place.

What is Data Debt?

Data debt is like technical debt, only visible in bad data, poor data quality, and an underuse of data-oriented technology to help make better decisions. It is the overhang of projects, whether they went well or poorly.

Technical debt is the implied cost of additional rework caused by choosing an easy (limited) solution now instead of using a better approach that would take longer. It also applies to data debt; the  cost of ignoring the data by choosing the quick or easy solution earlier at a previous point, which now takes longer to resolve.

At some point, however, we need to pay back data debt if we are to succeed in making better decisions.

Together, the issues all add up to make data debt, which is often a hot mess which nobody wants to resolve. Businesses can develop the Bystander stance; it is someone else’s problem.

What can you do about data debt?

Finance and IT need one another so that the whole organization can be successful, but there can be issues in communication and priorities that cause divisions. There can be seen very clearly in the sphere of business intelligence, where the business and IT need to work together to produce an essential component of the function of the organization: reporting, data and information flow.

Business Intelligence projects can inherit data debt from previous projects, which means that projects do not start from the starting line. Instead, the projects can start with a deficit before any progress can be made. In the long term, technical debt can exacerbate issues that exist between the Finance and IT departments. Take, for example, the situation where Finance don’t believe that their requirements and priorities are understood properly and that their voice isn’t being ‘heard’ in the IT strategy. From the Finance lens, this situation could be shown where IT won’t respond to support calls quickly enough, for example, or won’t allow the purchase of some new software. On the other hand, IT can hold the viewpoint that Finance are using uncontrolled data sources, such as Excel or Google Sheets spreadsheets. If something goes wrong with these artifacts, Finance team members involve IT to help fix something that they do not own, and inherit the technical debt for something that they did not create. This means that IT can end up owning ‘cottage industry’ artifacts, and this can create frustration. The ‘cottage industry’ artifacts arise partly as a result of Finance not getting the data or the software that they need from IT, so team members create their own data sources. The cycle of frustration between IT and Finance can continue without resolution, and since the pain isn’t owned by either department, there is no real ownership of resolving the issues. Instead, it simply gets absorbed into the business processes and it becomes part of everyday work. The real cost is never identified since it is a productivity cost, and it isn’t measured as stringently as direct costs that people can see or hold.

Conflicts between departments can arise out of differing priorities, but the overall vision should be the same for each department within the organization. As IT pushes forward into new projects, technical debt can be pushed to the side, simply to accumulate. For Finance, this issue can develop into unresolved conflict and ongoing productivity issues, as well as concerns over data governance and data veracity. The reality is that IT and Finance need to work together for the success of the whole organization. There are risks if the departments cannot work together. For example, if the departments cannot work together then this could put security at risk, for example, due to a failure to communicate successfully. Ultimately, this could land the company in court. Other consequences include an adversarial working environment which can make the company unable to recruit and retain key talent. In turn, this can impact technical debt as people leave without resolving it, and it just continues to accumulate. Although it’s hard to quantify, the difficulties in IT and Finance working together will cost the organization money, whether it is calculated in terms of lost productivity, or direct costs such as poor software selection. By contrast, if the teams can work together and work better, then both departments can increase their contribution to the success of the whole organization.

Blueprint for paying back data debt

How can the teams move forward from the impasse? We need to put the business back into business intelligence and move forward, working more collaboratively.  Fundamentally, this is an issue of how information can be tailored to different audiences who are specialists in their own field, and need to work together for a specific outcome that is seen across the organization.

Both Finance and IT departments need to work together to improve understanding and collaboration between each department. Brown bag lunch sessions, where team members present bitesize summaries of relevant concepts to both teams, can provide a good experience of working together for all team members. This may also help to break down stereotypes of the characterization of each department. Further, it can help team members to enlarge their tribe to include people from other teams, as well as their own team.

Finance and IT leaders need to work together and set an example to the rest of the organization. Personal conflicts between department managers is not acceptable, and the dispute will affect processes and practices within the company and disrupt the entire workflow.

Set up an informal wiki or blog, where team members can add every day. This blog can be used to identify the data and the information each department needs every day. By identifying key information, the wiki can be used to help understand each department better, and to share this crucial information in a straightforward, recorded way. This will help the teams to see the impact that they can have on each other, and identify and prioritize effort to maximise benefit.

Creating a community of excellence as a cross-department initiative that has representatives from each department working on joint initiatives, with progress reports and clear objectives. This will take some time to set up, and there may be resistance due to previous projects that have failed or not delivered.

Inherit technical and data debt. Every new IT project should inherit some technical debt from previous It work. This could involve data cleansing, or improved reporting, or the review of a support contract of cloud vendors.

Requirements can be hard to write and accept. One way to increase teamwork is to accept that prototyping is a valid way of developing and articulating what Finance need. Further, it means that IT can take on board It’s considerations, such as security, governance and upgrades. When developing, IT will want to prototype; at first, the efforts will seem awry but constant communication and feedback will be a good investment of time.

Conclusion

It isn’t going to be easy, but it is achievable if the example and direction is set at the Csuite level. Communication is key.

And finally, you can see the end of the data debt.