Author Archives: Jen Stirrup

AWS RDS vs Microsoft Azure SQL Database: What does it mean for the business?

Posted on by

As a freelance industry analyst who has worked with GigaOm, I’m pleased to see the GigaOM Transactional Field Test derived from the industry-standard TPC Benchmark™ E (TPC-E) report which compares Amazon Web Services Relational Database Service (AWS RDS) and Microsoft Azure SQL Database. It’s written by William McKnight and Jake Dolezal from GigaOm.

From the business intelligence perspective, it is incredibly useful to compare AWS RDS and Microsoft Azure SQL Database. Before we dive in, what do they have in common? They are both fully-managed cloud SQL Server offerings, and from the Business Intelligence perspective, it can be difficult for business decision makers to choose between them. In this post, we’ll distill the technical language so that business decision makers find it easier to make the right decision for them.

Why SQL Server?

SQL Server is one of the most mature, well-known and common databases in the world, according to data from DB-Engines ranking. In fact, there is some evidence to suggest that SQL Server is going to overtake Oracle, potentially due to the cheaper price tag.

SQL Server offers fantastic business intelligence features, and this is probably one of the key areas where it wins over Oracle. In my experience, this is where I saw SQL Server ‘creep’ into IT estates due to its superior business intelligence solutions in SQL Server, which were easier to use. In particular, the SQL Server Reporting Services solution solved problems for Finance departments; they could get repeatable financial reports very easily, and Microsoft SQL Server solved a business problem. So it took hold.

Why SQL Server in the cloud?

Generally speaking, the cloud is attractive to businesses due to its lower costs, scalability and pay-as-you-go pricing model.

For IT Departments, cloud databases are mainly used for read-intensive, data intensive applications such as data warehousing, data mining and business intelligence operations which need elasticity and scalability. Cloud databases offer reliable computing, storage, backup and network facilities at the lowest cost, which is particularly important when the IT Department is regarded as a cost center.

AWS RDS and Microsoft Azure SQL Database are both web services that makes it easy to set up and scale a relational database in the Cloud. They are designed for developers or businesses that want cloud databases.

AWS RDS vs Microsoft Azure SQL Database

GigaOm have held a GigaOM Transactional Field Test which aims at comparing the two databases. In the test, the performance was scaled to meet the needs of 80,000 customers, and the performance was tuned to meet the database engine’s optimal capability to process data. The tests consisted of five test runs that lasted a duration of two hours each for both platforms. You can read more detail about the test runs in the GigaOM Transactional Field Test.

Cost Differentiation

The paper found that Microsoft Azure SQL Database was considerably less expensive to run in comparison to AWS RDS over a month. Of course, the reader should try out different scenarios in order to make the decisions that are right for their environment. That said, the cost savings are compelling, and if you have not looked at Microsoft Azure SQL Database before, then it is worth reviewing the cost differences. This held to be true over different reviews of the solutions, such as cost, licencing and time reservations.

The paper concluded that the database, along with the cloud, matters to latency which is
the killer for important transactional applications. Microsoft Azure SQL Database presents a compelling proposition for the modern transactional workload, meeting the need for data warehousing, data mining and business intelligence operations which are the engine of many of today’s businesses.

Conclusion

The reality is that we live in a multi-cloud world. Apart from cost, how can organizations show that they are creating uniquely desirable products and services rather than simply discussing cost?

Since Microsoft are the creators and owners of Microsoft Azure SQL Database, the differentiation here is that Microsoft lead with research, development and innovation in SQL Server. This means that Microsoft lead with the ability to design, deliver and support SQL Server in its different guises, whether on-premises, in the cloud, or a hybrid mixture of both platforms.

Since Microsoft own SQL Server, they are uniquely placed to understand the dynamics of the customer needs and the market, thereby meeting those needs to develop uniquely well-specified products for the market. Therefore, it makes sense that Azure would be finely tuned to rise up and meet these needs. Hence, it makes sense that Microsoft SQL Azure Database costs are optimized to be lower, and this is reflected in the costing differences found in the study.

To read the GigaOM Transactional Field Test derived from the industry-standard TPC Benchmark™ E (TPC-E) report, please refer to the article. Please feel free to leave your thoughts in the comments.

Keynote and Session highlights at ASG Evolve19

Posted on by

I’m excited to be attending ASG Technology’s Evolve19 event in Dallas, Texas on October 21 – 23.

I’m particularly interested in learning more about metadata management. As I’ve written previously, I believe that metadata management is often overlooked in data science. It’s an important part of obtaining full value from data, and it should be part of the data maturity model. I’m looking forward to seeing how businesses can securely use data to make better business decisions faster, and compliance teams can reduce data related risks and resolve any data problems.

I’ve blogged more about my favourite sessions and the keynotes over at the Data Relish site and I look forward to seeing you there! If you are going to be at the event, please don’t hesitate to get in touch. I’d love to say hello!

 

Future Decoded: Microsoft imbues ethics and responsibility into AI as a core part of Azure AI technology.

Posted on by

Microsoft are designing AI to be trustworthy requires creating solutions that reflect ethical principles that are deeply rooted in important and universal values. This is becoming a core principle and practice, and Microsoft are doing a great deal of research in this area.

Why is this important? Information technology is ever-growing, and becoming more sophisticated by the day. We are living in an era that is more defined by its rapidly evolving technology and how this technology is becoming a part of every person’s everyday life, regardless of their own abilities or experience. Because of this, it is the responsibility of the organisations behind this technology to ensure that they are taking responsibility for the massive impact that they cause. The future is likely to be even more defined by the technology that is currently evolving – and if companies neglect to take a practical, thoughtful and responsible approach to implementing and developing this software, they run the risk of not being able to catch up with the consequences. 

Technology can be both empowering and threatening. Issues such as breaches of privacy and data protection, cyber-attacks, and irresponsible programming are present threats that run the risk of becoming even more serious problems. The rise of AI poses another type of threat that is different but equally as important to recognise, wherein the developers and programmers behind the technology pose the risk of embedding societal prejudices and their own judgements into the algorithms that determine the AI’s actions. There are moral and ethical conundrums arising as technology becomes more sophisticated and ingrained in our daily lives.

Human decision makers are susceptible to many forms of prejudice and bias, such as those rooted in gender and racial stereotypes. Evidence in research as well as publicized news stories have found that machine learning systems can inadvertently discriminate against minorities, historically disadvantaged populations and other groups. One would hope that machine learning would overcome the bias, but, unfortunately, it learns from decisions that are made in the world.

Microsoft are devising interesting and innovation solutions to tackle bias. For example, Microsoft research have created a “fairness enforcer” which uses machine learning to create a process that yields a classification rule that is fair, according to the fairness definition while minimizing the error involved.

Microsoft are currently attempting to help companies by embedding a practical and considerate approach into their software. Last year, Microsoft released a statement titled Responsible Bots: 10 guidelines for developers of conversational AI (2018), where they outlined their ethos for responsibly creating and maintaining technology that people are most likely to use day-to-day for real-world, consequential tasks and problems. They also published Six principles for the responsible development of AI in a 2018 book, The Future Computed (2018). With Microsoft working on systematically and intentionally aiming to prioritise social responsibility and ethical decision-making as the most integral factors in their transformation as a company, and making their choices and actions public and transparent. 

While Microsoft are currently leading the way on implementing practical strategies, it is becoming more important that any company who is developing AI need to be aware of the risks and how to prevent and handle them.

Tay_bot_logoA strong example of how conversational AI can be used irresponsibly and become harmful is Microsoft’s ‘Tay’ experiment. Tay was a Microsoft AI chatbot who was launched on Twitter  in 2016. After less than 24 hours, Tay was shut down completely, as it had begun to generate a huge amount of inappropriate tweets filled with racist, sexist and anti-Semitic language. This was due to the fact that Tay learned from the interactions that it had with the public. The lack of responsibility on the developer’s part was a failing in itself – but Microsoft have learned this lesson and they have done the right thing in doing so. Miller et al state in their article ‘Why We Should Have Seen That Coming: Comments on Microsoft’s ‘Tay’ Experiment, and Wider Implications’, 

We contend that these incidents are symptoms of a deeper problem with the very nature of learning software (LS-a term that we will use to describe any software that changes its program in response to its interactions) that interacts directly with the public, the developer’s relationship with it, and the responsibility associated with it. We make the case that when LS interacts directly with people or indirectly via social media, the developer has additional ethical responsibilities beyond those of standard software. There is an additional burden of care. (Miller, Wolf and Grodzinsky, 2017)

Cases like this make it clear that developers need to re-think their approach to conversational AI, and begin to prioritise a more proactive ethical stance. Microsoft have suggested a number of ways for developers and companies to begin their journey to more responsible, practical AI practices.

It is important to have a ‘break out in case of emergency’ option present when conversational AI is interacting with the public. This allows the opportunity for conversations to be interrupted or shut down if they become inappropriate. In the case of companies and products using AI virtual helplines, it also presents the chance for the conversation to be flagged to the moderator if the customer expresses that they are dissatisfied or have a problem which goes beyond the AI’s capabilities.

One of the most interesting, but also most potentially harmful features of conversational AI is that it can take on a human-like persona, or appear to have a personality. This gives extra weight to the things that it says in conversation; it needs to interact appropriately with users, respect cultural norms, and be designed to avoid norms violations wherever possible. For example, a chatbot designed to help users navigate an online banking site should not have opinions on religion or politics, and should not engage at all with any questions that are not relevant to its purpose. It is also useful to have a ‘code of conduct’ for users where possible, explicitly prohibiting hate speech or any form of harassment. Techniques such as keyword filtering come in useful when designing a bot to pick up on inflammatory language and inappropriate subjects.

Data Relish have recently released a series on how to tackle prejudice in machine learning, and this is something that also needs to be considered in this case. It is essential that AI is programmed to treat every user fairly and equally. To help ensure this is the case, the development team should be diverse and home to a wide range of perspectives and experiences. Furthermore, data should be consistently assessed by humans – while bias detection tools can be helpful, they should not be relied on entirely.

toolsandweapons_3d_new-1-671x1024Overall, it is essential that the development of AI does not lead to us feeling as if their job is now their job only and is therefore out of our hands. Human intervention is essential throughout the entire process. For AI in more sensitive fields, such as health or law enforcement, experts from relevant fields should be brought in and encouraged to give their input. Customer service AI should be programmed to ask users for feedback, which should be fed back to colleagues at the business. To ensure the AI is accessible for all, disabled people should be invited to test the bots before they are released to the public. Ideally, AI should learn from diversity and effectively recognize and shut down issues.

At Data Relish, we are currently reading Brad Smith’s new book on ethics and AI, called Tools and Weapons. We will summarize in the future, and, in the meantime, we are glad to see that Microsoft are taking these issues seriously.

 

Miller, K.W; Wolf, Marty J; Grodzinsky, F.S. (2017). Why we should have seen that coming. ORBIT Journal, 1(2).

Future Decoded: Business opportunity and sustainability in the era of AI

CindyRosePeople don’t like change, and they don’t like to be changed. That’s what makes Digital Transformation so challenging.

Digital Transformation is one of the key topics at Future Decoded, where I’m working as an analyst this week. The Day 1 event is aimed to help visitors learn about the cutting-edge technologies that are shaping the modern workplace across industries. The event is intended to promote and provoke ideas for business opportunity and sustainability in the era of AI. This becomes increasingly important as UK companies risk falling behind foreign rivals unless they use more AI.  

Interestingly, the Financial sector leading the way in using AI. For example, NatWest is using AI to understand how financial markets behave. For people who are interested in applying AI to their organization, it is worth taking a look at the latest Microsoft’s AI report – Accelerating Competitive Advantage with AI

Planet, Profit and People – and AI

Organizations are taking environmental impact very seriously and, at Future Decoded, Microsoft cracks down on plastics at Future Decoded. At Future Decoded, the keynotes have had a clear emphasis on ethics being at the centre of working with technology, people and process. This emphasis also chimes with the new focus on Diversity and Inclusion at Future Decoded.

One key theme was the need for diversity and inclusion as a key part of the philosophy of a non-biased way of managing AI. Diverse teams, whether inadvertently or mindfully, do challenge each other, which means that people can also challenge themselves. In challenging themselves in an empathetic environment, the bar for success can be raised. In teams where everyone is similar, people don’t always question themselves or each other since there does not seem to be a need in these environments; the ‘like me’ impact means that the implication is that everyone is right, because they are ‘like me’ so they must be right, so therefore…. I am right. That’s not the way forward, however. If you want to raise the bar of success for your teams and your product, you need to include diversity and inclusion as part of the DNA of your organization in order to challenge ourselves and each other. It is these friendly challenges which raise the bar to success. You can’t just slap an AI label on a weak business proposition, and expect it to be successful.

In the field of AI in business, there are plenty of so-called ‘experts’ but it is important to understand how the Dunning–Kruger effect is involved in planning out projects, such as those projects that include AI. In case you haven’t heard of it, the Dunning-Kruger effect is a cognitive bias in which people mistakenly assess their cognitive ability as greater than it actually is. That’s why we need people around us, to challenge us, to encourage us, and to help us to see the ‘data’ to guide us towards making better decisions and having a sustainable impact on our business. Recognizing our cognitive biases, and working to overcome them, can help us to do better. Listening to the Future Decoded keynotes today was an interesting experience, because it was thoughtful about the impact of technology on people and planet. It was clear about how this practical ethics translates into authenticity in the behaviour of an organization, which, in turn, can  help an organization to scale up by increasing profits by focusing on using less impactful resources.

Aside from that issue, what else can you do in order to have a scale up attitude to AI?

  • Create a roadmap for 12 to 18 months, based on what the business values and perceives as key criteria for success.
  • Follow the value chain of your business. Think of the operations that are required to make AI succeed in the long term, and think about automation.
  • Think about sustainability for your organization as well as the environment. What is going to have a lasting impact for good for your business?

Digital Transformation is everywhere, but it is an ongoing process rather than a box to be ticked. Every company has to be a software company, whether they like it or not. Eventually, is every company is going to be a cloud and AI company? An analytics company?

Digital Transformation is about showing what is possible, and giving people a vision that they feel that they can be a part of, and a vision of the organization that they feel they belong to. This is an additional important theme, and it was good to hear that the stakeholders in the business are part of the Digital Transformation and AI stories.

The Future Decoded keynote ended on a note of recasting Big Data with a view to using it for quantum computing. The end user gets information and insights that they were never able to have before, which brings better choices and outcomes. The keynote is exciting because it allows for longer-term future results using Microsoft as a platform. It’s not science fiction, and quantum is helping us to solve problems that were never solvable before with classic computing.

Transformation will mean that every company is a software company, focused on what your organization does best. Trust is the currency of the new economy; it is a foundation for what makes a good partner.

For this reason, data is so important in earning trust. Data is a vague term, a catch-all which is skimmed over by organizations and people. Bad decisions are based on bad data; good decisions are down to expertise. Perhaps we all have a Dunning-Kruger impact? Data is the moments of our lives; our books, what we read, our photographs. Companies have taken the trust for granted and we suffer a techlash. With great power, comes great responsibility and the decisions need to be an ethical one, from the data right up to impact on the environment. In line with the thinking about ethics, the keynote ended with this thought; it is your data, and we must respect it. In doing so, it shows a respect for the people behind the data, and that should be a core pillar of any organization.

Future Decoded – see you there?

Posted on by

I’m excited to be attending Future Decoded in London this week. I’m attending in my role as an industry analyst, so I am not presenting. Instead, I will be interviewing Microsoft UK executives

What will be happening at Future Decoded? The event itself is aimed at business decision makers and leaders, rather than a technical event for IT Pros or developers. Due to that objective, Future Decoded features unique perspectives from industry leaders. I’m personally excited to meet astronauts Major Tim Peake CMG and Dr Helen Sharman CMG, OBE, HonFRSC and I’m interested to hear their perspective on the priorities for technology, given the challenges that the planet is currently facing. Additionally, Red Bull F1’s Team Principal Christian Horner OBE is delivering a keynote on technology and what it means for the F1 success.

I’m also excited to meet Michael Wignall, Azure Business Lead, Microsoft UK and I’ll be talking with Michael about the importance of AI as the fourth industrial revolution. Following on from that topic, I’ll be speaking with Paolo Costa, Microsoft’s Principal Researcher, Microsoft Research Cambridge and I’ll be learning more about the AI projects and innovation coming out of Cambridge.

I’m also super excited to be meeting Mitra Azizirad, Corporate Vice President, Microsoft AI again. Mitra leads Microsoft’s portfolio of AI innovation, and I’ll be asking her about Microsoft’s vision to democratise access to AI for ultimately empowering every individual to transform society with AI.

So I’m going to have a busy few days at the event. Check back in later this week for more news and deeper insights on these issues, and more!

 

 

 

 

Join me at Live! 360 Orlando to learn more about Applied Business Analytics

Posted on by

I’m delighted to be speaking at Live! 360 Orlando and I’m presenting a workshop on From Business Intelligence to Business Analytics with the Microsoft Data Platform. The conference is held on November 17-22, 2019 at the Royal Pacific Resort at Universal Orlando in Florida.

Data becomes relevant for decision making when we start to use it properly, so this workshop will demonstrate the use of analytics for real-life use cases. This means that you can make use of the knowledge you learn from the session when you go back to the office. In this workshop, we will make analytics relevant to your business so your organization will feel the benefit. If you’d like to register, please click here to register and please use code SPKLIVE87 and receive $400 off the standard pricing for both the 6-Day and 5-Day Packages.

Here are the topics we’ll cover:

  • Introduction to Analytics with the Microsoft Data Platform
  • Essential Business Statistics for Analytics Success – the important statistics that business users use often in business spheres, such as marketing and strategy.
  • Business Analytics for your CEO – what information does your CEO really care about, and how can you produce the analytics that she really wants? In this session, we will go through common calculations and discuss how these can be used for business strategy, along with their interpretation.
  • Analytics for Marketing – what numbers do they need, why, and what do they say? In this session, we will look at common marketing scenarios for analytics, and how they can be implemented with the Microsoft Data Platform.
  • Analytics for Sales – what numbers do they need on a sales dashboard, why, and what do they say? In this session, we will look at common sales scenarios for analytics such as forecasting and ‘what if’ scenarios, and how they can be implemented with the Microsoft Data Platform.
  • Analytics with Python – When you really have difficult data to crunch, Python is your secret Power tool.
  • Business Analytics with Big Data – let’s look at Big Data sources and how we can do Big Data Analytics with tools in Microsoft’s Data Platform.

Power BI and Marketing Data

What kind of applicable analytics will we cover? One important area in data is marketing. The primary aim of a Digital Marketing Analytics Report is to provide Digital Marketing Managers, Product Owners, and other relevant teams and stakeholders with a clear and easily-usable overview of the KPIs related to their campaigns. They should be designed in such a way that whoever is looking at them can interact with them in an easy, productive way. A Digital Marketing Analytics Report will generally have a wide range of visuals, of different levels of complexity, that display data in different ways.

The average attention span is only eight seconds. Therefore, it makes perfect sense to gather your data onto one canvas in a visually-compelling way, rather than relying on a lengthy PowerPoint or a cluttered Excel spreadsheet. Power BI is a hugely effective way of gathering and connecting to various data sources and using them to create a more engaging experience. 

When it comes to marketing specifically, there are some major advantages that come with processing your data in Power BI. Visualizing your data in a Power BI report lets you recognise patterns and trends that might have otherwise gone unnoticed if the data was in a text-based format. It is also an effective way to identify and create your company’s own best practices vs the common best practices of the industry; you can estimate traffic by creating a click-through-rate curve for your site, rather than just using the industry average.

Power BI can also make it easier to answer the questions that your clients and other stakeholders ask of you. Being able to explore and interact with live data means that, when you’re asked a question, rather than responding by saying “I don’t know, but I’ll get back to you” you can say “Let’s find out”. Not only does this mean that your client doesn’t have to wait for an answer, but it means that you are showing that you’re being honest and transparent about your data and how you analyse it.

You can also interact with your data in other ways — for example, it’s easy to isolate data from a specific time period in one visualization, and have this be reflected in the other visualizations on the Dashboard. You could select the spendings from a particular week, and have the KPIs from this week be highlighted, allowing you to better see the relationship between these two statistics.

Colour can also be used in a Dashboard to have certain effects, such as showcasing relationships clearly and drawing attention to the most important data, either overall or during user interaction. Colour can also be used to illustrate performance — for example, green statistics to represent above target, yellow for on target, and red for below target.

You can use a Power BI Dashboard to give an overview of impressions, campaign clicks, and average spend for a digital campaign. These statistics combined gives an overview of the effectiveness of and costs related to the campaign — seeing these statistics side by side can give a better idea of how they relate to each other.

Marketers are already learning how to use better tools for bigger datasets, and the environment is quickly evolving. Digital marketers working in eCommerce are experiencing over 45% increases year-on-year (1). People are becoming more digitally literate, and are learning how to search more effectively across more devices — often even beginning a purchase on one device and completing it on a different one. There are more search engines, more sites, and overall more options for users, and many users are responding to this by becoming better, more specific searchers. This means that, as marketers, instead of worrying about 1,000 keywords, you need to worry about 1,000,000. You need to have a much better idea of your marketing initiatives, click-throughs and engagements, and you need to convey this in a way that will be both understandable and engaging. 

If there are larger initiatives or recurring marketing programs that you need to track, it may be beneficial to setup specific reports, but in general, a Power BI dashboard is a great way to share your insights in a way that doesn’t end at a basic display of your data, but allows the data to be further investigated on-the-spot and when needed.

Join me in Live! 360 Orlando to learn more about Power BI and marketing, along with other topics.

Putting the Business back into Business Intelligence: reduce Data Debt by reconciling IT Department with Finance Professionals

Posted on by

It’s ironic that there are so many technical tools ways to communicate, there are still problems in achieving effective internal communication in companies. This is particularly evidence in relationships between Finance and IT. In this topic, the issues will be explored further  The issue with having so many technical tools is that the quantity of communication may increase, so this can give the appearance that communication is going well. However, the quantity of communication does not always reflect the quality of communication. IT and Finance concepts are difficult for non-specialists to understand, and messages may not be received as they were intended when working across different teams. It is hard for each department to see through the lens of the other department, and this issue can be further exacerbated if there is a culture of blame in place.

What is Data Debt?

Data debt is like technical debt, only visible in bad data, poor data quality, and an underuse of data-oriented technology to help make better decisions. It is the overhang of projects, whether they went well or poorly.

Technical debt is the implied cost of additional rework caused by choosing an easy (limited) solution now instead of using a better approach that would take longer. It also applies to data debt; the  cost of ignoring the data by choosing the quick or easy solution earlier at a previous point, which now takes longer to resolve.

At some point, however, we need to pay back data debt if we are to succeed in making better decisions.

Together, the issues all add up to make data debt, which is often a hot mess which nobody wants to resolve. Businesses can develop the Bystander stance; it is someone else’s problem.

What can you do about data debt?

Finance and IT need one another so that the whole organization can be successful, but there can be issues in communication and priorities that cause divisions. There can be seen very clearly in the sphere of business intelligence, where the business and IT need to work together to produce an essential component of the function of the organization: reporting, data and information flow.

Business Intelligence projects can inherit data debt from previous projects, which means that projects do not start from the starting line. Instead, the projects can start with a deficit before any progress can be made. In the long term, technical debt can exacerbate issues that exist between the Finance and IT departments. Take, for example, the situation where Finance don’t believe that their requirements and priorities are understood properly and that their voice isn’t being ‘heard’ in the IT strategy. From the Finance lens, this situation could be shown where IT won’t respond to support calls quickly enough, for example, or won’t allow the purchase of some new software. On the other hand, IT can hold the viewpoint that Finance are using uncontrolled data sources, such as Excel or Google Sheets spreadsheets. If something goes wrong with these artifacts, Finance team members involve IT to help fix something that they do not own, and inherit the technical debt for something that they did not create. This means that IT can end up owning ‘cottage industry’ artifacts, and this can create frustration. The ‘cottage industry’ artifacts arise partly as a result of Finance not getting the data or the software that they need from IT, so team members create their own data sources. The cycle of frustration between IT and Finance can continue without resolution, and since the pain isn’t owned by either department, there is no real ownership of resolving the issues. Instead, it simply gets absorbed into the business processes and it becomes part of everyday work. The real cost is never identified since it is a productivity cost, and it isn’t measured as stringently as direct costs that people can see or hold.

Conflicts between departments can arise out of differing priorities, but the overall vision should be the same for each department within the organization. As IT pushes forward into new projects, technical debt can be pushed to the side, simply to accumulate. For Finance, this issue can develop into unresolved conflict and ongoing productivity issues, as well as concerns over data governance and data veracity. The reality is that IT and Finance need to work together for the success of the whole organization. There are risks if the departments cannot work together. For example, if the departments cannot work together then this could put security at risk, for example, due to a failure to communicate successfully. Ultimately, this could land the company in court. Other consequences include an adversarial working environment which can make the company unable to recruit and retain key talent. In turn, this can impact technical debt as people leave without resolving it, and it just continues to accumulate. Although it’s hard to quantify, the difficulties in IT and Finance working together will cost the organization money, whether it is calculated in terms of lost productivity, or direct costs such as poor software selection. By contrast, if the teams can work together and work better, then both departments can increase their contribution to the success of the whole organization.

Blueprint for paying back data debt

How can the teams move forward from the impasse? We need to put the business back into business intelligence and move forward, working more collaboratively.  Fundamentally, this is an issue of how information can be tailored to different audiences who are specialists in their own field, and need to work together for a specific outcome that is seen across the organization.

Both Finance and IT departments need to work together to improve understanding and collaboration between each department. Brown bag lunch sessions, where team members present bitesize summaries of relevant concepts to both teams, can provide a good experience of working together for all team members. This may also help to break down stereotypes of the characterization of each department. Further, it can help team members to enlarge their tribe to include people from other teams, as well as their own team.

Finance and IT leaders need to work together and set an example to the rest of the organization. Personal conflicts between department managers is not acceptable, and the dispute will affect processes and practices within the company and disrupt the entire workflow.

Set up an informal wiki or blog, where team members can add every day. This blog can be used to identify the data and the information each department needs every day. By identifying key information, the wiki can be used to help understand each department better, and to share this crucial information in a straightforward, recorded way. This will help the teams to see the impact that they can have on each other, and identify and prioritize effort to maximise benefit.

Creating a community of excellence as a cross-department initiative that has representatives from each department working on joint initiatives, with progress reports and clear objectives. This will take some time to set up, and there may be resistance due to previous projects that have failed or not delivered.

Inherit technical and data debt. Every new IT project should inherit some technical debt from previous It work. This could involve data cleansing, or improved reporting, or the review of a support contract of cloud vendors.

Requirements can be hard to write and accept. One way to increase teamwork is to accept that prototyping is a valid way of developing and articulating what Finance need. Further, it means that IT can take on board It’s considerations, such as security, governance and upgrades. When developing, IT will want to prototype; at first, the efforts will seem awry but constant communication and feedback will be a good investment of time.

Conclusion

It isn’t going to be easy, but it is achievable if the example and direction is set at the Csuite level. Communication is key.

And finally, you can see the end of the data debt.