Customers want their data good, fast, cheap and easy. A tall order, right?
One of the biggest challenges that I see with data warehousing in the cloud is that customers are concerned with cost. I was interested to see the Gigaom report on the topic of Data Warehousing in the cloud, which contained a number of benchmarks, including cost.
The study by GigaOm showed that Azure SQL Data Warehouse is now outperforming the competition up to a whopping 14x times at up to 17 times cheaper than the competition, namely, Google BigQuery and AWS Redshift. This is an incredible achievement and the Azure team should be proud!
As part of my work in Business Intelligence, often, this involves a move to the cloud by default. Simply put, customers want quick Business Intelligence and they don’t want to spend time or effort in looking after kit. They want to delegate the responsibility. This means that cost is a key differentiator, since they want their data, good, fast and cheap. I’m glad to see that the Azure SQL Datawarehouse is competing on cost and performance since customers do want their data good, fast, cheap and easy.
Customers also want their data easy and this is where Power BI comes in. If a customer wants to use Power BI, I generally recommend that they put their data into Azure so that the data is traversing the Azure network. This means that the customer is not paying to extract or access their data from another cloud system and then put it into Power BI.
I’m excited about Power BI dataflows, since I believe it’s a great way forward for companies to sort out the issue that nobody likes to talk about: cleaning data. I’m passionate about this topic since I believe in inheriting technical debt as much as I can, so I leave the organization’s technical debt in a better place than I found it. I’m taking the long view when I de
I tried to connect my Azure Data Lake gen 2 storage to Power BI, and I ran into this error message:
”Your Azure storage account must be in the same Azure Active Directory tenant as your Power BI tenant.’
I checked, and my Azure storage account was definitely in the same Azure Active Directory tenant as Power BI. So I was confused. What a pesky error! So I decided to investigate.
It turns out that I had missed a step when setting up the connectivity between Azure Data Lake gen 2 storage account and Power BI. I had missed assigning the Reader role to the Power BI service on the storage account.
Microsoft’s instructions are here and make sure you follow them to the letter. I missed a step because I rushed through it, and then spent time trying to figure it out, so it was my fault since I scrolled past a step. I’m recording this issue here in case anyone else gets this error message, and wonders what’s going on.
December is a great time for setting personal and business goals for next year. I’d hoped for some downtime at this time of year, but I’ve picked up a crucial data science project which needs to be delivered over Christmas so there isn’t much downtime. I’m also starting to write another book, which will be my fourth published book, and that will require focus. So I will be working every day except Christmas Day, and that includes weekends as usual.
As part of my MBA program, I’ve done lots of strategic analysis modelling for businesses, starting with the default SWOT and a PESTLE analysis through to different models: Porter, Business Canvas Model, SOAR, and so on. I am not going to share here. The reality is, with Brexit, it’s more difficult to plan. Markets, and businesses, do not like uncertainty. Businesses, like countries, benefit in the longer term from open trade. Increasingly, global competition will occur on a more level playing field but there has to be open access for that to occur. Brexit is a plan for the right by the right, and it is stopping the open access that we enjoyed previously.
For what it is worth, here is my take on Brexit: I believe that May’s Remainy Brexit plan will never get through Parliament because it pretty much keeps us in the Customs Union. This means that the People’s Vote option will become increasingly likely. We are no longer talking about the big red bus lie and we see the chaos. If there is another vote, I hope that we will stay in the EU as we are now. So I am going to assume that all other things remain equal.
More than talking a good game
I was contacted me recently, by a former colleague who explained to me that someone had spoken to them about me. This ‘someone’ had told them that I was too small to deliver a project, that I didn’t know what I was talking about but I talked a ‘good game’, and that I would never deliver on time or on budget. The former colleague was so concerned about what they’d heard, that they invited me to lunch to explain the accusations which were stated behind my back. I was so horrified and fairly mystified; I had never worked with this individual and I didn’t really know them. I just asked for specific examples where I had just ‘talked a good game’ and not delivered on time or on budget, because I had no idea where they got their evidence from. It was all fairly alarming but of course the ‘someone’ hadn’t given any basis for their assertions at all.
In any case, the former colleague simply said that they would continue to work with me and they enjoyed working with me in the past; they had their own evidence to go on, and it is a mark of our good relationship that they talked to me about it at all. I was hugely relieved but I do wonder how often these tactics work, so I’ve decided to explain myself here and discuss this in terms of my goals.
So if you’re here because you’ve heard something similar, please let me know and we can have an honest discussion. I’ve got over twenty years as a Consultant, and I have postgraduate degrees plus industry experience in delivering Artificial Intelligence solutions. I’m writing my fourth book in Business Intelligence and I’ve edited another two. I have presented to five thousand people in the past month. So I do talk but I also deliver.
How was 2018 for you?
Data Relish Ltd was originally set up as Copper Blue Consulting, and was rebranded three years ago in order to make the offerings more clear. I have employed people in the past, and I had a business partner for a few years but that relationship is no longer in place. I do work for myself but note the following points:
Contracts and Partnerships
Current Status for 2018: I have partners who help me to deliver effectively. I have contracts in place with large partner organizations which allow me to scale; I have partnerships for 24 by 7 support, for example. Partnerships allow me access to sales, marketing and legal backup when I need it. Some of these partnerships are with Azure resellers and organizations, and I do not directly sell Azure so that my customers can benefit from the support that my partner organizations offer.
I am not releasing partner names here; I have a degree of concern, originating from the conversation with my customer, that there is a strong need for confidentiality on the basis that I’ve had the aforementioned stunt pulled on me. I am not giving any names away so that I inadvertently give detractors the opportunity to repeat their performance.
Customers
Current Status for 2018: 50% repeat customers with 50% new customers generated by word–of-mouth from previous customers
I am able to provide references on request from organizations of different sizes, verticals and global impact. Some of my customers are multi-billion turnover, right down to startups. I don’t make a huge effort to market myself because all of my work comes from word-of-mouth. Since I am so dependent on word-of-mouth marketing, it can mean I’m susceptible to stunts like the one mentioned above since I don’t have the flashy marketing that might give some people comfort. I have started to look more closely at marketing and I have set up a partnership with a small consulting firm to help me. It works well since they give me help, but I also pass on leads that I receive, and we have jointly delivered projects on occasion.
Business Community
Current Status for 2018: I started more business networking last year but it tailed off in February. Reason: I was not picking the right events well enough and I got disheartened.
I have started to attend Chamber of Commerce events again, and I met with David Gauke MP (Minister for Justice) and Richard Harrington MP (Minister of Business and Industry) this year. I am interested in learning from the local business leaders here in Hertfordshire, and also looking at ways that I can feed back into the business community. Networking is important and it builds trust. Additionally, being part of the local Chamber groups is offering the route of having our voices heard jointly in Government, and I was surprised to learn that our local MPs have regular and frequent meetings with Chamber representatives. Business is crucial to the success of society by keeping cashflow moving, providing jobs and stability and homes. I advise businesses strategically because I believe in putting the business back into business intelligence, and at the front-and-centre of artificial intelligence efforts.
Charity
I was a Data Ambassador for DataKind this year. Wow! What a rewarding, creative experience and it was incredibly satisfying to work with people who love data and want to do good with data science to help charities.
Current Status for 2018: I got involved with DataKind. You know who you are and I love you all; you have brought such knowledge, fun and great company to my life and you shine your light far out. You cancel out the darkness where people behave as in the anecdote above. Thank you for your healing.
So what are my Goals?
Here is the list of goals for 2019:
Contracts and Partnerships
Goals for 2019: I have partners who help me to deliver effectively and I will continue to honour the trust that they have put in me.
In 2019, I do not intend to employ people now since it takes me away from doing things that I love. I may do so in 2020 but it is not the right thing for me at this time. I have previously lost sleep over making people redundant and I do not want that burden again. I work on projects, often, that are simply too big for me to run out and hire a whole team of people and it would be madness to try. How could I even support things like 24 hour support? These are big projects and I need help quickly. Customers need help with AI, BI and Data Science leadership. So it works well.
Customers
I have done extremely well this year and I am proud of my customer list. Through partnerships and on my own, I have punched well above my diminuitive size to have a customer list where my customers have billions and billions in turnover and it is a mind-boggling amount when I add up the turnover amount of all these customers combined.
I do help them to be even more successful and I do help them make technology choices as part of the vision and strategy that I help them to formulate and devise.
Goal: More repeat custom and generate new customers. 50%/50% split between regular customers and new customers is a good split, and it worked well for me this year. As I lead customers to success, they drop off my customer list because they don’t need me any more; but that’s what I want! Self-sufficient, satisfied customers who are enabled and empowered to move on, and who recommend me to their colleagues and community.
Business Community
I am going to set myself the target of attending more business events this year. I am currently researching the business events that have value, and the ones that do not offer value to me.
Goal: I intend to re-ignite my business networking in a more targeted way. I have been pleased to meet leaders that I can learn from, and i hope to contribute more.
Charity
I was a Data Ambassador for DataKind this year. Wow! What a rewarding, creative experience and it was incredibly satisfying to work with people who love data and want to do good with data science to help charities.
Goal for 2019: More of the same. I get enormous richness from philanthropic efforts and I want to help people.
I wish you all the best for 2019! And don’t let anyone steal your pixie dust.
I was very pleased to appear on the Izenda website along with five other Business Intelligence experts, discussing the past, present and future of self-service Business Intelligence. I was delighted and honoured to appear with luminaries such as Wayne Eckerson, John Myers, Kevin Smith, Rich Ghiossi, and Ron Powell.
Self-service Business Intelligence is a much larger topic than you might think, and it’s clear that some organizations who market themselves as ‘self-service’ aren’t really meeting the criteria. I recommend that you head over to the post in order to read it all. I’m interested in the idea of self-service analytics as well as self-service business intelligence, and I do think that will become increasingly relevant as the industry matures.
Thank you to Izenda for having me along. Please let me know what you think; I look forward to your comments.
I’m holding a series of training courses around the UK, more details will be published. In the first instance, on 15th September, I’ll be holding a day-long practical workshop on Working with Business Data for Busy Executives in SME Organisations in Hertfordshire, England. The cost will be £100 pounds plus VAT, food and workshop materials included, and you can also network and share experiences with other attendees who will also be running businesses, like you.
I don’t believe in a ‘stack ’em high’ approach, which doesn’t give a pleasant experience for learning. So, classes will be restricted to 12 people only, unless otherwise stated. This means that you will get a good amount of attention.
I’m doing the Executive MBA at the University of Hertfordshire Business School, I’ve also been a NED (Non Executive Director) for PASS, who are based in the United States. As I’ve been spending time leading organisations, I’m keen to share this knowledge and expertise with the community from a data-driven, data leader perspective. The following blog post will give you a flavor of the workshop. along with some of my thoughts on Microsoft Data Insights Summit. If you have any questions, please pop them in the comments box and I’ll read them from there.
Here are my slides from Microsoft Data Insights Summary, combined with some of the slides from the keynote by James Phillips, held in June 2017.
For those of you who know me, you’ll know that I have extensive experience in Tableau as well as Power BI. However, most of my consulting data visualisation is in Power BI suite of products. Why is that?
Tableau is wonderful at data visualisation, as is Power BI, of course. However, for enterprise customers, where I’m building a data warehouse, I prefer having analytics closer to the data source, perhaps in a data warehouse or data lake. I like to think about the overall business intelligence architecture. Tableau is superb at data visualisation and it also cleans and integrates data, but to a much lesser extend, which is why they partner so well with Alteryx. I don’t like cleaning data or doing repeatable analytics so close to the end reporting layer and business people seem to want to do it there, without thinking of issues such as robustness, repeat-ability and longevity in the analytical formula that they are creating. I prefer to hand off clean data and analytical formula to the reporting tool as far as possible.
I’m not thinking about Business Intelligence in terms of a spot solution for data visualisation or reporting, for example. I’m looking at the whole canvas. I prefer to clean the data and have it all fixed closer to the source, so that I can get the same number for the same report, regardless of the reporting technology that I use. With Power BI, I can stay within the Microsoft playpen of technologies. I do note however that Tableau Server is in Azure and if you are looking at analytics, that’s another option so that the analytics formula isn’t contained in disparate workbooks. Instead, they are published to Tableau Server and people can should download their workbooks there, for ‘one version of the truth’.
As an external consultant, I work with Power BI because I think it has an astonishing reach technically as well as geographically. Some of my customers are global and I really need the certainty of global resiliency.. Gone are the days when Microsoft had a lot of disparate reporting technologies that didn’t talk to one another very well and we had lots of different interfaces that used to overlap. Customers got really confused about what to use. For example, do you put your KPIs in Analysis Services, or in Reporting Services? Now:
The answer is always Power BI! Take a look:
Apart from Data Visualization, what is Power BI useful for?
Power BI is particularly useful for:
businesses that are acquiring other businesses and they need somewhere to put the data, and keep the business running in the meantime
cost savings
GDPR – if you don’t know what this is, you need to contact me to find out more. Microsoft are in the forefront of working with customers to make sure that they are compliant.
Do I still see Tableau?
Yes – some of my customers don’t need public cloud because they pop up their own data centres if and when and where they need them. So, for them, they tend to stick with what they know, and what works for them.
What Business Intelligence tools do I see less of?
I see Qlikview less and less, as customers look to align their reporting and they can replicate their Qlik scripts in SQL Server and SSRS.
I also don’t see Pyramid Analytics appearing much, and I don’t get asked often about them. According to the Gartner report, 2017 may represent a critical period for the company and, rightly or wrongly, the Gartner Magic Quadrant does carry enormous weight when customers are looking for solutions. With many solutions, customers don’t use the full range of features contained in any technical solution, and Pyramid are going to have to work hard to explain how they compare / compete with the Power BI on-premise solution, which is going to go from strength to strength.
However, for others, particularly in the SME market, the Azure offering is extremely compelling. Power BI and Azure together mean that you can focus on the business, rather than working on the technology to support the business. I can also see that more and more data is going into the cloud, and I am part of projects where I am doing exactly that – cloud business intelligence. Cloud Business Intelligence is a real growth offering for me and I plan to keep being ahead of the curve.
Are people using Power BI or is it simply good Microsoft Marketing?
People are using it, yes. Here are the numbers, produced by James Phillips during the Power BI Keynote:
Power BI and the C-Suite
Given it’s reach within the organisation, Power BI can reach the C-suite level as well as the rest of us, in the organisation. Before continuing, it’s probably worth reading about linear vs exponential business growth models e.g. HBR.
You can watch the video below, or read on for some of the headlines:
Here are some headlines:
Gross and Net Profit
Net profit
Progress Towards Targets
Revenues and revenue growth rate
Expenses
Employee Engagement
Let’s get started!
Gross and Net Profit
Why do CEOs care? As part of the Digital Transformation process, the CEO must develop a guiding philosophy about how he or she can best add value whilst showing ongoing strategic assessment and planning. However, it is difficult for them to allocate time to the collection, cultivation and analysis of data. Instead, they need to focus on strategic decisions, and they need data to run their business, to understand how their customers behave and measure what really matters to the organization. Power BI can help to bring clarity and predictability to the CEO, and this session is aimed at CEOs, and those who support them with data, in order to see how they can be empowered by Power BI, and see it as a key asset within the organisations short and long term future.
Net Profit
This goes without saying, but keeping an eye on net profit at all times is essential for business leaders. This might be visualized as a line graph or quarterly chart. However you decide to represent the data, it needs to provide detailed, regularly updated information. You can get added value by allowing this data to be broken down.
With Power BI, you’d be able to tap your chart and see real-time data on profits by region, product type or team. First you calculate your gross profit, then your expenses, subtract expenses from gross profit, and you have net profit.
Calculate Gross Profit first:
Gross profit, also called gross margin, shows you how much money you made from selling a product.
It subtracts the selling price from your wholesale cost to calculate the difference. It does not take into account expenses from rent, personnel, supplies, taxes or interest. Gross profit is a required step toward calculating the company’s income or net profit.
Progress Towards Targets
You can use EXPON.DIST function in Microsoft Excel to help measure progress towards your targets.
Use EXPON.DIST to model the time between events, such as how long from the order placement takes to actual delivery. For example, you can use EXPON.DIST to determine the probability that the process takes at most 1 minute.
Revenues and revenue growth rate
By being able to instantly visualize how fast (or otherwise) your business is growing its revenues, it’s much easier to find out what’s going right and what’s going wrong. Need to lose some dead weight? Invest in a growing department? Respond to a new trend among consumers? Tracking your revenues closely is crucial and will help with those decisions. A line graph would again be particularly clear in this instance.
Think of a Rubik’s cube – people instinctively know how to use them, and to arrange the cube into colors. We also interact with colour and data in the same way; intuitively and quickly.
Expenses
Whether it’s staff, machinery, IT or property, your expenses are one of the biggest drains on your long term success. A dashboard can break these down instantly so you can see where your biggest outgoings are, and then make decisions about what’s costing too much.
Revenue per employee
Revenue per employee is a little like Return on Investment. Are your people actually making enough revenue to justify hiring them? Are they working at 100% capacity or is there room for them to work more, instead of employing new workers? A revenue per employee dashboard helps you make these choices rationally.
Employee Engagement
Measured by an anonymous survey, employee engagement is a key BI factor for any CEO. If your people are motivated, enthusiastic and giving their work 100%, you can be sure your company will grow. By contrast, unengaged colleagues will be a detriment to productivity. It’s essential to keep regular tabs on how employees are feeling about their work.
Summary
As part of the Digital Transformation process, the CEO must develop a guiding philosophy about how he or she can best add value whilst showing ongoing strategic assessment and planning. However, it is difficult for them to allocate time to the collection, cultivation and analysis of data. Instead, they need to focus on strategic decisions, and they need data to run their business, to understand how their customers behave and measure what really matters to the organization.
Power BI can help to bring clarity and predictability to the CEO, and this session is aimed at CEOs, and those who support them with data, in order to see how they can be empowered by Power BI, and see it as a key asset within the organisations short and long term future.
As a consultant, I get parachuted into difficult problems every day. Often, I figure it out because I have to, and I want to. Usually, nobody else can do it other than me – they are all keeping the fires lit. I get to do the thorny problems that get left burning quietly. I love the challenge of these successes!
How do you get started? The online and offline courses, books, MOOCs, papers, blogs and the forums help, of course. I regularly use several resources for learning but my number one source of learning is:
Doing the ‘do’ – working on practical projects, professional or private
Nothing beats hands-on experience.
How do you get on the project ladder? Without experience, you can’t get started. So you end up in this difficult situation where you can’t get started, without experience.
Volunteer your time in the workplace – or out of it. It could be a professional project or your ‘data science citizen’ project that you care about. Your boss wants her data? Define the business need, and identify what she actually wants. If it helps, prototype to elicit the real need. Volunteer to try and get the data for her. Take a sample and just get started with descriptive statistics. Look at the simple things first.
Working with dat means that you will be challenged with real situations and you will read and learn more, because you have to do it in order to deliver.
In my latest AzureML course with Opsgility, I take this practical, business-centred approach for AzureML. I show you how to take data, difficult business questions and practical problems, and I show you how to create a successful outcome; even if that outcome is a failed model, it still makes you revise the fundamental business question. It’s a safe environment to get experience.
So, if this is you – what’s the sequence? There are a few sequences or frameworks to try:
TDSP (Microsoft)
KDD
CRISP-DM
SEMMA
The ‘headline’ of each framework is given below, as a reference point, so you can see for yourself that they are very different. The main thing is to simply get started.
This girl! I am super excited and please allow me to have one little SQUUEEEEEEE! before I tell you what’s happening. Now, this is a lifetime achievement for me, and I cannot begin to tell you how absolutely and deeply honoured I am. I am still in shock!
I am working really hard on my demo and….. I am not going to tell you what it is. You’ll have to watch it. Ok, enough about me and all I’ll say is two things: it’s something that’s never been done at PASS Summit before and secondly, watch the keynote because there may be some discussion about….. I can’t tell you what… only that, it’s a must-watch, must-see, must do keynote event.
We are in a new world of Data and Joseph Sirosh and the team are leading the way. Watching the keynote will mean that you get the news as it happens, and it will help you to keep up with the changes. I do have some news about Dr David DeWitt’s Day Two keynote… so keep watching this space. Today I’d like to talk about the Day One keynote with the brilliant Joseph Sirosh, CVP of Microsoft’s Data Group.
Now, if you haven’t seen Joseph Sirosh present before, then you should. I’ve put some of his earlier sessions here and I recommend that you watch them.
I hear you asking… what am I doing in it? I’m keeping it a surprise! Well, if you read my earlier blog, you’ll know I transitioned from Artificial Intelligence into Business Intelligence and now I do a hybrid of AI and BI. As a Business Intelligence professional, my customers will ask me for advice when they can’t get the data that they want. Over the past few years, the ‘answer’ to their question has gone far, far beyond the usual on-premise SQL Server, Analysis Services, SSRS combo.
We are now in a new world of data. Join in the fun!
Customers sense that there is a new world of data. The ‘answer’ to the question ‘Can you please help me with my data?‘ is complex, varied and it’s very much aimed at cost sensitivities, too. Often, customers struggle with data because they now have a Big Data problem, or a storage problem, or a data visualisation access problem. Azure is very neat because it can cope with all of these issues. Now, my projects are Business Intelligence and Business Analytics projects… but they are also ‘move data to the cloud’ projects in disguise, and that’s in response to the customer need. So if you are Business Intelligence professional, get enthusiastic about the cloud because it really empowers you with a new generation of exciting things you can do to please your users and data consumers.
As a BI or an analytics professional, cloud makes data more interesting and exciting. It means you can have a lot more data, in more shapes and sizes and access it in different ways. It also means that you can focus on what you are good at, and make your data estate even more interesting by augmenting it with cool features in Azure. For example, you could add in more exciting things such as Apache Tika library as a worker role in Azure to crack through PDFs and do interesting things with the data in there. If you bring it into SSIS, then you can tear it up and down again when you don’t need it.
I’d go as far as to say that, if you are in Business Intelligence at the moment, you will need to learn about cloud sooner or later. Eventually, you’re going to run into Big Data issues. Alternatively, your end consumers are going to want their data on a mobile device, and you will want easy solutions to deliver it to them. Customers are interested in analytics and the new world of data and you will need to hop on the Azure bus to be a part of it.
The truth is; Joseph Sirosh’s keynotes always contain amazing demos. (No pressure, Jen, no pressure….. ) Now, it’s important to note that these demos are not ‘smoke and mirrors’….
The future is here, now.You can have this technology too.
It doesn’t take much to get started, and it’s not too far removed from what you have in your organisation. AzureML and Power BI have literally hundreds of examples. I learned AzureML looking at the following book by Wee-Hyong Tok and others, so why not download a free book sample?
How do you proceed? Well, why not try a little homespun POC with some of your own data to learn about it, and then show your boss. I don’t know about you but I learn by breaking things, and I break things all the time when I’m learning. You could download some Power BI workbooks, use the sample data and then try to recreate them, for example. Or, why not look at the community R Gallery and try to play with the scripts. you broke something? no problem! Just download a fresh copy and try again. You’ll get further next time.
