Microsoft Ignite interview with Kevin Farlee on Azure SQL Database Hyperscale

Azure SQL Database is introducing two new features to cost-effectively migrate workloads to the cloud. SQL Database Hyperscale for single databases, available in preview, is a highly scalable service tier that adapts on demand to workload needs. It auto-scales up to 100 TB per database to significantly expand potential for app growth.
What does this mean? It’s one of the most fundamental changes to SQL Server storage since 7pm. So this is big: big news, and very big data stores. I am very lucky because I got to interviewe Kevin Farlee of the SQL Server team about the latest news, and you can find the video below.

I am sorry about the sound quality and I have blogged so that the message is clear. When I find the Ignite sessions published, I will add in a link as well.
What problem are the SQL Server team solving, with Hyperscale? The fundamental problem is how do you deal with very large databases in the cloud. VLDBs is the problems that people want to do with normal operations. All the problems with VLDBs occur due to the sheer size of data, such as backups, restores, maintenance operations, scaling. Sometimes these can take days to conduct these activities, and the business will not wait for these downtimes.  If you are talking tens of terabytes, that takes day and ultimately Microsoft needed a new way to protect data and VLDBs. The SQL Team did something really smart and rethought very creatively on how they do storage, in order to take care of the issues with VLDBs in the cloud.
So, the Azure SQL Server team did something that is completely in line with one of the main benefits and key features of cloud architecture: they split out the storage engine from the relational engine. Storage implementation was completely rethought and remastered from the ground up. They took the viewpoint over how you would go about architecting, designing and building for these solutions in the cloud, if you were to start from scratch?
The Azure SQL Server database team did a smart thing: Azure SQL Server is using microservices to handle VLDBs.
The compute engine is one microservice which is taking care of it’s role, and then another microservice that is taking care of the logging, and then a series of microservices that handle data. These are called page servers, and they interface at the page level. The page servers host and maintain the data files. Each page server handles about a terabyte of data pages. You can add on as many as you need.
Ultimately, compute and storage are decoupled so you can scale compute without moving the data. This means it’s possible to keep adding more and more data, and it also means that you don’t have to deal with the movement of data. Moving data around when there are terabytes and terabytes of data isn’t a trivial task. The page servers have about a terabyte of data each, and the page servers have about a terabyte’s worth of SSD cache.
The ultimate storage is Azure Blob Storage, because blob storage is multiply redundant and it has features like snapshots, so this means that they can do simultaneous backups by just doing a snapshot across all of the blobs. This has no impact on workload.
Restores
Restores are just instantiating a new set of writeable disks from a set of snapshots, and works with the the page servers and the compute engine to take care of it, working in symphony. Since you’re not moving the data, it is faster.
I’m personally very impressed with the work that the team they’ve done, and I’d like to thank Kevin Farlee for his time. Kevin explains things exceptionally well.
It’s worth watching the video to understand it. As well as the video here, Kevin goes into detail in his Microsoft Ignite sessions, and I will publish more links when I have them.
Community
One advantage in doing the MIcrosoft Community Reporter role is that I get to learn from the experts, and I enjoyed learning from Kevin throughout the video.
It seems to me that the Azure SQL database team have really heard the voice of their technical audience and they’ve worked passionately and hard to tackle these real life issues. I don’t know if it is always very clear that Microsoft is listening but I wanted to blog about it, since I can see how much the teams take on board the technical ‘voice’ from the people who care about their solutions, and who care enough to share their opinions and thoughts so that Microsoft can improve their solutions.
From the Azure architecture perspective, it works perfectly with the cloud computing concept of decoupling the compute and the storage. I love watching the data story unfold for Azure and I’m excited by this news.

Why I changed my mind about #PowerApps and became a fan… and why you should, too

This blog post title was initially titled ‘Why I disregarded PowerApps but now I radically changed my mind’. I changed my mind and I now think that PowerApps is a very powerful tool and basically it does a lot of the app form functions that Access does but way better. I get it now and I want to explain how I changed my mind about PowerApps.

PowerApps performs the same function as Access can do: building the business apps you need and extend or customize the apps you already use. Only, PowerApps does it way better because it has the robustness, scalability, security and performance of the Azure cloud with Office365 inbuilt in the app from the start.

In this blog post, I’ll set out where I think PowerApps fits in the business, and why it’s way better than Access. There are plenty of PowerApps templates but I hope that this post will convince you to give it a try.

TL;DR – I care about producing business-friendly solutions that are maintainable, robust, perform well, have potential for scale, and that I can hand off to an IT support team. PowerApps gives me these things. Access does not because it is not always backed up, or placed on a network so people can update it, and it is not always secured properly. So you can think about migrating your Access apps to Power Apps, for example. In this blog post, we will answer the following questions:

  • What can PowerApps do for you?
  • How do you get started with PowerApps?
  • What I’d like to see PowerApps do next

contact-icon
Any questions, please let me know or get in touch to schedule some time to talk further.

 

What can PowerApps do for you?

PowerApps allows you to design, create, use and share custom business apps with your team and organization. PowerApps installation means that you can use apps that have been shared with you, and it also means that you can create your own apps.

What kind of apps can you create? Intelligent laziness is very important! The automation of tasks means that you can focus on the activities that are more meaningful and add more value to the organization. If you are leading a team, then it means that you and your team can work wherever you happen to be, on any device.

What kind of apps can you create? Many Access forms could be redone in PowerApps since they involve data entry. Here are some examples:

  • Connect apps to your existing data, such as SharePoint, Salesforce, Dropbox, Google Drive and more
  • Create and update customer invoice, receipt and order forms
  • Create Helpdesk apps
  • Create Expense apps

As a BI person, I have spent time curing businesses from cottage industry Access databases and Access forms. Often, in my experience of rescuing Access databases and forms, I find that they proliferate through the business, usually unmonitored, untested properly and aren’t always production ready. They are often built in an ad-hoc fashion. In short, when I find some Access, I perform an internal sigh and I immediately start to mastermind it’s replacement. Access is often a the sticking plaster to solve a genuine business problem and it is quick to produce something quick. As Brent Ozar puts it, Access has Play-Doh factor. Yes, it’s an old post from Brent Ozar ( b ¦ t ) but it is still holding true.

play-doh-3308885_1920

So when I saw PowerApps at first, my reaction was ‘Oh no, it’s the new Access’. My initial vision was that developers would hate it, and I’d spend my life running around chasing after apps as well as Access to make them production ready.

So here’s what I found:

PowerApps has the IT guardian role built in

PowerApps solves the Access problem of scaleability, robustness and performance. PowerApps is built in Azure as part of Office365 so that means that Azure takes care of these factors for you. For me, this is a massive relief because I care about producing solutions that are maintainable, robust, perform well, have potential for scale, and that I can hand off to an IT support team. When I leave site, I want to leave with a job well done. I don’t intend to be there forever; it’s not what I do. So I like to enable others and bring people on a journey that they can continue without me. So what does PowerApps mean for the small business?

 keyboard-2553530_1920

PowerApps for the Small Business 

PowerApps is robust and secure. If you are using Access or Excel to run your business, then you should consider PowerApps. It provides a much more robust way for small companies to track data and projects than Excel or Word because it is in the cloud so you don’t need to worry about losing or screwing up your Excel spreadsheet of inventory, orders and so on. Also, it means that you can be more confident that your data is secure because it is in the cloud, and not on a USB.

PowerApps is mobile. It is also an easy way to mobilise apps as well. The functionality is built in. This is another big win over Access, in my book. This means that PowerApps has the most value added for tracking your organization in terms of projects, budgets, and growth.

PowerApps lets IT do their job more easily. PowerApps will give your IT people an easier way to manage your business-built apps rather than having them hidden on a laptop somewhere. PowerApps lets IT do their job, of being the guardian of the data and the IT function. It’s easy to maintain, which is idea for the organic business user turned power user turned ad-hoc techie. The idea of creating databases and trying to maintain them seems like an unnecessary use of resources; however, for small businesses, PowerApps creates a database for you so you can focus on what you need to do for your organization: managing growth and delivering success.

PowerApps has Play-Doh factor but in a good way. Creating apps in PowerApps allows a business to customize its own apps, thereby streamlining business processes. For example, you can use it to track crucial business information for each customer, including contact information, addresses, order information, invoices, and payments. Since it is in the Azure cloud, team members can access and update it, so the information can stay current and secured. Since the frm is a central idea to Power Apps, this helps small businesses ensure that employees enter data accurately and consistently so your data is cleaner at source.

How do you get started with PowerApps?

At a high level, here are the steps:

  • Choose from professionally designed templates to start quickly
  • Start with a blank canvas, and customize to suit your needs
  • Add business logic and intelligence using the power of Excel-like expressions
  • Create interactive visuals and media to build unique, professional apps

PowerApps is an easy jump for the BI Developer who already knows DAX. To create more custom apps, PowerApps has some object-oriented ‘formula’ that you can change, configure and set. It also has intellisense so you are guided to program correctly.

PowerApps is constantly being updated. For information about what’s new in this release, please see this topic: https://aka.ms/powerapps-releasenotes

Accessing the Azure Wordpress Installation using FTP

Do you need to do some back-end work to secure your WordPress site on Azure? If so, you will need to do some back-end work using an FTP facility, such a FileZilla.

You will need:

  • ftp site
  • username
  • password

 

So let’s get to it!

Finding the FTP and FTPS site details for Azure WordPress

The FTP URL will look something like this:

ftp://waws-prod-aa10-000.ftp.azurewebsites.windows.net

You can find it by searching for the Diagnostics Logs section of the App service. Can you see the sections for FTP and FTPS? They are marked in red.

Find FTP and FTPS details

Finding the User Name and Password

Again, you can find these from the Azure portal. Go to the App service, and look for Deployment Credentials. You can reset them here, and use those details as the FTPS username and password.

Deployment Credentials

 

Using FileZilla

I use FileZilla Pro because it connects to Azure, Google Cloud, Amazon and other sources as well. It’s not expensive and I like having everything in one place.

Here is a sample of the Site Settings in FileZilla.  The FTP URL goes into the Host settings. You can leave out the Port number. The FTP User goes in the User Box, and the Password goes into the Password box.

FileZilla

After that, you should be good to go. I hope you manage to fly with WordPress in Azure!

 

Fixing ‘Could not create Directory’ in Azure WordPress installation of Elementor Pro

I have ported my Data Relish company website over to Azure. It isn’t finished yet, and I’m making the final touches. I’m using WordPress on Azure and it’s been good experience in understanding how the moving parts of Azure, Office 365 and WordPress all hang together.

In trying to update Elementor Pro, I got the following error message:

An error occurred while updating Elementor Pro: Could not create directory. elementor-pro/modules/assets-manager/asset-types/fonts

After some digging around, it turns out that this is due to the length of the directory name. The long filename was causing an issue, and sub-directories increased the length of the filename so they simply could not get created.

In order to sort the issue, you have to use the Windows Compatibility Fix

Once the Fix is applied, it’s possible to create directories, and the upgrade to the latest edition of Elementor Pro should proceed as expected.

I hope that helps you to fly farther with WordPress!

Being a Microsoft Regional Director: faith, trust and pixie dust for good

I’m still learning about being a Microsoft Regional Director and I’m figuring things out. I’d like to thank Microsoft here for this opportunity and I’d like to thank the great RD team at Microsoft for their seemingly-endless patience with my questions!

Here is my opinion. I don’t represent anyone other than myself, and this is in no way official. I am extremely honored to be a Regional Director and an MVP and I think that the RD role is worth exploring further. This is just an opinion, and that I’m still learning about the RD role since I am new to it. I might add that i’m still figuring out being an MVP as well. Actually, I’m still wondering what job I want to do when I grow up!

Let’s take an example. Recently, an email popped up in my mailbox from a senior executive and decision maker, who asked for a hiring strategy for Azure team members and a commentary about POs for Azure, including Power BI. So I made a huge impact at that customer site, which was a large organization and a big ship to steer around. In fact, it takes faith, hope and a little dash of pixie dust as well as joining hands with the team in order to make the jump in digital transformation; people, processes and technology. And then, I rinse and repeat at other organizations so that everyone has a good leap of faith in the direct direction.

Recently I was on the BBC, talking about a different client where I am helping with a data science for good project, which focuses on homelessness and other aspects of social care. I’ve put the video here, in case you’re interested:

You probably think that any one-woman-band projects mean much, but they do. In fact, it’s huge. I have been working with the first client for months, on and off, combining my time with other customers in an ad-hoc way. I am convinced that Azure is the right solution and the role was born out of the roadmap to Azure that I had worked with them to produce, as part of a larger strategy piece; and it’s just the first role and more will be added later.

For the second customer case, the work we are doing, using Microsoft technologies, is going to have a good impact on people’s lives. The data overrides your perceptions. When we think of homeless people, we think of the tramps on the street, right? Wrong. What about victims of domestic violence, who become victims of unexpected homelessness because they are in fear for their lives? What about their children? That’s how hard people have it in their lives, and in the tech world, we are so blessed, often. What are we complaining about, really?

I don’t think that the RD role or the MVP role are sales roles at all. I don’t benefit financially from these recommendations. I am entirely independent and, if I recommend a solution, it’s because I believe that it is the correct solution.

So I think an RD is partially about having that strategic impact that Microsoft can really see and feel, in a good way. There will be nothing to tie me to the purchase of Azure at all, because I didn’t receive anything and I don’t sell Azure, and I didn’t make anything from the sale. I’m an independent consultant so I get paid for my time, not the fruits of my recommendations. But people will feel the results; the new hires, for example.

So I think an RD is partially about having that strategic impact that other people can really see and feel, in a good way. In these digital transformation pieces, I’m making people’s work easier for them through better processes, great technology and mentoring, supporting and helping people. For the work I’m doing in data science for good, I’m using Microsoft Data Science technologies as part of an amazing, amazing team who are doing great things and making people’s lives better. I think that is it, really: about using your pixie dust to do good things. It’s not about ‘bigger is better’ – bigger business, higher github admissions, higher turnover, larger number of hires, bigger number of Azure VMs, bigger number of forum answers or bigger profile on Stackoverflow. I think it’s about having the same pixie dust as anyone else, but throwing it liberally on the right things.

Rule your mind or it will rule you – Buddha

I think it’s about personal growth. It’s also about striving to have a maturity of outlook and a cool head, and I am trying very hard to heal and be the clean person I’d like to be. I’m doing my MBA and it’s all about personal growth and development. It’s unlike any other course I’ve done, since it means I get really hard feedback about myself as a person as well as my work. Some of the feedback is great, and other feedback is uncomfortable and provokes cognitive dissonance, but the self-honesty means that I can work on it through reflexive and reflective leadership techniques. For example, I’ve written before about having Imposter Syndrome but now I am learning to watch my thoughts better (mindfulness and my Buddhist journey) which means I’m starting to understand better if it really is Imposter Syndrome, or perhaps it’s a reality check, or perhaps I am just being silly? I have grown so much in the past few years and my Buddhist journey tells me that I have a long way to go.

When others go low, you go high

Kirk D Borne, who is an immensely insightful gentleman, asked me a deceptively simple question: what does this actually mean for you? I’d like to thank Kirk here because his generous and insightful question provoked me to think of  for days. I love it when someone challenges me with a wise question and one that I hadn’t considered before, which was kind of the point! I’ve decided on an answer: what being an RD means for me is the opportunity to network, learn and share with people who are brilliant, mature, optimistic, knowledgeable, willing to share freely and with no reward in it, know when to speak and when to stay silent, experienced in business and in the tech sphere. I’m with a great set of people who I admire.

Accountability

Accountability is a very tough thing to learn and it’s something that I ask myself every day: who is accountable? Professionally or personally, you can’t shrug off personal accountability. To lead by example, you have to be accountable, which means that people can have faith and trust in you.

It’s about people you can have faith and trust in, and striving to be that person. The RD program inspires me to work towards being all of these things and to consider accountability.

It also means that I am working to make sure that nobody steals my pixie dust. Michelle Obama inspires me here: when others go low, I go high. Words to live by!

Don’t let anyone steal your Pixie Dust

Following on from accountability, it’s about being an authentic you and striving to be a better  you. On my office wall, I have a picture of Tinkerbell, as follows.

pixiedust

My onboarding to the RD Program has been incredible and people outside and inside of Microsoft have been amazing. So I’d like to thank everyone who has congratulated me and I can promise that I will do my best.

Modelling your Data in Azure Data Lake

One of my project roles at the moment (I have a few!) is that I am architecting a major Azure implementation for a global brand. I’m also helping with the longer-term ‘vision’ of how that might shape up. I love this part of my job and I’m living my best life doing this piece; I love seeing a project take shape until the end users, whether they are business people or more strategic C-level, get the benefit of the data. At Data Relish, I make your data work for different roles organizations of every purse and every purpose, and I learn a lot from the variety of consulting pieces that I deliver.

If you’ve had even the slightest look at the Azure Portal, you will know that it has oodles of products that you can use in order to create an end-to-end solution. I selected Azure Data Lake for a number of reasons:

  • I have my eye on the Data Science ‘prize’ of doing advanced analytics later on, probably in Azure Databricks as well as Azure Data Lake. I want to make use of existing Apache Spark skills and Azure Data Lake is a neat solution that will facilitate this option.
  • I need a source that will cater for the shape of the data…. or the lack of it….
  • I need a location where the data can be accessed globally since it will be ingesting data from global locations.

In terms of tooling, there is always the Azure Data Lake tools for Visual Studio. You can watch a video on this topic here. But how do you get started with the design approach? So how do I go about the process of designing solutions for the Azure Data Lake? There are many different approaches and I have been implementing Kimball methodologies for years.

cellar

With this particular situation, I will be using the Data Vault methodology. I know that there are different schools of thought but I’ve learned from Dan Lindstedt in particular, who has been very generous in sharing his expertise; here is Dan’s website here. I have delivered this methodology elsewhere previously for an organization who have billions USD turnover, and they are still using the system that I put in place; it was particularly helpful approach for an acquisition scenario, for example.

 

Building a Data Vault starts with the modeling process, and this starts with a view of the existing datamodel of a transactional source system. The purpose of the data vault modelling lifecycle is to produce solutions to the business faster, at lower cost and with less risk, that also have a clear supported afterlife once I’ve moved onto another project for another customer.

 

Data Vault is a database modeling technique where the data is considered to belong to one of three entity types: hubs, links,and satellites:

 

  • Hubs contain the key attributes of business entities (such as geography, products, and customers)
  • Links define the relations between the hubs (for example, customer orders or product categories).

 

  • Satellites contain all other attributes related to hubs or links. Satellites include all attribute change history.

 

The result is an Entity Relationship Diagram (ERD), which consists of Hubs, Links and Satellites. Once I’d settled on this methodology, I needed to hunt around for something to use.

How do you go about designing and using an ERD tool for a Data Vault? I found a few options. For the enterprise, I found  WhereScape® Data Vault Express. That looked like a good option, but I had hoped to use something open-source so other people could adopt it across the team. It wasn’t clear how much it would cost, and, in general, if I have to ask then I can’t afford it! So far, I’ve settled on SQL Power Architect so that I can get the ‘visuals’ across to the customer and the other technical team, including my technical counterpart at the customer who picks up when I’m at a conference. This week I’m at Data and BI Summit in Dublin so my counterpart is picking up activities during the day, and we are touching base during our virtual stand-ups.

StockSnap_DotsSo, I’m still joining dots as I go along.

If you’re interested in getting started with Azure Data Lake, I hope that this gets you some pointers from the design process.

I’ll go into more detail in future blogs but I need to get off writing this blog and do some work!

Cloud computing as a leveler and an enabler for Diversity and Inclusion

I had the honour and pleasure of meeting a young person with autism recently who is interested in learning about Azure and wanted some advice on extending his knowledge.
It was a great reminder that we can’t always see people who have conditions such as autism. It also extends to disability, particularly those that you can’t see; examples include epilepsy or even Chronic Fatigue Syndrome.

Diversity gives us the opportunity to become more thoughtful, empathetic human beings.

dyslexia-3014152_1920

Credit: https://pixabay.com/en/users/geralt-9301/

I love cloud because it’s a great leveler for people who want to step into technology. It means that these personal quirks, or differences, or ranges of abilities can be sidestepped since we don’t need to all fit the brogrammer model in order to be great at cloud computing. Since we can do so many things remotely, it means that people can have flexibility to work in ways that suit them.

In my career, I couldn’t lift a piece of Cisco kit to rack it, because I was not strong enough. With cloud, it’s not a problem. The literally heavy lift-and-shift is already done. It really comes down to a willingness to learn and practice. I can also learn in a way that suits me, and that was the main topic of conversation with the autistic youth that I had the pleasure to meet.

I believe that people should be given a chance. Diversity gives us the opportunity to become more thoughtful, empathetic human beings. In this world, there is nothing wrong with wanting more of that humanness.