Want to learn how to light up Big Data Analytics using Apache Spark in Azure?

Businesses struggle with many different aspects of data and technology. It can be difficult to know what technology to choose. Also, it can be hard to know where to turn, when there are so many buzzwords in the mix: analytics, big data and open source. My session at PASS Summit is essentially talking about these things, using Azure and Apache Spark as a backdrop.

Vendors tend to tell their version of events, as you might expect, so it becomes really hard to get advice on how to have a proper blueprint to get you up and running. In this session, I will examine strategies for using open source technologies to improve existing common Business Intelligence issues, using Apache Spark as our backdrop to delivering open source Big Data analytics.

Once we have looked at the strategies, we will look at your choices on how to make the most of the open source technology. For example, how can we make the most of the investment? How can we speed things up? How can we manipulate data?

itoa-illustration-1200x572

These business questions are translated into technical terms. We will explore how we can parallelize your computations across nodes of a Hadoop cluster, once your clusters are set up. We will look combine use of SparkR for data manipulation with ScaleR for model development in Hadoop Spark. At the time of writing, this scenario requires that you maintain separate Spark sessions, only running one session at a time, and exchange data via CSV files. Hopefully, in the near future, we’ll see an R Server release, when SparkR and ScaleR can share a Spark session and so share Spark DataFrames. Hopefully that’s out prior to the session so we can see it, but, nevertheless, we will still look at how ScaleR works with Spark and how we can use Sparkly and SparkR within a ScaleR workflow.

Join my session at PASS Summit 2017 to learn more about open source with Azure for Business Intelligence, with a focus on Azure Spark.

What I learned from David Bowie, and Statement of Support for Wendy Pastrick for PASS Board of Directors

Firstly, I don’t speak for PASS generally. Denise McInerny has already written a statement of support and you’re welcome to read it. I encourage you to learn about all the candidates campaign platforms on the PASS Elections site and to vote. The voting period is Oct. 5-11.

87598346_bowie

“Don’t be the first to do something. Be second.” – David Bowie

 

 

I’ve decided to write this statement because Wendy cares about the same things that I do, and I am going to summarise them here.

EMEA

I’d like more support on the PASS Board for Global Growth. Grant Fritchey has started to take this on board, and I look forward to more support and input from the EMEA perspective on the Board itself, and throughout the community. Ideally, I’d like ‘lessons learned’ to be at the heart of the LATAM onboarding so that it is set up for success.

I’ve been on the Board for nearly three years, and it’s been a consistent heartache for me that PASS isn’t growing in Europe. I have tried my best. I haven’t been able to do as many SQLSaturdays this year after the financial difficulties wrought by the failure of SQLSaturday Edinburgh, and the problems that precipitated. Further, the Board have to declare precons, which is fine, in order to be transparent. However, it’s made me nervous of offering to do PASS precons because I am worried that I will be accused of financially profiting from the community in some way. Since I am nervous of the accusations, it means that I don’t do PASS precons anymore and this compounds my ability to travel. I often do precons for free in return for travel being paid, actually, to help the event do the success, but I’m aware of the perception.

So Wendy has understood that PASS has so much potential to grow outside of North America, and one of her key election missions is to support Global Growth. For me, it’s music to my ears and I’d like to have a friend on the Board right beside me, who prioritises it. I haven’t been able to do this by myself and I hope that Wendy might be more successful in highlighting it.

Business Analytics

I need more support to work on Business Analytics as well. This is the second thing where I am first. Wendy’s been instrumental in delivering something that we will announce shortly but it’s good news. No spoilers here so you will have to wait!

What I learned from David Bowie

In business, sometimes you don’t want to be first. For Europe and BA, I was first, and being first isn’t easy. I need a ‘second’ and I think Wendy can help to break the back of some of the work that needs done in EMEA and in Business Analytics. I need another voice that will be heard, and two voices will carry further.

What is hard about being first

man-489744_960_720To be first is to be brave and a pioneer. Being first means that you have the burden of expectation, and that the target isn’t always clear.

It also means that people can withdraw from you and what you’re doing, because they are not sure if you are going to be successful. It’s easier to get behind the second person because it means that the first person has already absorbed the pain of the initial journey. It also means that there is someone to blame; everyone just blames the person that just left, right?

102636981_smSeymour Cray, father of modern day supercomputers, is quoted as saying “I’m certainly not inventing vector processors. There are three kinds that I know of existing today. Those three were all pioneering processors. One of the problems of being a pioneer is you always make mistakes and I never, never want to be a pioneer. It’s always best to come second when you can look at the mistakes the pioneers made.” If my example serves as anything, it will be to show some of the mistakes and issues that have been clarified through the process, and can be picked up and resolved.

I’ve put this photo here. There are plenty of PASS official ones but I’ve borrowed it from Hope Foley’s blog. It’s a shame that Wendy’s not facing the camera, but this is Wendy the person; joining in, strongly participating in the community, and bringing wisdom, fun, joy and friendship along with her. This is always how I will think of her.

20140809_211752

So I’ll vote for Wendy.

I wish the other candidates well, of course. I’d like to thank them for their courage in going forward. They have done a great thing in putting themselves forwards, and they are winners purely for that.

race

 

Are you a data Thought Leader? Call for speakers for Thought Leadership podcast series

thought-leadership_fish_131437046

Credit: MPI Group

As part of the Business Analytics Portfolio, I am spearheading a series of Thought Leadership podcasts and I am looking for people to be interviewed in a ‘fireside chat’ format.

 

I am bringing together experts from our community to share insights, ideas, and tips on helping data executives lead the way to becoming more data-driven.

The podcasts are intended to speak to senior executive people in the organisation, and they aren’t technically oriented. PASS already had a wealth of opportunities to speak at the Virtual Chapters to share deep technical expertise.

The first episode with Ken Puls is on the PASS website, and please do listen to his session. I am looking for more episodes, and I’d love to interview people in the PASS community.  I will be your friendly interviewer, and the topic is YOU, how you got to this stage in your career, what’s your data story, and what wisdom would you share to a  younger you? What do you think is happening in the industry now, and where is it going? What books do you recommend for people who want a more data-driven organisation?

I am looking for Thought Leaders and Budding Thought Leaders. This is your chance to showcase your expertise. It’s an informal podcast, so there are no slides. It’s just you, me and twenty minutes of your time.

Do you have a data story to share? If so, please email me at jen.stirrup@datarelish.com and let’s try to make it happen!

 

 

Jen’s PASS Diary: Two Pillars for Leadership

As always, I don’t represent PASS. Here are my two pillars:

Say Thank You.

The number one item is to say ‘Thank You’ to people for their efforts, even if you don’t like what they did. They are on a path; and so are you. Saying Thank You is the glue to healthy communities, particularly on social media.  The duties of gratitude are perhaps the most sacred of those which the beneficent virtues prescribe to us. (18th Century Scottish economist and philosopher, Adam Smith, Theory of Moral Sentiments, 1759)

Empirical science agrees: gratitude, even a simple “thank you”, is a basis of leadership; without it, you can’t lead effectively. People will remember that they didn’t get a Thank You. I hear this often from volunteers, unfortunately, and it’s fairly universal across geographies and events, both within and outside of SQLFamily. Say it, and mean it. I don’t hold to Machiavelli’s maxim, written in the 16th Century: It is better to be feared than loved.

Expect criticism, both justified and unjustified.

I’ve taken a lot from John Donne’s 17th Century poem, the Prohibition. Here is an excerpt:

Take heed of hating me,
Or too much triumph in the victory ;
Not that I shall be mine own officer,
And hate with hate again retaliate ;
But thou wilt lose the style of conqueror,
If I, thy conquest, perish by thy hate.
Then, lest my being nothing lessen thee,
If thou hate me, take heed of hating me.

Donne was a metaphysical poet and you can take from this poem what you will.

One aspect for me is being isolated; outside a circle of friends, family, community, whatever you want to call it. When people criticise, occasionally, what they can actually mean is: I said that you were x, and that means that I must be, by definition, y. Example: you’re not a team player (because I said this, this must mean that I am a team player), you are not good at a particular thing (because I said this, that must mean that I am good at it). This means that they have the ‘style of conqueror’ because I give them something to point at, and therefore, I serve a purpose. When you recast criticism into these terms, suddenly, it becomes much less meaningful and what you realise is that you need to stay true to yourself, and not ‘hate with hate retaliate’. Really, it’s not worth it. The real trick is to work out whether it’s valid criticism or not, and look at the motivation.

So, what have I been working on?

I pulled two nighters, to try and pull together a draft strategy for PASS Business Analytics. At 43 years old, I thought that my days of working right through the night were over; well, they are not. I don’t believe that anyone else does this, particularly not for a volunteer role, and I thought I’d point it out here to show the level of commitment I have to the community.

Now that we are not doing a full PASS Business Analytics Conference in 2017, people must be wondering what we are doing next, and what the strategy is? Well, I am trying my best to define it and we will release when we have agreed and signed it off. This is going to take some time, unfortunately. Business Analytics touches all parts of PASS: finance, marketing, and the other Portfolios will also have input. We will also need to have an eye on things globally.  I’m also working on a few other things. I am doing Thought Leadership podcasts and if you want to give me a podcast, then please get in touch!

I am assisting some of the sponsors at the moment; I won’t say whom, at this point. Basically, I want them to have a good experience of dealing with PASS, and me personally of course, so I am trying to juggle to make sure that everything works out well for them. I am not a great fan of the word ‘sponsor’ – personally, I prefer ‘Partner’. It feels more equitable.

Given that I have a good pillar and a bad pillar of Leadership, and this is a ton of hard work, why am I doing it? Well, it really is lonely at the top, even if you don’t see that you are at the top of anything at all, others do, and that’s when it starts.

Truth is, I have learned a lot of lessons, certainly more than just two! and I’m left with very few real friends. Now, the thing is, when you realise that you have really nobody left, then it actually gives you a certain freedom and a latitude. That realisation is a gift. Along with that gift, I’ve come to withstand criticism a lot better, but it’s also made me determined that I will not ‘hate with hate again retaliate’. People’s actions speak for themselves, and I don’t need to say a single word about it. I will just continue to try to do good things, and hopefully you will join me on the way.

I think that you put into it, what you get out of it. It’s not all bad. I have met some wonderful people who continue to shine a light out. To them, I say Thank You. Some people have got Thank Yous coming their way, and I will let them know that they are heartfelt.

With leadership, you have to stick true to yourself because, actually, it’s the only way to be. I think that these two pillars feed into that, for me. What do you think?

I keep this diary so that you come on a journey with me, and I wonder if you’d reach the same conclusions? I’d be interested in your thoughts.

Love,

Jen

 

 

 

 

 

 

Jen’s PASS Diary: A week in the life of a PASS Non Executive Director

As always, I don’t officially represent PASS here. As you will know, this week the application opens to be on the Board of Directors for PASS. I’ve held my post for two and a half years, and I am not up for election this time. If anyone has any questions, please don’t hesitate to get in touch. I also help run the London Power BI User Group, where I am responsible for the sponsors and I help recruit and manage speakers.

If you are considering the role of PASS Director, I thought you might be interested in what a typical week looks like, for a PASS Director.

First things first; please don’t think that the role involves kudos.With PASS, you aren’t building your own empire, but you are building an empire for the community as a whole. This can mean putting PASS first, and putting the team before yourself. If you are doing it because you think it will help build a business, then you need to rethink that. As an independent consultant based in Europe, I have had no business at all from being a PASS Board Director. PASS are still fairly nascent in Europe, and organisations over here don’t seem very interested in that part of my life; they are mainly concerned if I can deliver for them whilst having this commitment.

I certainly do not feel that I have kudos from it, at all. If you have an ’email signature’ career strategy whereby your objective which is led by having more and more titles on your email signature, I can see that the role might be attractive. However, I personally don’t see any kudos from the role itself and I think that I get kudos and thanks for the things that people can see; events, speaking sessions, books, webinars and podcasts. If your career strategy is led by titles, then I think you overstate the importance of hierarchical roles and job titles; people are more interested in what they can hold in their hands, and see, as a consequence of what you do.

So what about the stuff I do, that you can’t see?

I have had three evenings this week, used up with PASS Meetings: Monday, one with the BA team, Thursday, with the Exec, and Friday (Today) I will have another meeting with one of the PASS Summit Sponsors over one of their activities at PASS Summit 2016.  I am going to try and squeeze in a fourth meeting with another PASS HQ member tonight as well, but that has to be confirmed.

I say ‘tonight’ because PASS are based out in the PST timezone, and I am based in London so they are eight hours behind me. So basically, the PASS ‘day’ starts at 5pm for me. I try to hold my meetings from 8pm onwards (12pm PST) because I usually have to travel home, and this can mean problems in participating effectively over Skype. I also try to get ‘family time’ in the early evening and I try to carve out this time so that my family don’t lose out.

Note that this doesn’t include the work I do outside of these meetings:

  • setting up Thought Leadership podcasts for PASS BA
  • creating a BA strategy
  • setting up a PASS BA Advisory board and interacting with the potential members for that
  • preparing for the sponsor Skype meeting on Friday.

It’s a lot of work and it’s stuff that people can’t see, which is why I mention it here.

What am I working on?

200px-strategy_concept-svgA Business Analytics strategy document that outlines where I think PASS should go. This is particularly important now that we are not having a PASS Business Analytics conference in 2017. What does this involve, in terms of skill set?

Defining a strategy is based on knowing:

  • where your organisation is today
  • where you want it to be
  • how you want to get there

The risk of not changing and improving can be as significant as the risks which may affect your plans to develop and grow an organisation. I take after one of Steve Jobs’ Crazy Ones; this is a risk in itself. Although I will never achieve as much as the Crazy Ones in the video, these are the crazy attitudes that I bring with me; the people who are crazy enough to think that they can change the world, are the ones who do.

There is a lot of ‘noise’ in the data world, and I am particularly concerned with trying to ensure that PASS continues to change and move ahead in terms of efficiency, reputation and meeting the PASS Goals of disseminating high quality community education in data. It is an important exercise since it forms the blueprint of the strategy. To do this, I need to  learn lessons and appreciate what factors may influence the likely success in delivering your goals and success for PASS. This is what forms the basis of my thinking for BA.

Defining a strategy is a process, and this is one key area where a PASS Director adds value to the organisation. The objective of the process is to pull together the activities of the various areas of PASS that touchpoint BA, so that it is in a good spot to achieve its organisational objectives. Once the strategy is in place, it will help to specify how PASS organises to incorporate BA, set objectives and point community and team members towards those objectives through a commonly held vision.

traditional-vennYou’ll notice that this isn’t a technical skill; this is all people and process. As a Business Intelligence professional, I think about the people and the process, too. I’ve said before that Business Intelligence is often change management in disguise, and part of defining a strategy is that change management necessarily follows. I lean towards a more agile delivery, and I am working with PASS right now to point the ship towards a more agile delivery where the BA Strategy definition and process is concerned. It’s not enough just to be agile; you have to pay attention to people and process, too, and it can be more difficult when the people are volunteers who don’t need to do anything that you ask of them. It’s a sensitive balancing act.

To achieve this well, the execution process needs to be separated from the creative process of generating the strategy. As a ‘doer’, it can be hard to excise yourself from the execution and it is tempting to do that, because you see short term results. However, I am focused on long-term results, and strategy definition is a longer-term process that looks further into the distance, and has the objective of pointing the organisation towards that vision.

To summarise,  hope that gives some insight into what I have been doing this week. Any questions, let me know!

 

 

 

Jen’s PASS Diary: The Happy Prince and the Swallow

tumblr_n1tk0vugdz1tnqouoo1_500

Picture Credit: checanty

Oscar Wilde was well known for his writing for adults, but he’s probably less well known for his excellent children’s writing, too. He tackles strong themes, and they are well worth the read for adults as well as children. In one short story, The Happy Prince, Wilde writes a rather sad tale about a statue of a Happy Prince, who is somewhat misnamed because he’s never really known true happiness. In the tale, there is a swallow who was left behind after his flock flew off to Egypt. Saddened by what he saw around him, The Happy Prince tasked the swallow with giving everyone the jewels of himself, until, eventually, there was only the basic iron left. The swallow flies around, giving out the precious stones in the statue, but eventually dies, breaking the Happy Prince’s heart. There’s more to it than that, and it’s a sad tale, and for various reasons it has always been special to me.

I think that being on the PASS Board is a bit like being both the Happy Prince and the Swallow. Like everyone, I have talents, experience and wisdom in some areas, and not others. I happen to try to use my talents, small as they are, for the benefit of SQLFamily. I have won both of my PASS elections outright as the winner with the most votes. This means that SQLFamily gave me a position, but also a mandate to try and help the community via PASS. I am a volunteer and I try my best, and I give my ‘jewels’ away for free where I think they are best needing to be spent, and I am also the messenger that takes them there.

It’s a lot of work, however, inside the tent. I’d like to explain a little about how much effort I put in normally. Since I’m in Europe, attending PASS calls means that I’m on the phone late in the evenings. If I have a few calls a week, then that takes out a few of my evenings. It all adds up, and anyone in IT knows it’s not just about the meeting, there is work outside of that as well. If I was based in the US, I’d take the calls during my working day; however, it’s a different story when you have your evenings taken out. I’m just putting that here so people understand that being on the Board, from this part of the world, is actually a huge commitment and this is my third year of doing it. Fortunately I am single so it’s not impacting time with a spouse, although it does impact my newly-found Netflix addiction.

What am I working on? A few things:

There’s a difference between PASS BA the event, and PASS BA the strategy. Both require a lot of work. Vision is ‘why’, strategy is ‘what’, and execution is ‘how’. The strategy is a follow on from a vision, a mission statement of where PASS would like to go. This diagram might help, and I sit along all three of these elements:

vse-leadership-management-474x234

 

Credit: Goulston Group‘s image.

The PASS Mission Statement has to translate into a strategy, which then translates into execution plans. Execution is crucial, but it shouldn’t be mistaken for a strategy. A strategy is all about making a decision about where to play, and the way to play. A strategy tackles more fundamental questions:

  1. What organisation should PASS be?
  2. How does this add value to PASS?
  3. Who are the target audience for the PASS BA proposition?
  4. What are your value propositions for the BA audience?
  5. What capabilities are essential to adding value to the PASS BA organisation, and differentiating their value proposition?

In future blog posts, I will try to speak to each of these questions from my input as the PASS Business Analytics portfolio holder. A strategy provides a foundation for decision-making. It’s a garden for growth and where to cut costs, and determining priorities. The strategy gives a signpost and a guide to prevent drift, or scope creep. Personally, I have never seen scope ‘creep’ – it usually gallops! So there is a lot to think about, as I try to help PASS continue to be successful, and move forward to further success.

A strategy is particularly critical in volatile environments, and there is none more volatile than the world of data at the moment. You just need to see the Apache top level projects at the moment. Apache Spark is à la mode, but now there is also Apache Flink and Apache Arrow to consider, which also play in some of Spark’s space. Also, you could consider Apache Apex which is designed to improve the performance and speed of big data components that work together as part of a larger system. How would an architect decide, and put these bits together?

I am continuing to make sure that my voice is heard and I’ve already made the following points:

  • Strategy – I have been working a lot on the PASS BA strategy. More details on this will be ongoing, but here are some details which I’ve previously posted. I’m supporting the team as we move forward to tell our story, and that’s involved a lot of research and teamwork. Thank you Teresa C for your help 🙂
  • PASS are working on the BA Marathon, as promised in the last blog post. I’ve been having input on that.
  • I’ve raised the question of greater engagement and activity outside of the US. I’ve sent through my thoughts and ideas, and hopefully that will lead into more growth in that area, through strategy and execution. This is crucial; growing PASS will mean greater support and engagement outside of the US. My EMEA seat is supposed to give the ‘voice’ outside of the US to the rest of the Board, and I’ve already made these points on a number of occasion. In my own capacity, I spoke at SQLSaturday Vienna, SQLBits and I’m speaking at SQLSaturday Paris and SQLSaturday Dublin in June. I’m also speaking at Digital Pragmatism: Delivering Real World Improvements in Mental Health. I also spoke at Microsoft TechDays, a UK event.
  • Also, I’m supporting SQL Server Geeks in my own capacity and I’m delivering a precon for the team out there in India to help support their wonderful event and community. How much do I believe in the SQL Server Geeks event? I have 100% faith in the team out there – and SQL Server Geeks is going to be the highlight of my year.I’m delighted to see the growth in the community there. Amit Bansal, Manohar Punna and the team are doing a wonderful job. Pinal Dave runs a wonderful blog out of India. These are only a few people in that part of the world who are doing wonderful things, and they all contribute to make the Data Platform world better, both in person and online.

So, that’s a roundup. As always, please feel free to get in touch at jen.stirrup@sqlpass.org

 

 

 

 

 

See you in Paris? Presenting at SQLSaturday Paris?

Fancy joining me in Paris? 25th June, I’ll be there! The details are below.It’s the place to be, to learn Microsoft Data Platform technologies e.g. Azure, Power BI, SQL Server – and there’s a range of sessions for beginners to experts. Register here – http://www.sqlsaturday.com/510/eventhome.aspx 

Also, a big Thank You to the SQLSaturday Paris team for having me along again. They always do a professional, world-class job of organising the event. Contact Jean-Pierre Riehl for more information ( twitter and GUSS site  )

They are also offering workshops and here’s the information for you to check out. Kevin Kline‘s session will be in English.

unnamed

I will be attending Marco Russo’s session on DAX, Markus Ehrermueller-Jensen‘s session on data visualisation, Isabelle Van Campenhoudt and Serge Luca‘s session on Power BI, Kevin Kline’s session on Troubleshooting, and then I’ll be presenting on Cortana Analytics.

L’agenda

SQLSatParis Agenda

20 sessions, 25 experts internationaux
http://www.sqlsaturday.com/510/Sessions/Schedule.aspx

Le SQLSaturday, c’est LA conférence technique internationale de l’année. Avec plus de 25 speakers de 10 nationalités, vous retrouvez le meilleur des technologies Data de Microsoft : SQL Server, Power BI, Azure, deep dive, etc.

Les préconférences

Cette année, les préconférences sont de retour. Imaginez passer la journée entière avec un expert internationalement reconnu pour creuser un sujet particulier ? A mi-chemin entre la formation et la conférence, les préconférences vous permettent de découvrir en profondeur une technologies ou un savoir-faire.

Les premiers sponsors du SQLSaturday Paris

Microsoft, Pyramid AnalyticsData by Design, Jobly, AZEO, DCube et SUPINFO Paris.

Vous pouvez devenir sponsor en nous contactant: sponsors@guss.pro ou en passant par le site de l’événement: https://www.sqlsaturday.com/510/Sponsors/SponsorSignup.aspx