HRExaminer Radio Executive Conversations Badge Podcast Logo

HRx Radio – Executive Conversations: On Friday mornings, John Sumser interviews key executives from around the industry. The conversation covers what makes the executive tick and what makes their company great.

HRx Radio – Executive Conversations

Guest: Andrew Marritt, Founder & CEO, OrganizationView
Episode: 335
Air Date: August 16, 2019


Join John Sumser at this year’s HRTech conference



Important: Our transcripts at HRExaminer are AI-powered (and fairly accurate) but there are still instances where the robots get confused (or extremely confused) and make errors. Please expect some inaccuracies as you read through the text of this conversation and let us know if you find something wrong and we’ll get it fixed right away. Thank you for your understanding.

Full Transcript with timecode

[00:00:00] Today’s Show is brought to you by the Human Resources Executive Magazines HR Technology Conference and Exposition held October first through fourth at the Venetian in Las Vegas. Join me and thousands of your colleagues at the world’s largest exhibition of HR Technology Act Now using the code HREX and you can receive a $300 dollar discount on your ticket.

[00:00:39] Thanks. We’ll see you there. And by the way, don’t miss the Women In Technology segment.

Good morning, and welcome to HRExaminer’s Executive Conversation. I’m your host John Sumser and today we’re going to be talking with Andrew Marritt who is the founder of OrganizationView. [00:01:00] OrganizationView is one of the earliest European people analytics practices and Andrew is I believe the global expert on how to use natural language processing to understand sentiment and large texts of survey data. How are you?

[00:01:16] Yeah, I’m very well, it’s a good Friday and I”m looking forward to a summer weekend.

Cool. So I want to start with I’m going to ask you to do yourself in a second, but you built a computer in the 70s.

Well, actually I think yeah. Well, I built it with my father. I think he was he was much more since this technology family and he wants to own computer you bought a kit and built it yourself. [00:01:45] So that’s how I got involved in technology.

So take a moment and introduce yourself.

As you said I’m founder and CEO of organization view. We are one of Europe’s first people analytics [00:02:00] firm founded in in 2010 for the last few years. We’ve become a lot more focus and there’s a niche and work mostly working with large corporate clients to help them understand their vast quantity of unstructured and mostly text Data as way of building that into their practices of learning and Improvement.

[00:02:22] So what do you do to that point? How did you how did you end up doing? Yes, sir. She mentioned I can bring me some technology background and it is Wednesday the 17th. Let me see probably about 3 K, you know, I spent. Academic setting by my teaching. So I went until we [00:03:00] knock this walking to the economics when I want to do something more applies and most of the 1990s working management consultancy, and it’s a lot of work around process re-engineering also the same that I was working in did not work around process simulation and I was often with on the morale of the parts of those big process change.

[00:03:20] I then went into industry in 10 and spent 10 years working with in a child and probably spending at least as measured by time in marketing. So I was typically running Global project. They would typically involving some technology. They’re using data and a lots of the approaches that we got there were very much followed.

[00:03:42] Well Stone in the tech we want to look at this from what was happening in marketing department. So my first experience doing machine learning it was in 2004 working with a bank where we used to use purchasing data on employees counts, [00:04:00] too. Six and optimize the benefits package. We I spent two years running the programs and play experience working really closely with my colleagues in consumer experience.

[00:04:12] And then when I left corporate HR 22 found organization views pretty much natural. We started you started with using our experience on my experience in terms of applying marketing analytics into into HR. We’ve been.

We’re talking about 10 years and I’m covered pretty much the transition of analytics from an interesting subject of maybe was really doing in 2010 to now when I suspect, you know, this data is clean just intensive in terms of doing this type of work.

[00:04:50] I think we’ve always been. There’s more technology focused mentally Advanced. So a lot of my peers and from the psychology background and we very much came [00:05:00] from data science type of approach and that we through our interesting text. So when you work with clients on building predictive models, five six years ago, we would typically find that you get to a stage where the way of in making improvements was not to.

[00:05:19] Improve the quality of algorithms but instead to improve the quality of events and these machine learning models of mining models would be able to build some sort of problematic approach to saying what is going to happen to when and to who wouldn’t be able to answer in itself why it was going to happen and do that.

[00:05:39] We needed to catch touch typically employ perception and increasingly. You can call us today when I was working with Executives.

What I noticed was they were disproportionally likely to make actions based on that qualitative data rather than the stuff, you know reams of really high quality with this simple analysis and we didn’t [00:06:00] and whilst some of our clients have the enthusiasm approach to this is one of our Telecom science here instant Chief Executives to read every single one of the employee survey.

[00:06:11] We realized that the much larger firms and so most organizations. We need to find a technological are scaling. So we. Last one five years ago.

We started focusing on text and actually can be suddenly in approaching and we got a very very strong position there and that’s where 90% of what we do these days.

[00:06:31] We’re also a little piece the heart of the business sir. Yeah, and if he’s the heart of the business it is it’s and it’s something that a lot of clients. Well, it’s a very large amounts of text as you and I are discussing. So increasingly employers will be asked questions as we go more towards shorter.

[00:06:56] Paul surveys the weighty one of the key ways [00:07:00] you counter the lack of richness in asking us for one more question since I asked me open text questions, so we increasingly seeing large going to take place. We can also look at other HR systems performance reviews. We can come lot of work understanding both objectives and also 360 reviews even down to HR said Services centers and Health Care Systems and trying to get much more granularity out of the information that action capturing our expertise and NLP is of pretty.

[00:07:31] It’s a series of very narrow field specialist in by the answers given to a particular question that requires the fine-tuning of the general. So, how does it help you work? I think I think if you can explain that simply, it’ll be a home run with the audience. Let me try let me try to text analytics.

[00:07:57] I know he’s done by analyzing the [00:08:00] words in the word frequencies. Give you a call me to pick up any beginners experts say that’s probably one of the outputs of that which is actually counting the most frequent words and visualizing those commercial modern commercial uses of tech today are not based around this approach used around assessment techniques called word embedding.

[00:08:24] So these embeddings numerical representations effectively. Definitely of of the text. Changing Words, which this millions of words and especially when you think that I miss that a computer will see a misspelled word is a loading itself into numbers and numbers are in Henry’s we have computers there with him these day large ones the Praises of their abilities like he planned which is why they’ve been popular over the last few years the way that designing The Works means that we the members.

[00:08:56] Semantic understanding. [00:09:00] So if we look at words for patents reasonably close to the work the dog, for example in the first major paper on this came out of bounds of the people do go. We also showed if you took the vector represents the world tank and you subtracted the word protector for the map for man.

[00:09:21] And as if the death of this woman you get the computer would say actually this is the answer that is the best of the queen and it’s learning semantic understanding of the text. Just by the way that the algorithms work and this is V2 powerful this enables to do some really smart thing with that in the last especially 18 months.

[00:09:39] We’ve seen a very large. Birds in trees algorithms for what embedding each which spaced on using large base set and more and more processing power for us and our application. We use these embedding to spot happens in so in total. I like to suggest is that.

[00:10:00] My areas of service which provides Maps want 10,000 that means you can see every single building on the map of the dot shape of the building if you would take away everything else.

[00:10:11] They’re always contour lines and which is the woods and colors of them out and just have those God and then try to identify which Villages were there. That’s effectively what we’re trying to do with these embedding. So we have all the phrases and we try to use this identity in the heat map to identify where we talk about one topic.

[00:10:29] Nothing, we’re not really working at the word level. But instead using classical structure the back to extract short phrase as the provide a key information. And so we’ve simplified the sentence rephrasing it both to increase the size of the data also to concentrate the meaning to something simple and then we use that to to identify the the phrase when most research is out there and sort of the big Tech firms are doing this and using larger and larger data set.
[00:10:57] We Mor more computing power.

And then back to [00:11:00] generalize for models. All types of different we change the algorithms and we use small custom models based on very specific date. And again, it’s about concentrating the information. If you’ve learnt your approach based on all of the information in Wikipedia, you’ve got things about European battles pop star Dolphins were unlikely ever to see in the data metal you need very large data sets to learn everything for us and our clients.

[00:11:27] Okay, if we ask a question about customer satisfaction and in the stores, we know the answers are probably gonna be about customer satisfaction. So it’s not important to teach the little about the Battle of Trafalgar. It’s not going to see that database. So we can’t use concentrated approaches and this enables us to build really super accurate models with just for that one.

[00:11:48] So what are the what are the things that I wonder a lot about about this technique is it seems to me that you can do a great deal with what has been [00:12:00] said. I wonder if you believe that you can get to what was meant by think. There’s often the difference between what people say and what they mean and if you were looking at the output of a large group of people you’d be more interested in what they meant than what the show.

[00:12:16] Right, and maybe you call that sentiment and I was just but I’d love to understand how you see the difference between the data that you get and the underlying meaning of the data that you. Yeah, I can wrestle really important Point sir. I think the problem is you need to take action in the survey or form.

[00:12:39] I think if you are having a conversation with somebody especially conversation with somebody from a different culture where some of those inherent things that you expect aren’t pressing then.

This is just a natural human part. Of course, it’s a it’s a human part of computers and humans find [00:13:00] reasonably.

[00:13:00] The computer side inhaling is the way that we deal with that is that we keep the human in the loop. So we’ve developed an approach mixing in a genius called Active Learning where effectively our algorithms. Identify where they’re not sure about a meaning and they passed those sentences to a human who can you say human superpowers?

[00:13:25] We have toxins and immediately say about this and we’ll know it’s about something completely different I think for us the view that computers are able to take, you know to look at these optimum’s your speech to text and. Make a decision is are you mentioned sentiment analysis? And you technically that’s about understanding positive on the negative aspects of the text and Harry we might think that’s relatively easy to do this cook to humans to view a big one to attack academic studies show.

[00:13:58] They get that [00:14:00] 70% of agreement maximum. And that’s about a level of performance is useless on consent. I wrote an article a few weeks ago about 10 cents an ounce and I gave a few examples of the realization that we have we see issues as right. So if an employee says on the survey, I used to have a really good manager.

[00:14:20] The technical sentiment analysis and say yes, some has a really good manager, but actually by them saying that you have a really good manager, they’re implying that they no longer have really good manager and it’s that sort of sort of phrases in the text that you or I would understand the computer really really struggled government.

[00:14:42] So there was a there was a reason project we got on the phone to talk about the MIT promoted. Analysis of glass door data that claim to be able to identify culture from reviews. So they took all of the blast or data and made some [00:15:00] claims about culture. What do you think about that? There’s a few little wasteful of it.
[00:15:06] So this is the coldest of culture 500. It was published in what’s to by MIT Sloan management review and a team out of MIT and presumably working hand-in-hand with the glass store data Glassdoor team because they have. I’m sorry, sweetie, interesting. We have lots of experience with the type of questions last all ask often used internally within organizations.

[00:15:32] And also we’ve looked at other public review Dayton and the because the research also in their documentation to have really good job of documenting their methodology which has lots of similarities to the approach that we use. So I always like researchers and say how you’ve done it rather than just saying got amazing results.

[00:15:52] What we have done is identify expressions and information that they think Twisted into nine Dimensions, which they can [00:16:00] score each firm against now for some degree. This is similar to what we’re doing. We’re identifying the looking the dates or identifying themes and then because typically with identify maybe a hundred beams on on a question like the top cons question is last year you then have to simplify because no human can make sense and store on the good things in there.

[00:16:20] Did they said that years which was 1.2 million reviews as it is a decent size, but it’s not enormous, right? So a lot of our time to go, you know, which would come to official data size of well over 1000 reviews from one question. And that’s once they have 500 pounds and they have 1.2 million things.

[00:16:37] They said they have Rich firm in your study had 2000 reviews. Good weight distribution means and sometimes we’ll have an awful lot dating is twenty thirty thousand views and some might have tens or maybe a so when you started getting this more volumes of data you then end up having lots more uncertainty because you haven’t asked the employee or [00:17:00] the worker bee X worker how this person have exterminated intense communication or performance of the other schemes like that.

[00:17:07] You’ve you’re looking at cancer and Cancers. So I think it’s I think it’s an interesting approach is certain that would certainly technically o a good approach and chosen it now talk about whether this actually is called shown and whether we can Circle to based on on the stack data exactly. That’s that’s the next question.

[00:17:29] It seems like. It seems nonsensical to be that you would go to a place like glass door, which has some notable bias in the data set. So it’s like it’s like looking into a mirror, but maybe a mirror that you’d find in a fun house and and a praising praising yourself based on the funhouse mirror and walking away saying well now I know who I am.

[00:17:59] Yeah, sorry, I [00:18:00] think that’s I think that’s true. I mean the last all day. Which much higher quality than other review date on the right is tax amounts of information and research showing biases on in terms of web research and the glass guilty in the past of publish data, which shows you know the on a relative basis, they perform better than Amazon review.

[00:18:21] So if you look at plotting out there scale recommendations much more like you would expect to see in an employee survey that you would expect the analysis team. Thanks. I think the data is is not perfect, but it’s better than many types of days mentally. There will be many Mexican culture in terms of how this how they Define the culture and I’m you know, I’m sure the seven of our listeners know far more about which owns my do.

[00:18:48] I guess it’s fair to say that many different definitions of organization called Chinese. No Foreman shared view. I don’t have some think about when I look at some desk research on [00:19:00] the common models and the most popular one their model didn’t seem to fit anyone cancer. So it’s as if they’ve created a definition of call Jeff probably partly based on the data those boundaries in in the glass door data.

[00:19:15] So they also claim to be able to identify the performance the data. Performance. And again, I think the evidence on culture performance is inconclusive but I think like engagement would expect their leader is a strongly my guess is an analyst is that good cultures necessary but not sufficient to drive to drive performance.

[00:19:37] So we are doing through our time together. I’m going to skip ahead a little bit because we were going to talk here. Deeper about the measurement of culture and what you can and can’t measure but I’m going to skip to what I think is the fundamental question here, which is that the measurement of culture is going to increase I [00:20:00] believe and I’m curious about how you see that unfolding because because what I’m seeing is that there are projects like this.

[00:20:09] Good actually seem to ignore all of the prior work on what culture is or is it and there’s in the states? I don’t know if this is a global trivia, but there’s there’s an idea called Columbus sharing and Columbus thing is when you get to a brand new land and you declare that you’ve discovered it even though they’ve been people living there for 500 years.

[00:20:32] Yeah, right, right. I only see this a lot in data science of people, you know from the research that you’ve been doing the fax number of firms, especially young firms coming onto the market to potentially have found one interesting result and then try to generalize this into inter product. I think fundamentally the issue that this were and.

[00:20:59] Many of [00:21:00] you have seen the world has is that when we doing sentence analysis, so we deal with data was captured for one of those operators of that. We’re trying to ensure so we’re trying to make a best guess estimate of culture or incentive plans. Entenmann’s from data, which is never got action that we can do that reasonably.

[00:21:22] Well, so the accuracy of sentence now, this really has been an answer to that question. So all you have is this Foundation, it’s probably the best you can work with if you want exactly the biggest fan I and want to understand culture and thoroughly recommend you go to a culture specialist your org psychologist.

[00:21:46] And it was something specifically designed for measuring coach that you can get you get more accurate. Inside [00:22:00] engagement from Attack somebody that level of level of Engagement. And so I think there’s also a key verse we see new date and especially novel forms of theta as as being new and shiny and potentially Innovative Ambassador.

[00:22:15] We have analysts always need to be thinking about what to say to show. How is it captured? What are the biases that are inherent in this quantity boring stuff? It’s unfortunate tempers down, you know, just how much. To squeeze out of out to some data. So I think my recommendation on this type of this approach.

[00:22:34] What we also have done is you know, and and you agree in their definition of culture what the others have done is really good. So there’s a whole bunch of caveat. If you’re job hunting you haven’t got the apps the data better days and then you should rely on this time, if your recruitment executive and want to develop a differentiate [00:23:00] free position between you and your tongue competitors, there isn’t a.

[00:23:03] It’s difficult to say to a friend. They probably could start using somebody like this who can feel the specific model for your filial yourself and probably means that form this type of model in a very short period of time. I think there’s a natural human Tendencies towards these models were making.

[00:23:22] 30 countries are understanding and they successful resetting extremely valuable. However, we need to take part citations for the consumer constraints and Harrington in the right data structure pervert. Thank you. So it’s been great having you to do this. We should do it again soon. There’s so much more to talk about would you would you take a moment and reintroduce yourself and tell people how to get hold of.

[00:23:46] Yes, sir. I’m Andrew Marathon the founder and CEO of organization view a text Focus people analytics firm based in Switzerland. You can get hold of me UPS by LinkedIn [00:24:00] and I’ll be the easiest way is to go to organization or or, which is fixed by the server the tech service and the contact forms.

[00:24:11] There will come we’ll go straight to me. I guess the every single one of them. I’m always willing to. – a knowledge about text analytics and answer any questions you have and for the large firms, we usually can can show them quite quickly what we can do with our data in terms of Pilots here. We like working as a hotel right now.

[00:24:30] Thanks. It was really great to talk with you this morning Andrew. Thanks for making the time to do it. You for listening to HR examiner’s executive conversations. We’ve been talking with Andrew Merit em ARR Itt from organization view of Switzerland based company that specializes in sentiment analysis and text processing.

[00:24:51] Thanks for tuning in and we will see you back here next week. Bye.

[00:25:00] Bye now.

Join John Sumser at this year’s HRTech conference

Read previous post:
Fiddling While Rome Burns

“Nero is a role model for doing the exact wrong thing at the exact wrong time. The HR profession is...