HRExaminer Radio

HRExaminer Radio is a weekly show devoted to Recruiting and Recruiting Technology airing live on Friday’s at 11AM Pacific

HRExaminer Radio

Guest: Dr. Tomas Chamorro-Premuzic, CEO, Hogan Assessment Systems
Episode: 130
Air Date: November 18, 2015

 

Dr. Tomas Chamorro-Premuzic is an international authority in psychological profiling, talent management, and people analytics. He is the CEO of Hogan Assessment Systems, Professor of Business Psychology at University College London (UCL), and visiting Professor at Columbia University. He has previously taught at New York University and the London School of Economics.Dr Tomas has published 8 books and over 120 scientific papers (h index 41), making him one of the most prolific social scientists of his generation. His work has received awards by the American Psychological Association and the International Society for the Study of Individual Differences.

Dr Tomas is also the director of UCL’s Industrial-Organisational and Business Psychology programme, and an Associate to Harvard’s Entrepreneurial Finance Lab.

Over the past 15 years, he has consulted to a range of clients in the financial services (JP Morgan, HSBC, Prudential), advertising (Havas, Fallon, BBH), media (Yahoo!, MTV, Endemol), consumer goods (Unilever, Reckitt Benckiser), fashion (LVMH, Net-a-Porter) and government (British Army, Royal Mail, National Health Service) sectors.

Dr Tomas’ media career comprises over 70 TV appearances, including the BBC, CNN, and Sky, and regular features in Harvard Business Review, the Guardian, Fast Company, Forbes, and the Huffington Post. Dr Tomas is a keynote speaker for the Institute of Economic Affairs and the co-founder of metaprofiling.com, a digital start-up that enables organisations to identify individuals with entrepreneurial talent. He lives in New York.

 

 

Audio MP3

 

Transcript

Begin transcript

John Sumser:                        Good morning and welcome to HRExaminer Radio. I’m your host John Sumser and we are coming to you as usual live from beautiful downtown Occidental California. Occidental is where Leland Stanford the guy who founded Stanford University and built the [rail roads 00:00:34] did his major engineering for the rail road between [inaudible 00:00:41] in California. That winding rail road was the foundation for the national lines. Today we’re going to be talking with Dr. Tomas Chamorro Premuzic who is the CEO of Hogan Assessment Systems. Tomas, why don’t you introduce yourself to the audience.

Dr. Tomas C-P:                      Hi John. Well, first it’s a pleasure to be here and yes, my name is Tomas Chamorro Premuzic. I’m the CEO of Hogan Assessments. I’m also a professor of business psychology at Colombia University and UCL. My main area of interest is personality assessments. I got into this mostly because I was interested in understanding myself and the people around me but now in our business what we do is we create tools that enable businesses and people to understand and predict human behavior at a very large scale.

John Sumser:                        Great. These days it seems like assessment is the [inaudible 00:01:47]. There’s a big deal of emphasis on pre-employment assessment in particular but also I’m going looks at people as companies try to optimize the productivity of their teams and try to reduce the risk associated with the [inaudible 00:02:05]. Are you seeing that?

Dr. Tomas C-P:                      Absolutely and I would go even further than that, beyond that because there is of course a clear increase in interest for assessment for evidence based tools for pre-hiring, for hiring, for really higher ends of the kind of management spectrum, at mid level managers and high potential identification and development and of course in the leadership development space. That is just HR. There is also a clear increase in interest for assessment in other industries: marketing, financial services, online and mobile dating. It seems that people have suddenly woken up to the idea that you can leverage a combination of psychological knowledge or science and technological tools including data analytics to work out what people are and predict what they do.

On the other hand, I think this is still the beginning, this is still the tip of the iceberg. There’s probably just around forty million assessments which we can consider relatively serious assessments. Not like [buzz feed 00:03:25] quizzes or things that you can pick up for free on the internet. Around forty million assessments that are sold for HR purposes every year, we probably cover 5% of that. That may sound like a lot but it’s only 1% of the potential market because there’s four billion people who work in the world who really should get assessed. We will continue to see an increase and although by positional standards, the industry is doing really well. This is just the beginning.

John Sumser:                        Assessment has as long there has been groups of people we’ve been trying to figure out who fits, who doesn’t fit and what they are going to do next? You might consider that the roots of assessment are astrology in the early days of civilization. There’s a ton of really bogus stuff out there and some of them looks very smart. How do you think about the quality of assessment and how to make traces [about 00:04:37] assessment based on validity or utility rather than marketing [instinct 00:04:46]?

Dr. Tomas C-P:                      That is a really good question. Unfortunately the average user or potential buyer doesn’t have the capabilities or the time or the interest in really scrutinizing these different tools. As you’re suggesting, there is a lot of shiny new objects out there that might look really cool, great user interface, [game-fied 00:05:12] stuff, interesting but those things are mostly not very accurate. Mostly people need to understand that there are two key components to a good or valid assessment. The first is it has to predict relevant outcomes. It has to predict fit. That is really a statistical question.

It’s not a matter of opinion. The problem is that a lot of assessment publishers base their validity it’s on their own white papers, studies that are not verified. They make claims that for the average naïve user sounds really great but they are just unrealistic. The first component is it needs to predict the relevant outcome. This is about mathematics, statistics and probability. The second one it needs to have a theoretical basis that is really grounded on the independent science of what we know about people. Application of these tools to HR maybe relatively recent but the science that underlines this practice is over a hundred years old.

It only takes five or ten minutes of somebody to dive into Google scholar and read about the factors or competencies that predict leadership, that predict talent and reap and search for independent scientific evidence that will put things into context and help them understand if the claims that commercial publishers or vendors who are feasible or not.

John Sumser:                        On the subject of prediction, I heard somebody say the most interesting thing the other day, which is that no predictive system can predict the fact that your plane is late. What that means predictions are statistical probabilities. That means that you don’t don’t really know if person X is going to do thing Y. You just know the likelihood that they [buy it 00:07:32]. While that helps you, it doesn’t [de-risk 00:07:37] the question entirely. A lot of the claims that are bogus are the claims that say they can take all of the risk to of the [environment 00:07:46]. Do you think that’s right?

Dr. Tomas C-P:                      Yeah. There’s some truth in that. The way I would phrase it is slightly different is that you can have … Well, first of all even if you have the most rigorous and scientifically valid assessment in the world today at most you can expect to predict performance or any behavior by about 30 to 35% above the chance [or 00:08:18] baseline level. Imagine, if flipping a coin gives me 50% chances of hiring the right candidate if I have two candidates that turn up. If on top of that I use a valid assessment my probability would not be 50%. It might be 75 to 85%. Okay? There is still a margin of error and there’s still a possibility that I make a mistake.

More importantly when you’re using these methods, even when they are valid, it is true as you were suggesting that you can’t necessarily predict individual outcomes. You will be able to make predictions for a group. What that means is that if a company puts a thousand people through a valid assessment they might make mistakes on different individuals and you can’t guess who those individuals are but at the same time, they will be able to quantify and estimate that using that valid assessment for a thousand people would help them or would lead to ending up with 30% more true positives than if they don’t do it.

On average, there will be on an ROI that can be quantified of about 30 or 35%. Now that still doesn’t enable them to know whether this particular guy or this particular girl will be good or bad. I think it’s good enough if you know that on the whole you’re going to end up with let’s say 35% more top performers than you will [inaudible 00:09:50]. This is what this is about.

John Sumser:                        That seems to me to suggest that the value of assessment tools is somehow proportional to the size of the organization. The reason I think that is if I’ve got twenty people on the team and you suggest that I can improve [of a 00:10:17] chance by another 50%, I’m still making so many mistakes that the business is going to fail. If I have a thousand people and I improve my good decision making productivity by that same value it probably makes me the top of the [inaudible 00:10:38] in competition. I wonder if the amount of risk reduction associated with using assessment tools is related to organization size.

Dr. Tomas C-P:                      I think what’s related to size is that the bigger an organization obviously the more scalable both the advantages and disadvantages are. More importantly, if you have large organizations that have a thousand employees or more and what that enables them to do is to actually test the assessments in the real environment of work and work out whether they are effective or not and how accurate they’re not. To go back to your previous question, how do organizations realize how good a tool is? The best answer to that is try it. Try different tools at the same time. Put two hundred or three hundred people through various tools, various methods and then see which one predicts performance more.

Otherwise, organizations are generally in competition with other organizations that have similar sizes. You can’t just say that because the company has five thousand employees assessment will be more useful to them than for a company that has two hundred employees because the two hundred employees firm is still trying to compete with other firms that have two hundred employees. The fight there is still for the best people they can gather. By the way for pre-hiring methods you can have a very small firm. We have worked with start ups of twenty or thirty employees who still develop pre-hiring processes or protocols that enable thousands if not millions of people to go through their pre-hiring assessments.

That level of quantity can also enhance the quality of people that you attract. Even if at the end of the day you have few employees. If thousands or millions of employees go through the pre-hiring tools you don’t need to have assessment methods that are super accurate to select the top because you can just slice it down from the top 2% or top 3%. Anybody you get at that level will be quite good. A simple way to say this is that quantity tends to lead to quality eventually. In the beginning it doesn’t look that way but if you attract lots of people, your chances of getting good people increase substantially.

John Sumser:                        That’s pretty interesting. I’ve also run across the notion that what really makes assessment valuable is that in order to adopt most assessment processes that work you actually have to adopt a framework of looking at the world. I think it’s right. You’re the expert here. I think it’s right that assessment companies sort of differentiate on the intellectual framework that they wrap around their work and that what makes a company successful is with assessment is it’s ability to incorporate that specific framework in the language associated with it. What do you think?

Dr. Tomas C-P:                      Yeah that’s a really good point. I certainly agree. We can discuss accuracy for hours and most of your listeners would get bored probably. Assessments specialists and experts like myself have a tendency to get fixated on accuracy and we can have different discussions about how to test what the best tools are. In reality that tells us more part of the equation. First of all, assessment tools tend to really provide two different types of products. Data is one of them but stories is the second one. When you look at the stories as you suggested what matters is the framework, the how you see the world or how you interpret behavior. That’s when you shift from coefficient at data point or a data matrix to a report.

There is okay John this is what you’re like, this is your bright side, this is your dark side, this is how you behave, this is how you annoy others and this is how you can become a better leader or a better employee. That really is a combination of the signs of personality but also the art of evaluation and it’s where theory meets data. It is true that that’s where you find slightly more differentiation between different assessment providers although I would say that a lot of them don’t really have a point to view, don’t have a theory or don’t have a framework. They just throw some numbers out here and say, “Okay …” Kind of not very intellectual, not very carefully thought typologies that might be catchy but they might not work.

The ones that have a framework and theory they might look superficially different but the good ones are based on the same independence signs. Mostly they re-label different qualities or traits with different names. Mostly when you scratch under the surface of all these competencies that become very fashionable in HR circles you see that they are new names for old qualities, old traits, old competencies.

John Sumser:                        What other ways that I’m trying to think about is what you described as re-labeling may actually be the place where there is fit or not fit for a given assessment tool and a given culture. Is that a sensible way to think about it or is it really just piecing different names of the exact same stuff?

Dr. Tomas C-P:                      There’s good and bad re-labeling. An example of good re-labeling would be for me emotional intelligence. Everybody in the HR role today has heard of emotional intelligence or EQ. It has been a very strong concept or competency both in practice and academia for over ten years now. All the signs show that it’s basically just a combination of personality characteristics that have been studied for sixty or seventy years in academia. If you have people how have high EQ or high emotional intelligence what that means is that they have [inaudible 00:17:37] optimism, high extroversion, high openness to near experience, high agreeableness and they are slightly more conscious that the average person.

However, if I talk about people using these five these very academic and geeky labels, I lose interest so it’s fine to talk at the level of EQ or emotional intelligence. If on the other hand you think about concepts such as learning agility to date we have no theoretical evidence or no conceptual framework that can make sense of that concept. It seems more like a new generic label or umbrella term to refer to potential. To say of somebody that they have high learning agility seems to indicate just that they have potential for something but then when you see what underlies learning agility it’s a combination of things that don’t naturally go together and that certainly have no scientific underpinning.

There’s good and bad re-labeling. I would say, just to finish on this point that, [if commpanies 00:18:49] want to put their own names, their own labels to qualities or competencies because their HR and marketing teams have spent a lot of time and money coming up with a list and if they want to call something the X-factor or [inaudible 00:19:05] or is awesome, that’s fine so long as under these labels there is a credible and valid tool that can actually predict the behaviors that they want to predict. I have no problem with labels as such.

John Sumser:                        Okay. If I head over the HR department somewhere and I want to think about putting my company in an assessment regime, how do I go about getting intelligent enough to make a good decision?

Dr. Tomas C-P:                      I would say, “Hire somebody in your team with some background in industrial organizational psychology.” That really provides people with the training to not only understand basic processes and principles around talent management and leadership but also around assessment and they have the ability to look at a technical manual, look at a tool and evaluate whether the claims are bogus or whether they have some substance. Then secondly … Three points. First hire an expert, a specialist that help you evaluate the tools. Secondly try it out, put people if you know who your top performers are and who your not so good performers are, put them through the assessment and see if they can blindly identify them correctly and classify them correctly.

Then, third, of course work out what your strategy is. There are really just two questions that HR directors or professionals need to ask around this area. First question is, “What should I look for? What should I assess?” The second one is, “How?” Too often people start thinking of the how before the what. If you’re really good at assessing the wrong thing, you won’t get very far. If you haven’t worked out what your leaders or managers should be like, if you haven’t worked out what profile or what competencies they should have don’t go out shopping for different tools or assessments because even if they are good they might not be evaluating what you need to evaluate.

John Sumser:                        There’s a level of readiness required before you [install 00:21:36] a system. Yes?

Dr. Tomas C-P:                      There is and most good vendors or companies would have some flexibility here between offering off the shelf tools that are ready and that would tap into generic attributes that are important across the board or across a specific industry and then custom products or custom solutions that mostly tweak those tools in order to focus on the attributes that are really relevant for that organization.

John Sumser:                        Got it. Got it. The other thing then is, the implication here is that installing an assessment system or an assessment process is a cultural change that requires readiness at the front end, some sort of beginning process and then firmly installing the framework inside of the company’s culture. Will you get to the place where you’re really trying to make the cultural change? What are the things that make that work and what are the things that make it fail?

Dr. Tomas C-P:                      I can start with the second one because that’s … We see very often, the same recurrent factors that threaten or inhibit success. The first is intuition, the degree to which managers and the decision makers overrate their intuition. It’s somewhat unfortunate that most people seem interested to some degree in psychology and in people because they all think they are experts. That doesn’t happen when you’re having discussions about organic chemistry or quantum physics with people, people don’t pretend to know. They are like, “Okay, there’s an expert and there’s me”.

Whereas with people, just because everybody interacts with people on a daily basis and everybody has opinions on others, which by the way are mostly prejudiced and wrong, they think they know. Often, even if they’re sold at a theoretical level on why they should use assessment and you show them beautiful correlations and statistics showing, quantifying the ROI, they would still tell you, “Well, when I see a good performer I can recognize it straight away,” or, “When I see somebody with a potential, I know.” There might be 2 or 3% of the people in the industry that actually know because they have amazing intuition but most people, our biggest enemy is their intuition. They think they know and in fact they don’t.

The second one is just politics, organizational politics. Organizations can end up using the right assessments for the right reasons but then they don’t necessarily act on it because there are political forces that essentially eclipse the knowledge or the recommendations that would come from the assessment. For example, you can put three or four people through the same assessment and the results would recommend that candidate A is promoted but everybody likes candidate B even if they know that the results are reliable and valid they would still go for candidate B.

Those are the two common enemies, what makes projects fail or derail. As for what makes it work, I think it helps when HR managers or directors have [buying 00:25:18] from the top. If they have a CEO or senior leaders or a board that truly believes in scientific tools than in evidence based practices, that helps. Mostly we have no problem selling to HR or [talent folk 00:25:33] is helping them sell the idea to the top. That is more difficult. Secondly it helps when they are fairly quantitative and they can make sense of data and they understand the results and the implications.

Then finally, it helps if their minds haven’t been polluted by old [BS 00:25:57] and nonsense that they are exposed to on an average day. If they don’t read much stuff that is written out in popular publications or forums where people just make up the same old things and you have self designated experts with no real knowledge or expertise, selling them ideas that are just populous and may sound nice but are just completely meaningless. For example, the idea that everybody has talent, that you should just focus on your strengths, that everybody can be  a great leader, that you can coach anybody to go from one extreme of the spectrum to another one and things that are mostly just wishful thinking but very toxic because then these people are brainwashed and when you present them with actual evidence or things that are supported by the independent scientific community they look at you with surprise. They’re like, “What?”

Same for [inaudible 00:27:03] that I met. They come across people who promise to predict, I don’t know, they present them with tools that they say are 99.9% accurate so when you tell them that yours is 30 or 40% accurate they think you’re a loser. Actually what they were told before is of completely meaningless and made up. You’re just coming up with an honest and real credible statistics. It thus help if people have not been exposed previously to all the nonsense that you can read if you spend two minutes online or believe even 5% of the claims that most people make in this area.

John Sumser:                        We sure have exhausted half hour here. I hear two great conversations that we should have sooner rather than later. One is, in general, about the relationship between politics or a social structure and individual assessment, I think that that would be a very interesting half hour. Then the other is we should take a walk through the false claims.

Dr. Tomas C-P:                      Oh, yeah, I would love to do that.

John Sumser:                        I hear you have a very strong opinion there and also some clear insight about each of those claims. Let’s plan to do that in the future. It was fantastic to have you on this show Tomas. Would you reintroduce yourself and let people know how to get hold of you?

Dr. Tomas C-P:                      Sure. It’s been a pleasure. My name is Tomas Chamorro Premuzic and I’m the CEO of Hogan Assessments. If people want to contact me they can email me, it’s Tomas with no H at Hogan assessments dot com or my twitter handle is at drtcp and there’s also a website they can visit, www.drtomas.com

John Sumser:                        Well, thanks so much for being here, Tomas and thanks everybody for tuning in. We will be here same time next week and have a great day. Thanks again for being here Tomas.

Dr. Tomas C-P:                      Thank you John.

End transcript



Tagged with:  
Read previous post:
square abstract photo cc0 HRExaminer feature image v6.47 Double Bind December 18, 2015
HRExaminer v6.47

HR and Recruiting leaders are required to maintain the status quo while imagining some sort of future. In our feature...

Close