“Most hiring managers and recruiters want accurate, online methods for assessing talent that are as short as possible. But is that always a good thing?” – Tom Janz.

Most hiring managers and recruiters want accurate, online methods for assessing talent that are as short as possible. But is that always a good thing? There are tradeoffs and disruptions. This post looks at the tradeoff between assessment accuracy and length. Then it examines a disruption in the form of taking assessment length down to zero through automated candidate social media analytics.

Accuracy vs. length

Here’s the thing about something that hiring managers really want—they will believe almost anyone that tells them they can do it. Add to that, test length can be easily and quickly measured. Measuring test accuracy is another thing altogether.

If the choice is between a long assessment that has proven predictive power, and a short one supported only by client endorsements, choose the long strong assessment. Recent field research examined 222,722 assessments across 69 systems, finding that candidates are not more likely to quit long assessments. Candidates that quit did so in the first 20 minutes. For those that kept going, the longer the test the higher the completion rate (Hardy, Gibson, Sloan, and Carr, Journal of Applied Psychology, 2017).

Shorter–Screening vs. Decision Assessments

Assessments with proven high prediction power often take more than 20 minutes to complete. Placing the longer valid assessments near the bottom of the selection funnel makes good dollars and sense. Positioned close to offers, candidates largely accept and even appreciate a thorough assessment.

It’s different at the top of the hiring funnel. That’s where employers face screening from 50-200 respondents down to a manageable pool of qualified candidates. That’s where talented respondents quickly resent having to answer lengthy application forms and assessments. Panful experience tells them that the most likely outcome of all this boring work is nothing at all. Or maybe a personalized email telling them how much the company appreciates their application.

Tom Janz, HRExaminer.com Editorial Advisory Board Contributor

Conditions at the top of the hiring funnel call for engaging, short, inexpensive but predictive screening assessments. There are hundreds of online screening assessments. Some are short. Some are engaging. Some are inexpensive. But how many are all that and proven predictive? In the past, crickets- leading to a marketplace need.

Some intriguing solutions have evolved. Pymetrics offers a collection of short, online, neuroscience-based games to directly measure constructs related to job performance and retention. Traitify offers a simple, engaging set of photo choices to measure personality and career interest. HireVUE offers online mental ability games and video interviews. HireVUE can also objectively score the audio and video cues from the interviews via advanced analytics. These new solutions have yet to report enough field validation studies to lock down their prediction power with confidence, but it is early days. No doubt many other short, valid, screening solutions have arrived or are on the way.

Shortest—Automated Candidate Social Media Analytics

If shorter screening assessments are good, what about shortest? The shortest screening assessments take no time and don’t ask respondents to do anything at all. How could they be predictive?

Such an assessment would need to analyze information on respondents that already exists. It would need to be available easily and quickly for almost all respondents. The pervasive presence of text in posts, tweets, and profiles on social media sites provides that information. Studies have shown that the text entered in tweets and posts can be scored to reveal important personality characteristics (Tskhay and Rule, Journal of Research in Personality, 2014).

Several new online assessment solutions download the social media text for that person, analyze it, and produce scores on common personality scales (i.e. DeepSense¹, Crystal Knows, Receptiviti). These solutions generally use the target person’s name, email address or LinkedIn profile ID to find the publicly available social media text. Other solutions, such as IBM BlueMix, require the user to enter a text passage authored by the target person.

Some solutions like IBM BlueMix, return personality scores on the big five personality dimensions (1 Openness to Experience, 2 Conscientiousness, 3 Extroversion, 4 Agreeableness, 5 Neuroticism). Others like Crystal Knows provides scores on the four Disc dimensions (1Dominance, 2 Influence, 3 Steadiness, and 4 Conscientiousness). Crystal links the Disc type to coaching suggestions for how to talk, write, and interact with the target person. DeepSense offers both OCEAN and Disc dimension scores, along with additional dimensions of workplace behavior such as Action Orientation, Need for Autonomy, Learning Ability, and Teamwork.

How well do these zero length assessments work? We don’t know yet. DeepSense made a start on finding out by gathering the average of three or four ratings on the DISC, OCEAN, and work behavior dimensions for 10 people. Then the DeepSense predictions of those scores were generated and compared. The sample is too small to place confidence in these initial findings. DeepSense is now working on increasing the sample size.

The first chart shows the correlations between the DeepSense AI predictions, and the average ratings provided by the evaluators for the 12 dimensions. ‘Attitude and Outlook’ means staying positive during trying times and confident about transforming tough situations into good solutions. ‘General Behavior’ means friendliness and ability to get along with others.

In prior research, Neuroticism has correlated lowest with social media based prediction. DeepSense suggested that the very low correlation for Openness to Experience was caused by limited data training cycles afforded the algorithm for that dimension.

The true scores (average of 3-4 evaluators), the DeepSense direct prediction of the true scores, the DeepSense Disc prediction, and the Crystal Knows Disc prediction were available for all ten subjects. Three of the four Disc dimensions that related to the true scores were analyzed. Those were: 1 Action Orientation linked to the Disc Dominance scale, 2 Likeability linked to Influence, and 3 Stability Orientation linked to Stability. Conscientiousness was not included since there wasn’t a related true score dimension.

The above graphic charts the percent of times out of 60 comparisons that the two DeepSense methods, and the Crystal Disc method, were the best and worst fit for predicting true scores. Again, small sample and it was necessary to transform the Crystal Disc types into percentile scores. The method is complicated but the result needs no further explanation.

Will candidate social media analytics requiring zero candidate effort replace the growing set of short, engaging valid screening solutions? If everyone completes the short, valid assessments and they win big on proven prediction power, probably not. If the social media solutions can come close on prediction power, and the text content they feed on remains public, then perhaps. One thing for sure, it won’t be boring.

1 Disclosure: The author of this post advises the DeepSense team.