Epic Debate on the Science of OMG's Sales Assessment

Posted by Dave Kurlan on Mon, Mar 09, 2015 @ 06:03 AM


Copyright: 123RF Stock Photo

Sometimes things happen in ways that you never plan for.  Last week, a blog post appeared on another site that listed, 8 Things that the Top 1% of Salespeople Do Differently.  In response, I posted a simple counter argument on my blog.  The extremely popular article was syndicated by CustomerThink.com, where the conversation picked up comments from both doubters and supporters alike.  It was a perfect storm except in this case, it was more like Objective Management Group's (OMG) Sales Candidate Assessments on trial.  You won't believe some of the things that were said!  In my opinion, that very conversation is now the ultimate, defining conversation comparing the science behind OMG's award-winning sales assessments, to gut instinct, faith, intuition and experience.  The conversation explored whether or not the science was accurate, valid, predictive, consistent, and reliable.  The contrarions weighed in, the know-it-alls spoke up, and eventually, the supporters arrived in droves.  If you read only one article/discussion on sales selection tools in your lifetime, this must be the one.  Read and Join the discussion here, but I warn you, it contains a LOT of very compelling and highly-charged reading.

In February, I wrote another extremely popular article which won awards for best article of the day, week and month.  Depending on where it appeared, it had a title of either The 25 Ways That Selling Has Changed or How Dramatically Has Selling Changed?  One of the comments, by Chris Bealle, CEO of ConnectAndSell, asked a similar question about sales management, so last week I wrote How Dramatically Has Sales Leadership Changed for EcSell Institute's in advance of their Spring Coaching Summit (I'll be there speaking about The Four Keys to Selling Value).

As OMG celebrates its 25th year of pioneering, growing and perfecting the science of sales evaluation and sales assessments, I will have a lot more to say on this subject...starting right now.  For many years, Neil Rackham has long been considered the father of sales research.  After all, his body of work includes research on more than 10,000 salespeople, he wrote SPIN Selling, and he has had an impressive career on this side of sales.  As someone who loves comparison data, I would like to remind people that my data and research at OMG is nearing 1 million salespeople evaluated and assessed.  That's almost 100 times more data than Neil Rackham has and I have used it to write several award-winning White Papers.  He has sold more copies of SPIN Selling than I have of my book, Baseline Selling, but he had a 20-year headstart on me...  By the way, if you haven't read Baseline Selling, it continues to be a very popular 5-star read and I receive notes from people every single day telling me how much they love it and the impact it has had on growing their revenue.  Have you read Baseline Selling?

Topics: sales assessment, Dave Kurlan, personality, top sales books, sales selection tool, Validation, sales science, OMG Assessment, Customer Think

Validation of the Validation of the Sales Assessment

Posted by Dave Kurlan on Fri, Oct 04, 2013 @ 06:10 AM

validatedSome companies need to validate our validation.  Objective Management Group (OMG) uses Predictive Validity - the most time-consuming and expensive form of validation.  Unlike simpler methods of validation, Predictive Validity requires that we prove a connection to on-the-job performance.  The challenge is that our predictive validity is so good, some people just don't believe it and they want to revalidate it themselves.

One of the companies that insisted on validating our validation is moving forward with a license to hire 200 salespeople using our Sales Candidate Assessment.  I'll share the results of their own validation:

They conducted a 7-day pilot in April of 2012 and hired 23 salespeople.  

Our assessment recommended 13 of them, and did not recommend 10.  They reported that 9 of the 10 hired (who were not recommended) failed, and that 12 of the 13 (who were recommended) succeeded.  How do those numbers compare to our historical statistics?  

Historically, 75% of the candidates we don't recommend, who somehow get hired anyway, fail inside of 6 months.  In this pilot, 90% of the candidates that were not recommended failed.

Historically, 92% of the candidates we do recommend, who eventually get hired, succeed.  In this pilot, their success rate was also 92%.

Does it always work out like this?  Of course not.  Some companies just don't have the right sales management, sales process and systems in place so even the best candidates can fail or leave.  On the other hand, some companies, who have been using our processes, systems and tools for a while, consistently exceed these results.

The most common scenario where companies wish to do their own validation is when they are located overseas.  Despite the fact that our expansion overseas began more than a decade ago, some companies located outside of the US don't believe that a US-based tool will work in their country.  They have cultural differences to be sure, but those are more about relationships and the proper times and appropriate ways for people to interact in business settings.  Selling and what it takes for salespeople to succeed doesn't actually vary from culture to culture.  

Some countries lack selling sophistication - they're way behind - and are still selling very transactionally.  But if the company is ready to change, and their markets are ready for them to change, then they must be able to select salespeople who can make those changes as well.

Validation is an interesting process and if you look into it, you'll find that none of the personality or behavioral styles assessments use predictive validity because there simply isn't a correlation between their findings and on-the-job performance.

Topics: sales assessment, Dave Kurlan, sales evaluation, sales profile, Validation, sales test, objective management group

The Sales Assessment Client Who Didn't Renew after All These Years

Posted by Dave Kurlan on Mon, Mar 05, 2012 @ 10:03 AM

correllationHe has been a client of Objective Management Group (OMG) for over 20 years.  He had a license to use OMG's Sales Candidate Assessments and, as most clients do, had renewed it each year.  When we met for breakfast recently, he told me that he had a new VP of Sales and would not be renewing his license this year.  I was surprised for two reasons:  

  1. OMG clients nearly ALWAYS renew their candidate assessment licenses unless they are hiring just one or two salespeople, after which time they are done - they find who they are looking for and then they are finished hiring.  Companies that hire salespeople on an ongoing basis always renew their licenses because the salespeople they hire with our assessments are much more effective.
  2. He told me that the reason for not renewing was that the assessment did not correlate with their performance.  Seriously?  Our top-rated, highly-predictive assessment didn't predict success and failure?  "Send me the names and the outcomes".
I reviewed everything he sent me.  He gave me their names along with their scores for the OMG Assessment , Predictive Index Assessment, Industry Knowledge, Industry Experience, Group Interview and Performance Results.  Each score was rated on a 1-3 scale with 3 being the best.
They hired seventeen salespeople in the past five years.  I ran a simple face-value correlation analysis: How predictive was each score - at face value - at predicting the performance outcome?  Here are the results:
Step Face Value Correlation
OMG Sales Candidate Assessment 68%
Predictive Index 50%
Group Interview 54%
Industry Knowledge 38%
Industry Experience 38%

Face value correlation alone isn't enough. At OMG we have a mantra that we expect our clients to follow.  A candidate that is "Not Recommended" should never be hired.  A "Recommended" candidate can easily become a "No" based on their phone interview, whether their experience meets your requirements, and how well they perform during a face-to-face interview.  "Recommended" is not a license to automatically hire without further due diligence.  It is simply a recommendation that the candidate belongs in the pool of candidates that move to the next step.  This client was hiring candidates that were not recommended!

I reviewed the seven assessments that did not correlate at face value.  Two of the five met performance expectations and four were recommended by the OMG assessment, but with warnings and conditions.  In the table below, the scores in columns two and three are how the client conducted the scoring, not how OMG scores its assessments.  The client's scale is beneath the table.  Here are the findings:

ID OMG Score Performance Comments
1  3  1 Candidate's Assessment showed that while she had strong DNA, she is an excuse maker, isn't motivated by money, and had zero skills other than top-of-the-funnel skills and had will prospect as a weakness. It was there in black and white that while she met the criteria,  Salesperson #1 would not build a pipeline or move opportunities along!
2  3  2 Assessment was nearly identical to Salesperson #1 but with additional skills that went beyond top of the funnel.
 3  1 Assessment showed that he lacked direction - no goals, plan, tracking and would be unable to work independently.  He was assigned a remote territory!
 3  1.5 Assessment also showed will prospect as a weakness and not money motivated and 7 of the 10 skills we identified were top of the funnel skills.  Like Salesperson #1, Salesperson #3 would not build a pipeline or move opportunities along!
 2  1.5 Salesperson #5 was assessed twice - the first time not hirable, the second time with a huge red flag saying less than ideal. 
 3 Salesperson #6 had the lowest group interview score, industry experience score, and the lowest overall score of any candidate.  That correlated with the assessment's finding showing his inability to develop rapport early in the process. 
 1.5 Salesperson #7 met expectations only because of how likeable she was and her high scores in food service, and industry experience - she had a following.  She would have been more successful, but as the assessment showed, she wasn't a hunter, made excuses, wasn't money-motivated, and had zero selling skills other than her top-of-the-funnel skills.
3 - Exceeded    2 - Meets    1 - Failed

Conclusions: Inclusive of our recommendations not to hire, and the warnings and skill gaps associated with recommended candidates, the assessments accurately predicted the results in 16 of 17 cases between 2006 and 2010 - a batting average of 96%.  In the 17th case, Salesperson #6, the client failed to follow his own hiring process and would not have hired the individual based on his non-OMG scores.

The client assessed 2,500 candidates over this time period.  As a result, they were saved from having to speak with at least 2,300 candidates.  If they spent just five minutes reviewing 2300 resumes (191 hours) and ten minutes on the phone with one third of the candidates (126 hours), the assessment saved them 317 hours (two business months) of manual filtering.  If they value their time at $100 an hour, the license more than paid for itself in the value of time saved over five years ($41,700).

Monday morning quarterbacking is easy if you take the time to do it.  I chose to take the time.  This client is probably similar to many clients who either don't read the warnings, or don't factor the warnings into the decision-making process for bringing a salesperson on board. The warnings were certainly incorporated into the development plan or accountability requirements for success.

Message to Clients and Would-Be Clients of OMG:  Our assessments are incredibly predictive, but you need to pay attention to more than a single finding, recommendation or score!  Every assessment tells a story and if you take the time to read the story on the candidates who are recommended, you'll be able to determine whether your current sales management resources are up to the task at hand for borderline candidates:

  • How much hand-holding will they need?
  • Will they fill the pipeline on their own?
  • Will they be able to move opportunities through the process?
  • Will they be able to generate urgency?
  • Will they qualify thoroughly?
  • Will they be able to close?
  • Will they self-start?
  • Can they work independently?
  • Will they take short cuts?
  • Will they be ineffective because of weaknesses?
  • How long will it take them to ramp up?
  • Will they stick?
It's all there - it's all accurate - it's all predictive - one just has to read it!

Topics: sales assessment, omg, kurlan, Validation, Correlation, sales candidate assessment

Content not found
Subscribe via Email

View All 2,000 Articles published by Dave

About Dave

Best-Selling Author, Keynote Speaker and Sales Thought Leader,  Dave Kurlan's Understanding the Sales Force Blog earned awards for the Top Sales & Marketing Blog for eleven consecutive years and of the more than 2,000 articles Dave has published, many of the articles have also earned awards.

Email Dave

View Dave Kurlan's LinkedIn profile View Dave Kurlan's profile


Receive new articles via email
 to the Blog on your Kindle 



Most Recent Articles


Top 50 Sales & Marketing Blogs 2021

Sales & Marketing Hall of Fame Inductee

Hall of Fame

 Hall of Fame


Top Blog Post

Expert Insights

Top 50 most innovative sales bloggers


Top Blog

Hubspot Top 25 Blogs


2021 Top20 Web Large_assessment_eval