Recruiting rankings operate in a similar vein. That is, the people that are strong believers in the system have plenty of information available to defend their claims, while skeptics also possess a wealth of data to disprove the rankings' legitimacy.
Scott Kennedy, Scout.com's Director of Scouting, summed up the entire circus pretty well in an interview with Inside Carolina this time last year, saying, "People that say that recruiting rankings is everything are wrong, and people that say that recruiting rankings don't mean anything are wrong. Obviously, it means something. There's a pretty strong correlation between the top-ranked recruiting teams every year and the top teams every year."
The evidence in that regard is telling. The last eight BCS national champions all had at least one top-five recruiting class on their roster, and in most cases, there were numerous top-10 classes. Florida's 2007 national championship included four recruiting classes ranked in the top-8, LSU's 2008 title team had four classes that finished seventh or better, and Alabama's 2009 championship squad included two classes ranked in the top-two on National Signing Day.
But winning a national championship is out of the realm of possibility for most programs, leaving conference titles as the primary goal when teams take the field each August. The accuracy of the team recruiting rankings is even more apparent at that level. Southern Cal consistently lands the top class in the Pac-10, while national powers such as Ohio State (Big Ten) and Texas (Big 12) typically headline their respective conferences.
With all of the recent talk concerning non-BCS programs such as Boise State, TCU, BYU and Utah, the evidence is still readily available. The Broncos have landed the top recruiting class in the WAC in three of the last five seasons, while securing the highest average star ranking (star count divided by number of recruits) in all five seasons. TCU, BYU and Utah have dominated the Mountain West Conference over the past four years, finishing 1-2-3 on Signing Day each February.
Even so, there are plenty of head coaches that scoff at the notion of recruiting rankings, such as N.C. State head coach Tom O'Brien. But there's little doubt that O'Brien's former team – Boston College – benefitted by signing the top-rated class in the Big East in '04 after landing the third-ranked class in '03.
As with all things, there will always be exceptions. Wake Forest delivered recruiting classes ranked 10th or worse in the ACC three straight years prior to its 2006 ACC Championship and Orange Bowl berth, while Cincinnati finished dead last in the Big East in '05 and '06 before making back-to-back trips to the BCS the past two seasons.
Then again, there's a reason Brian Kelly recently landed the most high-profile job in America at Notre Dame. Great coaching can overcome talent deficits, just like bad coaching often fails to take advantage of the talent on hand – just look at former N.C. State coach Chuck Amato's ninth-ranked class and former UNC coach John Bunting's 13th-ranked class back in 2004.
Talent doesn't guarantee success, but it is extremely difficult to be successful without talent.
Which leads us to individual rankings. It's one thing to label Florida's 2010 recruiting class as the best in the nation, but how can someone accurately state that Player A is two spots higher than Player B in the top-100 nationally or even at the state level? That answer is easy – they can't.
"The rankings are extremely subjective," InsideCarolina.com recruiting reporter Don Callahan said on Tuesday. "They are influenced heavily by perception. Also, the people who do the rankings don't have the proper materials to "accurately" rank the 2,000-plus recruits that sign with Division I-A programs every year."
According to Scout.com, 2,448 high school players signed college scholarships with D-1 schools for the class of 2009 last February. While that may sound like a large number, it's important to note that there are tens, if not hundreds, of thousands of high school seniors playing football each fall.
With so many prospects to grade and only a handful of scouts available to classify each player, the process takes a serious turn into the direction of skepticism. The increase in high school combines have helped to replicate the gathering of national talent that AAU ball has done for basketball, but for players that are unable to attend these events for any given reason, their rankings are often based upon film study.
In Callahan's opinion, that process opens the door for a wealth of scouting mistakes.
"There are a lot of intangible attributes you can witness in-person that you can't see on film," Callahan said. "For example, how the recruit acts on the sideline or in between plays. Is he a leader, does he get frustrated, etc. You can also observe a lot during pre-game and post-game. Additionally, certain positions, most notably quarterback, need to be evaluated in-person. No matter the quality of the video, it's tough to judge attributes, such as throwing velocity, on film."
Quite possibly the best judge of high school recruiting rankings can be found on the professional level, namely the NFL Draft. Inside Carolina enlisted the help of Matt Ritter, a doctoral candidate for Statistics at N.C. State University, to break down the ‘09 draft as it related to star value.
His initial response highlighted one of the glaring issues with recruiting rankings – lack of sample size.
The '09 draft primarily included recruits from the '04-'06 classes, ranging from third-year juniors to fifth-year seniors. The problem is that recruiting services were still in their infancy back in '04. Scout.com's first available rankings occurred in '02, but the large number of unevaluated prospects in those early years was detrimental to validity of the rankings. Things have improved dramatically in that regard over the years, but until there is more credible information available, the opportunity for a concrete correlation is not possible.
Even so, Ritter ran the numbers for the '09 draft and delivered some startling figures. While the odds of a 5-star recruit being drafted were six times higher than those of a 2-star recruit, the odds of a 4-star were only twice as high as those of a 2-star and it was essentially a wash between 3-stars and 2-stars.
"The difference seems to be that 5-stars have the best odds of being drafted, 4-stars have pretty good odds and after that, it's a crapshoot," Ritter said. "There is virtually no difference between the 3-stars and 2-stars in terms of getting drafted."
It's also worth noting that eight percent (21-of-256) of the draftees hailed from non-FBS programs, such as Abilene Christian, St. Paul's and Furman.
So when you're arguing your stance on recruiting rankings with a friend or co-worker following the Signing Day festivities on Wednesday, just know that he may have an equal amount of evidence on his side of the table to prove you wrong.