Jump to content




All Due Respect


85 replies to this topic

#1 Steve Wilson

    Needs 12 Step Group for Forum Recovery

  • CBBQA Members
  • PipPipPipPipPip
  • 1,530 posts
  • Interests:Sponsors:
    Fast Eddy Smokers by Cookshack, Q-Pellets, Brandt Beef, Dizzy Pig

Posted 30 March 2014 - 11:51 AM

I want to start by stating how thankful I am for the job our judges do to make competition BBQ so great in California. At the Sam's Club event in Sacramento, the promoter made a statement how every year the scores at the California events have always been the lowest in the country. He further said "it's not the food."

I've attempted to judge and it's a tough job. Everything comes at you so fast. As cooks we are now given score sheets that have the "Judge Average Score" along with the "Calculated Score" from the three scores we are given by each judge. We are also furnished the table # and our rank at that table for each meat. I have been tracking these numbers and comparing the results. There are two telling results I like to look at. First is the comparison of the Calculated score to the Judge Average Score. When I turn in what I think is a good entry I expect the Calculated Score to be higher than the Judges Average Score. When my Caculated Score is higher than the judges average I know I impressed the judge. As a team that is normally a bit better than average this is what I expect to see.

The second telling feature of the new score sheets is the comparison of each of the Judges Average scores to the other judges at my table. As an example my brisket table had these average scores for the 6 judges I was scored by: #1 @ 32.1664, # 2 @ 28.6842, # 3 @ 32.3448, # 4 @ 31.2770, # 5 @ 27.3009 and # 6 @ 30.6012. These scores are the judges averages for the day. The variance between judge # 6 and judge # 2 is 8.0228. The point here to judge # 2 is you are single handedly taking teams out of contension for the championship. An 8 point swing is the difference between 1st and 6th place. That's big money to the teams.

It's a shame that KCBS is not allowing our judges to look at the results the teams get. If you were able to see, hey my scores are a little high or a little low in comparison to my table mates, you might be able to self correct for your next assignment and the variance between the highest and lowest Judge Average Score could be reduced over time.

Keep doing the great job, I know the teams really appreciate the job you all do. Thank You.

#2 scott newcomb

  • CBBQA Members
  • 3 posts
  • Location:Hemet

Posted 30 March 2014 - 02:09 PM

I think you just showed your point.

#3 Scott Shimano

    Forum Problem Child

  • CBBQA Members
  • PipPip
  • 134 posts
  • Location:Redlands, CA

Posted 30 March 2014 - 03:13 PM

View PostSteve Wilson, on 30 March 2014 - 11:51 AM, said:

The second telling feature of the new score sheets is the comparison of each of the Judges Average scores to the other judges at my table. As an example my brisket table had these average scores for the 6 judges I was scored by: #1 @ 32.1664, # 2 @ 28.6842, # 3 @ 32.3448, # 4 @ 31.2770, # 5 @ 27.3009 and # 6 @ 30.6012. These scores are the judges averages for the day. The variance between judge # 6 and judge # 2 is 8.0228. The point here to judge # 2 is you are single handedly taking teams out of contension for the championship. An 8 point swing is the difference between 1st and 6th place. That's big money to the teams.

It's a shame that KCBS is not allowing our judges to look at the results the teams get. If you were able to see, hey my scores are a little high or a little low in comparison to my table mates, you might be able to self correct for your next assignment and the variance between the highest and lowest Judge Average Score could be reduced over time

Based on the example provided, it might be beneficial to make a comparison between the Judge's average score and what type of actual scores would be required to obtain that average. Judge #2 had an average of 28.6842, while Judge #6 had an average of 30.6012. Although I do not have an actual score that would produce the average, the following examples are close. 877 = 28.5600, 977 = 29.1200... Judge #2 was somewhere between theses scores as an average. 978 = 30.2628, 879 = 30.8456, 887 = 30.8572... Judge #6 was somewhere between these scores as an average.

Do your really believe it is so egregious that Judge #2 would give an 877, while Judge #6 gave an 887, that Judge #2 would need to "self-correct"??

#4 Kevin Barteaux

    Forum Nut Job

  • Registered User
  • PipPipPip
  • 956 posts
  • Location:Novato, CA
  • Interests:Anything to do with Cal Bears Football and Basketball, Fishing, and Barbecue.

Posted 30 March 2014 - 03:33 PM

Question: When I look at Judge 2 under pork ribs, is his average score for ribs only, or for all the meats combined?

One way to read the scores is to see where you finished at the table. If you finished first or second and ended up near the top, or third or fourth and ended up in the middle, or fifth or sixth and ended up towards the bottom, then you know things probably worked out were they should have.

#5 Steve Wilson

    Needs 12 Step Group for Forum Recovery

  • CBBQA Members
  • PipPipPipPipPip
  • 1,530 posts
  • Interests:Sponsors:
    Fast Eddy Smokers by Cookshack, Q-Pellets, Brandt Beef, Dizzy Pig

Posted 30 March 2014 - 04:03 PM

the judges avg is for all 4 meats and all six judges scored the same table all day. So when you have a variance, on average of 5 points it kills that team. Overall was 662.8116, if you lost 5 points because one judge has a low average, based on his or her table, that takes you to 657+/- and is the difference between a GC and 5th or 6th place. The tighter the spread between the high avg at a table and the low average at the table, the better the table. That table would be pretty consistant.

Part of the beauty of this whole thing is the judges opinion of your Q. I respect a judge that can render scores that are consistant, meaning they score good selection the same and poor selection the same regardless of the meat being scored. My thought is if there are good consistant judges that are continually high or continually low, the ones we see in our score sheets they can be made aware that they are a little high or a little low and possibly make adjustments.

I think this all starts with KCBS and their definitions for scoring. They list a 6 as average. I rarely get a six but when I do I am pissed. There is nothing average about a seasoned competition BBQ cooks Q. I think my worst competition BBQ blows the doors off Famous Dave's Restaurant BBQ. To me that is average, Competition BBQ is cooked for 6 people with the best ingredient and the best techniques. Average is an insult to a comp cook. Any Comp cook.

#6 Scott Shimano

    Forum Problem Child

  • CBBQA Members
  • PipPip
  • 134 posts
  • Location:Redlands, CA

Posted 30 March 2014 - 04:45 PM

I would like to think that in CA we have the best KCBS reps in the Country, and as such, their direction to the judge's is to judge within the KCBS guidelines. There are scores from 2 - 9 with words to define when any of those scores are to be assigned. If CA is scoring consistently lower than other states, why does one have to assume that the error is with CA judges, and not those elsewhere. Take a look at the following inquiry posted 1 1/2yrs ago on bbqcritic, and you will see why judging scores have potential discrepancies.

http://www.bbqcritic...ompetition.html

Fortunately, I have not heard these types of comments made while judging in CA. If someone submits an "average" entry (and there are many average, or below-average entries submitted during a typical contest), KCBS has conveniently provided direction about what an appropriate score would be. It would be unfair to give an average entry an "8" just because of the cooks hard work, while rewarding an excellent entry with a "9"... how would that 1 point variance provide an accurate evaluation of the entries submitted?

I would be interested to know if Big B's, Son of Smoke, Slap Yo' Daddy or Big Shot BBQ had any of their entries land on Pit Crew's brisket table. If they did, then the top 5 teams all faced the same Judge #2, and having a low scoring judge/table makes no difference to the results. If they didn't, then I have to question the purpose of this thread... to propose the need for judging changes, or to lament at the unfairness which resulted in "some teams" to win while others who clearly produced much better BBQ to be punished

#7 Steve Wilson

    Needs 12 Step Group for Forum Recovery

  • CBBQA Members
  • PipPipPipPipPip
  • 1,530 posts
  • Interests:Sponsors:
    Fast Eddy Smokers by Cookshack, Q-Pellets, Brandt Beef, Dizzy Pig

Posted 30 March 2014 - 09:14 PM

Big B's in Pork was the only other team you mentioned to land on the table.

#8 Matt Dalton

    Needs to Get a Life

  • CBBQA Board Of Directors
  • PipPipPipPip
  • 1,241 posts

Posted 31 March 2014 - 10:13 AM

Steve take the score sheet out and burn it..it's useless now and all you can do is cook on..trust me

#9 Bill Bain

    Forum Nut Job

  • CBBQA Members
  • PipPipPip
  • 943 posts
  • Location:Roseville, CA
  • Interests:CBJ
    Kilted Pig BBQ Team
    'Life's too short to drink cheap liquor or eat bad BBQ"

Posted 31 March 2014 - 11:34 AM

Having done double duty a number of times over the years I have seen that as a rule the newer judges seem to average one or two points lower than most experienced judges. There is no perfect scoring system but if you look at the scores over time you will see there are a few teams that seem to score higher year after year so all in all the system seems to be consistent. Perhaps the most interesting thing I have noticed is that team cooks who judge once in a while seem to judge the harshest.

#10 peter Zeman

    New to this forum

  • Registered User
  • 11 posts
  • Location:Moorpark, CA

Posted 01 April 2014 - 01:51 PM

With all due respect BBQ is NOT a math science. Its more about KISS and good times with family and friends.

#11 sterling ball

    Forum Problem Child

  • CBBQA Business Members
  • PipPip
  • 123 posts

Posted 02 April 2014 - 05:04 AM

at sams the top 5 rib scores came from the same table.....662 won it.

Sacramento Sams has a history of bizarre scoring....one year Steph won chicken with a 662

I am so happy to once again disagree with Kevin Winning your table is a mostly meaningless stat that you get to talk about....("I won all my tables and came in 6th!, All you can do is win your tables!" You can have a bad cook against 5 newbies or poor cooks.


Steve once again not looking deep enough...We all have a handicap so to speak...our running average of our scores both in a category and overall....A judge does too...SO unless you know the relative strength of the teams on the judges table its hard to say if it was a bad table. If a judge had an average score that matched the average cook of said teams then they are probably pretty close.

#12 Kevin Barteaux

    Forum Nut Job

  • Registered User
  • PipPipPip
  • 956 posts
  • Location:Novato, CA
  • Interests:Anything to do with Cal Bears Football and Basketball, Fishing, and Barbecue.

Posted 02 April 2014 - 06:43 AM

View Poststerling ball, on 02 April 2014 - 05:04 AM, said:

I am so happy to once again disagree with Kevin Winning your table is a mostly meaningless stat that you get to talk about....("I won all my tables and came in 6th!, All you can do is win your tables!" You can have a bad cook against 5 newbies or poor cooks.
Hence the word "probably" in my post. There are exceptions to just about everything.

#13 Scott Shimano

    Forum Problem Child

  • CBBQA Members
  • PipPip
  • 134 posts
  • Location:Redlands, CA

Posted 02 April 2014 - 07:18 AM

View Poststerling ball, on 02 April 2014 - 05:04 AM, said:

Steve once again not looking deep enough...We all have a handicap so to speak...our running average of our scores both in a category and overall....A judge does too...SO unless you know the relative strength of the teams on the judges table its hard to say if it was a bad table. If a judge had an average score that matched the average cook of said teams then they are probably pretty close.

It would be interesting to do a review of the judging scores at King of the Smoker, as this event had 24 of the best teams in the country present. If there were noticeably cold tables of judges there, it is more likely due to the judges than due to the teams themselves, as all of the teams had to present one of their entries before each judge

#14 Steve Wilson

    Needs 12 Step Group for Forum Recovery

  • CBBQA Members
  • PipPipPipPipPip
  • 1,530 posts
  • Interests:Sponsors:
    Fast Eddy Smokers by Cookshack, Q-Pellets, Brandt Beef, Dizzy Pig

Posted 02 April 2014 - 08:03 AM

Perhaps I did not make my original post clear. This is about the variance from the top scoring judge at a table to the lowest scoring judge at the same table. In a perfect world all 6 judges would score a turn in equally. But that is not feasible in the real world. My point was the judges variance(the difference between the high & low scores) on the average scores for that day could be improved upon by letting judges see their scores as compared to their tablemates. I believe most judge's are fair and pretty consistant through out a given day. Take a judge that has an average of around 28 for the day, when the table is averaging 31 per judge. If a judge was able to see that they were just a hair low, consistantly, they might be able to use this information to adjust their scoring in the next contest.

KCBS has created a great new score sheet that teams are using to see where we all stand, It's time judge's see the information to see if they are inline with their fellow judges.

#15 Abel Tirre

    CBBQA ToY Commissioner

  • CBBQA Members
  • PipPipPipPip
  • 1,144 posts
  • Location:South Orange County! Just follow the blue smoke!
  • Interests:To sin by silence when they should protest makes cowards of men.

Posted 02 April 2014 - 08:08 AM

View PostScott Shimano, on 02 April 2014 - 07:18 AM, said:

It would be interesting to do a review of the judging scores at King of the Smoker, as this event had 24 of the best teams in the country present.

You also have to keep in mind the rules of KOS didn't allow teams to use devices like stokers. Some teams cooked on borrowed equipment, some cooked after traveling across the country. All the meat was provided for the teams and they had to cook using just that meat. So although it was in fact some of the best teams in the country it was for certain that only the best cook would win under those circumstances.

#16 Kenny Lee

    Forum Irregular

  • Registered User
  • Pip
  • 68 posts
  • Location:San Jose, CA
  • Interests:BBQ, San Jose Sharks, BMX, Football, NASCAR

Posted 02 April 2014 - 08:10 AM

Why do you think the low judge was incorrect? Maybe the other 5 judges need to be retrained and are too generous?

The judging system is incredibly subjective. It is what it is. Until kcbs makes changes to the way judges are trained, this will continue to happen.

The lowest score is always dropped. You got dinged by two judges.

Edited by Kenny Lee, 02 April 2014 - 08:13 AM.


#17 Steve Wilson

    Needs 12 Step Group for Forum Recovery

  • CBBQA Members
  • PipPipPipPipPip
  • 1,530 posts
  • Interests:Sponsors:
    Fast Eddy Smokers by Cookshack, Q-Pellets, Brandt Beef, Dizzy Pig

Posted 02 April 2014 - 08:18 AM

View Poststerling ball, on 02 April 2014 - 05:04 AM, said:


Steve once again not looking deep enough...We all have a handicap so to speak...our running average of our scores both in a category and overall....A judge does too...SO unless you know the relative strength of the teams on the judges table its hard to say if it was a bad table. If a judge had an average score that matched the average cook of said teams then they are probably pretty close.
Perhaps you are looking too deep. I don't think the relative strength of the teams at a specific table is critically important. When we turn in good BBQ, we know it. Sometimes judges know it too and reward us with good scores. If you are at a table and you turn in great food but the 5 others at the table do not, this would be a weak table, but your food scored great, relative strength has nothing to do with it. My point is if you get 2 or more judge's that are tough judge's, meaning everything they judge they judge lower than the average at the table, a team is at a great disadvantage. One low scoring judge can have their scores dropped. Get two or more low scoring judges at a table and it's over.

#18 Sylvie Curry

    Needs 12 Step Group for Forum Recovery

  • CBBQA Members
  • PipPipPipPipPip
  • 1,703 posts

Posted 02 April 2014 - 10:03 AM

If you are at a table where all 6 samples score in the top overall for that category does that tell you that you turned in some great product though you may have been 5th or 6th on that table or does it tell you that table over scored compared to the judges at the other five tables? Sometimes you hit sometimes you miss. There were a few judging irregularities at SAMs Club from judges scoring low on what they thought was a raw rib to one table producing the top rib scores which was a primary game changer for the six who moved on to Las Vegas. With all due respect.

#19 jason miranda

    What can I say I like it here

  • CBBQA Members
  • Pip
  • 46 posts

Posted 02 April 2014 - 10:13 AM

View PostScott Shimano, on 02 April 2014 - 07:18 AM, said:

It would be interesting to do a review of the judging scores at King of the Smoker, as this event had 24 of the best teams in the country present. If there were noticeably cold tables of judges there, it is more likely due to the judges than due to the teams themselves, as all of the teams had to present one of their entries before each judge

Even the best teams in the country have “off days” or worst catastrophic days; that may result in a team having lower scores and/or even a disqualification. You forgot to mention, KOS had 24 of the best judges in the country, along with the best reps in the business.

#20 Matt Dalton

    Needs to Get a Life

  • CBBQA Board Of Directors
  • PipPipPipPip
  • 1,241 posts

Posted 02 April 2014 - 10:24 AM

View PostSteve Wilson, on 02 April 2014 - 08:18 AM, said:


Perhaps you are looking too deep. I don't think the relative strength of the teams at a specific table is critically important. When we turn in good BBQ, we know it. Sometimes judges know it too and reward us with good scores. If you are at a table and you turn in great food but the 5 others at the table do not, this would be a weak table, but your food scored great, relative strength has nothing to do with it. My point is if you get 2 or more judge's that are tough judge's, meaning everything they judge they judge lower than the average at the table, a team is at a great disadvantage. One low scoring judge can have their scores dropped. Get two or more low scoring judges at a table and it's over.
Are you crazy?..who you land on the table with has EVERYTHING to do with winning it..it's human nature to know which entry is best and that's the way it should work and it does..think about this though if you land on a table with 5 teams that are inexperienced your chances of doing well or winning the table are much higher than if you ended up on a table full of top level teams..so the strength of the table is critical IMO..also I'll use Laughlin last year when you came home with the car..that day there was a table of death in play and an epic one to say the least..the highest finishing team that hit that table was17th overall..you escaped that one that day and I'm sure your cook was a great one to win but you see my point..there will always be hot and cold tables and until KCBS takes the data being collected at puts it to use I don't think we'll see too much change under the current rules..there is no accountability on the judges part yet as cooks we can be dq'd over a tiny speck of foil...once the judging has some accountability then things will get better.




Reply to this topic



  


1 user(s) are reading this topic

0 members, 1 guests, 0 anonymous users