Being able to predict the next few question types – or knowing how much longer until the next Memory Verse question will be asked – allows you to make strategic choices to put your team in the best possible position for success.
Impossible, you say? What if I told you there are patterns inherit in the algorithm that generates competition questions? For certain conditions, it is not the totally random selection process as I had believed.
Here are the answers to the 6 questions I asked to better understand how the question set generator works:
As it turns out, the answer is “it depends”.
If all of the question sets for the event were created as a single file download, then the first 150 sets or so contain very few – if any – questions that repeat. Quiz directors will do this to ensure all of the quizzers in all of the rooms hear the same questions over the course of the event.
However, if sets of competition questions are generated separately for each room then questions are repeated between the rooms. This happens because the questions chosen for Room 1 are put back in the question pool used to generate questions for Room 2. While Room 1 will not likely ask any question twice, Room 1 may ask questions asked in Room 2 (and vice versa).
In my study, 14% of the questions between 2 file downloads containing 10 competition sets each are repeats. With 5 files (or Rooms) of 10 competition sets each, 33% of all questions are repeats (or about 6 per game). Just keep in mind that your team will not hear all of these questions that get repeated because they will be competing in 1 room while the other rooms are using different questions. As you change rooms throughout the event, if each room is using a separately-generated batch of questions you can expect to hear 1-2 questions per game that were asked in one of your previous games.
So, if you have a bye round cheer on a fellow team and pay attention to the questions – you may hear some of them again!
Interestingly, Memory Verse questions rarely occur on the first question. If it was statistically random, 15% of the games should start with a Memory Verse (3 memory verses divided by 20 available questions). Instead, Memory Verses only started 3% of the games. As the game progresses, the odds increase for the appearance of the first Memory Verse question. After question #5, there is about an 80% chance that question #6 will be the first Memory Verse. In none of the games I studied did the first Memory Verse question show up after question #10.
Conversely, According To questions represent 30% of all #1 questions (instead of the expected 20%). The first According To question of a game has a 78% of appearing on or before question #5.
Memory Verses, Book & Chapter, and Context questions are strangely under-represented on questions 1 and 20. Thus, there is a 96% chance that Question 1 is either an According To or General question and a 91% chance that Question 20 is an According To or General question.
Book & Chapter questions tend to occur in the first half of the game (61% of the time). Further, they appear between questions #5 and #9 more than half of the time.
During the game, for the purposes of planning substitutions you may want to know when the next According To or Memory Verse question will appear.
In the totality of this study, there were absolutely no back-to-back Memory Verse questions. Nor did Memory Verses appear with 1 question between them. Instead, 2 questions is minimum and most frequent gap between Memory Verses. The average gap between Memory Verses is 4.3 questions and 90% of the time the gap between Memory Verse questions is 7 or less.
About once a season (or 1 out of every 100 games), a team may encounter a game where two According To questions are asked back-to-back. More likely, however, According To questions are separated by up to 12 questions with an average gap of 3.7 questions. 91% of the time the gap between According To questions is 7 or less.
For General questions, this one is just math. Simply divide the number of General questions remaining by the number of questions remaining in the game. If the previous question was a Memory Verse or According To question, the chances that the next question is a General Question increases.
The higher the current gap since the last Memory Verse or According To question, the more likely those types of questions will be asked next.
Here is an example:
- Question 1 was a General question.
- Question 2 was an According To question
- Question 3 was a General question
- Question 4 was an General question
What will Question 5 most likely be?
- Memory Verse = 69% probability (from the previous “First MV Question” chart)
- General = 50% probability (8 General questions remaining/16 questions available)
- According To = 41% probability (from the “AccTo Gap” chart)
- B&C = 30% probability (from the previous “Book & Chapter Location” chart)
- X = 21% probability (from the “Context Location” chart)
No, the question set generator does not show favoritism. In repeated trials, the questions that get repeated between the files vary. While this is good news, it does in effect debunk this persistent myth believed by quizzers and coaches alike.
ARE CHAPTERS EQUALLY REPRESENTED IN EACH GAME?
Having completed this study, I am pleased that the data indicates the question set generator is working as intended to “balance” the game: Questions within a file of question sets do not tend to repeat, According To and Memory Verse questions are purposely spread throughout the game, and specific questions do not appear more than others when multiple files are generated.
As a Quiz Director, I now know how to minimize the number of repeat questions during a tournament.
As a Coach, the timing and effectiveness of my player substitutions will be greatly improved now that I have a better feel regarding what types of questions will be asked soon.
Naz Quiz Scoresheet for iOS Numbers (2-team quizzing)
Naz Quiz Scoresheet for iOS Numbers (3-team quizzing)
On an iPad, after the link opens choose the export button (the button to the right of the address bar) and select "Copy to Numbers". If you have trouble, try the Excel link below and export that to Numbers.)
Naz Quiz Scoresheet for MS Excel (2-team quizzing)
Naz Quiz Scoresheet for MS Excel (3-team quizzing)