TD guidelines

This is where all of the games are discussed.

Moderators: alietr, trainman, econgator, dhkendall

User avatar
RandyG
Founder of the Royal House of JBoardie of the Month
Posts: 2006
Joined: Wed Jul 13, 2011 6:23 pm
Location: Marana, AZ
Contact:

TD guidelines

Post by RandyG »

As the keeper of the TD queue and archive I get a lot of questions PMed to me about writing and running TDs, particularly from people running them for the first time. Now that we've got a regular schedule going, I was thinking I would write up a set of guidelines from past experience. Before I get around to writing and then posting for comment, if you have ideas/recommendations that you'd like to see included, go for it here. Issues that frequently come up, or are inherent in running a quiz:

- Was asked most recently about multiplicative, rather than additive, scoring.
- Other scoring issues, incorrects, lifelines, complicated scoring rules
- Open for entry, reveal periods
- Methods for tabulating scores and formatting the results for posting
- Themed TDs vs. general
- Independent reviewer/editor
- Time involved
.
.
.
User avatar
dhkendall
Pursuing the Dream
Posts: 8789
Joined: Fri Jul 01, 2011 11:49 am
Location: Winnipeg, Manitoba
Contact:

Re: TD guidelines

Post by dhkendall »

RandyG wrote:As the keeper of the TD queue and archive I get a lot of questions PMed to me about writing and running TDs, particularly from people running them for the first time. Now that we've got a regular schedule going, I was thinking I would write up a set of guidelines from past experience. Before I get around to writing and then posting for comment, if you have ideas/recommendations that you'd like to see included, go for it here. Issues that frequently come up, or are inherent in running a quiz:

- Was asked most recently about multiplicative, rather than additive, scoring.
- Other scoring issues, incorrects, lifelines, complicated scoring rules
- Open for entry, reveal periods
- Methods for tabulating scores and formatting the results for posting
- Themed TDs vs. general
- Independent reviewer/editor
- Time involved
.
.
.
I would like to see people put up (on Google Docs or another publicly available site) a scoring spreadsheet to help potential smiths. I had one (before my hard drive crash this summer) and Vermonter and someone else said they made ones as well.
"Jeopardy! is two parts luck and one part luck" - Me

"The way to win on Jeopardy is to be a rabidly curious, information-omnivorous person your entire life." - Ken Jennings

Follow my progress game by game since 2012
User avatar
econgator
Let's Go Mets!
Posts: 10673
Joined: Fri Jul 01, 2011 10:32 am

Re: TD guidelines

Post by econgator »

RandyG wrote:As the keeper of the TD queue and archive I get a lot of questions PMed to me about writing and running TDs, particularly from people running them for the first time. Now that we've got a regular schedule going, I was thinking I would write up a set of guidelines from past experience. Before I get around to writing and then posting for comment, if you have ideas/recommendations that you'd like to see included, go for it here. Issues that frequently come up, or are inherent in running a quiz:

- Was asked most recently about multiplicative, rather than additive, scoring.
- Other scoring issues, incorrects, lifelines, complicated scoring rules
- Open for entry, reveal periods
- Methods for tabulating scores and formatting the results for posting
- Themed TDs vs. general
- Independent reviewer/editor
- Time involved
.
.
.
Quite simply, the correct answer to most questions is: your game, your rules (granted, the more difficult you make those rules, the less likely people are to enter).
User avatar
Magna
Hooked on Jeopardy
Posts: 3079
Joined: Thu Jan 19, 2012 2:37 pm

Re: TD guidelines

Post by Magna »

Maybe something about writing and selecting good questions, and also what order the questions should go in? Question order is a little overlooked, but keeping the first and last questions straightforward and easy to score is helpful. Maybe also something on basic fairness and etiquette expected of whoever runs the TDs - mainly just to people know what types of behavior would attract or repel people from entering any future TDs they might run.
User avatar
RandyG
Founder of the Royal House of JBoardie of the Month
Posts: 2006
Joined: Wed Jul 13, 2011 6:23 pm
Location: Marana, AZ
Contact:

Re: TD guidelines

Post by RandyG »

econgator wrote: Quite simply, the correct answer to most questions is: your game, your rules (granted, the more difficult you make those rules, the less likely people are to enter).
No disagreement, and your points are what it generally comes down to when I'm asked for recommendations. I lay out the pros and cons based on my experience and let the quizmaster take it from there. My calling this guidelines wasn't intended to put together a list of rules, rather "I wish I had known this prior to writing and hosting" and "these are things that will help you" lists.
Bamaman
Also Receiving Votes
Posts: 12897
Joined: Sat Jul 02, 2011 1:39 pm

Re: TD guidelines

Post by Bamaman »

I have seen games with bizarre mathematical formulas to determine scores. While I play every game, I think games where the scoring gets too confusing turns some people off. The simplest to me are ones where you get a point for everyone who answers with some sort of penalty (though not too high) for a wrong answer.

I've never really concerned myself with question order, though I can see where a really tough one right off the bat is going to scare off some players. Letting people get their feet wet seems to make for a better game.

Also, I prefer questions with around 10-15 or so possible answers. Obviously, keeping every question within that limit isn't always an option. But I'd suggest trying to keep most of them within that limit. If there are too few answers, everyone gets a high score and it is the opposite problem with a question with infinite answers.

I also like games with a connecting theme, it makes it more interesting to me.

Basically, I think the games that work best are straightforward....here are the questions, answer them. No need to get too fancy.
User avatar
Magna
Hooked on Jeopardy
Posts: 3079
Joined: Thu Jan 19, 2012 2:37 pm

Re: TD guidelines

Post by Magna »

Bamaman wrote:I've never really concerned myself with question order, though I can see where a really tough one right off the bat is going to scare off some players. Letting people get their feet wet seems to make for a better game.
I think it's a good idea to begin and end with questions you're confident can be scored easily and straightforwardly. If you start with one that requires you to make tough judgment calls or calculations, or to ask people to BMS, the reveal may be delayed while you sort it out or wait for responses. If you end with one that might need a lot of corrections or changes, you might inadvertently announce final results only to have to change it later.

As an example, "Name a U.S. state that starts with M" is going to be easy and quick to score. There's a limited set of correct responses, no chance you'll have to decide whether two answers are the same or different, and little chance you'll need to ask someone to BMS. A couple of examples of hard-to-score questions are "Name a primate" or "Name a word that rhymes with 'red'." The former will undoubtedly draw some overlapping answers like "gorilla," "mountain gorilla," and "lowland gorilla," which will take time to sort out. The latter will draw disputable answers like words you might or might not consider real words (foreign words, proper names, unusual inflected forms, etc.), or words whose rhyme is questionable (like "wanted" or "infrared").
User avatar
goforthetie
(username no longer operative)
Posts: 1337
Joined: Tue Oct 18, 2011 5:01 pm

Re: TD guidelines

Post by goforthetie »

Ideally, one would steer clear of questions that partly rely on some criterion that isn't widely known. This would include objective but fairly technical criteria (did a movie make over $50M at the box office? did a single hit the top 10, or just the top 40? did a pitcher pitch more than 2000 innings?) as well as opinion-based lists. However, sometimes this is unavoidable, in which case you will want to have at least a few mitigating factors. Here are some that I've seen:

* If the possible answers all come from a relatively small, well-known set, that's good. For instance, US states, countries of the world, Best Picture Oscar winners, that sort of thing.

* Choose a cutoff wisely. Let's say you want to ask about the countries that produce the most titanium. Don't just pick a round number (top 10); rather, look at the data and pick a cutoff where there is a big gap - maybe country #13 produces 253 tons whereas country #14 produces only 68. Then, tell us that you want the top 13, give the stats for #13 and #14, and maybe even tell us the identity of country #14.

* Make sure your source is reputable and reliable, and describe its methodology if appropriate.

* Anticipate wrong but highly plausible answers, and find some way to head them off. For instance, I once asked a question about acts that won the Best Rap Album Grammy, and gave a list of artists that had been nominated in the category but not won. If I had not done that, I suspect there would have been a huge number of negs.

* Give players some way to plausibly guess at answers. Sherm did this quite well with his question about ESPN's top 15 athletes, by telling us what sport(s) each was associated with.

* If you need to use a subjectively compiled list, make sure it is as close to "critical consensus" as possible. Here's a question I used that turned out pretty well (it also incorporates the "anticipate wrong but highly plausible answers" technique):
In November 2008 the British magazine Gramophone published the results of a survey of classical music critics, who were asked to name the best symphony orchestras worldwide. Separately, the Encyclopedia Brittanica Almanac 2009 gave a list of 25 World-Class Orchestras. Name a city that is the home base of an orchestra ranked on both lists.

There are 12 orchestras on both lists. When I asked my brother, a professional orchestra member, about this question, he said 8 of the 12 are canonical consensus picks that most anyone in the business would fully expect. (Of course, this is not to say that the other four are out of left field or anything.)

Brittanica did not include festival orchestras or opera company orchestras in its list; Gramophone did. Neither had orchestras that only rose to prominence very recently (in the last 5-10 years). To help you out, here are the orchestras that are on one list only:

Brittanica:

Israel Philh. Orch.
London Philh. Orch.
Nippon Hoso Kyokai (NHK) Symph. Orch.
Orch. de la Suisse Romande
Orch. de Paris
Orch. National de France
Orch. Symph. de Montreal
Oslo Philh. Orch.
Philadelphia Orch.
Philharmonia Orch. (London)
Pittsburgh Symph. Orch.
Royal Philh. Orch. (London)
St. Louis Symph. Orch.

Gramophone:

#6: Bavarian Radio Symph. Orch.
#9: Budapest Festival Orch.
#10: Saxon State Orch.
#14: Mariinsky Theatre Orch.
#16: St. Petersburg Philh. Orch.
#18: Metropolitan Opera Orch.
#19: Saito Kinen Symph. Orch.
#20: Czech Philh.
* I'm sure others will add to this list...
User avatar
econgator
Let's Go Mets!
Posts: 10673
Joined: Fri Jul 01, 2011 10:32 am

Re: TD guidelines

Post by econgator »

goforthetie wrote:* Choose a cutoff wisely. Let's say you want to ask about the countries that produce the most titanium. Don't just pick a round number (top 10); rather, look at the data and pick a cutoff where there is a big gap - maybe country #13 produces 253 tons whereas country #14 produces only 68. Then, tell us that you want the top 13, give the stats for #13 and #14, and maybe even tell us the identity of country #14.
I got bit by that a while back. I think I asked for hockey players who scored 600+ goals (I think there were 10 or 15 -- a nice cut-off), not bothering to look slightly down to see the next guy had 599. :oops:

Also, when dealing with things like population, size, etc, make sure you give exact criteria. Don't ask for the 20 most populous cities in Texas. Ask for The 20 most populous cities in Texas within the defined city limits as listed on the 2010 Census.
Austin Powers
Watches Jeopardy! Way Too Much
Posts: 1783
Joined: Tue Nov 15, 2011 8:09 pm

Re: TD guidelines

Post by Austin Powers »

I would add, "Don't make a question that's basically about Jeopardy minutiae." I don't play TD that have such questions - I don't know them, there's no reason to think I should know them, I'm not a more well-rounded person or even a better trivia player for knowing them. I realize that this is a site devoted to the show... but the reason I watch the show is because I like trivia, not because I can't wait to see if the current champ can go a on a five game streak where he wins >$10,000, then >$20,000, then >$30,000, and then >$40,000 in consecutive episodes, becoming the first person to do that since [someone else I don't care about]. And I think it's fine that people know this stuff... it's just not what a TD should be about, IMHO.
teapot37
Watches Jeopardy! Way Too Much
Posts: 2054
Joined: Mon Aug 01, 2011 10:02 pm
Location: Louisville KY

Re: TD guidelines

Post by teapot37 »

econgator wrote:Also, when dealing with things like population, size, etc, make sure you give exact criteria. Don't ask for the 20 most populous cities in Texas. Ask for The 20 most populous cities in Texas within the defined city limits as listed on the 2010 Census.
This. Pin your answers.

My rule of thumb for questions is 8-16 correct answers, two or three of which most people should know. Try not to use subjective questions (most famous, most influential, best _____). If there are any borderline answers, either explicitly allow them or explicitly disallow them (and say what they are). I may not always follow these rules, but these are rules I try to stick to.
Not many people can say they've lost four times on Jeopardy!.
User avatar
dhkendall
Pursuing the Dream
Posts: 8789
Joined: Fri Jul 01, 2011 11:49 am
Location: Winnipeg, Manitoba
Contact:

Re: TD guidelines

Post by dhkendall »

Austin Powers wrote:I would add, "Don't make a question that's basically about Jeopardy minutiae." I don't play TD that have such questions - I don't know them, there's no reason to think I should know them, I'm not a more well-rounded person or even a better trivia player for knowing them. I realize that this is a site devoted to the show... but the reason I watch the show is because I like trivia, not because I can't wait to see if the current champ can go a on a five game streak where he wins >$10,000, then >$20,000, then >$30,000, and then >$40,000 in consecutive episodes, becoming the first person to do that since [someone else I don't care about]. And I think it's fine that people know this stuff... it's just not what a TD should be about, IMHO.
I would definitely agree (don't even care for Jeopardy minutiae outside of TDs), but something like that can be answered by those like us that don't care for inconsequential Jeopardy trivia and still be about Jeopardy. For example: "Name someone who has won more than 7 games on Jeopardy!" Even a non-boardie would have one answer (we all know which player), and boardies could most likely add a few more. Those who do know the ins and outs of show trivia could list all of them.
"Jeopardy! is two parts luck and one part luck" - Me

"The way to win on Jeopardy is to be a rabidly curious, information-omnivorous person your entire life." - Ken Jennings

Follow my progress game by game since 2012
User avatar
DadofTwins
Watches Jeopardy! Way Too Much
Posts: 1206
Joined: Fri Jul 01, 2011 10:11 am
Location: Fortress of SHC-itude
Contact:

Re: TD guidelines

Post by DadofTwins »

The one suggestion I would add is genre variety. I like the idea behind themed TD's, but I really enjoy the ones that call for a broad range of knowledge. If your theme will allow it, try to include a literature question, a history question, a geography question, a pop culture question or two, etc.

I also enjoy scoring systems that are a little more than "one point per person giving this answer" if they call for some strategy. That said, I learned the hard way that weirdness for its own sake can make scoring a TD . . . uh, complicated. So be creative, but use some discretion.
User avatar
StevenH
Not J! Contestant Material
Posts: 2524
Joined: Fri Jul 01, 2011 6:24 pm
Location: Atlanta, Georgia

Re: TD guidelines

Post by StevenH »

Austin Powers wrote:I would add, "Don't make a question that's basically about Jeopardy minutiae." I don't play TD that have such questions - I don't know them, there's no reason to think I should know them, I'm not a more well-rounded person or even a better trivia player for knowing them. I realize that this is a site devoted to the show... but the reason I watch the show is because I like trivia, not because I can't wait to see if the current champ can go a on a five game streak where he wins >$10,000, then >$20,000, then >$30,000, and then >$40,000 in consecutive episodes, becoming the first person to do that since [someone else I don't care about]. And I think it's fine that people know this stuff... it's just not what a TD should be about, IMHO.
I would add that, in general, it is better to avoid questions like this that are too esoteric. This can be hard to avoid, though, especially if you are trying to come up with TD questions that have never been used before. I'll admit that I have done this myself. For example, in the first TD game that I hosted (TD 17--I can't believe that it has been nearly 5 years!), one of my questions was "Name a character who Dan Castellaneta voices on The Simpsons." For a themed TD on The Simpsons, that question would have been okay. However, for a general knowledge TD it is too hard. I am sure that most boardies could make reasonable guesses, but if you are not a fan of The Simpsons (or even if you are) then you are probably likely to guess a character who is voiced by Harry Shearer or Hank Azaria. I think that a better question might have been "Name someone who has provided the voice of a character in five or more episodes of The Simpsons." That was my first of five regular TD games that I hosted, so I would like to think that I learned my lesson and did better on the next four.

Since I kind of mentioned this, too, I will add that it is probably a good idea to do a search and see if any question that you want to use has already been used. I do think that it is okay to reuse questions if they were last used more than one year prior to when you are running your TD contest, but I think that it is better to try to be original.
User avatar
dhkendall
Pursuing the Dream
Posts: 8789
Joined: Fri Jul 01, 2011 11:49 am
Location: Winnipeg, Manitoba
Contact:

Re: TD guidelines

Post by dhkendall »

DadofTwins wrote:The one suggestion I would add is genre variety. I like the idea behind themed TD's, but I really enjoy the ones that call for a broad range of knowledge. If your theme will allow it, try to include a literature question, a history question, a geography question, a pop culture question or two, etc.
Like my "Trivial Pursuit" themed TD a while back - it was themed, but it had questions from the categories in the Genus edition, making it a wide range of topics.
"Jeopardy! is two parts luck and one part luck" - Me

"The way to win on Jeopardy is to be a rabidly curious, information-omnivorous person your entire life." - Ken Jennings

Follow my progress game by game since 2012
User avatar
Magna
Hooked on Jeopardy
Posts: 3079
Joined: Thu Jan 19, 2012 2:37 pm

Re: TD guidelines

Post by Magna »

teapot37 wrote:
econgator wrote:Also, when dealing with things like population, size, etc, make sure you give exact criteria. Don't ask for the 20 most populous cities in Texas. Ask for The 20 most populous cities in Texas within the defined city limits as listed on the 2010 Census.
This. Pin your answers.
You can be strict or lenient about what you'll accept, but make clear what it is you are asking for. You'll have to make some tough calls sometimes, but you should always aim to be consistent and fair. Trick questions or changing the question without notice (or worse, after the entry deadline has passed) are not well-received.

Also, be prepared for some sly answers that technically answer the question correctly but aren't exactly what you were thinking of. If you want to eliminate those, do it by pinning the answers in some way. If someone manages to outwit you with a sly answer that's still correct, congratulate them and give them the points.
Bamaman
Also Receiving Votes
Posts: 12897
Joined: Sat Jul 02, 2011 1:39 pm

Re: TD guidelines

Post by Bamaman »

Have someone review your question list. A question can make perfect sense to you, but it might confuse the heck out of the rest of the world. I've used Randy for a couple of my games and it has been a big help. He's also served as a sounding board in debating whether to take a certain answer or just to chew the fat when dealing with the stress and hassle of scoring the game.
bpmod
Rank
Posts: 5424
Joined: Fri Jul 01, 2011 12:26 pm
Location: Hamilton Ontario

Re: TD guidelines

Post by bpmod »

Bamaman wrote:... dealing with the stress and hassle of scoring the game.
By George! Don't say things like that! You'll scare away those that haven't yet run one.

Brian
...but the senator, while insisting he was not intoxicated, could not explain his nudity.

If I had 50 cents for every math question I got right, I'd have $6.30 by now.
User avatar
RandyG
Founder of the Royal House of JBoardie of the Month
Posts: 2006
Joined: Wed Jul 13, 2011 6:23 pm
Location: Marana, AZ
Contact:

Re: TD guidelines

Post by RandyG »

Some very good suggestions...... appreciated by all. Keep them coming as you deem appropriate and I will start posting a running summary soon.
econgator wrote:
goforthetie wrote:* Choose a cutoff wisely. Let's say you want to ask about the countries that produce the most titanium. Don't just pick a round number (top 10); rather, look at the data and pick a cutoff where there is a big gap - maybe country #13 produces 253 tons whereas country #14 produces only 68. Then, tell us that you want the top 13, give the stats for #13 and #14, and maybe even tell us the identity of country #14.
I got bit by that a while back. I think I asked for hockey players who scored 600+ goals (I think there were 10 or 15 -- a nice cut-off), not bothering to look slightly down to see the next guy had 599. :oops:
When I was putting together a themed quiz on 1960s Pop/Rock Music, I had a concern that there would either be little interest in the subject or potential players' fear of being asked for details about 50 year old Billboard charts, the latter closely illustrated by econgator's and gftt's comments above. So I decided to add a degree of flexibility in accepting answers, which evidently helped as 66 people entered when I expected perhaps 25-30. Here's what I did:

- First, I specifically asked in a post whether there was interest in the subject, which a number of quizmasters have also done on other themed TDs.
- I also emphasized in that post and spelled out later in the rules that "close" would be "good enough," that a song in the range 1958-1971 (1960 +/- 2 yrs) would be close enough to be considered 1960s and also that a question with a Top 10 or Top 40 "cutoff" could be satisfied by being close to those targets. This obviously required some subjectivity on my part as to what was considered close, but it didn't turn out to be an issue at all. (This also enhanced the ability of players to take a chance on perhaps marginal answers with somewhat less risk.) I also specifically avoided asking about what went to #1 on the charts as opposed to those loser songs that only peaked at #2 or #3.

I'm not sure how applicable "close enough" would be for most other quizzes, but it definitely helped when the whole quiz was inherently based on rankings, even though they weren't arbitrary rankings.
User avatar
Vermonter
2003 College Champion
Posts: 1956
Joined: Wed Sep 18, 2013 4:57 pm

Re: TD guidelines

Post by Vermonter »

goforthetie wrote:Choose a cutoff wisely. Let's say you want to ask about the countries that produce the most titanium. Don't just pick a round number (top 10); rather, look at the data and pick a cutoff where there is a big gap - maybe country #13 produces 253 tons whereas country #14 produces only 68. Then, tell us that you want the top 13, give the stats for #13 and #14, and maybe even tell us the identity of country #14.
Agreed - my "mile-high world capitals" in TD 171 gave a nice round number, but threw a lot of people for a loop with Kathmandu, just a few hundred shy.
Hate bad wagering? Me too. Join me at The Final Wager.
Post Reply