Timing of Audition Invite Emails

This is where all of the games are discussed.

Moderators: alietr, trainman, econgator, dhkendall

Post Reply
Golf
Watches Jeopardy! Way Too Much
Posts: 2723
Joined: Thu Jan 10, 2013 12:27 pm

Re: Timing of Audition Invite Emails

Post by Golf »

boson wrote: Mon Jun 12, 2017 2:24 pm The problem with this argument is that Jeopardy has changed the audition and test mechanisms several times recently. There is no reason why who gets invited to audition isn't changed as well. Rather than thinking these most recent auditions are a big outlying statistical blip (given previous estimated numbers, a very unlikely result) , the simplest explanation is the J! is inviting approximately the same number of women as men to audition.
Well hell, everything is changing. 44 of 50 right on the online test is now a passing score, only boring stiffs at auditions get the call, and there is now a 15 question test on wagering strategy. Starting next season Alex will host the show wearing suspenders with no shirt.

No proof required, yet a very simple explanation.


Please stop getting sucked in by recency bias.
Prasutagus
Newbie
Posts: 9
Joined: Sun Jun 04, 2017 6:56 pm
Location: Norfolk

Re: Timing of Audition Invite Emails

Post by Prasutagus »

Golf wrote: Mon Jun 12, 2017 11:23 am
boson wrote: Mon Jun 12, 2017 10:56 am Did you just read the two posts above? The last two NYC auditioners said there were more women than men at the audition.
I did. Two auditions as opposed to hundreds discussed over the years. Forgive me if I ignore a meaningless sample size. Now, if this were to continue over most auditions over the next year or two, then we'll have something to discuss.
Golf, you are incorrect about this. If you apply Bayes' Theorem you will understand why. Suppose from a population of 70% men and 30% women, we take a random sample of 20 individuals. What is the probability there will be 10 or more women in the sample? It's a binomial probability calculation. There's only a 4.8% chance of having 10 or more women in the sample of 20. Now if we take two separate samples from the same population, what is the probability that each sample will have 10 or more women? It's 4.8% squared: 0.0023 or about one chance in 434.

Now we apply Bayes' theorem to estimate what is the probability, given the information from these two recent auditions, that the invitations for auditions are still selected randomly (without regard to sex) from a population whose ratio of men to women is approximately 70:30.

The prior probability of this we can set quite high, let's say 0.95, considering that this is (according to consensus in this thread) how it has been done in previous years and there was no indication that it was about to change. The prior probability of seeing two consecutive auditions each with ten or more women would then be around 0.02, or 1 in 50. So we multiply 0.0023 by 0.95 and divide by 0.02 to get 0.109 or about 1 in 9. This represents the revised probability (given the new information reported in this thread) that the Jeopardy staff are still inviting applicants to auditions without regard to sex. The probability has declined from 95% (before we were aware of the new information) to just 11% (accounting for the new information). That's how Bayes' Theorem works. If you have a pre-existing theory about how some process works (in this case how Jeopardy invites people to auditions) and then you see one or two events that would be very unlikely to occur if your prior theory is correct, you usually should conclude that your prior theory is now likely incorrect.

(Of course "likely incorrect" does not mean "impossible". My calculation shows an 11% probability, i.e. a nontrivial chance, that the process has not changed, in which case these two auditions with more women than usual are just statistical anomalies. But that leaves an 89% probability that the process has changed.)
raccoonleaf
Contributor
Posts: 18
Joined: Tue Nov 03, 2015 12:24 am
Location: Houston, TX

Re: Timing of Audition Invite Emails

Post by raccoonleaf »

Prasutagus wrote: Tue Jun 13, 2017 1:16 pm The prior probability of this we can set quite high, let's say 0.95, considering that this is (according to consensus in this thread) how it has been done in previous years and there was no indication that it was about to change. The prior probability of seeing two consecutive auditions each with ten or more women would then be around 0.02, or 1 in 50.
What does this mean, exactly?
Prasutagus
Newbie
Posts: 9
Joined: Sun Jun 04, 2017 6:56 pm
Location: Norfolk

Re: Timing of Audition Invite Emails

Post by Prasutagus »

raccoonleaf wrote: Thu Jun 15, 2017 9:13 amWhat does this mean, exactly?
If we have an estimate of the probability of a certain statement being true, and then receive new information, Bayes' Theorem provides a handy way of revising our probability estimate to account for the new information. The formula is usually expressed P(A|B)=P(B|A)*P(A)/P(B), where A is the statement whose probability we are evaluating, and B is the new information. It's explained nicely in this brief video: https://www.khanacademy.org/partner-con ... es-theorem

The example in the video, estimating the probability you have a certain rare illness, begins with a very small prior probability and shows that, when applying the formula, the revised probability is still pretty small (though larger than the prior probability).

Bayes' Theorem can also provide striking results when the prior probability is quite high, as discussed in my previous post. The statement whose probability we want to estimate is this: "Invitations to Jeopardy auditions continue to made without regard to sex, from a population whose ratio is approximately 70:30 male to female." A couple of months ago we would have estimated the probability of this statement as perhaps 95%, based on reports over a long period from boardies attending auditions. (We wouldn't have said 100% because we'd have known that just because it's been done this way in the past, does not mean it will certainly continue to be done this way.) But then we received new information: two recent auditions in New York City where the majority of participants were female. Golf derisively called this a "meaningless sample size". Applying Bayes' formula, however, we find that the new information has dramatically changed the probability of the statement we're considering: from 95% (prior probability) to 11% (revised probability accounting for new information). That's what I was getting at in my earlier post. (Incidentally, please don't take that 11% figure as "scientifically accurate", because it's based on some very rough estimates. Rather than stating "the probability is 11%" it's better to state it as a range, "5% to 15%" or something like that.)
raccoonleaf
Contributor
Posts: 18
Joined: Tue Nov 03, 2015 12:24 am
Location: Houston, TX

Re: Timing of Audition Invite Emails

Post by raccoonleaf »

Prasutagus wrote: Thu Jun 15, 2017 4:31 pm If we have an estimate of the probability of a certain statement being true, and then receive new information, Bayes' Theorem provides a handy way of revising our probability estimate to account for the new information. The formula is usually expressed P(A|B)=P(B|A)*P(A)/P(B), where A is the statement whose probability we are evaluating, and B is the new information. It's explained nicely in this brief video: https://www.khanacademy.org/partner-con ... es-theorem

The example in the video, estimating the probability you have a certain rare illness, begins with a very small prior probability and shows that, when applying the formula, the revised probability is still pretty small (though larger than the prior probability).

Bayes' Theorem can also provide striking results when the prior probability is quite high, as discussed in my previous post. The statement whose probability we want to estimate is this: "Invitations to Jeopardy auditions continue to made without regard to sex, from a population whose ratio is approximately 70:30 male to female." A couple of months ago we would have estimated the probability of this statement as perhaps 95%, based on reports over a long period from boardies attending auditions. (We wouldn't have said 100% because we'd have known that just because it's been done this way in the past, does not mean it will certainly continue to be done this way.) But then we received new information: two recent auditions in New York City where the majority of participants were female. Golf derisively called this a "meaningless sample size". Applying Bayes' formula, however, we find that the new information has dramatically changed the probability of the statement we're considering: from 95% (prior probability) to 11% (revised probability accounting for new information). That's what I was getting at in my earlier post. (Incidentally, please don't take that 11% figure as "scientifically accurate", because it's based on some very rough estimates. Rather than stating "the probability is 11%" it's better to state it as a range, "5% to 15%" or something like that.)
You're telling me how the watch is built. I just want to know the time. In your scenario:

1. What is P(A)?
2. What is P(B)?

Edit: As in, how are you defining them, and how are you calculating P(B)?
User avatar
morbeedo
Loyal Jeopardista
Posts: 3065
Joined: Wed Jan 18, 2012 12:58 pm

Re: Timing of Audition Invite Emails

Post by morbeedo »

Audition confirmation received today for NYC! I'll make a point of computing the female to male ratio at my audition

The application asks if I know or have communicated with anyone who's appeared on Jeopardy! This question strikes me as odd. How many former contestants have I interacted with on JBoard alone?? 10? 20?
User avatar
Volante
Harbinger of the Doomed Lemur
Posts: 9254
Joined: Thu Jul 14, 2011 11:42 pm

Re: Timing of Audition Invite Emails

Post by Volante »

morbeedo wrote: Thu Jun 15, 2017 10:06 pm Audition confirmation received today for NYC! I'll make a point of computing the female to male ratio at my audition

The application asks if I know or have communicated with anyone who's appeared on Jeopardy! This question strikes me as odd. How many former contestants have I interacted with on JBoard alone?? 10? 20?
I think the consensus answer is to just mention you're a JBoard member. They know who's here.
The best thing that Neil Armstrong ever did, was to let us all imagine we were him.
Latest movies (1-10): Everything Everywhere All at Once (10), Ruby Gillman: Teenage Kraken (6), Black Sunday /1960/ (6), Marcel the Shell with Shoes On (7)
Johnblue
Watches Jeopardy! Way Too Much
Posts: 1626
Joined: Tue Mar 06, 2012 3:55 am

Re: Timing of Audition Invite Emails

Post by Johnblue »

I mentioned it so let's hope it helps! Does anyone think the J6 results help? I usually get all 6 so I'm hoping it aids me.
Prasutagus
Newbie
Posts: 9
Joined: Sun Jun 04, 2017 6:56 pm
Location: Norfolk

Re: Timing of Audition Invite Emails

Post by Prasutagus »

raccoonleaf wrote: Thu Jun 15, 2017 6:02 pm1. What is P(A)?
2. What is P(B)?

Edit: As in, how are you defining them, and how are you calculating P(B)?
P(A) is the prior probability that invitations to Jeopardy auditions continue to be made without regard to sex, from a population whose ratio is approximately 70:30 male to female. Estimate of P(A) is 0.95.

P(B) is the prior probability that at least 10 auditioners are women in each of two given auditions of 20.

My calculation estimating P(B) uses a formula (where the tilde symbol means "not"):

P(B) = P(A)*P(B|A) + P(~A)*P(B|~A)
= 0.95*0.0023 + 0.05*0.3459
= 0.002185 + 0.017295
= 0.019480
= 0.02 (rounding)

That's where I got P(B)=0.02 or 1 in 50, which I mentioned in my previous post. If you're wondering how I estimated P(B|~A)=0.3459, it's based on the assumption that any new audition-selection procedure would see each spot in an audition equally likely to be filled by a man as a woman. Then it's easy to calculate (using the binomial formula) the probability of having zero to nine women out of twenty in a given audition: 0.411901. Hence the probability of having ten or more women is 0.588099. Since we're talking about two given auditions, not one, you square 0.588099 to get 0.3459.
raccoonleaf
Contributor
Posts: 18
Joined: Tue Nov 03, 2015 12:24 am
Location: Houston, TX

Re: Timing of Audition Invite Emails

Post by raccoonleaf »

Prasutagus wrote: Fri Jun 16, 2017 6:27 pm That's where I got P(B)=0.02 or 1 in 50, which I mentioned in my previous post. If you're wondering how I estimated P(B|~A)=0.3459, it's based on the assumption that any new audition-selection procedure would see each spot in an audition equally likely to be filled by a man as a woman. Then it's easy to calculate (using the binomial formula) the probability of having zero to nine women out of twenty in a given audition: 0.411901. Hence the probability of having ten or more women is 0.588099. Since we're talking about two given auditions, not one, you square 0.588099 to get 0.3459.
I get where you're going with this, but this is mathematically identical to saying there's a 95% chance that it's random and 70/30 and a 5% chance that it's random and 50/50 (with no other options in between). Why are you sure the gender mix hasn't changed?

Also, two random draws in a row are no more significant than any two random draws. Central Limit Theorem. The general odds of getting blackjack are also about 4.8%, but I wouldn't assume the dealer was cheating if he got it two times in a row.

You, and many men on this board, have a bias (whether conscious or unconscious) that says women must be treated better than men, when the answer could just as well be that women are doing better. Check out the Jeopardy Facebook page. Check out Jeopardy on Twitter. Women are engaged with this show. And I, as a man... I'm for it.
Prasutagus
Newbie
Posts: 9
Joined: Sun Jun 04, 2017 6:56 pm
Location: Norfolk

Re: Timing of Audition Invite Emails

Post by Prasutagus »

If the casino is so sketchy that there's only a 0.95 probability you'll be dealt an honest game, and you think one form of cheating might be repeatedly dealing blackjacks to the house, and then the first two hands are dealer blackjacks, you can say it's now likely you are being cheated. With the new information, your estimated probability of being dealt an honest game has dropped from 0.95 to something much lower.

(But if you're at a reputable casino, the prior probability of being dealt an honest game is so high, and moreover if they do cheat it's likely to be much subtler, that two dealer blackjacks does not cause concern about the integrity of the game.)

As for Jeopardy auditions, you have misinterpreted my argument. I stated three times that "A" in my example is the event that invitations to Jeopardy auditions continue to be made without regard to sex, from a population whose ratio is approximately 70:30 male to female. Therefore a significant change in the 70:30 ratio, for any reason, falls under "not A". I've shown that if P(A) was initially 95%, it's now around 5% to 15%. I never suggested that this decrease is because "women must be treated better than men". One way in which "A" might no longer be true is a significant change in the sex ratio of those who pass the online test (which might be caused either by a relative increase in the number of women who take the test, or by a relative increase in the average score of women who take it).

You pointed out that in my calculation of P(B|~A)=0.3459 I assumed that under "~A", the ratio would be 50:50. This was a simplifying assumption and does not dramatically change the results. If you think that my estimate of P(B|~A) is too high, you can replace the 0.3459 by, for example, 0.25, which results in a value of P(B)=0.015 instead of 0.02, and then P(A|B) is 14.6% rather than 10.9%. Or if you think that my estimate of P(B|~A) is too low, replace the 0.3459 by, say, 0.6, resulting in P(B)=0.032 and P(A|B)=6.8%. In any case, we get the result that P(A|B) is much less than P(A)=95%.

Over the last several years (up until a few weeks ago) it's been observed that men outnumber women at in-person auditions, but there are roughly the same number of male and female contestants appearing on the show. In other words, women who audition get the Call at a higher rate than men who audition. That doesn't necessarily mean there is an affirmative-action effect. Another explanation might be this: if men are more likely to cheat on the online test, then women would be more likely than men to pass the in-person written test. Another possibility is that women at the auditions have had on average more energy, better enunciation, or some other quality the contestant coordinators are looking for. The underlying reasons, we do not know. We do know that (over the past few years) women at in-person auditions are selected for the show at a higher rate than men. Therefore, it's plausible that the contestant coordinators recently decided to invite more women to auditions, since they expect to select an equal number of men and women to appear on the show. That's one possibility. Another possibility is that significantly more women are taking the test, resulting in a change of the ratio from 70:30 to something else.

The point is, with Bayes' theorem we can infer that these two recent examples are probably indicative of a new pattern, rather than just being outliers.
raccoonleaf
Contributor
Posts: 18
Joined: Tue Nov 03, 2015 12:24 am
Location: Houston, TX

Re: Timing of Audition Invite Emails

Post by raccoonleaf »

This is not a productive conversation. I'm done bothering everyone with it.
QBall
Valued Contributor
Posts: 74
Joined: Tue Nov 29, 2011 1:48 am

Re: Timing of Audition Invite Emails

Post by QBall »

To get this topic nack on track, I've seen at least two people mention they just got invites for auditions in Seattle next month.
John Boy
Watches Jeopardy! Way Too Much
Posts: 2981
Joined: Fri Jul 15, 2011 7:11 am

Re: Timing of Audition Invite Emails

Post by John Boy »

New topic:

If ten people are invited to a discussion of a proposed new senate health-care bill, what is the probability that at least one of them will be a democrat?

What is the probability that at least nine of the ten will be white male republican millionaires over the age of 55?
TomFromMD
Valued Contributor
Posts: 57
Joined: Fri Apr 07, 2017 8:34 am

Re: Timing of Audition Invite Emails

Post by TomFromMD »

Prasutagus wrote: Sat Jun 17, 2017 7:26 pm but there are roughly the same number of male and female contestants appearing on the show.
Just to keep it off topic, I'd wager a whole lot of money that the actual people that are making it to Culver City are intentionally 50/50, and it's only the fact that a few "extras" that get bumped that keep it from being 50/50 for new contestants each week (but that would even out).

Looking in the archive, going back from the latest game to the beginning of April, and ignoring the Teacher Tournament (which was as close to 50/50 as you could come, since there was an odd number of contestants), there are exactly 45 men and exactly 45 women. The odds of that are pretty darn small if it were random.
User avatar
triviawayne
Hoping I don’t drown in this contestant pool
Posts: 2677
Joined: Tue Feb 17, 2015 9:50 pm

Re: Timing of Audition Invite Emails

Post by triviawayne »

TomFromMD wrote: Thu Jun 22, 2017 9:32 am
Prasutagus wrote: Sat Jun 17, 2017 7:26 pm but there are roughly the same number of male and female contestants appearing on the show.
Just to keep it off topic, I'd wager a whole lot of money that the actual people that are making it to Culver City are intentionally 50/50, and it's only the fact that a few "extras" that get bumped that keep it from being 50/50 for new contestants each week (but that would even out).

Looking in the archive, going back from the latest game to the beginning of April, and ignoring the Teacher Tournament (which was as close to 50/50 as you could come, since there was an odd number of contestants), there are exactly 45 men and exactly 45 women. The odds of that are pretty darn small if it were random.
because it is not random who is on the show, it is only random who gets an in person audition. It's not just male or female that is used by casting (remember, they are casting a show here), it can be skin tone, professional background, location of residence, etc.

the process of creating their pool of potential contestants is not as complicated as so many people want to think it is simply because there is no need for said complications as the show gets what they looking for from a very uncomplicated process.
Total game show career losings = $171,522
reddpen
Thrice Unplucked from the Jeopardy! Pool
Posts: 844
Joined: Fri Jul 01, 2011 11:51 am
Location: Seattle
Contact:

Re: Timing of Audition Invite Emails

Post by reddpen »

QBall wrote: Thu Jun 22, 2017 7:24 amTo get this topic nack on track, I've seen at least two people mention they just got invites for auditions in Seattle next month.
I've seen just one (StevensonM on the LL msg board), but that's one more invite than has appeared in any of my folders since 2011, and it's not cuz I'm not passing the test. Don't get me wrong, I'll keep checking, but optimism is not high.

Next year before the online test, will someone please remind everyone--especially those whose scores tend toward the upper end of the scale--that Roger Craig didn't get on the show till his online test score dropped--quite deliberately--to 40 from the high 40s?
In and out of the pool four times
User avatar
acthomas
Watches Jeopardy! Way Too Much
Posts: 304
Joined: Wed Sep 14, 2016 2:29 pm
Location: Durham, NC
Contact:

Re: Timing of Audition Invite Emails

Post by acthomas »

reddpen wrote: Thu Jun 22, 2017 2:27 pm
QBall wrote: Thu Jun 22, 2017 7:24 amTo get this topic nack on track, I've seen at least two people mention they just got invites for auditions in Seattle next month.
I've seen just one (StevensonM on the LL msg board), but that's one more invite than has appeared in any of my folders since 2011, and it's not cuz I'm not passing the test. Don't get me wrong, I'll keep checking, but optimism is not high.

Next year before the online test, will someone please remind everyone--especially those whose scores tend toward the upper end of the scale--that Roger Craig didn't get on the show till his online test score dropped--quite deliberately--to 40 from the high 40s?
Citation please?
User avatar
morbeedo
Loyal Jeopardista
Posts: 3065
Joined: Wed Jan 18, 2012 12:58 pm

Re: Timing of Audition Invite Emails

Post by morbeedo »

reddpen wrote: Thu Jun 22, 2017 2:27 pm
QBall wrote: Thu Jun 22, 2017 7:24 amTo get this topic nack on track, I've seen at least two people mention they just got invites for auditions in Seattle next month.
Next year before the online test, will someone please remind everyone--especially those whose scores tend toward the upper end of the scale--that Roger Craig didn't get on the show till his online test score dropped--quite deliberately--to 40 from the high 40s?
So you're saying that the highest scorers on the online test are excluded by the selection algorithm? I don't buy it ;)
User avatar
triviawayne
Hoping I don’t drown in this contestant pool
Posts: 2677
Joined: Tue Feb 17, 2015 9:50 pm

Re: Timing of Audition Invite Emails

Post by triviawayne »

morbeedo wrote: Thu Jun 22, 2017 3:41 pm
reddpen wrote: Thu Jun 22, 2017 2:27 pm
QBall wrote: Thu Jun 22, 2017 7:24 amTo get this topic nack on track, I've seen at least two people mention they just got invites for auditions in Seattle next month.
Next year before the online test, will someone please remind everyone--especially those whose scores tend toward the upper end of the scale--that Roger Craig didn't get on the show till his online test score dropped--quite deliberately--to 40 from the high 40s?
So you're saying that the highest scorers on the online test are excluded by the selection algorithm? I don't buy it ;)
me either...see above
Total game show career losings = $171,522
Post Reply