0:42

You're being asked to make a decision, and there are associated payoffs and

Â losses that you should consider.

Â We'll summarize the payoff/loss information in a decision table.

Â Remember, the true state of the population can ether be 10% yellow M&Ms or

Â 20% yellow M&Ms.

Â Say that you decide that the percentage of yellow M&Ms is indeed 10% and

Â that is the true state of the population.

Â Then you made the correct choice and your boss gives you a bonus.

Â Say on the other hand, you decided that the true proportion of yellow M&Ms is 10%,

Â but the true state of the population is 20%.

Â You made the wrong choice, and you loose your job.

Â You might also decide that percentage of yellow M&Ms is 20% and

Â if that is your decision and

Â the true percentage of yellow M&Ms is 10%, once again you've made a mistake.

Â So you'd loose your job.

Â 1:38

Or if you decide that the true proportion is 20% and that is indeed the case,

Â you make the right decision and your boss gives you a bonus.

Â Obviously, you're going to be making this decision using data.

Â You can buy a random sample of M&Ms from the population.

Â Your data, as I said, will be your random sample of M&Ms from the population.

Â Each M&M is going to cost you $200 which is indeed pretty steep, but

Â remember that data collection can be pretty costly.

Â You pay $200 for each M&M and you must buy in $1,000 dollar increments.

Â That is 5 M&Ms at a time.

Â You have a total of $4,000 dollars to spend so

Â you may buy 5, 10, 15, or 20 M&Ms.

Â What is the cost?

Â Or the benefit of buying fewer or more M&Ms.

Â The benefit obviously is that as you increase your

Â sample size your decisions are going to be more reliable and remember that the cost

Â of making a wrong decision is pretty high, you could lose your job.

Â So you want to be fairly confident of your decision.

Â At the same time, though, the data collection is costly as well.

Â So you don't want to pay for a sample larger than you need.

Â If you believe that you could actually make a correct decision using a smaller

Â sample size, you might choose to do so and save money and resources.

Â 3:13

Let's start with the frequentist method.

Â Our null hypothesis is that the proportion of yellow M&Ms is 10%.

Â Remember the two choices were 10% or 20% within the frequentist framework

Â since we cannot set the parameter equal to a value in the alternative hypothesis,

Â we define that alternative as p is greater than 10%.

Â That's closer to the 20.

Â We also need to decide what our decision threshold.

Â In other words, the significance level should be.

Â A significance level of 5% is customary to use in literature, and in practice.

Â But there may be very good reasons for using a different significance level.

Â Remember from earlier courses in this specialization that the significance level

Â is a probability of a Type I error.

Â That is the probability of rejecting the null hypothesis

Â when the null is actually true.

Â So it makes sense to keep this rate as low as possible.

Â However, at the same time,

Â there may be benefits to using a slightly higher significance rate.

Â As this would mean that we would be reducing our Type 2 error rate.

Â That is the probability of failing to reject the null hypothesis

Â when it is actually false.

Â If the P value we calculate ends up being smaller than our significance level,

Â we reject our null hypothesis in favor of the alternative and

Â conclude that the data provide convincing evidence for the alternative hypothesis.

Â We mentioned that we would be working with a sample size of five, so n is five, and

Â since there's only one yellow M&M in the sample, k is equal to one.

Â Our test statistic is the number of yellow M&M's in this sample.

Â Remember that the P value is the probability of observed or

Â more extreme outcome given that the null hypothesis is true.

Â In context, this is the probability of one or more yellow M&M's in a random sample of

Â five M&M's assuming that the true proportion of yellow M&Ms is 0.10,

Â we can calculate this probability as the compliment of no successes in five trials.

Â Let's pause for a moment and think about why this is the case.

Â In a sample space with five trials, you could have zero successes, one success,

Â two successes, three successes, four successes or five successes.

Â If your interested in the number of successes being greater than or

Â equal to one that means that the only outcome that you're not interested

Â in is the number of successes being equal to zero.

Â Hence, the two probabilities, the probability of at least one, and

Â the probability of none are compliments of each other.

Â The probability of no successes in five trials with a probability of success for

Â each trial is 0.1 Is 0.90 to the 5th power.

Â So the overall probability of at least one success, comes out to be 0.41.

Â With such a high P value, we would fail to reject the null hypothesis and

Â conclude that the data do not provide convincing evidence

Â that the proportion of yellow M&M's is greater than 10%.

Â This means that if we had to pick between 10% and 20% for

Â the proportion of M&M's, even though this hypothesis testing procedure does not

Â actually confirm the null hypothesis, we would likely stick with 10% since we

Â couldn't find evidence that the proportion of yellow M&M's is greater than 10%.

Â Next we'll try to answer the same question using a Bayesian approach.

Â Once again we start with our hypotheses.

Â The first hypothesis is that the proportion of yellow M&Ms is 10%.

Â And the second hypothesis is that the proportion is 20%.

Â Note that in the Bayesian method we can actually evaluate

Â the probabilities of both these models we're considering

Â as opposed to having to choose one of as our null.

Â And tailor our alternative hypothesis around that.

Â We also need to place prior probabilities on these hypotheses.

Â I really don't have a reason to believe one is more likely than the other so

Â I'm going to place a 0.5 probability on each one.

Â We're still working with the same data set of 5 M&Ms,

Â where one is yellow The next step is to calculate the likelihood of this outcome.

Â One success in five trials, under the two models,

Â the two hypotheses that we're considering.

Â We can use the binomial distribution to calculate these probabilities.

Â The probability of one success in five trials,

Â where p is equal to 0.10, is roughly 0.33.

Â 8:45

Here we summarize what the results would look like

Â if we had chosen larger sample sizes as well.

Â That is, if we had a sample size of 10 with 2 yellow, or

Â 15 with 3 yellows, or 20 with 4 yellow M&Ms.

Â Under each of these scenarios,

Â the frequentist method yields a higher P value than our significance level,

Â so we would fail to reject the null hypothesis with any of these samples.

Â On the other hand, the Bayesian method always yields a higher posterior for

Â the second model where P is equal to 0.20.

Â So the decisions that we would make are contradictory to each other.

Â However, note that if we had set up our framework differently in the frequentist

Â method and set our null hypothesis to be P is equal to 0.20 and

Â our alternative to P is less than 0.20, we would obtain different results.

Â This shows that the frequentist method is highly sensitive to the null hypothesis,

Â while in the Bayesian method,

Â our results would be the same regardless of which order we evaluate our models.

Â