Learn fundamental concepts in data analysis and statistical inference, focusing on one and two independent samples.

Loading...

From the course by Johns Hopkins University

Mathematical Biostatistics Boot Camp 2

42 ratings

Learn fundamental concepts in data analysis and statistical inference, focusing on one and two independent samples.

From the lesson

Discrete Data Settings

In this module, we'll discuss testing in discrete data settings. This includes the famous Fisher's exact test, as well as the many forms of tests for contingency table data. You'll learn the famous observed minus expected squared over the expected formula, that is broadly applicable.

- Brian Caffo, PhDProfessor, Biostatistics

Bloomberg School of Public Health

Hi, my name is Brian Caffo and this is mathematical

bio statistics boot camp, lecture seven on Fisher's exact test.

So in this lecture we're going to talk about Fisher's exact test.

We're going to talk about the hypergeometric distribution which

plays a central role in Fisher's exact test.

And we'll talk about some practical implementations and then we'll talk

about how you can execute Fisher's exact test using Monte Carlo.

Fisher's exact test is a historically very famous test, and it's

going to be of the, you know, one of the first instances

where we'll able to test. Equality of say binomial proportions

using a formal exact test rather than relying on asymptotics.

So what does exact mean?

So Fisher's exact test is exact because it guarantees the alpha rate.

So when you do a asymptotic test and you

use the nominal type one error rate, say a 5%.

If you say for example calculate a 95% confidence interval for the risk

difference and declare the differences in the proportions as as

being significant if the confidence interval for the difference doesn't

include zero, that's a nice valid testing procedure.

However, that doesn't guarantee you a 5% err

rate, it only guarantees you a 5% limit as the sample sizes go to infinity.

Fisher's exact test and contrast guarantees you the 5%

limit provided the IID assumptions are met for each of the two groups.

And the,the background on Fisher's exact test is that the um,the

famous example is that of the so called Lady tasting tea.

And, in this case, Fisher was at a party and there was a wager

about whether or not a lady at the party could determine whether the cream

had been put in place first or the milk

had been put in place first into her, her tea.

And Fisher devised a blinded experiment where he put tea

in first for a couple of cups and put the milk in first for a couple of, of

examples and then had her declare which she thought was the case.

So you know here is a kind of more um,um

medically oriented version of that so here we have

chemically toxicant with ten mice and we treated um,um,

ten of them I'm sorry, we treated five of

them with the toxicant, and five with the control.

And then here we have the counts of, in each group, of

tumours versus in each, group of treated versus control, what number had tumours.

So here just looking

at this four exposed to the toxicant received tumors,

two controls received tumors out of five in both cases.

So there's maybe some amount of indication that there's a

difference that the toxicant may have some association with the tumors.

But we'd like to test that formally.

there's a lot of ways you, there's actually a surprising

number of ways you can, wind up at Fisher's exact test.

and so,

we're going to go through a particular development.

So, the way we're going to develop

Fisher's exact test, is we're going to assume

that we have two binomials, and we want to test equality of the proportions.

When Fisher originally developed this test, it was quite interesting.

He, so in our case were going to margins these in the five and the five fix, so

we have five treated and five controlled and we are going to model the probability

of random mouse from this population of the treated mice has a tumor

as being having binomial probably p1 and simialriy p2 for the controls.

Fisher thought about this problem differently, he said

I know that I random, let's just assume

this was the tea tasting experiment and he had randomized five cups to have the tea

put in first and five cups to have the milk put in

first, and he says I know that these margins should be fixed.

Then, then it would have made sense if the, given that the,

the, the lady guessing would also know that this margin was fixed.

She would, it would have made sense for her to fix the

second margin, instead of six and four, she would force five and five.

So we wanted to create a procedure that

would force both margins. Right?

And it turns out the Fisher's exact test constrains the margins, right?

It looks at all tables matching the margins,

and, and, that, that, that gave him

the motivation for designing the test this way.

So he looked at so-called hyper-geometric distribution, which is,

is a way in which we can characterize this table.

in, in, in terms of having fixed margins.

Now when we actually implement the test now we don't have to have

five and five on both margins the way Fisher was thinking about it.

But he was analyzing a particular experiment with in

a particular way but it, it, it the, the, the result of Fisher's exact test

is constraining both of these margins and

then looking at what tables satisfy the margins.

And we'll go through the formal mathematical development of

the way in which we're thinking about Fischer's exact test.

There's other ways and maybe I'll ellude to some more of them um,later on,

okay.

Okay on this slide so imagine that we want to

test that p1 equals p2 where p1 is the probability that

a mouse a treated mouse, had, has a tumor, and

then p2 is the probability the control mouse has a tumor.

And so they, we're going to want the null hypothesis that

these are equal which let's just call the common proportion p.

and you know just as a matter of practicality, we can't use a

normal distribution or a chi square test because the sample size is small.

But also, we don't have a specific value of p which

is taken care of in the chi square and the Z test.

They figure out a way to do that using

the asymptotics, but we want to use small sample distribution.

But making the small sample distribution using a

small sample distribution is hard because we don't actually know this this

probability p. Okay so

one way wha, one way to think about Fisher's exact what's testing

is imagine if we were to create the observed data.

List out the observed data as the individual data points.

So mouse one was treated and got a tumor, mouse

two was treated and got a tumor, mouse three was

treated and got a tumor, mouse four was treated and got a tumor.

Mouse five was treated and had no tumor, six

was a control and got a tumor, and so on,okay.

now, one, so, so what we, what I'm going to do right now

is go through another way to kind of develop Fisher's exact test.

and it winds up with the same test.

All these different developments yield the exact same test.

so, in this case imagine if the treatment and control were randomised.

And we wanted to explicitly use

the randomization process, in analyzing the data.

So treatment and control are were randomized.

Treatment and control status were randomized.

Well then, if the null hypothesis is true, And it should

be exchangeable for any mouse, whether or not it got a tumor.

As to whether or not it was from the treated group or the control group, right?

The, the, there should be nothing outrageous about

the particular collection of treated and control mice,

that we see how they line up with the tumor and non-tumor status that we saw.

So what we could do is take this top

row and permute the treat, treatment and control labels.

Right?

Permute the Ts and the Cs.

And we see the second row here is one example

of a permutation Okay, and the result of

that is that the total number of treated, and the total

number of controls remained fix, and the total number to

tumors and total numbers of non tumors also remained fixed.

And thats exactly, maintaining the margins of the, of the table.

This seems like kind of a reasonable null distribution

to investigate right, this that in other

words that treatment in control statistics is exchangeable

relative to tumor status so we'll look

at some test statistic relative to this distribution.

And, the consequence of this is, every time we permute treatment and

control labels, if we were to reform the little two by two table

we had, it would have the same margins, it would have five and

five on the row margins and six and four on the column margins.

And this is exactly one way to develop the null distribution for Fisher's exact test.

Test and this is an interesting way to develop

it because it explicitly uses the idea of randomization.

There's more than one way to think about it.

We can think about this rather than explicitly using the randomization,

we can say well maybe treatment and control status weren't randomized.

Maybe just the first five mouse mice got the

treatment, the latter five mouse mice got the control.

And we believe that there

is no ordering effect, like there is no light in the lab

for the first five mice, causing tumors; and not for the controls.

So in that case we might think of not

the relabeling as a process of re-implementing the randomization scheme,

but instead it's a process of, well we think,

if the null hypothesis is true, relative to tumor status,

treatment and control status are a bunch of exchangeable, permutable labels.

And you could do the same procedure to coming up with

the null distribution and that's another way to think of it.

Yet another way to think about it is the development

we going to go through next which is a Mathematical treatment.

But it is interesting that these very

different methods of thinking about the problem

results in the same task and you see this very often in Statistics where you

wind up with procedurally the same approach

but that the interpretation differs quite a bit.

I would argue that explicitly using

the randomization saying well, this treatment and

control was randomized and so we're

going to explicitly work that into our analysis.

It's a very fundamentally different process than what we're about to do.

Which is to assume that the data are binomally distributed, I impose a model,

a super population model for the mice, and to

work with that lined up with the same procedure,

but the interpretation I think is vastly different, and

I hope you think so too after taking this class.

Coursera provides universal access to the world’s best education,
partnering with top universities and organizations to offer courses online.