0:04

In this video we're going to dig into data analysis and

Â talk about how researchers use statistics to test their hypothesis.

Â Right now I'm imagining that most of you skipped the result section in papers

Â because there's a bunch of numbers and charts.

Â My hope is that by the end of this video you'll feel more comfortable

Â knowing how to understand this information.

Â 0:24

Remember that there are two types of hypotheses and analyses, descriptive and

Â inferential.

Â Descriptive hypotheses are hypotheses where we're just describing trends.

Â We're not trying to manipulate any data but just observing the sample and

Â understanding what's happening.

Â 0:39

In contrast inferential hypothesis which we'll cover in this video

Â a hypothesis where you're interested in the difference between groups or

Â testing the relationship between two variables.

Â You'll hear party talk this week about whether those who participated in his

Â mental toughness intervention.

Â Improved in their grit and optimism which is an example of looking for

Â differences between groups, and

Â you'll hear Angela she's been interested in and tested over the years.

Â She'll talk about how grit is related to other variables, like happiness.

Â Growth mindset and purpose.

Â Statistical analysis can help you identify the relationship between two variables

Â of interest or the difference between two groups.

Â 1:21

With inferential statistics we want to draw inferences about populations from

Â samples.

Â So why do we do this?

Â Well we would really like to talk about a large set of people But

Â it's not practical to survey everyone in our population.

Â So we want to be able to generalize what we know about a small set of people

Â to the larger group.

Â So in inferential statistics we're really making a numerical

Â guess about the population based on what we know about a sample.

Â 1:47

So when ever were doing analysis were assessing samples,

Â with the hopes of generalizing to our population of interest.

Â With this means that there's always a possibility that what we observe is do

Â the chance and not actually a different a relationship in the population.

Â So release statistical analysis is about assessing chance and probability.

Â 2:09

To use stats to test our hypotheses there are a few steps.

Â First, as we've already discussed,

Â we develop hypotheses which will either support or reject our predictions.

Â Then we collect data from a sample and test our claims,

Â based on this sample, using probability to say something about the situation.

Â And we do this by using what's called a null hypothesis.

Â 2:32

Remember that the null hypothesis is the statement that an intervention has

Â no effect on an intended outcome, or

Â that there is no relationship between variables.

Â The alternative hypothesis is our prediction

Â that there is a relationship or a difference.

Â If the data that we collect are consistent with our null hypothesis,

Â then we reject our predictions.

Â But if we have enough data to support our predictions,

Â then we reject the null hypothesis.

Â 3:11

So let's take an example.

Â Let's say we implemented an intervention to increase grit with teachers.

Â And we want to know if teacher is participating in our intervention

Â have higher grit scores than teachers in the district

Â who did not participate in our intervention.

Â We would collect grit scores from these two groups and compare the results.

Â To do this, we would test the null hypothesis that there is

Â no difference in grit.

Â Between teachers and our intervention and other teachers in the district.

Â 3:39

So let's say this slide represents our results.

Â The average grits score of teachers not in our intervention is 3.5 and

Â the dot represents the average from our intervention Group of 4.5,

Â so there is a difference, and

Â it appears as though teachers in our intervention have higher grit scores.

Â But the question we would then ask is whether that difference is

Â just due to chance, or likely a real difference in teachers across the district

Â 4:06

because this score falls far enough out from the average it could be in what's

Â called the significance region.

Â which is where we would think the results weren't just to the chance or

Â error in our data collection, but actually a difference worth paying attention to.

Â 4:21

This is where the concept of statistical significance or P values comes in.

Â You'll see P values reported in parts of psychology articles.

Â The P value tells you the probability of rejecting the null

Â hypothesis when it's true.

Â 4:35

So in simpler terms, it's telling you the likelihood that your results were

Â just due to chance or error in the data collection.

Â In positive psychology the standard that's applied is .05.

Â So what that means is that there's a 5% likelihood that the difference or

Â relationship was just chance or error.

Â Anything under 0.05 is determined to be statistically significant.

Â So when you hear that a result was significant that means that a statistical

Â test of some sort was performed and a P value of less than 0.05 was obtained.

Â And that there's only a 5% likelihood that these results would have been due to

Â chance alone.

Â So this means that the difference or relationship was different enough from

Â the null hypothesis that we should pay attention to it.

Â So in the case of the visual of our great intervention,

Â the great score we observed was far enough out on that curve or

Â different enough to not likely have been due to chance.

Â 5:28

This is where terms and statistics can get tricky because the term signficant

Â has different meaning in common language then in statistics.

Â In common language significant means something is large, but in statistics

Â the term significant doesn't say anything about the size of magnitude of the effect.

Â It just tells you whether or not the results were likely due to chance.

Â 5:50

So, let's say you have a really big sample.

Â Increasing the size of your sample makes it more likely that your results would be

Â replicated if you run the study again and not just stood a chance.

Â So, if you have a large sample you might find results to be statistically

Â significant even if they aren't very large.

Â 6:09

To get at the actual size at the difference or relationship, or

Â the practical significance, you look at the measure called effect size,

Â the effect size will tell you about the size or magnitude of the relationship.

Â If you're interested in testing the difference between groups,

Â as [INAUDIBLE] might be if he was looking at the difference between participants

Â in his mental [INAUDIBLE] intervention in a control group.

Â You would use one type of test or

Â if you're interested in the relationship between grit and other variables like

Â Angela has been over the years then you would run different types of test.

Â 6:45

This Chart outlines the types of tests and results you get from these types of tests.

Â Now, I'm not expecting that you become an expert on these in this course, but

Â you'll see them sided in articles.

Â So, I do want to help you be a more critical consumer of the research.

Â 7:00

Relationship analyses tell you the association between variables.

Â So, for example, if researchers are testing hypothesis about the relationship

Â between grant and other variables, like happiness or optimism.

Â Then they're either running correlation or

Â regression in the effect size measure they would get is the Pearson's R for

Â correlation and a beta value for regression.

Â These measures tell you the strength of the relationship between variables.

Â 7:28

Group difference analyses allow you to understand the relationships between

Â different groups.

Â So if researchers are testing hypotheses about group differences say between

Â an intervention In a control group, then they are either running a T-Test for

Â two groups or an ANOVA for multiple groups.

Â So think about the example I shared earlier about teachers who

Â participated in our Grit Intervention as compared to teachers in the district who

Â did not participate in our Grit Intervention.

Â The effect size measure they get is a Cohen's D

Â which tells you the size of the difference between groups.

Â So regardless of the type of analysis, researchers are testing hypotheses and

Â will get a p value.

Â Which again, remember, tells them whether their results are due to chance or

Â likely represent a real difference in the population of interest.

Â 8:16

These effect size measures assess the magnitude of the effect, or

Â the practical significance.

Â I'm going to give you some tips on how to interpret these measures,

Â if you see them in articles.

Â 8:32

And our value of 0.1, or -0.1 is small, a value of 0.3 or

Â -0.3 is medium and 0.5 or -0.5 is large.

Â Calls in D which tells you the size of the difference between groups

Â doesn't have the same limitation in range as R.

Â The calls in D of 0.2or -0.2 are small 0.5 or

Â -0.5 is medium, and 0.8 or -0.8 is large.

Â But D can be over 1.

Â So I know I'm throwing a lot of numerical values at you, so

Â let's actually look at an example.

Â Here's a table from the True Grit article.

Â This table is showing the correlation or relationship based task where we

Â looked at how grit was correlated with many other variables.

Â If you look across the first row in the table you'll see the values reported for

Â the relationship between grit and all other variables we tested.

Â The values represented here are the Pearson's correlation coefficient, or

Â that r value I told you about so what you'll see is that most of

Â the correlations are very small with R values of 0.02, .16, and 0.808.

Â The one exception is the relationship between Grit and

Â Leadership which has an R value of .36.

Â Now, remember our rule of 0.01, 0.03, and 0.05.

Â This means that there's a moderate positive relationship between Grit and

Â Leadership such that greedier teachers also had higher leadership scores.

Â The second thing you will see in this table is the P value or

Â the likelihood that results are just due to chance, the stars let you know that

Â the results have been found to be statistically significant,the more stars

Â the more significant and the less likely the results were just due to chance.

Â 10:14

So, if we are looking at what variables are significantly related to grid.

Â The only variable is leadership because that's the only variable where

Â there are stars by the value.

Â You can see at the key at the bottom of the table

Â that the three stars mean that the P value is less than 0.001.

Â This means that there's a less than a 0.1% chance that

Â the results were just due to chance.

Â So, we can conclude from this table that there's positive moderate relationship

Â between grit and leadership.

Â That is unlikely due to chance, or just an error in our data collection.

Â Let's look at another example,

Â this time where we're testing the differences between groups.

Â In this table from true grit, we're looking at the difference in grit and

Â other variables between teachers who were retained for

Â the year, and those who quit mid year.

Â 11:11

And the results actually provided evidence to support our prediction.

Â So, if you look at the first row of this table, youâ€™ll see the difference in

Â the grit rating between retain teachers and those teachers who resigned mid-year.

Â The mean or average of Grit for retaining teachers was 3.98 compared to the mean or

Â average of 2.79 for teachers who resigned.

Â The Cohen's d was last as I remember the size of that difference.

Â The the rule of 0.2, 0.5 and 0.8.

Â A Cohen's d of 0.79 is a large effect because it's close to 0.8.

Â Indicating that the size of the difference in grit between teachers who are retained

Â and those who resigned was practically relevant.

Â The fact that there are threes stars means the results are very

Â unlikely to be dued a chance.

Â You can see that the P value is less than 0.01, which again means that

Â there's less than a point one percent chance that the result are a fluke.

Â So, we conclude from this table that there is a large difference in grit

Â between teachers who stayed through the year and those that quit mid year.

Â That is unlikely due to chance or an error in our data collection.

Â 12:21

So in sum, I hope that you now won't just skip over the result section of positive

Â psychology articles.

Â When you're looking at tables and articles, you can look for three things.

Â First the P-Value or statistical significance.

Â This will tell you how likely it is that the results are due to chance.

Â If the P-Value is less than .05 you'll see stars represented which means you can have

Â competence that is unlikely that the results are due to chance.

Â Second the effect size.

Â We've talked about two types of effect size.

Â Cohen's D and Pearson's R.

Â But, there are other effect sizes you'll see reported in articles depending on

Â the type of test that is being run.

Â The effect size measures tell you how practically significant the results are.

Â And finally, always look for the sample size and information on the sample.

Â This would tell you how generalizable the results are to other population.

Â 13:14

So, this brings us to the end of the final video on analysis.

Â There are a few other components for this week, as usual.

Â Again, researcher videos,

Â featuring Angela Duckworth applying these concepts to her work.

Â Practitioner videos,

Â featuring practitioners applying these concepts to their work.

Â 13:32

A short quiz on key concepts and two extension activities.

Â A peer review activity where you discuss what type of analysis you would run to

Â test the hypothesis you submitted in week one.

Â And in advanced activity, where you look at table four in the true grid study.

Â And interpret our results.

Â