A practical and example filled tour of simple and multiple regression techniques (linear, logistic, and Cox PH) for estimation, adjustment and prediction.

Loading...

From the course by Johns Hopkins University

Statistical Reasoning for Public Health 2: Regression Methods

74 ratings

A practical and example filled tour of simple and multiple regression techniques (linear, logistic, and Cox PH) for estimation, adjustment and prediction.

From the lesson

Module 4: Additional Topics in Regression

- John McGready, PhD, MSAssociate Scientist, Biostatistics

Bloomberg School of Public Health

So in this lecture set,

Â Lecture 10, we're going to actually talk about something called propensity scores.

Â Which will give us another approach to estimating adjusted associations above and

Â beyond doing traditional multiple regression.

Â Where we include all potential comfounders as predictors in

Â the multiple regression model.

Â So, let me give you an overview of what we're going to do in

Â the next three sections.

Â In some non randomized studies there is a very

Â specific outcome/predictor association of interest.

Â But because of the study design, confounding is a threat.

Â In such situations there may be many other potential predictors that can

Â also be confound.

Â The primary outcome exposed a relationship.

Â But the research interest in the other predictors is only for adjustment of the,

Â out, outcome primary predictor of interest association.

Â The researcher isn't necessarily concerned with the associations otherwise, and

Â he isn't concerned with quantifying them, etc.

Â Well, what is the potential problem when there's a lot of potential confounders?

Â Well using a finite set of data to estimate a multiple regression model with

Â many predictors to best adjust.

Â May compromise the precision of the main outcome predictor association.

Â So for example, if we have 200 observations and

Â we're trying to estimate a regression that relates an outcome.

Â Whether it be continuous, binary, or timed event to a single predictor.

Â Like participation in a program or not.

Â Expose, we'll call exposed or unexposed.

Â Where people self select to be in the program.

Â And we try to adjust for all other measurements we have on the persons.

Â Well if we have a fixed amount of data and we have to estimate a bunch of

Â other slopes, quantifying the relationship on some scale.

Â And if some of these things aren't necessarily related to the outcome per se.

Â Then especially in that case we may end up compromising or

Â watering down the uncertainty of our main predictor of interest.

Â Because we have to estimate some things that aren't statistically significantly

Â associated with the outcome.

Â However, these things may help uh,uh, get us a better adjusted comparisons for

Â the outcome main predictors association.

Â So in this situation of potential solution is propensity scoring, a method for

Â creating a single measure.

Â Such that subjects or observations with similar scores are similar in

Â terms of their entire set of confounder values.

Â So in this lecture set we will briefly discuss,

Â we're just going to give a conceptual approach to this.

Â The definition and computation of propensity scores.

Â We'll talk about straightforward adjustment using the propensity score as

Â the adjustment factor as opposed to multiple predictors.

Â And then, we'll talk briefly about an approach called propensity score

Â matching of subjects in the exposed and unexposed groups.

Â And the reasons why that may be employed in certain study situations.

Â So this first section, section A,

Â we're just going to set up the idea of propensity scores, define them.

Â And give and example of adjustment with propensity scores,

Â which will continue on and give more examples in section B.

Â So hopefully by the end of this section,

Â you'll be able to define a propensity score.

Â And define how it's computed with regards to a primary predictor of interest.

Â And explain how a propensity score can be used to adjust for

Â multiple predictors at once.

Â And the potential benefits over including each predictor separately in

Â a multiple regression model.

Â In other words,

Â using a propensity score approach to adjust an outcome exposure relationship.

Â As opposed to putting the potential confounders of interest in separately,

Â one at a time, in a multiple regression model.

Â So let's just motivate the situation.

Â Suppose a researcher is interested in the association between the outcome.

Â Whether it be continuous, binary, or timed to an event, and a binary predictor.

Â Whether the subject was exposed or unexposed.

Â And this has to be estimated from an observational study.

Â The exposure of interest cannot be randomized.

Â And let me just note that we can employ the approach we're

Â talking about in these lecture sets.

Â To situations were the predictor is multi-categorical as well.

Â But we won't expand on the mechanics of

Â that because conceptually it's exactly the same as what we'll be doing.

Â So because of the observational study design, confounding is a threat.

Â However, the researcher has collected data on many potential confounders and

Â is interested in adjusting for these.

Â How can she or [SOUND] he proceed?

Â Well, option one is what we've been talking about in the last three lectures.

Â Multiple regression where we include the predictor of interest, let's call it x1.

Â And, then put in potential confounders as separate predictors in the model.

Â And this might be fine, especially if the researcher were interested in

Â the adjusted association between the outcome and each of these predictors.

Â But suppose he or she isn't, all they really want to do

Â is get the best most adjusted estimate of the outcome x1 association.

Â What are some potential problems with doing the approach we had just talked

Â about in the last three lecture sets?

Â Well the researcher has limited amount of data.

Â The researcher is only interested in adjusting for

Â the confounders, x2 through xp we'll call them.

Â But is not interested in associations between the outcome in each of

Â these things.

Â So having to include so many extra x's in the regression model,

Â may compromise the the precision of our estimated interest.

Â If some of these confounders are related to each other, and

Â some of them are nonstatistically significantly related to the outcome.

Â But we want to include them for adjustment nevertheless.

Â We may be estimating things that don't actually add

Â information about the outcome.

Â And that's going to pull away from our ability to

Â estimate the outcome exposure interest association with the most precision.

Â So another option under this scenario with the researchers only interested in

Â that one association really in terms of quantifying.

Â Is to create a single numerical summary,

Â such that subjects with similar values on this numerical scale are similar.

Â In their values of a potential multitude of confounders,

Â x2 through xp repeated's just a number.

Â So for example if we had nine potential confounders we'd be looking at

Â x2 through x10.

Â So one such approach to creating this single

Â numerical summary measure is called a propensity score.

Â And like I alluded to before this can be estimated and

Â created if the main predictor of interest is binary.

Â Can also be extended to multi-categorical.

Â If our main predictor of interest is continuous,

Â it would have to be dichotomized or categorized.

Â In order to proceed with a propensity score approach to adjustment and

Â we'll see why in a minute.

Â So, how do we create propensity scores?

Â Well, here's what we do.

Â If the main predictor is binary, we'll call it a 1 if subject is exposed and

Â the 0 if they are unexposed.

Â And we have potential confounders,

Â I'll call them again x2 through xp where p is just the number of x's total.

Â Then what we would do to estimate a propensity score for

Â each observation is perform a logistic regression.

Â But what we're doing here, our outcome in this logistic regression,

Â happens to be our predictor of interest for the main analysis, x1.

Â And we'd estimate with the logistic regression is the log odds of being a 1.

Â So if 1 is exposed and

Â 0 is unexposed, we're estimating through logistic regression.

Â The log on to being in the exposed group as a function of

Â the potential confounders.

Â So what we end up doing is estimating this multiple logistic regression and

Â getting an equation.

Â And then what we can do with this equation is for each observation in our sample.

Â Use the logistic regression results to estimate the predicted probability of

Â being in the exposed group from the logistic regression values.

Â Given the observation's values of x2, x3, etc.,

Â . up through x p.

Â So, we just plug those values into this equation.

Â Get an estimated log odds for the observation.

Â Turn it into an odds.

Â Then turn it into the estimated probability.

Â So it's the estimated probability that observations with these values of

Â the potential confounders are in the exposed group.

Â This is called the propensity score for each observation.

Â It's the propensity or

Â probability of being in the exposed group, given the confounder values.

Â So let me give you an example of this based on some research I've done.

Â And I actually didn't use propensity score adjustment for reasons I'll discuss.

Â But I'll show you and compare and contrast the results I

Â got the traditional way versus using the propensity score adjustment.

Â This'll be of interest to many of you, hopefully.

Â This is a study I did a few years back.

Â I was very curious about comparing course outcomes in

Â statistical reasoning between the online versus on-campus sections.

Â Knowing that the, potentially the student profiles were very different in

Â terms of demographic and educational characteristics.

Â So I did a survey on students.

Â And they consented to be, and not everybody participated,

Â where I collected information on demographics, educational history.

Â Had students do a baseline test to get a baseline knowledge score, etc.

Â And so, here's the abstract from the article published on this.

Â The objective was to compare student outcomes between concurrent online and

Â on-campus sections of introductory bio-statistics course offered.

Â And this was actually back in 2005.

Â It took awhile to publish because initially there wasn't much

Â interest in this.

Â And then when all these MOOCs, massive online open

Â courses became popular through Coursera etc etc, there was rejuvenated interest.

Â And so what the methods, we had 95 students in the online section and

Â 92 in campus.

Â Overall invited to participate in a confidential online survey.

Â And then these were linked to the course outcomes by a data manager.

Â And the course outcomes were bec, prepared between the two sections adjusting for

Â differences in student characteristics.

Â And I'll present the results in tabular format in a bit,

Â but there were 72 participants from the online section.

Â Participation rate of 76%,

Â and 66 from the on campus section of participate rate of 72%.

Â And they can be unadjusted final exam scores for the online and

Â on campus sections for 85.1 out of 100 86.3, out of 100 respectively.

Â These are the averages, and 87.8 and

Â 86.8 respectively between the online and on campus sections.

Â So the differences unadjusted were on the order of one point on average.

Â After adjustment for student characteristics,

Â the average difference in scores between the two sections was negative 1.5.

Â Slightly lower mean score and average from the online section, but

Â is not statistically significant.

Â And then 0.8 for the online section, compared to

Â the on-campus in term two 0.8 points higher on average for the online section.

Â But that was not statistically significant.

Â And so we concluded, and there's more discussion of this in the article,

Â that the results demonstrate that online.

Â An online campus.

Â Online, you know on-campus course formats of an introductory biostatistic course in

Â a graduate school of public health.

Â Can achieve similar student outcomes despite

Â potential characteristic difference between the enrollees.

Â So let me just show you what I mean by characteristic differences.

Â Here's some information that we collected.

Â So comparing the results between the online and campus groups.

Â So, so let's look at a couple things.

Â Look at the sex distribution.

Â The on-campus course was less, less than a quarter were male,

Â versus 42% in the online course, and that difference was statistically significant.

Â This part time about nine, eight only about a fifth of

Â the students on campus were part time compared to 90% online.

Â The distributions of degree programs was statistically equivalent.

Â However there was a greater by 5% proportion of

Â MPH students online versus on campus.

Â Similar distribution of prior statistical coursework,

Â except when we looked at having had statistics in graduate school.

Â But, what was interesting was,

Â there were striking differences in the highest degree obtained.

Â About two-thirds, or 65%, on the on-campus section.

Â The highest degree they had at that point was a bachelor's degree.

Â Whereas in the online section, a little over a quarter,

Â their highest degree was a bachelor.

Â And nearly half had an MD, as composed to 20% in the on campus section.

Â And then, another striking difference was with the age distribution.

Â And subsequently, it was correlated with years in school and

Â years in this last math course.

Â The average age.

Â You know online section in that the average age in the on campus section that

Â year was 30 years versus 38.2 in the online section.

Â So if any of these things were also associated with course performance.

Â They may confound the unadjusted comparison in

Â exam scores between the online and on campus sections.

Â So I did not use propensity scores to adjust for

Â these other factors extensively.

Â Because I was also interested in how these other

Â factors associated with course outcomes.

Â Even if they didn't necessarily confound the difference between the online,

Â on campus courses.

Â But I could have used propensity score approach and

Â since I have these data, let me show you how that would work.

Â So if I wanted to do a propensity score approach, what I would first do is

Â create a propensity score by using the data to run a logistic regression.

Â Where my outcome would be a 1, if the student was in the online section,

Â and a 0, if they were on campus.

Â And what I do is estimate through logistic regression the log odds of being

Â enrolled in the online section.

Â As a function of some of those other characteristics I showed.

Â The potential confounders.

Â So, including, the x's would include things like age, sex.

Â Having an MD degree versus not, years since last in school,

Â baseline knowledge score and having had a previous course in statistics.

Â I would run this logistic regression model.

Â And once I had the results, I could ask the computer to estimate for

Â each of the observations in my dataset.

Â The probability of being enrolled in the online section for each student.

Â We know whether they were or not.

Â But what we are at extensively the estimating is the probability of that

Â students with the same confounder or potential confounder values.

Â Would be enrolled in the online section.

Â And this is the propensity score for each student.

Â And then, and then what we would do in the next part was to

Â estimate the relationship between exam scores and core section.

Â Adjusting for these other factors as I would do another multiple regression,

Â in the case a linear regression.

Â because my primary outcome of interest is exam scores which is

Â measured from 0 to 100.

Â And my primary predictor of interest is an indicator.

Â We'll call it x1 of whether the student was on the online course, or on campus.

Â And then I could one way to handle this I could stick the propensity scores in

Â as continuous.

Â But a more common way to do this, and

Â because of the underlying linearity assumptions.

Â What I could check, would be to put the propensity scores in quartiles.

Â The propensity scores themselves,

Â they're more interpretable in terms of relative comparisons.

Â People with propensity scores closer to each other are more similar in terms of

Â their potential confounder values.

Â So putting them in quartiles is a reasonable thing to do as well.

Â So I put them into quartiles and included them in this model.

Â Ostensibly to adjust for the other factors I

Â included in the propensity score computation by using the propensity score.

Â Let me show you what the distribution pf propensity scores look

Â like by course section.

Â These are box plots.

Â And again,

Â what we're estimating is the probability of being in the online section.

Â And we see there are you know there's,

Â there are some differences it's clear that you know, those in the online section.

Â Their characteristics were used to estimate the probability of being in

Â the online section.

Â And they tend to have higher scores then those in the on campus.

Â But there's a fair amount of crossover even though their shifted upwards for

Â the online students.

Â There's a fair amount of crossover in these scores between the two sections.

Â So let me just show you the results through three different

Â approaches to estimating.

Â I'll give the unadjusted comparisons, the unadjusted mean differences and

Â confidence interval intervals.

Â And I'll do this for each term, the final exams for term one and term two.

Â And then I'll give you the adjusted that I report in the article.

Â This was adjusted the traditional way.

Â By doing a multiple linear expression of exam scores on course section.

Â Plus entering each of the potential confounders as separated x's and

Â estimating their associations as well.

Â And then, I reanalyze these.

Â I reanalyze these.

Â Adjusting with the propensity score approach.

Â And so lets look at for

Â example the difference in online on campus in the first term.

Â The unadjusted mean difference shows a slightly lower score,

Â shows a negative 1.3.

Â A lower score of 1.3 points on average out of 100 point test.

Â For the online can, section compared to on campus, but

Â this was not statistically significant.

Â After adjustment, for the factors that I noted and

Â also used in the propensity score.

Â This difference was qualitatively similar, negative 1.5,

Â and still not statistically significant.

Â If I did the adjustment via propensity scores I got something a little larger in

Â absolute value.

Â Negative 1.8 versus negative 1.5.

Â But still, after accounting for

Â the sampling variability, it's not statistically significant.

Â And the term two,

Â the unadjusted mean difference favored the online section slightly.

Â With an average score difference of 0.9 higher scores for the online section.

Â But this wasn't statistically significant after adjustment the traditional way.

Â By including the other potential confounders as their own

Â x's in the regression model.

Â This difference was very similar at 0.8.

Â Still not statistically significant.

Â In terms of adjusting with the propensity score approach where the predictors of

Â the propensity score were the same things.

Â I adjusted for individually as individual x's in this first approach,

Â this difference was a bit larger, 2.4.

Â So this was the 20 point, 20 question exam worth 100 points so

Â about half a question 2.4.

Â So the magnitude increased with this different type of adjustment, but

Â it still wasn't statistically significant.

Â So generally speaking the conclusions I would make regardless of whether I

Â adjusted the traditional way.

Â Or with the propensity scores would be the same.

Â In this example there isn't any specific benefit of

Â using propensity scores that can point to.

Â The confidence intervals for this adjusted difference were not

Â necessarily narrower when I use propensity score adjustment.

Â Versus the traditional approach.

Â And again, I was interested in the contributions of the other factors.

Â But I wanted to give an actual demonstration of how this would be

Â done using data that I am privy to.

Â So, let's talk about potential benefits, first, of using propensity scores.

Â It may allow for a more precise estimate of the outcome, predictor of

Â interest association than traditional adjustment with multiple regression.

Â In the example I just gave you that wasn't the case.

Â The width of the confidence intervals for the adjusted mean difference in

Â test scores between the online and on-campus section were similar.

Â Whether I did the traditional adjustment by including all

Â minor adjustment factors individually as their own x's in a multiple regression.

Â Or whether I did the propensity score adjustment it didn't have much effect.

Â But suppose I had estimated, suppose I had measured 20 or

Â 30 potential confounders on these subjects.

Â Then, using the propensity score approach versus including the 20 or

Â 30 as individual predictors in the multiple regression.

Â Really might help me estimate the difference between the online and

Â on campus sections with a lot more precision.

Â The other potential benefit is it allows for a single measure that allows for

Â comparing observations.

Â In terms of their similarity on many factors.

Â And that's a nice index.

Â If we wanted to cluster and

Â look at graphics showing the relationship within different groupings.

Â Based on this we could actually visually look at adjusting for

Â multiple factors in a single graphic.

Â Using the propensity score to measure the similarity.

Â Potential drawbacks one potential drawback is to investigate effect modification.

Â We can't include the potential effect modifier in

Â the propensity score computation.

Â So lets suppose I wanted to see if there were any differences in

Â the outcomes between those who had a previous statistics scores versus not.

Â In other words, whether the relationship between exam scores in

Â course section was modified by having a previous statistic score.

Â So what I'd have to do for

Â my final analysis is I'd have re-run the propensity scores.

Â Not including previous statistics courses as a predictor.

Â And then when I did my multiple linear regression where

Â I just do with propensity scores I would include my predictor core section.

Â My predictor of having had a previous stats course

Â The interaction In the x 1 times

Â x 2 and then the propensity score quartiles,

Â where the propensity score no

Â longer included previous stats course.

Â So it is just a little bit more cumbersome when we want to involve interactions, but

Â it can be done.

Â Another drawback is the possibility of some observations having

Â similar propensity scores, but having very different values of some factor.

Â Let me give you a toy example based on this.

Â Suppose that increased age is associated with

Â increased probability of being in the online section.

Â And let's suppose increased baseline score.

Â This wasn't the case but just baseline knowledge score is also

Â associated with increased probability of being in the online section.

Â Well, we could have somebody who's very young with a very high score.

Â A high baseline score.

Â And, somebody who's older with a low baseline score.

Â Look similar in terms of these two things.

Â The contribution.

Â Combined with these two things to their estimated propensity score being

Â the online section would be similar.

Â Because one would be getting a bump

Â because of their high baseline score but no so much because of their age.

Â And the other would be getting a bump because of their age, and not so

Â much as their low baseline score.

Â So it's possible.

Â And I'm just.

Â This is an example with two predictors.

Â But it's possible that these things can operate in tandem to,

Â to give a false sense of closeness on these distributions.

Â There are ways to investigate that and make sure that doesn't happen.

Â But that's a potential threat.

Â But in general, I want to give you a sense of what propensity scores are.

Â And how they're estimated in the next section we'll look at some examples from

Â the literature.

Â But in some propensity scores provide an alternative approach to

Â traditional multiple regression.

Â For estimating an adjusted outcome predictor association.

Â And, this is especially useful when the predictor of interest is binary and

Â there are many potential confounders of the outcome/predictor relationship.

Â And the adjusted outcome/confounder relationships are not of

Â scientific interest.

Â Propensity scores are the estimated probabilities of being in

Â the exposed group.

Â As expose, the unexposed group.

Â Estimated from a multiple logistic regression with all potential

Â confounders as predictors.

Â These scores then are estimated for each observation in the sample.

Â In other words, the estimated probability of being the exposed group for

Â subjects with the same values of the confounders.

Â As that particular observation.

Â And then their relationship between the outcome and

Â predictor of interest can be adjusted.

Â Using the only the propensity scores in a regression model.

Â Coursera provides universal access to the worldâ€™s best education,
partnering with top universities and organizations to offer courses online.