Welcome back. In this module, we're going to talk about meta analysis. Some of its uses, and how to do it. So, why is meta analysis important? Well, we need to solve the crises of replicability and interpretability in order to build a cumulative science. And meta analysis is one of the principle ways of doing that. So, what is meta analysis? With any study, there's a risk that the conclusion is wrong. That it's a false positive or that it's limited in it's generalizability across experiments and contexts. So, meta analysis is the analysis of findings across publications from different research groups. It can help establish the likelihood of the true findings, the generalized ability across studies and populations and research teams, and it can also help evaluate the heterogeneity across studies that can point to moderating variables, things that are influencing the sizes of those effects or the incidence of those effects, that we hadn't previously thought of. So, what is meta-analysis? Meta-analysis often combines summary statistics like effect sizes across a population of studies or group of studies based on reported values in the publications themselves. So when you're imaging the most common practice is to analyze reported activation coordinates from published tables. This is called coordinate-based meta-analysis, and the coordinates are usually reported in standardized space in virtually every study or most studies. So, we get a pretty good sample of overture from this, and the coordinates usually reflect the peek statistic values. And they are often reported in Montreal Neurologic Institutes base MNI space, or what's called Talairach space, which is an approximation that's similar to MNI space but less precisely defined. So these x, y, and z coordinates refer here to coordinates in brain space. The 0 point is the anterior commissure, the small commissure that connects the hemispheres. And you can see on the structural scan and mark it off, and x is left to right. Y is posterior to anterior. And z is inferior to superior by convention. So, we can take all these reported coordinates and put them into a series of studies. So, coordinate-based meta-analysis will turn this collection of coordinates across the brain into a picture of where the consistent findings, are and where there's a significant density of reported coordinates that exceed what you expect by chance. So, we can use this for a number of things. One is we can use this to establish consensus across studies. We can evaluate the specificity of the activation across categories of mental events or types of studies. And we can come up with a priori hypotheses, which can be regions or patterns that can improve inferences in your study. Finally, we can use this as a guide to interpret findings in new studies as well and make better inferences about what brain maps mean. A similar concept is called mega analysis. It's often possible to combine image data and re-analyze image data from participants across many studies, for example. And this is often called image based meta analysis, or mega analysis. And this is actually preferred, where it's possible to do that, to coordinate-based meta-analysis, because it's much richer information. We have affect sizes across every vessel in the brain. So, for example, here's a pain meta-analysis done two ways, on top is an image based mega-analysis across a number of studies. And, on the bottom, is a coordinate based meta-analysis, and at it's heart, any coordinate based meta-analysis is going to attempt in some way to reconstruct what the maps look like, and then do a test for consistency across studies on those maps. Meta-analysis is increasingly popular, so the number of meta-analysis have been going up across the years. And that's in part a response to the huge number of studies that are coming out, and the need to synthesize and interpret findings. So, some of the topics of meta-analysis you can see below, they range from things like schizophrenia, depression, emotion, various disease categories and various other kinds of basic cognitive processes. So, let's look now at some tools for doing meta-analysis, and then how we can use them. So, one of the tools for coordinate-based meta-analysis that you can download and use is called multi-level kernel density analysis, MKDA, you can download it. Another tool is a web-based tool called neurosynth.org. And, finally, there's brainmap.org, which is a repository of coordinates and a tool for doing activation likelihood meta-analysis, which is a very similar method to MKDA. And so here's kind of how it works. We take the reported coordinates in the literature across a population of studies, and we can break them up into which study they came from and which specific contrast or map within that study that they came from. So, that's what we mean by contrast here. So, those are contrast-specific coordinates. We need to know which studies they came from, because we're going to take each of those. And the points are very sparse in space. So, we're going to convolve them with a kernel, a spherical kernel in this case, and then we end up with a reconstructed contrast map from each of those studies. Then we take a weighted average of those contrast maps, and from that we get a map of how consistent the activations are. We simulate a null hypothesis case by permuting the location, shuffling the locations of the activations and taking a weighted average again and again over many iterations and finding the maximum statistic value just like we would with multiple comparisons correction in other venues. And, finally, then we can apply that threshold and end up with results that are corrected for multiple comparisons. And what we're really then saying when we have a significant result is, here's an area where the consistency of activation across studies exceeds what we expect by chance, where chance is a random even distribution across the whole brain. So, the convolution with spherical kernel is really a smoothing kernel that gives us a interpretable metric, which is how many contrasts or studies activated in a local region. We weight by the sample size, the square root of the sample size, by whether it's a fixed or random effect study. Which influences the quality in statistic values and other custom quality metrics are possible as well. We don't like to weight by Z-scores, because Z-scores are high variance, and the smaller the study the more likely it is that you're going to find high Z-scores by chance. And so then, you're weighting by something that's actually exactly the wrong thing in that case. So it might seem sensible, but that's a choice that we've made, and you can make your own choices. We do thresholding in this case via blob level permutation. What that means is we take those reconstructed contrast maps and we move the whole blob around, so we're preserving the structure, the spatial structure or the activations, in order to identify signficant regions. And this reduces bias towards small sample studies and towards studies that report more peaks. In the worst case, we can't just summarize the number of peaks overall, because some studies report many, many peaks, others very few. And sometimes the smallest studies, the worst studies in many ways report more peaks, because they have high variability in their maps. So MKDA is also a gateway to the flexible use of meta-analytic data across many uses. So, we can do MKDA maps for one condition, like I showed you. We can make difference maps that compare different conditions to one another. We can do chi-square analysis, or logistic regression analysis. That give us complementary pictures of differences among maps. This really gives us a matrix of studies by voxels that we can do lots of things with then. So, we can use those to do multidimensional scaling or graph theoretic analyses of the relationships in the meta-analysis. We can use those to visualize regions and cluster them into networks and parcel them into brain regions. We can use them to examine the associations between the incidents of brain activity and the task type for decoding across a wide variety of different types of psychological and clinical processes and outcomes. So, this is really useful for reverse inference, in a formal setting. So, here's another tool. It's called neurosynth.org, and it's an online tool for meta-analysis that was built by Tal Yarkoni a few years ago. This contains published activation coordinates from 10,000 neuroimaging studies now, or more. And along with those coordinates, there is saved, the text or keywords and topics from each of those studies. So, then those studies and coordinates can be mined for relationships. So, if you do a term-based search in Neruosynth for something like pain, it knows what the related studies are that talk about pain a lot, and the coordinates, and then it will construct an automated meta-analysis of the studies that use pain quite frequently. And this is all done on the web. And they're actually maps now for 10,000 common terms. So, here's one for vision, social processing memory, reward, pain, memory retrieval, language, emotion, and so on. This is being actively developed, and it can be useful for connecting region of interest analysis or pattern of interest analysis that we're going to talk about later. It's called feature sets in Neurosynth. For creating co-activation maps, clustering of regions, and even recently genetic associations via matching up the Allen Brain Project genetic maps with meta-analysis maps from neurosynth. And so, there are lots of things that you can do with it. We won't go over all of them, but there are many uses. So that's the end of this module on meta-analysis, and thanks for your attention. And that was the first part of the lecture. Now onto the second part of the lecture. >> Tor, I said ten minutes! >> [LAUGH] >> [LAUGH] [SOUND]
