0:00

Hi, In the Sun Lectures we're talking about prediction, And here's the idea, we

Â want to think of individual people make prediction based on models. Those models

Â can be based on categories or linear models or Markov models, any of the models

Â you've learned in this class you could use to make some sort of prediction. And what

Â we want to talk about is where collective wisdom can come from. So if we have a

Â whole bunch of people using a whole bunch of different models, how does that enable

Â the crowd of models to do better at making sense of the world, making an accurate

Â prediction? Now, the essence of our argument is going to be something called

Â The Diversity Prediction Theorem. And, the Diversity Prediction Theorem is gonna

Â relate the wisdom of the crowd to the wisdom of the individual. So, in other

Â words the accuracy of the crowd, in relationship to the accuracy of the

Â individuals. Now one logic that should come to you right away is if I had more

Â accurate individuals, I should also get a more accurate crowd. But a logic that

Â might not come to you right away is that if I had a more diverse crowd, I should

Â also get more accuracy. So, we can think of things like the crowds accuracy is

Â going to depend on the individual accuracy Plus The diversity. Now the question is

Â how much do these things matter. How much does it depend on, individual accuracy and

Â how much does it depend on diversity. That's why we wanna use a model, to figure

Â out. So let's first do an example just to get some bearings, some inner bearings in

Â terms of what it means for a crowd to make a mistake versus an individual to make a

Â mistake and what diversity is. So we'll do an extremely simple example. We have three

Â people, Amy, Bell and Carlos. And let's suppose that they're picking the number of

Â people that come to our diner on a particular day for lunch. And so Amy

Â predicts it's gonna be ten. Val predicts it's gonna be sixteen and Carla predicts

Â it's gonna be 25. Now if I add these up, I'm gonna get 51 and divide by three, I'm

Â gonna get an aver age value of seventeen. So the crowd predicts seventeen. Now let's

Â suppose the actual value is eighteen. Now again, if, let's suppose the crowd is

Â pretty accurate, it's not gonna matter but this is just for the purpose of the

Â example, I'm gonna make it so the crowd actually does pretty well, I just wanna

Â work to the logic. So the first thing I'm gonna do is I wanna figure out how

Â accurate are these people. Well what I can do is I can compute the error of each

Â person. So remember the, the true value was eighteen, that's the number that

Â showed up. And I can ask what's the error of each individual? And remember we

Â computed errors by looking at variations, squared error. So INU predicted ten, the

Â truth was eighteen, so her squared error is 64. Belle predicted sixteen, The true

Â value is eighteen, So her score there is four. Carlos predicts 25, The true value

Â is eighteen, So his score there is 49. And if we add all those up, I get 117, And if

Â I divide that by three, I get the average there that's 39. So on average, these

Â people are off by 39. Some people, Belle, are really accurate, She's only off by

Â four. Other people, Amy, is off by quite a bit. She's off by, her error is 64, But

Â the average is 39. So this sort of gives us a sense of how accurate the individuals

Â are. The individuals off, are off by an average of 64, four and 49, for an average

Â of 39. Now we can ask how accurate was the crowd? Remember the crowd predicted

Â seventeen, because that was The average prediction of the three people. The

Â [inaudible] is eighteen, so we get the crowd was only off by one. So the crowd

Â here is, if you notice, better than anybody in it. So, we get "the wisdom of

Â crowds." Well, let's try and think about. Why that makes sense and to do that we're

Â going to look at diversity. So diversity [inaudible] is the variation in the

Â predictions. So how do we do the variation in the predictions? We look at each

Â person's prediction and its distance from the mean prediction not from the true

Â [inaudible], the mean prediction. So the mean predicti on was seventeen so. Amy's

Â contribution to this sort of total variation of predictions is ten minus

Â seventeen squared, which is 49. Belle's is sixteen. It should be seventeen minus

Â seventeen squared, which is one, and Carlos's is 25 minus seventeen square

Â root, which is gonna be 64. Now if I add all these up, I get 114, And if I divide

Â by three, I get 38. So the diversity of these predictions is 38. Well notice this.

Â The crowd's error was one. The average error was 39 and the diversity was 38. So

Â I look at that, I get one equals 39, minus 38. The crowd's error in this case equals

Â the average error minus the diversity. But I just Set this up, What turns out That's

Â always true. This is what the diversity prediction theorem says; That the crowd's

Â error equals the average error minus the diversity. Now, this isn't some, you know,

Â feel good setting, This is a mathematical fact, This is an identity. So no

Â assumptions have to be made here. This, there's no opposite [inaudible]. This is

Â just true. If I have a set of predictions, it will always be the case that the error

Â of the crowd to the average errors. Squared error, the average prediction

Â squared error is going to equal the average squared error of the people in

Â that crowd, minus the diversity of their predictions. Now the way to write that

Â formally is like this, Now this looks pretty scary, but let's just walk through

Â it. So let's let C be the crowd's prediction. Data be the truth, so data is

Â equal to the true value, And so this [inaudible] thing. This is the crowd

Â square [inaudible]. So it's the distance from the crowd to the truth. Let's let SI

Â here equal individual I's prediction So individual I's prediction. And so we're

Â gonna get... This is I's prediction minus the truth squared. And then we sum that

Â all up over all the individuals, and we divide by the number of individuals. So

Â that's just gonna be the average error. So crowd [inaudible] equals average error

Â minus... Now we take each person's prediction minus the crowd's prediction,

Â which is C. Remem ber, because C is the crowd. So this tells us how far people are

Â from the crowd on average. We sum those up and rate divide by N, So we get the crowds

Â there equals the average airlines diversity. Now if you take this equation

Â and expand all the terms and cancel everything out you'll see that it's an

Â identity. It's a mathematical identify. So it's always true. Crowds there equals

Â average year minus diversity. Let me give a famous example to sort of drive this on.

Â So in a book called The Wisdom of Crowds by Jim Surowiecki, he talks about the 1906

Â West of England Fat, Stock, and Poultry Exhibition. At this exhibition, 787

Â people, Guess the weight of a steer. Their average guess was I think, 197 pounds; the

Â actual weight of the steer was 198 pounds. So they're only off by a pound. So you're

Â looking at it and say, oh my gosh that's amazing, that's the wisdom of crowds. But

Â let's think about it, what's going on? We've got a bunch of predictions, there's

Â a true value, there's an average value, our theorem, this thing, this [inaudible]

Â theorem must hold. And, in fact, if you take Galton's data. And you plug it all

Â in, here's what you get. The crowd's error is actually a little bit less than a

Â pound, it's.6. The average error is 2956. Now, wait, that seems crazy, 'cause

Â remember, the steer only weighs 1100 pounds. So if this thing weighs 1100

Â pounds, how could they be up by 2956? Whenever these are squared errors, so if I

Â square 50 I get 2500 and if I square 60 I get 3600. So this is probably 55, 56

Â squared. Something like that. Well, that makes sense because people could probably

Â guess the weight of a steer within about 55, 56 pounds. Well, why is that? Well

Â think about it. A steer's five times the size of a person. If you can guess the

Â weight of a person within about ten pounds, you can probably guess the weight

Â of a steer to about 50 pounds. So what you've got is you've got some sort of, you

Â know people are reasonably good at guessing the weight of steers. They're not

Â geniuses, but they're also not crazy. They're not guessing 15,000 pounds. So

Â these are reasonably knowledgeable people who for whatever reason are you know

Â making these errors of about 55, 56 pounds. Not it's interesting is that there

Â diversity is 29 55. So, what you get is. The crowd is wise because they're

Â moderately accurate. I drop by 55, 56 pounds and they're are also diverse and

Â it's that accuracy plus diversity that makes the crowd do so well. Now if you

Â think about this book, the wisdom of crowds, see we can get a bunch of

Â examples, well that's the case. Let's think about in the context of our theorem,

Â so we've got crowd air, equals average air, minus diversity. Now in this book,

Â Sir Wiki says, here's what matters, diversity matters a lot. Well why does

Â diversity matter a lot f we're looking at the wisdom of crowds, let's see, this

Â actually, the math will tell us why. If you make it into the book, the wisdom of

Â crowds, what has to be true? This has to be small. The collective area has to be

Â small. So if the collective area isn't small it doesn't make the book it's not

Â the wisdom of crowds, it's the madness of crowds. So for the wisdom of crowds to

Â exist this has to be small, collective area has to be small. Let's think what

Â else has to be true, the average air has to be fairly large, why does it have to be

Â fairly large? If the average air is small, that means it was a easy thing to predict,

Â everybody can pretty much get it right. So if it's interesting enough to make a book

Â called the wisdom of crowds, where the crowd is smart and the people aren't, if

Â the people are not smart that means the average area has to be large. Well if you

Â got something small equal something large minus something else. This other thing has

Â to be large, Which means diversity has to be large. So when Surowiecki walks through

Â all these examples and he looks at what's going on, he says, look there's a lot of

Â diversity. And diversity seems to be a key component in the wisdom of crowds. You

Â want to encourage people to think about the world in different ways if you wanna

Â get the wisdom of crowds. And our model explains why that's the case. It's

Â collective error equals average error minus diversity. If people aren't that

Â smart, average error is gonna be big. If you want the crowd to be smart, the only

Â way to get it is by having that crowd be diverse. So if we look at the first

Â example of [inaudible]. Where we get 0.6, 29 56 and 29 55. We see that's exactly the

Â case, small crowd error, you know fairly large average error because it's not an

Â easy thing to do and then high diversity. And if you take examples of wisdom of

Â crowds from all over the place, you'll see they all look like this, they look exactly

Â like this, Small crowd error, large individual error, large diversity.

Â Question is, how do you get how do you get that diversity? Well you get that

Â diversity by people using Different categorizations. Different that your

Â models. Maybe people using entirely different models. Maybe, one person's

Â using a mark off model and one person's using a diffusion model. Maybe one

Â person's using a linear model. Maybe one person's got a non linear terminal model.

Â There is a lot of different variables. So, what you get is this how originating what

Â we see the world. In the boxes we use and the variables we use, and the models that

Â we construct. I would give Diversity to these collective predictions. In that

Â collective prediction, those collective predictions, then, lead to accurate

Â crowds, provided you've got reasonably accurate people who are reasonably

Â diverse. And what we've learned is by constructing a very simple model of that

Â predictive task, where the wisdom of crowds come from. And we've learned that

Â individual ability. And collective diversity matter equally, they're equal

Â partners. So if someone were to say to you, where does the wisdom of crowds come

Â from? You could say, well it comes from. You know, reasonably smart people who are

Â diverse, And you could also ask, where does the madness of crowds come from? How

Â could it be that a crowd could get something totally wrong? Well, that's not

Â har d either; cuz crowd error equals average error times diversity. Well, if I

Â want this to be large. I want large collective air, then I need large average

Â air, cuz I need people to, on average, be getting things wrong, and I need diversity

Â to be small. So the [inaudible] of crowds come from like-minded people who are all

Â wrong, and again, the equation gives us that result. Alright, thank you.

Â