As we just saw, local averaging has some very interesting mathematical properties.

It's a very simple operation as well and is very useful as we saw a few examples

before. But we have also observed that it can

blur the information in the image, it can blur the edges.

So, how do we solve that? We have a number of different directions

to go. One is to basically replace this local

averaging by a different local operation and we're going to see that very soon.

The other is, we may ask ourself, are we only limited to average in the local

neighborhood? Maybe we can do a smarter average.

Remember that averaging is like a heat flow.

So, shall we basically average between a very hot pixel and a very cold pixel or

shall we try only to average between pixels that are similar and try to stop

averaging pixels that are very different. And that's, a very interesting concept

that can actually be implementing a very, very variety of ways in a very different

number of directions. But let's just explain one which is

extremely elegant. In part it's elegant because it's very

simple and it's actually called non-local means.

Instead of looking at averages in the local neighborhood, we're going to look

at averages in the whole image. Let's see why we can do that and how we

do that. The idea is very simple.

Let's just observe this image. We actually see that images have

repetition. Sometimes these are called

self-similarities. So for example, if we look at this local

neighborhood, it's very similar to this local neighborhood, and very similar to

this local neighborhood. We also observed that here this local

neighborhood is very similar to this local neighborhood,

even similar to this. They're far away, but they're very

similar, the same here.

So, before, we were thinking about averaging only pixels inside here, maybe

we should actually average neighborhoods that are similar.

Maybe we should average these three, so we can replace this pixel by certain

relationship with others, but we should not mix this with these because the local

neighborhoods are very different. So, the basic idea of non local means or

non local averaging is to try to average, try to enjoy the properties of averaging

but not restrict that to a local neighbourhood.

So why are we doing that, and how are we doing that?

So, the basic idea is that if we assume that these are identical things,

identical structures, but maybe with different noise.

Let's see the effects of that. Let's assume that we have a certain

pixel, lets call it P, that's what we observed.

It's the clean pixel plus some noise. But the noise, basically is a random

variable, so it's always slightly different.

So, we observed the pixel a number of times.

We observed one with a certain noise. We observe it again, same pixel,

but with a different noise. And we can observe it multiple times,

let's say that we observe this n times, always the same pixel but with different

realizations of the noise. If we average all these,

we can easily show these are basically properties of signal analysis,

that the noise goes down proportional the square of the number of times that we

average. So we are averaging always the same

pixels, the noise is the random variable,

and basically, the noise goes down proportional to n^.

The more we average, the more we reduce noise.

Of course, it's very important that we always average exactly the same pixel.

The noise is the only thing that changes. So this is one of the motivations of non

local means to say, okay maybe I can find repeated structures that are identical or

at least very similar, the noise is different all around and I can reduce

that noise by averaging just because of this.

And the basic idea then, is to look for similar neighborhoods first and average

them. So if I want to replace this pixel in the

center, here, I look at the neighborhood. I look for similar neighborhoods and I

replace the pixel by the center, by the average of the center of the similar

neighborhoods. And how do I find similar neighborhoods?

So there's a whole literature on non local means,

that basically includes two parts. How do I find similar neighborhoods?

That's the first part. Once I found them, how do I average them?

Do I weighted average them? Do I compute a different relationship between them?

But once again, the simplest for both options is look for neighborhoods by

comparing, let's say, the mean square error between them and make sure that

it's very small. That's a very, very simple operation.

You go around the image you look at neighborhoods, lets say 3x3 or 5x5 or