In this course, you will learn the science behind how digital images and video are made, altered, stored, and used. We will look at the vast world of digital imaging, from how computers and digital cameras form images to how digital special effects are used in Hollywood movies to how the Mars Rover was able to send photographs across millions of miles of space. The course starts by looking at how the human visual system works and then teaches you about the engineering, mathematics, and computer science that makes digital images work. You will learn the basic algorithms used for adjusting images, explore JPEG and MPEG standards for encoding and compressing video images, and go on to learn about image segmentation, noise removal and filtering. Finally, we will end with image processing techniques used in medicine. This course consists of 7 basic modules and 2 bonus (non-graded) modules. There are optional MATLAB exercises; learners will have access to MATLAB Online for the course duration. Each module is independent, so you can follow your interests.
6 - Degradation Function - Duration 11:40
Loading...
From the course by Duke University
Image and Video Processing: From Mars to Hollywood with a Stop at the Hospital
496 ratings
Duke University
496 ratings
From the lesson
Image restoration
The goal of this unit is to complement Unit 3 by adding prior information about the sources of degradation. Students will learn that if we know about the degradation process, we can do better. The objective of this unit is to complete the training with basic and powerful classical tools.
Meet the Instructors
Guillermo SapiroProfessor
Electrical and Computer Engineering
Hello and welcome back to our Image and Video Processing class.
We are going to continue with image restoration, and so far we have dealt a
lot with the noise. So let us now work with the filtering
component. So let us remember that this is the
complete model of degradation that we are talking about.
We are talking about filtering the image and then adding noise. And so far we have
considered that there was no filtering, but just noise.
In general, what we have here as we have seen before
is g(x,y). The observed image is equal to f(x,y)
convolution with h(x,y) plus the noise, which, as we discussed before, is a
random violate that we add noise to the image.
And we have seen how to try to estimate the noise.
Now let us discuss how to try to estimate this blurring function.
In general it's called a blurring function, this h.
This is much harder than estimating the noise.
But before we discuss that why is this important.
Let me just illustrate it as a very, very simple formula why this is important.
And for that lets assume for a second that we don't have any noise.
So basically there is no noise. We're going take the Fourier Transforms
of this and the Fourier is g of u, v equal f of u, v times.
H of UV. So for those that are familiar with ,
this is very clear. For those that are not, just bare with me
for a moment. In Fourier, the convolution becomes a
product. So basically we have transformed our
image into a different domain where the convolution became a product.
And now look what we have, we want to estimate the original image so we divide,
so we say okay, very good. F u, v is equal to the observation,
basically the degrade, the degraded image divided by the filter.
So if we manage to estimate the filter, all very easy.
We take the observed image, we do our Fourier transform, we do our Fourier
transform of the estimated filter. We divide one by the other.
We got the original image, actually it's Fourier transform, we invert it, we get
the image back. This is what's called inverse filtering.
Has some fundamental problems, in particular because there is no guarantee,
that this H, U, V is always different than zero.
Actually for realistic scenario's, it has either some zeros, or is very close to
zero for some values of U or V, but the general idea is very, very good that we
basically if we have knowledge about this degradation function, then we can try to
invert it. And that's very cool.
I mean it's that, we want to try to estimate it.
As I say estimating it very, very hard, much harder than estimating the noise.
So let's see some examples where we can actually estimate it and also some
examples of what type of h functions do we have in reality.
For example, one thing that happens is, as I mentioned earlier this week, is
what's called blurring. Blurring means that I basically observe
an image that is the result of my original image.
Convolve, let's say, with the Gaussian function with zero mean uncertain
variance, or standard deviation. And we already saw the general function
of a Gaussian. Basically, this exponential with a
quadratic exponential function. We saw it when we were discussing noise.
Basically, it blurs my image. Now if you know that there is a Gaussian
function that is affecting your observation.
So this would be my observation. This would be my G.
If you know that G is the result of F. With a Gaussian filter.
And you also know that the F was actually a dot.
A delta function. Just a point, that in that case for
example, that very trivial case, you can actually estimate the blurring.
Actually, not very hard because a simple property of the convolution says that
basically if F is the delta function that is convolved, with G.
Then we get G, so basically this is actually the Gaussian. So a very simple
case that we can actually estimate our H function for the particular case,
that we have a Gaussian filtering. Now,
this might sound a bit ridiculous, as first.
I say, how do I know that there was a dot, that it was a delta function in my
image? But actually it is not so ridiculous,
because we can use this to calibrate a system.
If we know that our camera is blurring. Then I can put artificially an image that
has a delta function and then I estimate my blurring.
So this is very useful to, to calibration to do estimation of how the camera is
behaving. Very important.
Now I'm going to tell you something else that will further enhance the knowledge
and the concept why this is a hard function.
A hard function to estimate. If convolution G is equal to G
convolution F. So you might wonder is F my original
image? So was my original image a delta?
Am I blaring a Gaussian? Or was my original image a Gauissian?
Am I blaring a delta, which means no blaring.
So there is a lot of basic ambiguity here that we don't' know which one is what,
and that is yet another challenge that we have when we are trying to estimate the
filtering function. And normally what you do is you assume
models for the image and models for the filtering.
But this is just to illustrate how hard this problem is.
So this is an example where we can estimate.
Gaussian actually as I say is very important for blaring.
It's also a model that is very frequently used for turbulence.
So here is an image which is courtesy of, of, of NASA,
and it appears in the book that we are using so far.
And here we have basically the same image under different levels of turbulence,
from more turbulence to less turbulence. So at least you can observe here the
blurring effect and very often turbulence is modelled with a Gaussian filter.
So it's very, very important, we discussed that Gaussian noise
distributions are very important mathematically although they don't appear
really, in real physical scenarios very often.
Gaussian smoothing, Gaussian filtering actually is a very realistic model.
For a lot of things that are happening in physical cameras.
The other type of blurring that appears a lot is what's called motion blurring.
And the basic idea of motion blurring is that when you're taking a picture if the
object is moving. Pictures are taken by integrating the
light that comes into the sensor. So if the object is moving during your
integration time, basically you get a blurry picture.
And that's illustrated here before I write down the formulas.
So in this case, it's basically a simulation of instead of moving the
object, we move the camera. And the basic idea is, going back to our
GF functions, basically your G function that we are observing is actually the
integral from zero to T. That's a time of integration of your
camera, of your image at X. But the point X is moving, so I can call
that function of motion, let's say X of T,
to write down that basically it's moving, and it's also moving in the Y direction,
Y of T. And I'm integrating over T.
Okay? So basically, what I'm doing here is I'm
not taking one image. It's like I've taken multiple images and
I'm adding them, and what I see is the result.
This actually was simulated by taking this image here,
shifting it, let's say, to the right, just by a tiny bit. Shifting it again,
shifting it again, and then adding that. That's what this operation is doing, is
basically adding images on its own shifts.
And, this is how we're observation. Now, if we take, if we do the math and we
take the Fourier transform in both sides, and we use properties of the Fourier
transform. Then we can also write this in the form
that we have seen before, G equal H,
F. So, also this translation is actually a
filter, and I am not going to do that but you're more than welcome to do that as an
exercise, a very simple exercise, if you are familiar with basic concepts
in Fourier transform, you basically just as I say, take the Fourier in both sides,
and use properties of, of translation. And you get this filter, which of course
will depend on the velocities. Okay, so you could imagine that if the
object is not moving at all, there is no blurring.
If the object is moving a lot, there is much more blurring.
So, this filter will depend on these velocities.
And once again the question will be, just to show how hard it is, how do I estimate
from this? Both my image and my blurring effects.
Certainly not a trivial thing to do. Even if I knew that the result is a
result of motion blurring. The, the concept is that, from the sum of
multiple images, you have to estimate one.
Okay? So that's certainly not a trivial
concept. If I give you the number five, and I tell
you it's the sum of two numbers, there is no way for you to know which two
numbers I added unless I give you more information about that.
We're going to discuss this more when we talk in a few weeks more advanced topics
in image restoration. When we come back to this topic you'll
see different tools like sparsmalling and to expect that.
This is to illustrate that is a hard problem, and an extremely important
problem. Simple models like Gaussian de-blurring
or motion, or, Gaussian blurring, sorry. The blurring is the operation of, of
inverting, and motion blur are very, very important.
And we need to be able to invert them to go back to our original very, very sharp
image. What we're going to do next, is
basically, we're going to see one way of doing that,
and that's called Wiener filtering. There's one way of inverting a filter and
inverting noise, and we're going to see that in the next
video. See you soon.
Thank you.
Explore our Catalog
Join for free and get personalized recommendations, updates and offers.
Coursera provides universal access to the world’s best education,
partnering with top universities and organizations to offer courses online.
© 2019 Coursera Inc. All rights reserved.
