The lectures of this course are based on the first 11 chapters of Prof. Raymond Yeung’s textbook entitled Information Theory and Network Coding (Springer 2008). This book and its predecessor, A First Course in Information Theory (Kluwer 2002, essentially the first edition of the 2008 book), have been adopted by over 60 universities around the world as either a textbook or reference text. At the completion of this course, the student should be able to: 1) Demonstrate knowledge and understanding of the fundamentals of information theory. 2) Appreciate the notion of fundamental limits in communication systems and more generally all systems. 3) Develop deeper understanding of communication systems. 4) Apply the concepts of information theory to various disciplines in information science.
Chapter 11 - Section 11.8
Loading...
From the course by The Chinese University of Hong Kong
Information Theory
6 ratings
Meet the Instructors
Prof. Raymond W. YeungChoh-Ming Li Professor of Information Engineering, and Co-Director, Institute of Network Coding
In this section, we discuss a more general
channel model called the band-limited coloured Gaussian Channel.
[BLANK_AUDIO]
The schematic diagram for this channel is the same
as the schematic diagram for the band-limited white Gaussian Channel.
The only difference is the noise process Z(t).
[BLANK_AUDIO]
Specifically, Z(t) is a zero mean additive coloured Gaussian noise,
meaning that the power spectral density is not necessarily a constant.
X prime of t and Z prime of
t, are filtered versions of X(t) and Z(t), respectively,
both band-limited to the interval [0,W].
[BLANK_AUDIO]
The output process, Y(t) is equal to X prime of t plus Z prime of t.
And Z prime of t is a band-limited coloured Gaussian noise, with the power
spectral density S_{Z prime} of f greater than and or equal to 0
for f from minus W to W and equal to zero otherwise.
As for the case of the white Gaussian channel, we regard X prime of t
as the channel input and Z prime of t as the additive noise process.
We also impose a power constraint P on the input process X prime of t.
[BLANK_AUDIO]
Recall that the capacity of the white Gaussian channel
band-limited to the interval [0,W], is equal to W times
log of 1 plus P over N_0 times W bits per unit time.
For the white Gaussian channel, band-limited to the interval [f_l,f_h],
where f_l is the lowest frequency and f_h is the highest frequency,
and f_l is a multiple of the bandwidth of the channel W equal to f_h minus f_l,
we can apply the bandpass version of the
sampling theorem to obtain the same capacity formula.
This model is called the bandpass white Gaussian Channel.
When f_l, the lowest frequency, is equal to 0, the
bandpass white Gaussian Channel reduces to the band-limited white Gaussian Channel.
[BLANK_AUDIO]
Here is the filter associated with the bandpass white Gaussian Channel.
Note that the lowest frequency f_l is a
multiple of W, the bandwidth of the channel.
And this is the schematic diagram for the bandpass white Gaussian Channel.
[BLANK_AUDIO]
Before we derive the capacity of the band-limited coloured Gaussian
Channel, let us first recall the assumptions of the channel model.
Z(t), the noise process, is a zero mean additive coloured Gaussian noise.
X prime of t and Z prime of t are filtered versions of X(t) and
Y(t) respectively, both band-limited to the interval [0,W].
[BLANK_AUDIO]
And the input power constraints on the input process X prime of t is equal to P.
This is an illustration of the power
spectral density of a band-limited coloured Gaussian Channel.
We now derive the capacity of such a channel.
First, divide the interval [0,W] into k subintervals.
The width of each subinterval is delta_k equal to W over k.
[BLANK_AUDIO]
Let the i-th subinterval be
[f_l^i,f_h^i] for i from one up to k.
As an approximation, assume that the
noise power over the i-th subinterval is a constant S_{Z,i}.
Then the channel consists of k sub-channels with
the i-th sub-channel being a bandpass white Gaussian
Channel occupying the frequency band [f_l^i,f_h^i].
Let P_i be the power allocated to the i-th sub-channel.
Then the capacity of the i-th sub-channel, is delta_k times
log of 1 plus P_i, divided by 2 times S_{Z,i}, times delta_k.
The workout of this expression is as follows.
For a white Gaussian Channel, which is band-limited to the integral [f_l, f_h],
where f_l is a multiple of the bandwidth W prime equals f_h minus f_l,
with noise level N_0 over 2, and power
constraint P, the capacity is W prime log
of one plus P divided by N_0 times W prime.
For the i-th sub-channel,
W prime is equal to delta_k,
P is equal to P_i,
and N_0 over 2 is equal to S_{Z,i}, or N_0 is equal to 2 times S_{Z,i}.
With these, we can obtain the expression in step five.
The noise process Z_i prime of t of the i-th sub-channel is obtained
by passing the noise process Z(t) through the corresponding
ideal bandpass filter band-limited to the interval [f_l^i,f_h^i].
It can be shown that the noise processes Z_i
prime of t, i from 1 up to k, are independent.
We will leave this as an exercise.
By sampling the sub-channels at a Nyquist rate 2 times delta_k,
where delta_k is the bandwidth of each sub-channel,
the k sub-channels can be regarded as a system of parallel Gaussian channels.
Thus the capacity of the channel is equal to the sum of the capacities
of the individual sub-channels when the
power allocation among the k sub-channels is optimal.
Let P_i star be the optimal power allocation for the i-th sub-channel.
[BLANK_AUDIO]
In step five, we obtained the capacity of the i-th sub-channel
when P_i is the power allocated to that sub-channel.
Then the capacity of the system of parallel Gaussian channels
is equal to summation i from 1 up to k, delta_k times log
of 1 plus P_i star, divided by 2 times S_{Z,i} times delta_k,
where the fraction inside the logarithm can be written
as P_i star over 2 times delta_k divided by S_{Z,i}.
And by proposition 11.23, the values of P_i star
over 2 times delta_k is equal to the positive part of nu
minus S_{Z,i}, where the water level nu can be obtained from the
constraint summation P_i star, i from one up to k, equals P.
[BLANK_AUDIO]
As k tends to infinity, where k is the number of subintervals, the capacity
of the system of parallel Gaussian channels given by this summation
tends to the integral from 0 to W log of 1 plus the
positive part of nu minus S_Z(f) divided by S_Z(f) df.
This can be written as one half times the same integral from minus W to W,
because S_Z(-f), is equal to S_Z(f).
That is the power spectral density is symmetrical about f equals 0.
[BLANK_AUDIO]
On the other hand, as k tends to infinity, summation i P_i star is equal
to summation i 2 times delta_k times the positive part of nu minus S_{Z,i}.
And this summation tends
to 2 times the integral
from 0 to W the positive
part of nu minus S_Z(f) df.
Again, this can be written as the integral from minus W
to W, the positive part of nu minus S_Z(f)df because the
power spectral density is symmetrical about f equals 0.
Therefore, the constraint summation i P_i star equals
P, tends to the constraint, the integral from minus
W to W, the positive part of nu minus S_Z(f)df equals P.
So the capacity of the band-limited coloured Gaussian Channel is given by
this formula, where the water level nu can be obtained by this constraint.
The optimal power allocation given in equation one
has a water-filling interpretation given in the next figure.
[BLANK_AUDIO]
Here, the water filling is for the range from minus W to W
with the total water volume equal to P.
And so the water volume on each side is equal to P over 2.
Coursera provides universal access to the world’s best education,
partnering with top universities and organizations to offer courses online.
© 2018 Coursera Inc. All rights reserved.
