An Introduction to Statistical Signal Processing

X0, . . . , Xn, and hence also with *n. Rearranging terms for later use, we have that

E(*2n+1) = Φ2nE(*2n) − 2κnHnΦnE(*2n) − +κ2n(Hn2E(*2n) + Ψn) + Γn , (5.66)

Since we know that E(*20) = E(X02), if we knew the κn we could recursively evaluate the expected squared errors from the formula and the given problem parameters. We now complete the system design by developing a formula for the κn. This is most easily done by using the orthogonality relation and (5.65):

0 = E(*n+1Yn) = (Φn − κnHn)E(*nYn) + E(UnYn) − κnE(VnYn) .

Consider the terms on the right. Proceeding from left to right, the first

where we have used the fact that *n is orthogonal to Vn, to Y0, . . . , Yn−1 and

2

hence to Xn. The second term is zero by the assumptions of the problem. The third term requires the evaluation

E(VnYn) = E(Vn(HnXn + Vn)) = E(Vn2) .

Thus we have that

0 = (Φn − κnHn)HnE(*2n) − κnE(Vn2)

or

Thus for each n we can solve the recursion for E(*2n) and for the required κn to form the next estimate.

We can now combine all of the foregoing mess to produce the final answer. A recursive estimator for the given model is

3.Let {X(t)} be a stationary continuous time random process with zero mean and autocorrelation function RX(τ). The process X(t) is put into a linear time-invariant stable filter with impulse response h(t) to form a random process Y (t). A random process U(t) is then defined as U(t) = Y (t)X(t − T ), where T is a fixed delay. Find EU(t) in terms of RX, h, and T . Simplify your answer for the case where SX(f) = N0/2, all f.

4.Find the output power spectral densities in problems 5.1 and 5.2.

A discrete-time random process {Xn; n Z} is iid and Gaussian, with mean 0 and variance 1. It is the input process for a linear time invariant (LTI) causal filter with Kronecker delta responses h defined by

This filter (an FIR filter) is often referred to as a comb filter. A third process {Wn} is defined by

Wn = Yn − Yn−1 .

(a)What are the mean and the power spectral density of the process

{Yn}?

(b)Find the characteristic function MYn (ju) and the marginal pdf fYn (y).

(c)Find the Kronecker delta response g of an LTI filter for which

If so, to what?

5.Let {Xn} be a random process, where Xi is independent of Xj for i = j. Each random variable Xn in the process is uniform on the region

[−1/2n, +1/2n]. That is, fXn (xn) = n, when xn [−1/2n, +1/2n], and 0 otherwise.

Define {Yn} by

0 n = 0, 1

Yn = nXn − Yn−2 n = 2, 3, 4, . . .

(a)What is the expected value of Xn?

(b)What is the variance of Xn?

(c)What is the covariance function, KX(i, j)?

n

j=1 Xj. What is the expected value of Sn?

(e) Does {Xn} have a WLLN, i.e., does the sample mean n−1 n−1 Xn

k=0

converge in probability to the mean E[Xn]? If so, to what value does the sample mean converge? If it has no WLLN, explain why not. Make sure to justify your answer based on the definitions of WLLN and convergence in probability.

(f)Find EYn.

(g)Find RY (i, j).

(h)Find the cdf of Y4 + Y6.

6.Let {X(t); t } be a stationary continuous time Gaussian random process with zero mean and power spectral density function

Let {Z(t); t } be a stationary continuous time Gaussian random process with zero mean and power spectral density function

where we assume that B >> W , γ > N0, and that the two processes are mutually independent. We consider X(t) to be the “signal” and Z(t) to be the “noise.” The receiver observes the process {Y (t)}, where

Y (t) = X(t) + Z(t).

(a) Find and sketch the power spectral density of {Y (t)}.

(b)Find the conditional pdf fY (t)|X(t)(y|x), the marginal pdf fY (t)(y), and the conditional pdf fX(t)|Y (t)(x|y).

ˆ

(c) (4 points) Find the minimum mean squared estimate X(t) given the single observation Y (t) and compute the resulting mean squared error

(d)Suppose that you are allowed to use the entire observed signal {Y (t)} to estimate X(t) at a specific time and you can do this by linearly filtering the observed process. Suppose in particular that you pass the observed process {Y (t)} through a linear filter

˜

with output X(t), an estimate of X(t). (This filter is not causal, but all the results we derived for second order input/output relations hold for noncausal filters as well and can be used here.)

expressed as X(t) plus the output of the filter H when the input is Z(t).

7.Let {Xn} be an iid Gaussian random process with zero mean and variance RX(0) = σ2. Let {Un} be an iid binary random process, independent of the X process, with Pr(Un = 1) = Pr(Un = −1) = 1/2. (All processes are assumed to be two-sided in this problem.) Define the random process Zn = XnUn, , Yn = Un+Xn, , and Wn = U0 +Xn, all n. Find the mean, covariance, and power spectral density of each of these processes. Find the cross-covariance functions between the processes.

8.Let {Un}, {Xn}, and {Yn} be the same as in problem 5.7. The process {Yn} can be viewed as a binary signal corrupted by additive Gaussian noise. One possible method of trying to remove the noise at a receiver

2

is to quantize the received Yn to form an estimate Un = q(Yn) of the original binary sample, where

q(r) =

+1 if r ≥ 0 −1 if r < 0 .

9.Cascade filters. Let {gk} and {ak} be the pulse responses of two discrete time causal linear filters (gk = ak =) for k < 0) and let G(f) and A(f) be the corresponding transfer functions, e.g.,

∞

G(f) = gke−j2πkf .

k=0

Assume that g0 = a0 = 1. Let {Zn} be a weakly stationary uncorrelated random process with variance σ2 and zero mean. Consider the cascade of two filters formed by first putting an input Zn into the filter g to form the process Xn, which is in turn put into the filter a to form the output process Yn.

(a) Let {dk} denote the pulse response of the overall cascade filter,

that is,

∞

Yn = dkZn−1 .

k=0

Find an expression for dk in terms of {gk} and {ak}. As a check on your answer you should have d0 = g0a0 = 1.)

(b)Let D(f) be the transfer function of the cascade filter. Find D(f) in terms of G(f) and A(f).

(c)Find the power spectral density SY (f) in terms of σ2, G, and A.

(d)Prove that

1/2& '

|D(f)|2 df = 1 + |1 − D(f)|2 df ≥ 1 .

−1/2 −1/2

10.One-step prediction. This problem develops a basic result of estimation theory. No prior knowledge of estimation theory is required. Results from problem 5.9 may be quoted without proof (even if you

did not complete it). Let be as in problem 5.9; that is, {Xn} is a discrete time zero-mean random process with power spectral density σ2|G(f)|2, where G(f) is the transfer function of a causal filter with

k=1

Suppose that the linear filter tries to estimate the value of Xn based on the values of Xi for all i < n by choosing the pulse response {hk} optimally. That is, the filter estimates the next sample based on the present value and the entire past. Such a filter is called a one-step predictor. Define the error process {*n} by

(a)Find expressions for the power spectral density S (f) in terms of SX(f) and H(f). Use this result to evaluate E*2n.

(b)Evaluate S (f) and E(*2n) for the case where 1−H(f) = 1/G(f).

(c)Use part (d) of problem 5.9 to show that the prediction filter H(f) of (b) in this problem yields the smallest possible value of E(*2n) for any prediction filter. You have just developed the optimal one-step prediction filter for the case of a process that can be modeled as a weakly stationary uncorrelated sequence passed through a linear filter. As discussed in the text, most discrete time random processes can be modeled in such a fashion, at least through second-order properties.

(d)Spectral factorization. Suppose that {Xn} has a power spectral density SX(f) that satisfies

1/2

ln SX(f) df < ∞ .

−1/2

Expand ln SX(f) in a Fourier series and write the expression for exp (ln SX(f)) in terms of the series to find G(f). Find the pulse response of the optimum prediction filter in terms of your result. Find the mean square error. (Hint: You will need to know what evenness of SX(f) implies for the coe cients in the requested series and what the Taylor series of an exponential is.)

11.Binary filters. All of the linear filters considered so far were linear in the sense of real arithmetic. It is sometimes useful to consider filters

that are linear in other algebraic systems, e.g., in binary or modulo 2 arithmetic as defined in (3.65-3.66). Such systems are more appropriate, for example, when considering communications systems involving only binary arithmetic, such as binary codes for noise immunity on digital communication links. A binary first-order autoregressive filter with input process {Xn} and output process {Yn} is defined by the di erence equation

Yn = Yn−1 Xn , all n .

Assume that the {Xn} is a Bernoulli process with parameter p. In this case the process {Yn} is called a binary first-order autoregressive source.

(a)Show that for nonnegative integers k, the autocorrelation function of the process {Yn} satisfies

(b)Use the result of (a) to evaluate RY and KY . Hint: This is most easily done using a trick. Define the random variable

k

Wk = Xi .

i=1

Wk is a binomial random variable. Use this fact and the binomial theorem to show that

Pr(Wk is odd) − Pr(Wk is even) = −(1 − 2p)k .

Alternatively, find a linear recursion relation for pk = P r(Wk is odd) using conditional probability (i.e., find a formula giving pk in terms of pk−1) and then solve for pk.

(c)Find the power spectral density of the process {Yn}.

12.Let {Xn} be a Bernoulli random process with parameter p and let denote mod 2 addition as defined in problem 5.11. Define the firstorder binary moving average process {Wn} by the di erence equation

Wn = Xn Xn−1

This is a mod 2 convolution and an example of what is called a convolutional code in communication and information theory. Find pWn (w) and RW (k, j). Find the power spectral density of the process {Wn}.

13.Let {X(t)} be a continuous time zero-mean Gaussian random process with spectral density SX(f) = N0/2, all f. Let H(f) and G(f) be the transfer functions of two linear time-invariant filters with impulse responses h(t) and g(t), respectively. The process {X(t)} is passed through the filter h(t) to obtain a process {Y (t)} and is also passed through the filter g(t) to obtain a process {V (t)}; that is,

∞

Y (t) = h(τ)X(t − τ)dτ,

0

∞

V (t) = g(τ)X(t − τ)dτ .

0

(a)Find the cross-correlation function RY,V (t, s) = E(YtVs).

(b)Under what assumptions on H and G are Yt and Vt independent random variables?

14.Let {X(t)} and {Y (t)} be two continuous time zero-mean stationary Gaussian processes with a common autocorrelation function R(τ) and common power spectral densities S(f). Assume that X(t) and Y (t)

are independent for all t, s. Assume also that E[X(t)Y (s)] = 0 all t, s and that σ2 = R(0). For a fixed frequency f0, define the random process

W (t) = X(t) cos(2πf0t) + Y (t) sin(2πf0t) .

Find the mean E(W (t)) and autocorrelation RW (t, s). Is {W (t)} weakly stationary?

15.Say that we are given an iid binary random process {Xn} with alphabet ±1, each having probability 1/2. We form a continuous time random process {X(t)} by assigning

X(t) = Xn ; t [(n − 1), nT ) ,

for a fixed time T . This process can also be described as follows: Let p(t) be a pulse that is 1 for t [0, T ) and 0 elsewhere. Define

X(t) = Xkp(t − kT ) .

k

This is an example of pulse amplitude modulation (PAM). If the process X(t) is then used to phase-modulate a carrier, the resulting process is called a phase-shift-keyed modulation of the carrier by the

process {X(t)} PSK. PSK is a popular technique for digital communications. Define the PSK process

U(t) = a0 cos(2πf0t + δX(t)) .

Observe that neither of these processes is stationary, but we can force them to be at least weakly stationary by the trick of inserting uniform random variables in appropriate places. Let Z be a random variable, uniformly distributed [0, T ] and independent of the original iid process. Define the random process

Y (t) = X(t + Z) .

Let Θ be a random variable uniformly distributed on [0, 1/f0] and independent of Z and of the original iid random process. Define the process

V (t) = U(t + Θ) .

Find the mean and autocorrelation functions of the processes Y(t) and V (t).

16.Let {X(t)} be a Gaussian random process with zero mean and autocorrelation function

RX(τ) = N20 e−|τ| .

Find the power spectral density of the process. Let Y (t) be the process formed by DSB-SC modulation of X(t). Letting Θ be uniformly distributed in equation (5.37), sketch the lower spectral density of the modulated process.

17.A continuous time two-sided weakly stationary Gaussian random process {S(t)} with zero mean and power spectral density SS(f) is put into a noisy communication channel. First, white Gaussian noise {W (t)} with power spectral density N0/2 is added, where the two random processes are assumed to be independent of one another, and then the sum S(t)+W (t) is passed through a linear filter with impulse response h(t) and transfer function H(f) to form a received process

{Y (t)}. Find an expression for the power spectral density SY (f). Find an expression for the expected square error E[(S(t) − Y (t))2] and the so-called signal-to-noise (SNR)

E[(S(t) − Y (t))2] .