An Introduction to Statistical Signal Processing

Suppose that you know that SY (f) can be factored into the form |G(f)|2, where G(f) is a stable causal filter with a stable causal inverse. What is the best choice of H(f) in the sense of maximizing the signal to noise ratio? What is the best causal H(f)?

18.Show that equation (5.2) converges in mean square if the filter is stable and the input process has finitely bounded mean and variance. Show that convergence with probability one is achieved if the convergence of equation (A.30) is fast enough for the pulse response.

19.Show that the sum of equation (5.7) converges for the two-weakly stationary case if the filter is stable and the input process has finitely bounded variance.

20.Provide a formal argument for the integration counterpart of equation

(5.51); that is, if {X(t)} is a stationary two-sided construction time

t

random process and Y (t) = X(s) dx, then, subject to suitable

−∞

technical conditions, SY (f) = SX(f)/f2.

21.Prove that equation (5.58) holds under the conditions given.

22.Suppose that {YN } is as in example [5.1] and that Wn = Yn + Un,

where Un is a zero-mean white noise process with second moment E(U2) = N0/2. Solve the Wiener-Hopf equation to obtain a LLSE of Yn+m given {Wi; i ≤ n} for m > n. Evaluate the resulting mean squared error.

23.Prove the claim that if {Xn} and {Yn} are described by equations

(5.59) and (5.60) and if Xn is a LLSE estimate of Xn given Y0, Y1, . . . , Yn−1,

24.Prove the claim that the innovations sequence {νn} of example [5.7] is uncorrelated and has zero mean. (Fill in the details of the arguments used in the text.)

25. Let {YN } be as in example [5.2]. Find the LLSE for Yn+m given {Y0, Y1, . . . , Yn} for an arbitrary positive integer m. Evaluate the mean square error. Repeat for the process of example [5.3] (the same process with r = 1).

26.Specialize the recursive estimator formulas of equations (5.67) through (5.70) to the case where {Xn} is the {Yn} process of example [5.2], where Hn is a constant, say a, and where Ψn − N0/2, all n. Describe the behavior of the estimator as n → ∞.

27.Find an expression for the mean square error in example [5.4]. Specialize to infinite smoothing.

28.In the section on linear estimation we assumed that all processes had zero-mean functions. In this problem we remove this assumption. Let {Xn} and {Yn} be random processes with mean functions {mX(n)} and {mY (n)}, respectively. We estimate Xn by adding a constant to equation (5.52); i.e.,

2

(a) Show that the minimum mean square estimate of Xn is Xn = mX(n) if no observations are used.

(b) Modify and prove theorem 5.1 to allow for the nonzero means.

29. Suppose that {Xn} and {Zn} are zero mean, mutually independent, iid, two-sided Gaussian random processes with correlations

RX(k) = σx2δk ; RZ(k) = σz2δk ;

These processes are used to construct new processes as follows:

Find the covariance and power spectral densities of {Un} and {Wn}. Find E[(Xn − Wn)2].

30.Suppose that {Zn} and {Wn} are two mutually independent twosided zero mean iid Gaussian processes with variances σZ2 and σW2 , respectively. Zn is put into a linear time-invariant filter to form an output process {Xn} defined by

Xn = Zn − rZn−1,

where 0 < r < 1. (Such a filter is sometimes called a preemphasis filter in speech processing.) This process is then used to form a new process

Yn = Xn + Wn,

which can be viewed as a noisy version of the preemphasized Zn process. Lastly, the Yn process is put through a “deemphasis filter” to form an output process Un defined by

Un = rUn−1 + Yn.

(a)Find the autocorrelation RZ and the power spectral density SZ. Recall that for a weakly stationary discrete time process with zero mean RZ(k) = E(ZnZn+k) and

∞

SZ(f) = RZ(k)e−j2πfk,

k=−∞

the discrete time Fourier transform of RZ.

(b)Find the autocorrelation RX and the power spectral density SX.

(c)Find the autocorrelation RY and the power spectral density SY .

(d)Find the overall mean squared error E[(Un − Zn)2].

31.Suppose that {Xn; n Z} is a discrete time iid Gaussian random processes with 0 mean and variance σX2 = E[X02]. We consider this an input signal to a signal processing system. Suppose also that

{Wn; n Z} is a discrete time iid Gaussian random processes with 0 mean and variance σW2 and that the two processes are mutually independent. Wn is considered to be noise. Suppose that Xn is put into a linear filter with unit pulse response h, where

1 k = 0

to form an output U = X h, the convolution of the input signal and the unit pulse response. The final output signal is then formed by adding the noise to the filtered input signal, Yn = Un + Wn.

(a)Find the mean, power spectral density, and marginal pdf for Un.

(b)Find the mean, covariance, and power spectral density for Yn.

(c)Find E[YnXn].

(d)Does the mean ergodic theorem hold for {Yn}?

32.Suppose that {X(t); t R} is a weakly stationary continuous time Gaussian random processes with 0 mean and autocorrelation function

RX(τ) = E[X(t)X(t + τ)] = σX2 e−|τ|.

(a)Define the random process {Y (t); t R} by

t

Y (t) = X(α) dα,

t−T

where T > 0 is a fixed parameter. (This is a short term integrator.) Find the mean and power spectral density of {Y (t)}.

(d)Find the marginal pmf pYn .

(e)Find the probability Pr(Xn ≥ 2Xn−1) .

(f)Find the conditional expectations E[Yn|Zn] and E[Xn|Yn].

(g)Find the probability Pr(Yn ≥ 2Yn−1) .

34.Suppose that {Xn} is an iid random process with marginal pdf

Let N be a fixed positive integer.

(a)What is the probability that at least one of the samples X0, . . . , XN−1 exceeds a fixed positive value γ?

(b)What is the probability that all of the samples X0, . . . , XN−1 exceed a fixed positive value γ?

(c)Define a new process Un = ZXn, where Z is a binary random variable with the marginal pmf of equation (5.72) and and the Z is independent of all the Xn. Find the mean Un and covariance KU of Un.

Is Un weakly stationary? Is it iid?

(d)Does the sample average

converge in probability? If yes, to what?

(e)Find a simple nontrivial numerical upper bound to the probability

Pr(|Un − U| > 10σU ),

where σU2 is the variance of U0.

35.Suppose that {Xn} is a weakly stationary random process with zero mean and autocorrelation RX(k) = σ2α|k| for all integer k, here |α| <

1.A new random process {Yn} is defined by the relation Yn = Xn +

βXn−1.

(a)Find the autocorrelation function RY (k) and the average power E[Yk2].

(b)For what value of β is {Yn} a white noise process? I.e., the value of β for which SY (f) is a constant? This is an example of a whitening filter.

(c)Suppose that β is chosen as in the previous part so that Yn is white. (You do not need the actual value of β for this part, you can leave things in terms of β if you did not do the previous part.) Assume also that {Xn} is a Gaussian random process. Find the variance and the pdf of the random variable

1 N−1

SN = N i=0 Yi,

where N is a fixed positive integer.

36.Suppose that {Xn; n Z} is a Bernoulli random process with parameter p, i.e., it is an iid binary process with pX(1) = 1−pX(0) = p. Suppose that Z is a binary random variable with the pmf of equation

(5.72) and that Z and the Xn are independent of each other. Define for integers n > k ≥ 0 the random variables

Define a one-sided random process {Yn; n = 0, 1, . . . } as follows:

Z n=0

Yn = Yn−1(−1)Xn n = 1, 2, . . .

Note that for any n > k ≥ 0,

(a)Find the mean mY = E[Yn]. Show that pYn (1) can be expressed as a very simple function of mY and use this fact to evaluate pYn (y) for any nonnegative integer n.

(b)Find the mean, variance, and characteristic function of Wk,n.

(c)If you fix a positive integer k, do the random variables

Wk,n

n − k

converge in mean square as n → ∞? If so, to what?

(d)Write an expression for the conditional pmf pYn|Yk (l|m) for n > k ≥ 0 in terms of the the random variable Wk,n. Evaluate this probability.

Hint: Half credit for this part will be given if you get the general expression, i.e., a sum with correct limits and summand, correct. The actual evaluation is a bit tricky, so do not waste time on it if you do not see the trick.

(e)Find the covariance function KY (k, n) of {Yn}.

Hint: One way (not the only way) to do this part is to consider the case n > k ≥ 0, use equation (5.74) and and the fact

and try to make your formula look like the characteristic function for Wk,n.

37.(Problem courtesy of the ECE Department of the Technion.) Consider a process {Yt; t } that can take on only the values {−1, +1} and suppose that

pYt (+1) = pYt (−1) = 0.5 for all t. Suppose also that for τ > 0

(a)Find the autocorrelation function RY of the process {Yt; t }.

(b)Find the power spectral density SY (f).

38.(Problem courtesy of the ECE Department of the Technion.) A known deterministic signal {s(t); t } is transmitted over a noisy channel and the received signal is {X(t); t }, where X(t) = As(t) + W (t), where {W (t); t } is a Gaussian white noise process with power spectral density SW (f) = N0/2; f and A is a random variable independent of W (t) for all t. The receiver, which

is assumed to know the transmitted signal, computes the statistic

YT = 0T X(t) dt.

(a)Find the conditional pdf fYT |A(y|a).

(b)Assuming that A is N(0, σA2 ), find the MMSE estimate of A given yT .

(c)Find the MMSE resulting in the previous part.

Chapter 6

A Menagerie of Processes

The basic tools for describing and analyzing random processes have all been developed in the proceeding chapters along with a variety of examples of random processes with and without memory. The goal of this chapter is to use these tools to describe a menagerie of useful random processes, usually by taking a simple random process and applying some form of signal processing such as linear filtering in order to produce a more complicated random process. In chapter 5 the e ect of linear filtering on second order moments was considered, but in this chapter we look in more detail at the resulting output process and we consider other forms of signal processing as well. In the course of the development a few new tools and several variations on old tools for deriving distributions are introduced. Much of this chapter can be considered as practice of the methods developed in the previous chapters, with names often being given to the specific examples developed. In fact several processes with memory have been encountered previously: the Binomial counting process and the discrete time Wiener process, in particular. The goal now is to extend the techniques used in these special cases to more general situations and to introduce a wider variety of processes.

The development of examples begins with a continuation of the study of the output processes of linear systems with random process inputs. The goal is to develop the detailed structure of such processes and of other processes with similar behavior that cannot be described by a liner system model. In chapter 5, we confined interest to second-order properties of the output random process, properties that can be found under quite general assumptions on the input process and filter. In order to get more detailed probabilistic descriptions of the output process, we next further restrict the input process for the discrete time case to be an iid random process and

343

study the resulting output process and the continuous time analog to such a process. By restricting the structure of the output process in this manner, we shall see that in some cases we can find complete descriptions of the process and not just the first and second moments. The random processes obtained in this way provide many important and useful models that are frequently encountered in the signal processing literature, including moving-average, autoregressive, autoregressive moving-average (ARMA), independent increment, counting, random walk, Markov, Wiener, Poisson, and Gaussian processes. Similar techniques are used for the development of a variety of random processes with markedly di erent behavior, the key tools being characteristic functions and conditional probability distributions. This chapter contains extensive practice in derived distributions and in specifying random processes.

6.1Discrete Time Linear Models

Many complicated random processes are well modeled as a linear operation on a simple process. For example, a complicated process with memory might be constructed by passing a simple iid process through a linear filter. In this section we define some general linear models that will be explored in some detail in the rest of the chapter.

Recall that if we have a random process {Xn; n T } as input to a linear system described by a convolution, then as in equation (5.2) there is a pulse response hk such that the output process {Yn} is given by

k: n−k T

A linear filter with such a description — that is, one that can be defined as a convolution — is sometimes called a moving-average filter since the output is a weighted running average of the inputs. If only a finite number of the hk are not zero, then the filter is called a finite-order moving-average filter (or an FIR filter, for “finite impulse response,” in contrast to an IIR or “infinite impulse response” filter). The order of the filter is equal to the maximum minus the minimum value of k for which the hk are nonzero. For example, if Yn = Xn + Xn−1, we have a first-order moving-average filter. Although some authors reserve the term moving-average filter for a finite-order filter, we will use the broader definition we have given. A block diagram for such a filter is given in Figure 6.1.

Several other names are used to describe finite-order moving-average filters. Since the output is determined by the inputs without any feedback from past or future outputs, the filter is sometimes called a feedforward or