Welcome, fans, to 2002. We present another year of engineering, which we will again attempt to disguise as interesting reading material, with various amounts of success. While I don't often admit this in public, I have become a "Survivor" junkie with that show joining the X-files, the Phillies, and the Flyers on my personal must-see-TV list.
I got hooked into this Thursday night boob tube routine with the fascination of watching "team building" work as well on TV as it does in the corporate world. However, after a year like 2001, the word "survivor" has taken on a whole new meaning in a more real way on the world stage, national stage, and on the economic front. And, it is even more impossible to imagine what the view will be like just one year from today!
But, ready or not, 2002 is here - full of the promise of any New Year - so welcome to it, and bring it on!
Last month, we introduced the concept of the matched filter. There were some unfortunate incidents within that discussion where the introduction of that awful integral sign was unavoidable. I appreciate the coolheadedness that prevails in these circumstances. Not a single nasty e-gram was received the last time I dared to venture into calculus, and I am thus inspired and encouraged to dive deeper into the muck.
The gist of the December diatribe dealt with proving in an intuitively practical way (i.e. no greek symbols, except w for radian frequency) that a matched filter provides at its output a signal that, when sampled at the right time, results in the maximum signal-to-noise ratio (SNR) possible of the processed input signal. It does this by, not coincidentally, "matching" the input signal's shape in time or frequency.
The more intuitively appealing interpretation is to recognize that what is happening in the frequency domain is that the filter provides gain/attenuation across the spectrum in proportion to the signal's occupation of spectrum. Regions of spectrum that the signal distributes lots of power to are amplified a lot, while regions of the signal spectrum that are allocated little energy receive little gain.
Of course, there is the always-waiting-in-the-wings caveat with respect to the matched filter conclusions: these conclusions fall out of assuming that the noise is additive, white, and Gaussian. These three descriptors are three pieces of one of the most fundamental four-letter acronyms in communication systems engineering - AWGN. The N is left as an exercise for the student. I would whisper you a hint, but you might not hear me with all the 'noise' in the room.
All theoretical textbook bit-error-rate (BER) curves are shown as a function of Eb/No or SNR, where the noise part is always assumed AWGN. Curves that include other effects explicitly display what the impairments being considered are, against a baseline of the ideal AWGN channel. The A, W, and G are, for some unbeknownst to me reason - straightforward descriptors:
A = additive, meaning the signal enters the channel in which the noise can be modeled as an addition process; signal s goes into a channel, and signal y = s + n (n = noise) goes into the detector
W = white, meaning flat noise spectrum over a wide range of frequencies, or the range that matters; the term comes from white light, which includes all colors; white noise = all frequencies
G = Gaussian; this describes the amplitude behavior of the noise samples, and the statistical likelihood of them taking one amplitude level versus another. The concept of BER is statistical in nature - how many transmissions are likely to be in error over a particular number of total transmissions. A statistical result like BER requires a statistical description of the relevant variables. This noise distribution is the classic "bell curve" of randomness centered about some mean or average, where the spread of the curve describes the noise average.
The Color of Noise
Before a very engaging discussion ahead planned for the Gaussian piece of the puzzle, lets consider for just a moment the "white" piece. This is really the key item with respect to the matched filter assumptions used to derive the resulting filter structure.
The signal matching the filter idea becomes intuitively obvious because we didn't think about the fact that noise is to be passed through this filter too at the receiver side. We got away with not thinking - something I practice often - because having the spectral noise floor constant across the signal spectrum is, relatively speaking, the same as not having noise from the standpoint of the filter design. The filter can act on the signal as it likes, because the noise beneath it has no distinguishing characteristics that would alter the thinking of matching signal spectrum to filter spectrum as a way of using the signal power most effectively.
Mathematically, what happens is that the noise spectrum reduces to a constant term when white. Upon applying the Schwartz inequality that proves the SNR maximization, the constant factor is of no consequence to the resulting integrations over pulse and filter shapes that are a function of spectrum.
Should the noise not be white - which, not surprisingly, goes by the name "colored" noise - the matched filter changes too. The shape must account now for noise non-uniformity of spectrum. Again - this is intuitive under the thought that it doesn't make sense to amplify signal spectrum regions that are strong if the noise also spikes up in that region - perhaps more so than the signal does. It may be more desirable to accentuate other regions of the spectrum of less signal power because the channel noise in this region is also significantly lower.
Finally, it is important to recognize that noise can be Gaussian distributed and not white, and that white noise does not have to be Gaussian distributed.
Gaussian 101
In an early, but no less brilliant, column ("In the Noise", March 1997, www.CSDmag.com), a description of the Gaussian noise process was given. Any random variable, like noise, is quantified for analytical purposes by a probability density function (PDF), which describes the statistical nature of the randomly varying amplitudes. The curve for the Gaussian PDF is described by what is a seemingly cumbersome expression (inhale, exhale), given by the equation (1):
Again, ugly as it may be, this is the familiar "bell curve," with a width related to m, and location of its peak related to m (refer to the BB column above for a catalog of PDF shapes, including Gaussian).
For the usual system analysis model of AWGN in communication systems, the average value, or mean, is zero. The variable s is called the variance, and, in this case (m = 0, zero mean), is defined as the average of the squared noise value. Or, more simply, the variance of the commonly considered zero-mean AWGN is simply the power of the noise.
Furthermore, it is common to see the variable No in an expression, such as the familiar bit or symbol error rate expressions that are functions of Eb/No or Es/No. The term Eb is energy-per-bit, while Es is energy-per-symbol. The energy per symbol is the average signal power multiplied by the symbol period, while the energy-per-bit is the energy-per-symbol divided by the number of bits in a symbol.
The No term is the white noise power density, found as the noise power, s, divided by the bandwidth over which it exists. So, the comparison of schemes for BER performance turns out to be a contest of just how little energy we can consume for each bit sent - Eb - for a given noise floor.
While it may sound odd to only have the noise density in the expression, this simply points out that an optimal receiver structure, including the matched filter, is implicit in the theoretical error rate expressions. If for some reason you choose to implement the receiver in some other fashion, then these aren't the expressions to use as the theoretical baseline for performance.
The PDF comes into play in deriving the Eb/No expressions as follows. First, note that the probability that x falls between y and z is the area under the curve between those two points (Equation 2).
The function Q(y) is used to aid in the solution, where (3):
N(0,1) is shorthand for normalized Gaussian random variable, with (m = 0, s = 1). For a Gaussian random variable x with zero mean and variance s, (3) instead becomes Prob(x > Y) = Q(Y/s).
The good news is that everyone has known for a long time that the integral in equation 2 cannot be solved, yet everyone still needs to find solutions to the problem. Thus, many moons ago, some math nuts, inspired by self-torture rivaling the idea of must-see Phillies TV, developed tables and tables of solutions for Q(y). These tables are readily found in textbooks.
Of course, today, there are some more high tech ways of solving the integral - I will leave this as your homework assignment for next month.
The Backside View
We have discussed how and why the matched filter turns out to be the best way to optimize the SNR of a received signal comprised of the desired signal plus AWGN. What has not been demonstrated is why optimize SNR?
While it may seem intuitive that the best SNR gives the best BER, it has not been mathematically demonstrated that this is the case. Let's try and tackle this topic. To do this, in a sense, we will have work backwards by first recognizing that what we want at the end of the day is the lowest BER. With that as a goal, we can rule out receiver architectures. Here's how.
Sticking to the same notation from above, consider a receiver input y = s + n, where s represents the transmitted signal, and n represents the AWGN. Further, let's describe the input as y = sm + n, in recognition that the signal, s, can take one of many different waveshapes or values. The term sm refers to the fact that each transmitted symbol being observed and decided on is one of these, signal m.
The problem is very straightforward to express mathematically via probability terminology. Qualitatively, the receiver observes y, and calculates the probability that the underlying transmission corrupted by noise is sm, for all possible m. The receiver knows what all of the sm's are beforehand, as it is a receiver designed for this signal set. It looks at all of the calculated probabilities for each sm, and chooses the highest one - the one that has the highest probability of being the one sent. Quantitatively, this operation is expressed as a receiver that performs (4):
This form is known as the maximum a-posteriori (MAP) criteria, because it evaluates the probability sm after reception of y. However, as simple as this is to express, it is not made up of readily available information - the PDF of the transmit signal, given that the noise-corrupted y is received. Fortunately, there is a way out of this using a formulation known as Bayes' Rule for mixed variables. It states that (5):
For our case, then, the calculation required in Equation can be replaced by (6):
First, note that the denominator, the PDF of the received signal + noise, and is common to all calculations. Expressed as p(y), it is not conditioned on any particular signal, and therefore is the composite PDF for the family of possible transmitted signals through the channel. Thus, it is expendable as far as having useful information for this comparison of probabilities.
This reduces (6) to (7):
For simplification and under the reasonable assumption that all signals are equally likely to be transmitted for any sm, the Prob [sm] can be removed as a constant for all calculations. This situation may not apply, in which case this term must be lugged around and makes its way into the decision criteria. But, when the signals are equally likely to be sent, equation (7) becomes (8):
Recalling that y = s + n, the statistical part of y is tied up with the statistics of the noise, which we already know and are described in Equation 1. The deterministic (i.e. known) sm's, merely add some bias to the statistics, which in PDF speak is the same as effecting the mean, or average value. We know that the average value of the noise is zero. When a known signal, sm, is added to this noise, the first thing that is important to recognize is that the received signal y still has Gaussian behavior, since the noise is Gaussian and the signal is fixed, since we are considering p(y | sm). That is, the expression is conditioned on a known signal sm.
The new mean is simply the sum of the means. The mean of a deterministic variable, sm, is just sm. Also, the variance doesn't change because the signal is deterministic. In other words, for a given fixed signal, there is no additional uncertainty added. The statistics of y are thus Gaussian with a new mean, and can be written (9):
So, for a received signal y, the expression in equation 9 is calculated, and the maximum is chosen. Significantly, the function expressed in equation 9 is nonnegative for any sm. Additionally, the term 1/--(2ps) is constant for all sm, and therefore can also be dropped in a comparison. Thus, equation 9 is mathematically equivalent to maximizing their logarithms, which simplifies the expression to (10):
The constant term in the dominator can now be removed since it also appears in calculations for every sm. Furthermore, the maximization of a negative quantity is a minimization of the negative of the quantity. Doing these steps, and multiplying out the quadratic, we have (11):
This has been a lot of steps, but now we are really onto something. The term |y|? - the magnitude of the received signal vector - is common to every comparison and can be dropped. Let's further assume that all of the signal choices have the same energy, so that |sm|? is the same for every received possibility. This is not an unreasonable case, but plenty of cases also do not adhere to this simplification. However, the only impact is to offset the result by a bias term in the receiver comparison, and this is not material to the important conclusion. Doing both of these leaves us with (12):
Again, we have a negative sign, which can be reversed if the criteria is also reversed. The factor of two also is not important, although it is relevant without the equivalent energy assumption. The result is now (13):
This may not look like much at first but...ta-da! The vector dot product can be shown to represent a correlation operation when the vectors are represented as components projected onto the proper basis functions (another month's topic!). The correlation operation, as we have discussed in prior art in this column, represents an operation equivalent in result to the SNR - maximizing matched filter. In fact, last month, we pointed out that equivalency by showing how the correlator fits as an optimum filter, and then extended the idea to the matched filter.
That ends this month's adventure, pushing us further towards the deep end of the pool. Next month, we will move onto the high dive, dropping our simplifying one-symbol-at-a-time detection concept, and investigate sequence detection. In the meantime, practice your backstroke.
About the Author
Rob Howald is the director of systems engineering in the tranxission network systems group at Motorola's Broadband Communications sector in Horsham, PA. He has a B.S.E.E. and an M.S.E.E. from Villanova University and a Ph.D. from Drexel University. Rob can be reached at rhowald@gi.com.
---
Building Blocks Online
For past Building Blocks columns, visit www.commsdesign.com/main/arcmag.htm
http://www.CommsDesign.com
© 2002 CMP Media LLC.
1/1/02, Issue # 801, page 10.
