CMPE 320 MCRC The Functions of A Random Variable & the Received Signal Question

MEMO Number CMPE320-S22-0102DATE: 1 February 2022
TO: CMPE320 Students
FROM: EFC LaBerge
SUBJECT: Functions of a Random Variable
1
INTRODUCTION
This project will explore the pdf of functions of a random variable, including the pdf of a mixture
random variable.
Warning #1: Thinking is required!
Warning #2: Follow directions!
Warning #3: It isn’t on the web, so don’t bother looking. You may, however, look up background
material, such as the definition of terms, etc. All of the terms are well-defined in your textbook
and the lectures.
This project involves concepts spread across the various lectures in Module 2, and uses elements
of the solution to Project 1. It is, however, perfectly acceptable (and desirable) to start early, to do
what you can and then go back and do more as the course content expands.
Remember, there are no exams in S22 CMPE320, so I’m looking for you to develop and explain
concepts we have developed in class.
This project involves analytical computation as well as simulation. Express your math well! If
necessary, you may hand-write your equations and insert them as pictures in your writeup.
You may not collaborate with any other CMPE320 student or students, nor consult with any other
humans other than Dr. LaBerge. You may ask all the questions you like in office hours, review
sessions, but the preferred method of clarification is via the Ask the Professor discussion forum on
Blackboard. Do your own work!
2
FUNCTIONS OF A RANDOM VARIABLE
The engineers at Universally Marvelous Broadcasting and Communications (UMBC) are designing
how to detect the amplitude or the power of a bipolar signal of known amplitude but random sign
(or phase) that is corrupted by Additive White Gaussian Noise (AWGN)1. Three methods have
been suggested:
1) When the signal is received, it is passed the signal through a perfect diode detector, and
only the positive values are used; or,
1
The model for a signal with AWGN is
r(t) = (± A) + n(t), where r(t) is the received signal, (± A) is the desired signal, and n(t) ~ N (0,σ 2 )
2) When the signal is received, the processor computes the amplitude by taking the absolute
value of measured signal plus the noise; or,
3) When the signal is received, the processor computes the amplitude squared by taking the
square of the measured signal plus the noise, thus producing an estimate of the power.
The engineers have determined that method 1 will cost $10 in production, but that method 2 will
cost $20 in production and method 3 will cost $40 in production. Any of the methods will produce
a result that meets the product requirements.
For all of the following questions, assume that the known amplitude is A = 2V , that the known
9
2
amplitude is equally likely (hint!) to be + A or − A , and the noise variance is σ = .
16
2.1
Model R, the received signal
In this section we will build the model shown in Figure 1, which is known as the Additive White
Gaussian Noise Model for Binary Amplitude Shift Keying.
Random Variable X
!!
”
Random Variable R
+
0.5 for + = += #0.5 for + = −0 otherwise
!) (;) = ! = +- ; +- !* ++! = −- ; −- !* −-
by Law of Total Probability
Random Variable N
!#
$
=
1
278 %
9 &$
! /%( !
Figure 1 The Additive Gaussian Noise Model for BASK
Define a random variable, R , to be the sum of two other random variables, X and N , where
⎪⎧ 0.5 x = A
f X (x) = ⎨
⎩⎪ 0.5 x = − A
f N (n) =
1
,
( 1)
− n2 /(2σ 2 )
e
2πσ 2
so that R = X + N . Assume that X and N are independent. We can use the Law of
Total Probability to write
(
)
(
)
f R (r) = f R|+ A (r | X = A) f X X = + A + f R|− A (r | X = − A) f X X = − A ,
Page: 2
S22 CMPE320
( 2)
Project 2 rev0 220201.docx saved 2/27/22 1:40:00 PM printed 3/1/22 4:32:00 PM
where f R|A (r | X = A) and f R|− A (r | X = − A) are Gaussian with means A and − A,
respectively, and both have the same variance as the noise.
Using ( 2), write out the analytical form of f R (r) as a function of r and A . Then perform
a large number of simulations to get sample values of R , using the values for A and σ 2
given above. Plot a scatterplot of the simulated values of R as a function of the index in
your array (i.e., x-axis goes from 1 to the number of trials, and explain how that plot
represents the signal model you have. Do the samples cluster around the values of + A
and − A ? (If not, there’s something wrong!)
Then, on a new figure, use techniques from Project 1 to plot the appropriately scaled
histogram representing f R (r) . Plot your analytical result for f R (r) on the same set of
axes as your appropriately scaled histogram. Discuss why the plot looks like it does.
You may use your random samples of R as the input in 2.2, 2.3, and 2.4, below.
There are two plots required for Section 2.1.
2.2
Method 1
2.2.1
Analytical PDF
Let S be the signal the output from the perfect diode detector that is the first method considered
by UMBC engineers, as shown in Figure 2. S is a function of the random variable R , and,
therefore is itself a random variable. Using the CDF method developed in class, analytically derive
the probability density function. That is, find the CDF, FS (s) = Pr ( S ≤ s ) and then differentiate
with respect to S to get the pdf f S (s) =
dFS
.
ds
Hint 1: A detailed example computation is given in the appendix.
Hint 2: −∞ < R < ∞ , so what range of S must be considered? Hint 3: The derivative has different forms for S < 0 and S > 0 .
The analytical result from this section will be used in 2.2.2, below.
Page: 3
S22 CMPE320
Project 2 rev0 220201.docx saved 2/27/22 1:40:00 PM printed 3/1/22 4:32:00 PM
Random Variable R
!! (#) = ! & +( # +( !” +(
+! & −( # −( !” −(
by Law of Total Probability
&
Perfect
Diode
Detector
*
!#$%
Random Variable S
*=+ & =,
0 for & < 0 & for & ≥ 0 !!" Figure 2 Perfect Diode Detector and Model for 2.2.1 2.2.2 Simulated PDF Using your simulated data from Section 2.1, simulate the action of Method 1. On a new figure, provide a scatterplot of S vs. R, with R on the x-axis. Now, use your sample values that you just created to create a histogram version of the probability density function f S (s) . Plot the histogram-based pdf, and then plot the analytical pdf you derived in 2.2.1 on the same set of axes. Provide a professional plot with labels, proper scale, grid, a title, etc. Using the appropriate functional expression of Method 1 as S = g(R) , compute E[S] and g(E[R]), that is the function evaluated at the expected value of the random variable R Save this value for use in 2.5. You may use the sample mean computed from your random samples in both cases. Extra credit will be given for analytical computation of the expected values with correct numerical evaluation. If this analytical computation is performed, the writeup should compare the results of the sample means and the analytical means. 2.3 Method 2 2.3.1 Analytical PDF Let S again be the output value, but this time use the absolute value detector that is the second method considered by UMBC engineers, as shown in Figure 3. S is a function of the random variable R , and, therefore is itself a random variable. Using the analytical method developed in class, derive the probability density function. Hint 1: A detailed example computation is given in the appendix. Hint 2: −∞ < R < ∞ , so what range of S must be considered? Hint 3: The derivative has different forms for S < 0 and S ≥ 0 . Hint 4: You may check your answer using the CDF method. The analytical result from this section will be used in 2.3.2, below. Page: 4 S22 CMPE320 Project 2 rev0 220201.docx saved 2/27/22 1:40:00 PM printed 3/1/22 4:32:00 PM Random Variable R !! (#) = ! & +( # +( !" +( +! & −( # −( !" −( & Absolute Value Detector * !#$% Random Variable S * = + & = |&| by Law of Total Probability !!" Figure 3 Absolute Value Detector and Model for 2.3.1 2.3.2 Simulated PDF Using your simulated data from Section 2.1, simulate the action of the perfect absolute value detector. On a new figure, provide a scatterplot of S vs. R, with R on the x-axis. Now use your sample values of S to create an appropriately scaled histogram version of the probability density function f S (s) . Plot the histogram-based pdf, and then plot the analytical pdf you derived in 2.3.1 on the same set of axes. Provide a professional plot. Expressing the appropriate functional expression of Method 2 as S = g(R) , compute E[S] = E[g(R)] and g(E[R]), that is, the expected value of the random variable S, and then the value of the function g( ) evaluated at the expected value of the random variable R Save these value for use in 2.5. You may use the sample mean computed from your random samples in both cases. Extra credit will be given for analytical computation of the expected values (we call this the population mean) with correct numerical evaluation. If this analytical computation is performed, the writeup should compare the results of the sample means and the analytical (population) means 2.4 Method 3 2.4.1 Analytical PDF Let S be the signal the output from the square law detector that is the third method considered by UMBC engineers, as shown in S is a function of the random variable R , and, therefore is itself a random variable. Using the CDF method developed in class, analytically derive the probability density function. That is, find the CDF, FS (s) = Pr ( S ≤ s ) as a function of R and then differentiate with respect to S to get the pdf f S (s) = dFS . ds Hint 1: A detailed example computation is given in the appendix. Hint 2: −∞ < R < ∞ , so what range of S must be considered? Hint 3: The derivative has different forms for S < 0 and S ≥ 0 . The analytical result from this section will be used in 2.2.2, below. Page: 5 S22 CMPE320 Project 2 rev0 220201.docx saved 2/27/22 1:40:00 PM printed 3/1/22 4:32:00 PM Random Variable R !! (#) = ! & +( # +( !" +( +! & −( # −( !" −( & Square Law Detector * Random Variable S !#$% * = + & = &# by Law of Total Probability !!" Figure 4 Square Law Detector and Model for 2.3.1 2.4.2 Simulated PDF Using your simulated data from Section 2.1, simulate the action of the perfect absolute value detector. On a new figure, provide a scatterplot of S vs. R, with R on the x-axis. Now use your sample values of S to create an appropriately scaled histogram version of the probability density function f S (s) . Plot the histogram-based pdf, and then plot the analytical pdf you derived in 2.4.1 on the same set of axes. Provide a professional plot. Expressing the appropriate functional expression of Method 3 as S = g(R) , compute E[S] = E[g(R)] and g(E[R]), that is, the expected value of the random variable S, and then the value of the function g( ) evaluated at the expected value of the random variable R Save these value for use in 2.5. You may use the sample mean computed from your random samples in both cases. Extra credit will be given for analytical computation of the expected values (we call this the population mean) with correct numerical evaluation. If this analytical computation is performed, the writeup should compare the results of the sample means and the analytical (population) means 2.5 Looking Ahead: Jensen’s Inequality For each of three methods, compare the expected value of the simulated output data ( E[S] ) with the evaluation of the detection function ,at the expected value of the input ( g ( E[R]) ) . Is there a consistent inequality relationship that extends across the three cases? That is, is one of these values consistently larger or smaller than the other across the differences in the detection method? Can you guess the general rule, which is known as Jensen’s Inequality. Note: Demonstrations by simulated examples are only examples, never proofs. We will prove Jensen’s Inequality in class (or at least I will!). 3 3.1 INSTRUCTIONS FOR PROJECT REPORT Report Format The project report shall be in the same form as this document, with an introduction, simulation and discussion section, and a "what I learned" section. Each section shall contain the content identified in Section 2, and described in more detail below subsection below. The report shall be in Times New Roman 11 point font. MATLAB pictures shall be pasted in-line in the report (this is a useful Page: 6 S22 CMPE320 Project 2 rev0 220201.docx saved 2/27/22 1:40:00 PM printed 3/1/22 4:32:00 PM skill to know!); shall be numbered consecutively; shall be appropriately titled; the axes shall be appropriately labeled; the curves shall be appropriately identified by an appropriate legend. I’ll provide suggestions on professional-looking plots on Blackboard. Please follow them. 3.2 Section 2 Content Section 2 of the report shall be titled "Simulation and Discussion" and shall contain the required derivations, the required simulation plots and a discussion of each plot. The discussion shall address any points identified in Section 2 and any other interesting observations that occur to you. Remember, I know this stuff: you don't. So take a look at the plots and tell me what you see and what it means to you. A large part of the grade is based on what you observe, so take your time! I’m not particularly interested in really general observations like “the curve goes up” or “the curve is flat.” I’m interested in observations that reflect or illustrate concepts we have talked about in class, or that relate to the relationship between the normalized histogram and the analytical curve, or things like that. The subsections of Section 2 of the report should match the subsections of Section 2 of this document. 3.3 Section 3 Content Section 3 of the report shall be titled "What I learned" and shall contain a summary of what information you observed, what insights you gained, etc. Section 3 shall also contain a subsection critiquing the project and suggesting improvements that I could institute for next spring. Finally, Section 3 shall contain an estimate of how much time you spent on the project, including reading, research, programming, writing, and final preparation. 3.4 Questions I will accept questions regarding the project via the Ask The Professor Forum on Blackboard, so that I can reply to the entire class, and so that no student has an advantage by clever questioning. I will not do the project for you. I will not be answering (or even acknowledging) individual emails, so don’t ask. I will stop responding to questions at 12Noon on the day before the project is due. If you don’t start early, don’t ask for clarification, or read the earlier clarifications. 3.5 Project Grading The project shall be graded in the following way: 75% of the project score shall depend on the technical, theoretical, and graphical presentations of the tasks set out in Section 2 of this document. 25% of the project score shall be based on an evaluation of the technical writing against the Rubric on Technical Writing, posted on Blackboard, including grammar, clarity, organization, etc. For the purpose of this document, you can assume that the intended audience consists of your CMPE320 classmates. I will be assisted in grading by the TA/grader staff for this course. I will personally grade at least two projects for every student, spread over the five total projects. Page: 7 S22 CMPE320 Project 2 rev0 220201.docx saved 2/27/22 1:40:00 PM printed 3/1/22 4:32:00 PM 3.6 Project Delivery The project shall be delivered by 11:59 PM on the date indicated in the Detailed Schedule spreadsheet. Delivery shall be by submission of a PDF file as a Blackboard assignment. This is an individual assignment. You should also publish and deliver your MATLAB files in a single PDF file in the same assignment. 3.7 Academic Integrity The academic integrity provisions you signed at the beginning of class are in effect. You may discuss the interpretation of the assignment and approaches to solve the various problems amongst yourselves. You MAY NOT share MATLAB (or other) code, plots, text, etc. Do your own work. I will be looking at MATLAB (or other code) files for similarities, so please do not even attempt to copy work. You may not ask for assistance from the TA/grader staff, although you may ask me for help with the various concepts. So, for example, if you don’t understand a Gaussian pdf you may ask for help understanding pdfs in general and Gaussian pdfs in particular. You may not ask for help completing the tasks assigned in this document. You should ask me for assistance via the Ask The Professor forum. You may ask for help on the concepts during my open office hours. Do your own work! Page: 8 S22 CMPE320 Project 2 rev0 220201.docx saved 2/27/22 1:40:00 PM printed 3/1/22 4:32:00 PM Appendix A :Liebniz Rule (Calculus II) Remember that the pdf f X (x) is related to the CDF FX (x) = Pr ⎡⎣ X ≤ x ⎤⎦ by f X (x) = dFX (x) dx (A1) x FX (x) = ∫ (A2) f X (u) du −∞ If you have the form of (A2), and you want to apply (A1), as will happen with the CDF method I requested, you need to apply Liebniz Rule from Calc II: For any integrable function of two variables, f (x,u), and two differentiable functions of x, a(x) and b(x) b( x ) b( x ) ⎤ db(x) d ⎡ da(x) d × f x,b(x) − × f (x,a(x) + ∫ f (x,u) du ⎢ ∫ f (x,u) du ⎥ = dx ⎢⎣ a( x ) dx dx dx ⎥⎦ a( x ) ( ) ( ) (A3) (A4) If the function f (x,u) = f (u), that is, if it is not a function of x , then the derivative in the third term is zero. Similarly if either a(x) or b(x) are constant (e.g. 0 or ± ∞ ), then the corresponding derivatives in the first and second terms are also zero. Appendix B: Detailed Example of CDF method of determining fY ( y) for a transformation Y = g( X ) Assume we have the same setup for the RV X as in Section 2.1, but the random variable N is zero mean Gaussian with known variance, σ 2 , f N (n) = 1 2πσ 2 2 e− n /2σ 2 (B1) Assume that the “detector” that we wish to evaluate works on the received input signal, r, ⎧⎪ D = g(R) = ⎨ ⎩⎪ r > 0 , where r is the numerical value of the R.V. R
r ≤0
R
0
(B2)
By the Law of Total Probability
f R (r) = f R|+ A (r | + A) f X (+ A) + f R|− A (r | − A) f X (− A)
If we know that the value of X = A , then f R|A (r | A) = f N (r − A) =
1
X = − A, then f R|A (r | − A) = f N (r − A) =
f R (r) =
=
1
2
2πσ
1
2 2πσ 2
(e
2
2
e−(r− A) /2σ ×
−(r− A)2 /2σ 2
2πσ
2
2
(B3)
1
2πσ 2
2
2
e−(r− A) /2σ , and if
2
e−(r+ A) /2σ . Substituting in (B3) ,
2
2
1
1
1
+
e−(r+ A) /2σ ×
2
2
2
2πσ
2
+ e−(r+ A) /2σ
2
)
(B4)
The CDF for the new random variable, D = g( R) = R , or, equivalently, R = g −1 (D) = D 2
⎧⎪
0
d ≤0
FD (d) = ⎨
⎪⎩ Pr ⎡⎣ D ≤ d ⎤⎦ d > 0
⎧
0
r ≤0
⎪
=⎨
2
⎡
⎤
⎪⎩ Pr ⎣ R ≤ d ⎦ r > 0
⎧⎪
0
d ≤0
=⎨
2
F
(d
)
d >0
⎩⎪ R
⎧
⎪⎪
=⎨
⎪
⎪⎩
d
∫
⎧
⎪⎪
== ⎨
1
d >0
⎪
2
⎪⎩ 2 2πσ
d ≤0
0
2
f R (r) dr
−∞
d ≤0
0
∫ (e
d2
−(r− A)2 /2σ 2
2
+ e−(r+ A) /2σ
2
−∞
) dr
d >0
(B5)
We don’t actually need to evaluate (B5), because our goal is the pdf, f D (d) =
that differentiation for d > 0 needs Liebniz Rule, and looks like this
dFD (d)
. Performing
dd
d2
⎤
d ⎡
1
−(r− A)2 /2σ 2
−(r+ A)2 /2σ 2
f D (d) =
e
+
e
dr
⎢
⎥
∫
dd ⎢ 2 2πσ 2 −∞
⎥⎦
⎣
⎡⎛ dd 2 ⎞
⎤
−( d 2 − A)2 /2σ 2
−( d 2 + A)2 /2σ 2
×
e
+
e
⎢⎜
⎥
⎟
⎢⎝ dd ⎠
⎥
⎢ ⎛
⎥
1
d(−∞) ⎞
−( d 2 − A)2 /2σ 2
−( d 2 + A)2 /2σ 2
⎢
⎥ for d > 0
=
−⎜
+e
⎟× e
⎥
2 2πσ 2 ⎢ ⎝ dd ⎠
⎢ d2
⎥
⎢
⎥
d −(r− A)2 /2σ 2
−(r+ A)2 /2σ 2
+e
dr
⎢ + ∫ dd e
⎥
⎣ −∞
⎦
(
)
((
))
((
))
(
)
= 0 for d < 0 = δ (d) × FR (0) for d = 0 (B6) where the third expression comes from Lecture 8 on mixture random variable, and δ (d) is the Dirac Delta function. Simplifying the terms of (B6), ⎧ ⎪ ⎪ ⎪ f D (d) = ⎨ ⎪ ⎪ ⎪ ⎩ Page: 11 S22 CMPE320 d 0 Project 2 rev0 220201.docx saved 2/27/22 1:40:00 PM printed 3/1/22 4:32:00 PM We can evaluate FR (0) using the definition of our Q(x) function, where Q(x) = ∞ 1 2πσ 2 ∫e −u 2 /2σ 2 du . The computation looks like this: x ( ) ( ) 0 ⎡ 0 −(r− A)2 /2σ 2 ⎤ 2 2 FR (0) = e dr + ∫ e−(r+ A) /2σ dr ⎥ ⎢ ∫ ⎥⎦ 2 2πσ 2 ⎢⎣ −∞ −∞ A− r dr A Let v = , dv = − or − σ dv = dr,r = −∞ → v = +∞,r = 0 → v = σ σ σ r+A dr A Let w = ,dw = or − σ dw = dr,r = −∞ → w = −∞,r = 0 → w = σ σ σ Substituting 1 = = = ∫( 1⎡ σ ⎢− 2 ⎢⎣ 2πσ 2 FR (0) = 1⎡ 1 ⎢ 2 ⎢⎣ 2π 1⎡ 1 ⎢ 2 ⎢⎣ 2π ∫ (e ∞ − v 2 /2 A/σ ∫( ∞ e− v 2 /2 A/σ A/σ e− v ) /2 ∞ ) dv + ) 2 ∫ (e A/σ 1 2π 2πσ − w2 /2 −∞ ⎛ 1 dv + ⎜ ⎝ 2π ∫( ∞ 2 ∫ (e A/σ σ dv + 2 0 ) dw⎥⎥⎦ ⎤ ) e− w /2 dw − −∞ − w2 /2 ) dw⎥⎥⎦ ⎤ (note the σ cancelled ∫( ∞ 1 2π A/σ 1 σ2 ⎞⎤ 2 e− w /2 dw⎟ ⎥ ⎠ ⎥⎦ ) 1 1 ⎡Q( A / σ ) + 1− Q( A / σ ) ⎤ = ⎣ ⎦ 2 2 ( ) ) (B8) In the next-to-last step of (B8), we manipulated the equality 1 2π ∫( ∞ ) 2 e− w /2 dw = −∞ 1 2π ∫( e− w /2 dw + ∫( e− w /2 dw − A/σ 2 −∞ ) 1 2π ∫ (e ∞ − w2 /2 A/σ ) dw (B9) to get 1 2π ∫( A/σ −∞ 2 ) e− w /2 dw = 1 2π ∞ −∞ 2 ) 1 2π ∫ (e ∞ A/σ − w2 /2 ) dw . (B10) The first term on the right side of (B10) is the integral of an N (0,1) pdf over the region (−∞,∞) , which is equal to 1. That substitution gives us the final equality of (B8) This gives our final answer Page: 12 S22 CMPE320 Project 2 rev0 220201.docx saved 2/27/22 1:40:00 PM printed 3/1/22 4:32:00 PM ⎧ ⎪ ⎪ ⎪⎪ f D (d) = ⎨ ⎪ ⎪ ⎪ ⎪⎩ d 2πσ 2 (e d 0 (B11) Figure 5 show a plot of (B11). f D (d) for D = g(R) = sqrt(R) for R>=0, 0 otherwise
1
0.9
Impulse with area 0.5
0.8
0.7
f D (d)
0.6
0.5
0.4
0.3
0.2
0.1
0
-3
-2
-1
0
1
2
3
d Value of Random Variable D
Figure 5 Plot of f D (d)
Page: 13
S22 CMPE320
Project 2 rev0 220201.docx saved 2/27/22 1:40:00 PM printed 3/1/22 4:32:00 PM
Appendic C: Detailed Example of analytical method of determining fY ( y)
for a transformation Y = g( X )
Let’s apply the analytical method to the previous problem. Again, D = g( R) = R, R > 0, and 0
otherwise.
For d < 0, f D (d) = 0 , by the definition of the random variable D . 1 For d = 0, f D (0) = Pr ⎡⎣ D = 0 ⎤⎦ = Pr ⎡⎣ R ≤ 0 ⎤⎦ δ (d) = δ (d) , by the same derivation in (B8) . 2 For D > 0, the analytical method gives us
⎡ 1
⎤
f D (d) = ⎢
fR r ⎥
⎢⎣ dd / dr
⎥⎦ r=g −1 ( d )
()
(C1)
d = g(r) = r for r > 0
dd
1
1
=−
=−
dr
2d
2 r
(C2)
g −1 (r) = d 2
Substituting (C2) into Error! Reference source not found. and combining all of the results
⎧
⎪
0
⎪
⎪⎪
1
f D (d) = ⎨
δ (d)
2
⎪
2
2
2
2
2
2
2d
⎪
e−( d − A) /2σ + e−( d + A) /2σ
⎪
2
⎪⎩ 2 2πσ
(
which is the same as (B11).
d 0
(C3)