Y is Another Continuous Random Variable That is Uniform Between Y 0 and Y L

Uniform Random Variable

Monte Carlo Integration

Matt Pharr , ... Greg Humphreys , in Physically Based Rendering (Third Edition), 2017

13.1.1 Continuous random variables

In rendering, discrete random variables are less common than continuous random variables, which take on values over ranges of continuous domains (e.g., the real numbers, directions on the unit sphere, or the surfaces of shapes in the scene).

A particularly important random variable is the canonical uniform random variable , which we will write as ξ. This variable takes on all values in its domain [0, 1) with equal probability. This particular variable is important for two reasons. First, it is easy to generate a variable with this distribution in software—most run-time libraries have a pseudo-random number generator that does just that. ² Second, as we will show later, it is possible to generate samples from arbitrary distributions by first starting with canonical uniform random variables and applying an appropriate transformation. The technique described previously for mapping from ξ to the six faces of a die gives a flavor of this technique in the discrete case.

Another example of a continuous random variable is one that ranges over the real numbers between 0 and 2, where the probability of its taking on any particular value x is proportional to the value 2   − x: it is twice as likely for this random variable to take on a value around 0 as it is to take one around 1, and so forth. The probability density function (PDF) formalizes this idea: it describes the relative probability of a random variable taking on a particular value. The PDF p(x) is the derivative of the random variable's CDF,

$p\left(x\right)=\frac{\mathrm{d}P\left(x\right)}{\mathrm{d}x}\text{.}$

For uniform random variables, p(x) is a constant; this is a direct consequence of uniformity. For ξ we have

$p\left(x\right)=\left\{\begin{array}{ll}1\hfill & x\in [0,1)\hfill \\ 0\hfill & \mathrm{otherwise}\text{.}\hfill \end{array}\right.$

PDFs are necessarily nonnegative and always integrate to 1 over their domains. Given an arbitrary interval [a, b] in the domain, integrating the PDF gives the probability that a random variable lies inside the interval:

$P\left(x\in \left[a,b\right]\right)={\int }_a^{b}p\left(x\right)\mathrm{d}x\text{.}$

This follows directly from the first fundamental theorem of calculus and the definition of the PDF.

Read full chapter

URL:

https://www.sciencedirect.com/science/article/pii/B9780128006450500130

Simulation Techniques

Scott L. Miller , Donald Childers , in Probability and Random Processes, 2004

12.1.3 Generation of Random Numbers from a Specified Distribution

Quite often we are interested in generating random variables that obey some distribution other than a uniform distribution. In this case, it is generally a fairly simple task to transform a uniform random number generator into one that follows some other distribution. Consider forming a monotonic increasing transformation g() on a random variable X to form a new random variable Y. From the results of Chapter 4, the PDFs of the random variables involved are related by

(12.4) $f_Y(y)=\frac{f_X(x)}{dg(x)/dx}.$

Given an arbitrary PDF, f_X (x), the transformation Y = g(X ) will produce a uniform random variable Y if dg/dx = f_X (x), or equivalently g(x) = F_X (x). Viewing this result in reverse, if X is uniformly distributed over (0,1) and we want to create a new random variable, Y, with a specified distribution, FY (y), the transformation Y = F⁻¹(X) will do the job.

EXAMPLE 12.3: Suppose we want to transform a uniform random variable into an exponential random variable with a PDF of the form

$f_Y(y)=aexp(-ay)u(y).$

The corresponding CDF is

$f_Y(y)=[1-exp(-ay)]u(y).$

Hence, to transform a uniform random variable into an exponential random variable, we can use the transformation

$Y=F_Y^{-1}=-\frac{\text{ln}(1-\text{X)}}{\text{a}}.$

Note that if X is uniformly distributed over (0, 1), then 1 – X will be uniformly distributed as well so that the slightly simpler transformation

$Y=-\frac{\text{ln}(X)}{\text{a}}$

will also work.

This approach for generation of random variables works well provided that the CDF of the desired distribution is invertible. One notable exception where this approach will be difficult is the Gaussian random variable. Suppose, for example, we wanted to transform a uniform random variable, X, into a standard normal random variable, Y. The CDF in this case is the complement of a Q-function, F_Y (y) = 1 – Q(y). The inverse of this function would then provide the appropriate transformation, y = Q ⁻¹(1 – x), or as with the previous example, we could simplify this to y = Q⁻¹(x). The problem here lies with the inverse Q-function, which cannot be expressed in a closed form. One could devise efficient numerical routines to compute the inverse Q-function, but fortunately there is an easier approach.

An efficient method to generate Gaussian random variables from uniform random variables is based on the following 2 × 2 transformation. Let X ₁ and X ₂ be two independent uniform random variables (over the interval (0,1)). Then if two new random variables, Y ₁ and Y ₂, are created according to

(12.5a) $Y_1=\sqrt{-2\text{1n}(X_{\text{1}}\text{)}}cos(2\pi X_2),$

(12.5b) $Y_2=\sqrt{-21\text{n(}{X}_{\text{1}}\text{)}}sin(2\pi X_2),$

then Y ₁ and Y ₂ will be independent standard normal random variables (see Example 5.24). This famous result is known as the Box-Muller transformation and is commonly used to generate Gaussian random variables. If a pair of Gaussian random variables is not needed, one of the two can be discarded. This method is particularly convenient for generating complex Gaussian random variables since it naturally generates pairs of independent Gaussian random variables. Note that if Gaussian random variables are needed with different means or variances, this can easily be accomplished through an appropriate linear transformation. That is, if Y ∼ N(0,1), then $Z=\sigma Y+\mu$ will produce $Z\sim N(\mu ,{\sigma }^2).$

Read full chapter

URL:

https://www.sciencedirect.com/science/article/pii/B9780121726515500129

Stochastic Models

P.A. Ferrari , in International Encyclopedia of the Social & Behavioral Sciences, 2001

3.1 Coupling

Two or more Markov chains can be constructed with the same uniform random variables U _n . This is called a coupling and it is one of the major tools in stochastic processes. Applications of coupling include proofs of convergence of a Markov chain to its invariant measure, comparisons between chains to obtain properties of one of them in function of the other, and simulation of measures that are invariant for Markov chains.

Recent books on Markov chains include Chen (1992), Bianc and Durrett (1995), Fristedt and Gray (1997), Brémaud (1999), Durrett (1999a), Schinazi (1999), Thorisson (2000), Häggström (2000), and Ferrari and Galves (2000).

Read full chapter

URL:

https://www.sciencedirect.com/science/article/pii/B0080430767005921

Simulation Techniques

Israel Koren , C. Mani Krishna , in Fault-Tolerant Systems (Second Edition), 2021

9.3.1 Antithetic Variables

Suppose we run simulations to estimate some parameter (for example, mean time to data Loss (MTTDL) in a RAID system). In traditional simulation, we would run n independent simulations and use the results. If ${\mathrm{Z}}_1,{\mathrm{Z}}_2$ are the outputs from two independent runs, we can expect that

$\text{Cov}({\mathrm{Z}}_1,{\mathrm{Z}}_2)=0$

so that

$\text{Var}\left(\frac{{\mathrm{Z}}_1+{\mathrm{Z}}_2}{2}\right)=\frac{\text{Var}({\mathrm{Z}}_1)+\text{Var}({\mathrm{Z}}_2)}{4}.$

When the method of antithetic variables is used, we try to run simulations in pairs, coupled together in such a way that their results (any parameter that is estimated by the simulation, be it reliability, waiting time, etc.) are negatively correlated, and then treat $\mathrm{Y}=({\mathrm{Z}}_1+{\mathrm{Z}}_2)/2$ as the output from this pair of runs. If the simulation pair produces the outputs ${\mathrm{Z}}_1,{\mathrm{Z}}_2$ such that $\text{Cov}({\mathrm{Z}}_1,{\mathrm{Z}}_2)<0$ , the variance of Y will be smaller than it would be if the two runs were independent, and not coupled.

A good way to couple pairs of simulation runs is to couple the random variables used by them. Suppose the output of the simulation is a monotonic function of the random variables, and the first run of the pair uses uniform random variables ${\mathrm{U}}_1,{\mathrm{U}}_2,\cdots ,{\mathrm{U}}_{\mathrm{n}}$ , then the second run can use $1-{\mathrm{U}}_1,1-{\mathrm{U}}_2,\cdots ,1-{\mathrm{U}}_{\mathrm{n}}$ . The corresponding random variables in the two sequences are negatively correlated: if ${\mathrm{U}}_{\mathrm{i}}$ is large, $1-{\mathrm{U}}_{\mathrm{i}}$ is small, and vice versa. This applies even when the distributions of the random variables used in the simulation are not uniform. We are assuming that to generate such random variables, we will ultimately need to call uniform random number generators (URNGs), described later in Section 9.5.1. We can apply the coupling on the output of these URNGs. For example, if we need to generate exponentially distributed random variables by using $\mathrm{X}=-(1/\mathrm{\mu })\ln \mathrm{U}$ , the coupled simulations will generate U, and then use ${\mathrm{X}}_1=-(1/\mathrm{\mu })\ln \mathrm{U}$ and ${\mathrm{X}}_2=-(1/\mathrm{\mu })\ln (1-\mathrm{U})$ , respectively (see Section 9.5.3).

In other words, if we can write the simulation output as being a monotone function of the uniform random variables used, then it is possible to show that the simulation outputs will indeed be negatively correlated when the method of antithetic variables is used. Showing this is outside the scope of this book, see the Further reading section for details on where to find the proof.

Example: Consider a structure composed of k elements. Denote by ${\mathrm{S}}_{\mathrm{i}}$ the state of component i: a functional component is denoted by ${\mathrm{S}}_{\mathrm{i}}=1$ , whereas if it is down we have ${\mathrm{S}}_{\mathrm{i}}=0$ . A structure function $\mathrm{\varphi }({\mathrm{S}}_1,{\mathrm{S}}_2,\dots ,{\mathrm{S}}_{\mathrm{k}})$ is an indicator function (assumes the values $0,1$ ), which expresses the dependence of the functionality of the system on the functionality of its components: it is equal to 1 if the system is functional for given values of ${\mathrm{S}}_1,\dots ,{\mathrm{S}}_{\mathrm{k}}$ , and to 0 if it is not.

For instance, if the system consists of k elements connected in series, we have

$\mathrm{\varphi }({\mathrm{S}}_1,{\mathrm{S}}_2,\cdots ,{\mathrm{S}}_{\mathrm{k}})={\mathrm{S}}_1\times {\mathrm{S}}_2\times \cdots \times {\mathrm{S}}_{\mathrm{k}}.$

If it is a triplex system with a perfect voter, and ${\mathrm{S}}_{\mathrm{i}}$ denotes the state of the ith processor, then

$\mathrm{\varphi }({\mathrm{S}}_1,{\mathrm{S}}_2,{\mathrm{S}}_3)=\{\begin{array}{cc}1\hfill & \text{if}{\mathrm{S}}_1+{\mathrm{S}}_2+{\mathrm{S}}_3\ge 2\hfill \\ 0\hfill & \text{otherwise}.\hfill \end{array}$

Now suppose we want to simulate the reliability R, for some given length of time t, of a system with a very complex structure function that cannot easily be analyzed. Using traditional methods, we would run a simulation by generating random variables that would determine whether individual components were up or not, and then determine whether the overall system was functional during $[0,\mathrm{t}]$ . Using antithetic variables, we will run the simulations in pairs, with the random variables coupled, as described above. If ${\mathrm{Y}}_{\mathrm{i}}$ is the average of the values of the structure function from the two simulation runs in pair i, and we run a total of 2n simulations (or n pairs), then the estimated reliability of the system is

$\hat{\mathrm{R}}=\frac{{\mathrm{Y}}_1+{\mathrm{Y}}_2+\cdots +{\mathrm{Y}}_{\mathrm{n}}}{\mathrm{n}}.$

Furthermore, the variance of the estimate is likely to be far lower than would be obtained if we ran 2n independent simulations.

It is important to note that the ${\mathrm{Y}}_{\mathrm{i}}$ s are independent of one another, that is, although each run consists of paired simulations, there is no coupling between one pair and another. This allows us to use traditional statistical analysis on the ${\mathrm{Y}}_{\mathrm{i}}$ values.

By how much can we expect the variance of the estimate to drop? This depends on the covariance of the two outputs in each pair of runs. In the Exercises, you are invited to determine the usefulness of this approach in a variety of cases.

Read full chapter

URL:

https://www.sciencedirect.com/science/article/pii/B978012818105800019X

Scalar Quantization

Khalid Sayood , in Introduction to Data Compression (Fourth Edition), 2012

Uniform Quantization of a Uniformly Distributed Source

We start our study of quantizer design with the simplest of all cases: design of a uniform quantizer for a uniformly distributed source. Suppose we want to design an M-level uniform quantizer for an input that is uniformly distributed in the interval $[-X_{\max }\text{,}{X}_{\max }]$ . This means we need to divide the $[-X_{\max }\text{,}{X}_{\max }]$ interval into M equally sized intervals. In this case, the step size $\mathrm{\Delta }$ is given by

(9) $\mathrm{\Delta }=\frac{2X_{\max }}{M}$

The distortion in this case becomes

(10) ${\sigma }_q^2=2{\displaystyle \sum _{i=1}^{\frac{M}{2}}}{\int }_{(i-1)\mathrm{\Delta }}^{i\mathrm{\Delta }}{\left(x-\frac{2i-1}{2}\mathrm{\Delta }\right)}^2\frac{1}{2X_{\max }}\mathit{dx}$

If we evaluate this integral (after some suffering), we find that the msqe is ${\mathrm{\Delta }}^2/12$ .

The same result can be more easily obtained if we examine the behavior of the quantization error q given by

(11) $q=x-Q(x)$

In Figure   9.6, we plot the quantization error versus the input signal for an eight-level uniform quantizer with an input that lies in the interval $[-X_{\max }\text{,}{X}_{\max }]$ . Notice that the quantization error lies in the interval $[-\frac{\mathrm{\Delta }}{2}\text{,}\frac{\mathrm{\Delta }}{2}]$ . As the input is uniform, it is not difficult to establish that the quantization error is also uniform over this interval. Thus, the mean squared quantization error is the second moment of a random variable uniformly distributed in the interval $[-\frac{\mathrm{\Delta }}{2}\text{,}\frac{\mathrm{\Delta }}{2}]$ :

Figure 9.6. Quantization error for a uniform midrise quantizer with a uniformly distributed input.

(12) ${\sigma }_q^2=\frac{1}{\mathrm{\Delta }}{\int }_{-\frac{\mathrm{\Delta }}{2}}^{\frac{\mathrm{\Delta }}{2}}{q}^2\mathit{dq}$

(13) $=\frac{{\mathrm{\Delta }}^2}{12}$

Let us also calculate the signal-to-noise ratio for this case. The signal variance ${\sigma }_s^2$ for a uniform random variable that takes on values in the interval $[-X_{\max }\text{,}{X}_{\max }]$ is $\frac{(2X_{\max }{)}^2}{12}$ . The value of the step size $\mathrm{\Delta }$ is related to $X_{\max }$ and the number of levels M by

$\mathrm{\Delta }=\frac{2X_{\max }}{M}$

For the case where we use a fixed-length code, with each codeword being made up of n bits, the number of codewords or the number of reconstruction levels M is $2^n$ . Combining all this, we have

(14) $\mathrm{SNR}(\text{dB})=10{\log }_{10}\left(\frac{{\sigma }_s^2}{{\sigma }_q^2}\right)$

(15) $=10{\log }_{10}\left(\frac{(2X_{\text{max}}{)}^2}{12}·\frac{12}{{\mathrm{\Delta }}^2}\right)$

(16) $=10{\log }_{10}\left(\frac{(2X_{\text{max}}{)}^2}{12}\frac{12}{(\frac{2X_{\text{max}}}{M}{)}^2}\right)$

(17) $\begin{array}{cc}\hfill =& 10{\log }_{10}(M^2)\hfill \\ \hfill =& 20{\log }_{10}(2^n)\hfill \\ \hfill =& 6.02n\text{dB}\hfill \end{array}$

This equation says that for every additional bit in the quantizer, we get an increase in the signal-to-noise ratio of 6.02 dB. This is a well-known result and is often used to get an indication of the maximum gain available if we increase the rate. However, remember that we obtained this result under some assumptions about the input. If the assumptions are not true, this result will not hold true either.

Example 9.4.1

Image Compression

A probability model for the variations of pixels in an image is almost impossible to obtain because of the great variety of images available. A common approach is to declare the pixel values to be uniformly distributed between 0 and $2^b-1$ , where b is the number of bits per pixel. For most of the images we deal with, the number of bits per pixel is 8; therefore, the pixel values would be assumed to vary uniformly between 0 and 255. Let us quantize our test image Sena using a uniform quantizer.

If we wanted to use only 1 bit per pixel, we would divide the range [0, 255] into two intervals, [0, 127] and [128, 255]. The first interval would be represented by the value 64, the midpoint of the first interval; the pixels in the second interval would be represented by the pixel value 196, the midpoint of the second interval. In other words, the boundary values are {0, 128, 255}, while the reconstruction values are {64, 196}. The quantized image is shown in Figure   9.7. As expected, almost all the details in the image have disappeared. If we were to use a 2-bit quantizer, with boundary values {0, 64, 128, 196, 255} and reconstruction levels {32, 96, 160, 224}, we would get considerably more detail. The level of detail increases as the use of bits increases until at 6 bits per pixel, the reconstructed image is indistinguishable from the original, at least to a casual observer. The 1-, 2-, and 3-bit images are shown in Figure 9.7.

Figure 9.7. Top left: original Sena image; top right: 1   bit/pixel image; bottom left: 2   bits/pixel; bottom right: 3   bits/pixel.

Looking at the lower-rate images, we notice a couple of things. First, the lower-rate images are darker than the original, and the lowest-rate reconstructions are the darkest. The reason for this is that the quantization process usually results in scaling down of the dynamic range of the input. For example, in the 1-bit-per-pixel reproduction, the highest pixel value is 196, as opposed to 255 for the original image. As higher gray values represent lighter shades, there is a corresponding darkening of the reconstruction. The other thing to notice in the low-rate reconstruction is that wherever there were smooth changes in gray values there are now abrupt transitions. This is especially evident in the face and neck area, where gradual shading has been transformed to blotchy regions of constant values. This is because a range of values is being mapped to the same value, as was the case for the first two samples of the sinusoid in Example 9.3.1. For obvious reasons, this effect is called contouring. The perceptual effect of contouring can be reduced by a procedure called dithering [121].   ♦

Read full chapter

URL:

https://www.sciencedirect.com/science/article/pii/B9780124157965000090

Numerical Approximation of Predictive Distribution

Masashi Sugiyama Masashi Sugiyama , in Introduction to Statistical Machine Learning, 2016

19.3.1 Inverse Transform Sampling

Inverse transform sampling generates a one-dimensional random sample $\theta$ that follows a probability distribution with density $p\left(\theta \right)$ based on a uniform random variable $u$ on $[0,1]$ [62] and the cumulative distribution function of $p\left(\theta \right)$ . The cumulative distribution function of $p\left(\theta \right)$ , denoted by $P\left(\theta \right)$ , is defined as follows (Fig. 19.4):

Probability density function $p\left(\theta \right)$

Cumulative distribution function $P\left(\theta \right)$

FIGURE 19.4. Examples of probability density function $p\left(\theta \right)$ and its cumulative distribution function $P\left(\theta \right)$ . Cumulative distribution function is monotone nondecreasing and satisfies ${lim}_{\theta \to -\infty }P\left(\theta \right)=0$ and ${lim}_{\theta \to \infty }P\left(\theta \right)=1$ .

Let $\theta =P^{-1}\left(u\right)$ be the inverse function of $u=P\left(\theta \right)$ . Then, for the uniform random variable $u$ on $[0,1]$ , $\theta =P^{-1}\left(u\right)$ has probability density $p\left(\theta \right)$ (Fig. 19.5). Thus, for $n$ uniform random variables ${\left\{u_i\right\}}_{i=1}^n$ on $[0,1]$ ,

FIGURE 19.5. Inverse transform sampling.

are i.i.d. with $p\left(\theta \right)$ .

The validity of the above algorithm can be proved as follows. Since $\theta =P^{-1}\left(u\right)$ , for any $\tau$ ,

As illustrated in Fig. 19.6, $\theta \le {\theta }^{\prime }$ implies $P\left(\theta \right)\le P\left({\theta }^{\prime }\right)$ , and therefore

FIGURE 19.6. $\theta \le {\theta }^{\prime }$ implies $P\left(\theta \right)\le P\left({\theta }^{\prime }\right)$ .

Furthermore, since $u$ follows the uniform distribution on $[0,1]$ ,

holds and therefore

This means that the cumulative distribution function of $\theta$ generated by inverse transform sampling agrees with the target $P\left(\tau \right)$ .

Let us generate random samples ${\left\{{\theta }_i\right\}}_{i=1}^n$ that are i.i.d. with the standard Laplace distribution $\mathrm{La}(0,1)$ . The probability density function $p\left(\theta \right)$ , cumulative distribution function $P\left(\theta \right)$ , and its inverse function $P^{-1}\left(u\right)$ are given as follows (see Fig. 19.7):

Probability density function $p\left(\theta \right)$

Cumulative distribution function $P\left(\theta \right)$

Inverse cumulative distribution function $P^{-1}\left(u\right)$

FIGURE 19.7. Laplace distribution.

$p\left(\theta \right)=\frac{1}{2}{e}^{-\left|\theta \right|},P\left(\theta \right)=\frac{1}{2}(1+\mathrm{sign}\left(\theta \right)(1-e^{-\left|\theta \right|}))\text{,}$

$P^{-1}\left(u\right)=-\mathrm{sign}(u-\frac{1}{2})log(1-2|u-\frac{1}{2}|)\text{,}$

where $\mathrm{sign}\left(\theta \right)$ denotes the sign function:

$\mathrm{sign}\left(\theta \right)=\{\begin{array}{cc}1& (\theta >0)\text{,}\\ 0& (\theta =0)\text{,}\\ -1& (\theta <0)\text{.}\\ \end{array}$

A MATLAB code for inverse transform sampling is given in Fig. 19.8, and its behavior is illustrated in Fig. 19.9.

FIGURE 19.8. MATLAB code for inverse transform sampling.

Generated uniform samples $u$

Obtained Laplace samples

FIGURE 19.9. Example of inverse transform sampling for Laplace distribution.

As shown above, inverse transform sampling is a simple algorithm to generate samples following an arbitrary distribution. However, it can be applied only to one-dimensional distributions. Furthermore, the inverse cumulative distribution function $\theta =P^{-1}\left(u\right)$ needs to be explicitly computed, which can be difficult depending on the probability distributions.

Read full chapter

URL:

https://www.sciencedirect.com/science/article/pii/B9780128021217000303

Monte Carlo–Based Financial Market Value-at-Risk Estimation on GPUs

Matthew F. Dixon , ... Kurt Keutzer , in GPU Computing Gems Jade Edition, 2012

Performance Results

Table 25.3 illustrates the performance impact on a NVIDIA GeForce GTX 480 from applying kernel merging to two similar methods — the Box-Muller and Bailey's methods. ⁷ The performance highlights the sensitivity of the kernel merging optimization. The implementation of the Box-Muller method and the Bailey's method are almost identical, except for two extra parameters in the Bailey's method to produce a distribution with slightly higher weights for less likely events. These extra parameters caused the compiled kernel to maintain more registers; as a result the speedup for the Bailey method is less that that of the Box-Muller method.

Table 25.3. Implementation Cost in Seconds on a NVIDIA GeForce GTX480 for Setting Up 7.5 × 10⁵ Experiments Each with 4000 Risk Factors

Timing (s)	Standard (Separate)	Optimized (Merged)	Speedup
Box-Muller (step 1)	0.128	0.156	2.63x
(step 2)	0.282
Bailey (step 1)	0.128	0.441	1.16x
(step 2)	0.384

This optimization, however, can be sensitive to the amount of context each parallel thread of execution must maintain. It is found to be effective for the Box-Muller method, but not as effective for the Bailey's method, even though their functional form is very similar. Specifically, the merged Bailey's method resulted in a 1.16× speedup over two separate steps. This indicates some degree of register spilling, although this overhead is drastically reduced compared with the GTX200 generation of GPU architectures.

Read full chapter

URL:

https://www.sciencedirect.com/science/article/pii/B9780123859631000253

Perturbation Methods for Protecting Numerical Data: Evolution and Evaluation

Rathindra Sarathy , Krish Muralidhar , in Handbook of Statistics, 2012

4.2.2 Sullivan's model

Another type of nonlinear perturbation model was proposed by Sullivan (1989) in cases where the marginal distributions of the variables are not normal. Sullivan's approach tries to preserve the marginal distribution of the masked variables to be the same as that of the original variables, regardless of whether they are numerical or categorical. This approach transforms each observation into a uniform random variable using its empirical cumulative distribution function (cdf) that is then retransformed to a standard normal random variable.

Let $x_i^{\ast }$ represent the transformed variable where

(15) $x_i^{\ast }={\mathrm{\Phi }}^{-1}(F_i(x_i))\text{,}i=1\text{,}\dots \text{,}n\text{.}$

An appropriate level of noise is then added to the standard normal variable to result in $y_i^{\ast }$ as follows:

(16) $y_i^{\ast }=x_i^{\ast }+{\epsilon }_i^{\ast }\text{,}$

where ${\epsilon }_i^{\ast }$ represents the independent noise term. Once noise addition is completed, the entire process is reversed to yield perturbed values that have the same empirical distribution as the original confidential values as $y_i=F_i^{-1}(\mathrm{\Phi }(y_i^{\ast }))$ . The empirical nature of Sullivan's approach makes it difficult to predict its data utility and disclosure risk characteristics.

Read full chapter

URL:

https://www.sciencedirect.com/science/article/pii/B9780444518750000191

Design of experiments for uncertainty quantification based on polynomial chaos expansion metamodels

Subhrajit Dutta , Amir H. Gandomi , in Handbook of Probabilistic Models, 2020

4.1 Analytical problem: Ishigami function

The Ishigami function is a well-known benchmark problem for uncertainty quantification. It is a three-dimensional nonmonotonic and highly nonlinear analytical function, given by the following equation:

(15.8) $f\left(x\right)=\text{sin}\left(x_1\right)\mathit{a}{\text{sin}}^{\mathit{2}}\left(x_2\right)+b{x}_3^4\text{sin}\left(x_1\right)\text{.}$

The input vector consists of three independent and identically distributed (i.i.d) uniform random variables $X_i\sim \mathcal{U}\left(-\pi ,\pi \right)$ . The coefficient values (a   =   7, b   =   0.1) are chosen for this example.

The performances of the various sampling strategies with respect to the error estimates introduced in the previous section are compared. For validation purpose, an MCS-based approach with a validation set of size N _v   =   1000 is considered. Fig. 15.1A and B provide the MSE and ε _LOO calculated considering N  =   50 to 200 sizes of the ED.

Figure 15.1. Error estimates for various DoE: (A) Mean square error; (B) Leave-one-out error. DoE, design of experiment; IS, importance sampling; LHS, Latin hypercube sampling; MCS, Monte Carlo sampling; MSE, mean square error.

As expected, the LHS ED shows a decrease in both MSE and ε _LOO across all ED sizes. All the other design schemes show a consistent decrease in the error estimates, except MCS which shows steady values. In addition, LHS generally shows a more stable behavior with smaller variability between repetitions, especially as the size of the ED increases. In general, the metamodel accuracy seems to be more for larger EDs. Also, comparable results are obtained with IS and Sobol sequence.

Read full chapter

URL:

https://www.sciencedirect.com/science/article/pii/B9780128165140000151

Random Processes in Linear Systems

Scott L. Miller , Donald Childers , in Probability and Random Processes, 2004

11.4 Signal-to-Noise Ratios

Often the input to a linear system will consist of signal plus noise, namely,

(11.24) $X\left(t\right)=S\left(t\right)+N\left(t\right),$

where the signal part can be deterministic or a random process. We can invoke linearity to show that the mean process of the output can be viewed as a sum of the mean due to the signal input alone plus the mean due to the noise input alone. That is,

(11.25) ${\mu }_Y\left(t\right)={\mu }_S\left(t\right)*h\left(t\right)+{\mu }_N\left(t\right)*h\left(t\right).$

In most cases, the noise is taken to be zero-mean, in which case the mean at the output is due to the signal part alone.

When calculating the autocorrelation function of the output, we cannot invoke superposition since autocorrelation is not a linear operation. First, we calculate the autocorrelation function of the signal plus noise input.

(11.26) $R_{XX}\left(t_1,t_2\right)=E\left[\left(S\left(t_1\right)+N\left(t_1\right)\right)\left(S\left(t_2\right)+N\left(t_2\right)\right)\right].$

If the signal and noise part are independent, which is generally a reasonable assumption, and the noise is zero-mean, then this autocorrelation becomes

(11.27) $R_{XX}\left(t_1,t_2\right)=R_{SS}\left(t_1,t_2\right)+R_{NN}\left(t_1,t_2\right),$

or, assuming all processes involved are WSS,

(11.28) $R_{XX}\left(\tau \right)=R_{SS}\left(\tau \right)+R_{NN}\left(\tau \right).$

As a result, the PSD of the output can be written as

(11.29) $S_{YY}\left(f\right)=S_{XX}\left(f\right){\left|H\left(f\right)\right|}^2=S_{SS}\left(f\right){\left|H\left(f\right)\right|}^2+S_{NN}\left(f\right){\left|H\left(f\right)\right|}^2,$

which is composed of two terms, namely that due to the signal and that due to the noise. We can then calculate the output power due to the signal part and the output power due to the noise part.

DEFINITION 11.2: The signal-to-noise ratio (SNR) for a signal comprised of the sum of a desired (signal) part and a noise part is defined as the ratio of the power of the signal part to the power (variance) of the noise part. That is, for X(t) = S(t) + N(t),

(11.30) $\text{SNR}=\frac{E\left[S^2\left(t\right)\right]}{E\left[N^2\left(t\right)\right]}=\frac{R_{SS}\left(0\right)}{R_{NN}\left(0\right)}=\frac{{\int }_{-\infty }^{\infty }{S}_{SS}\left(f\right)df}{{\int }_{-\infty }^{\infty }{S}_{NN}\left(f\right)df}.$

EXAMPLE 11.5: Suppose the input to the RC lowpass filter of the previous example consists of a sinusoidal signal plus white noise. That is, let the input be X(t) = S(t) + N(t), where N(t) is white Gaussian noise as in the previous example and S(t) = a cos(ω _ot +Θ), where Θ is a uniform random variable over [0,2π) that is independent of the noise. The output can be written as Y(t) = S_o (t) + N_o (t), where S_o (t) is the output due to the sinusoidal signal input and N_o (t) is the output due to the noise. The signal output can be expressed as

$S_o\left(t\right)=a\left|H\left(f_o\right)\right|cos\left({\omega }_{o}t+\angle H\left(f_o\right)+\Theta \right),$

and the power in this sinusoidal signal is

$R_{S_o{S}_o}\left(0\right)=\frac{a^2{\left|H\left(f_o\right)\right|}^2}{2}$

From the results of Example 11.1, the noise power at the output is

$R_{N_o{N}_o}\left(0\right)=\frac{N_O}{4RC}.$

Hence, the SNR of the output of the RC lowpass filter is

$\text{SNR}=\frac{\text{2}{a}^{\text{2}}RC{\left|H\left(f_o\right)\right|}^2}{N_o}=\frac{2a^{2}RC}{N_o\left(1+{\left(2\pi f_{o}RC\right)}^2\right)}.$

Suppose we desire to adjust the RC time constant (or, equivalently, adjust the bandwidth) of the filter so that the output SNR is optimized. Differentiating with respect to the quantity RC, setting equal to zero and solving the resulting equation produces the optimum time constant

$R{C}_{\text{opt}}=\frac{1}{2\pi f_o}.$

Stated another way, the 3-dB frequency of the RC filter is set equal to the frequency of the input sinusoid in order to optimize output SNR. The resulting optimum SNR is

${\text{SNR}}_{\text{opt}}=\frac{a^2}{2\pi N_o{f}_o}.$

Read full chapter

URL:

https://www.sciencedirect.com/science/article/pii/B9780121726515500117

kentnowily.blogspot.com

Source: https://www.sciencedirect.com/topics/computer-science/uniform-random-variable

Share this post