# Dragon Notes

UNDER CONSTRUCTION
Latest content:
 Apr 05 Deep Learning Mar 19 Anomaly Detection - ML Mar 13 +Data Tables Mar 08 Clustering - Machine Learning Feb 28 Support Vector Machines - ML Feb 20 Regression - Data Science

# Randomness & Probability:Multiple WSS Random Processes

Jointly WSS rp's

$$\up$$Two rp's, $$X[n]$$ and $$Y[n]$$, are jointly Wide Sense Stationary when the below are met:
\begin{align}\mu_X[n] &= E[X[n]]=\mu_X \\ r_X[k] &= E[X[n]X[n+k]] \\ \mu_Y[n] &= E[Y[n]]=\mu_Y \\ r_Y[k] &= E[Y[n]Y[n+k]] \\ r_{X,Y}[k] &= E[X[n]Y[n+k]] \end{align}

Cross-correlation Sequence

$$\up$$The Cross-correlation sequence of two rp's $$X[n]$$ and $$Y[n]$$ is defined as
$$\ds r_{X,Y}[k]=E[X[n]Y[n+k]]\quad k=..., -1,0,1, ...$$
and is a measure of the similarity of two rp's as a function of relative displacement.
 [$$n$$-independence] If $$E[...]$$ depends on $$n$$, a CCS cannot be defined [Non-symmetry] $$\boxed{r_{X,Y}[-k]\neq r_{X,Y}[k]}$$ (CCS is not necessarily symmetric) [$$\t{max}\lrpar{r_{X,Y}}$$ can occur at any $$k$$] The maximum of the CCS can occur for any value of $$k$$ [Bounded] $$\boxed{\abs{r_{X,Y}[k]}\leq\sqrt{r_X[0]r_Y[0]}}$$ (The maximum value of the CCS is bounded) [Interchange-symmetry] $$\boxed{r_{Y,X}[k]=r_{X,Y}[-k]}$$ (Interchanging $$X[n]$$ and $$Y[n]$$ flips the CCS about $$k=0$$) [LSI relation] $$\boxed{r_{X,Y}[k]=\iisum{l}h[l]r_X[k-l]}$$ $$\Lra$$ $$\boxed{r_{X,Y}[k]=h[k]*r_X[k]}\$$ asm $$X[n]\ra\boxed{\t{LSI}}\ra Y[n]$$
Estimating CSS can be done with
$$\hat{r}_{X,Y}[k]=\left\{\nmtx{\frac{1}{N-k}\sum_{n=0}^{N-1-k}x[n]y[n+k]\ \ k=0,1,..., M\hspace{110px}} {\frac{1}{N-\abs{k}}\sum_{n=\abs{k}}^{N-1}x[n]y[n+k]\ \ \ k=-M,-(M-1),...,-1}\right.$$
where data records used for estimation were first shifted to place the maximum at $$k=0$$ (time alignment). CCS is then estimated for $$\abs{k}\leq M$$, where $$M$$ is some quantity for which $$r_{X,Y}[k]\approx 0$$ if $$\abs{k}>M$$.

Cross-Spectral Density

$$\up$$The Cross-Power Spectral Density (CPSD), or Cross-spectral Density (CSD), of two rp's $$X[n]$$ and $$Y[n]$$ (or $$X(t)$$ & $$Y(t)$$), is defined as
\ds \begin{align}P_{X,Y}(f)&=\ilim{M}\frac{1}{2M+1}E\lrbra{\lrpar{\sum_{n=-M}^{M}X[n]\t{exp}(-j2\pi fn)}^*\lrpar{\sum_{n=-M}^{M}Y[n]\t{exp}(-j2\pi fn)}} \\ P_{X,Y}(f) &= \ilim{T}\frac{1}{T}E\lrbra{\lrpar{\int_{-T/2}^{T/2}X(t)\t{exp}(-j2\pi Ft)dt}^*\lrpar{\int_{-T/2}^{T/2}Y(t)\t{exp}(-j2\pi Ft)dt}}\end{align}
and is a measure of power distribution over frequency of a joint rp.
 [CSD-CCS relation] $$\boxed{P_{X,Y}(f)=\iisum{k}r_{X,Y}[k]\t{exp}(-j2\pi fk)}$$ (CSD is the Fourier transform of the CCS) [Hermitian] $$\boxed{P_{X,Y}(-f)=P^*_{X,Y}(f)}$$ ($$g(f)$$ is hermitian if $$\Re\lrpar{g(f)}=$$ even and $$\Im\lrpar{g(f)}=$$ odd about $$f=0$$) [Bounded] $$\boxed{\abs{P_{X,Y}(f)}\leq\sqrt{P_X(f)P_Y(f)}}$$ (the maximum value of the CSD is bounded) [0 for zero-mean & uncorr. rp's] $$\boxed{P_{X,Y}(f)=0}\$$ asm $$\mu_X=0, \mu_Y=0$$, $$X[n]$$ & $$Y[n]$$ uncorr. [Complex conjugate-symmetry] $$\boxed{P_{Y,X}(f)=P^*_{X,Y}(f)}$$ (CSD of $$(Y[n],X[n])$$ is the complex conjugate of the CSD of $$(X[n],Y[n])$$) [Coherence / Complex corr.] $$\boxed{\gamma_{X,Y}(f) = \frac{P_{X,Y}(f)}{\sqrt{P_X(f)P_Y(f)}}}$$; $$\gamma_{X,Y} =$$ coherence function, giving the complex corr. coefficient at $$f$$; measures the correlation between the Fourier transforms of two jointly WSS rp's at $$f$$ [CSD-PSD relation] $$\boxed{P_{X,Y}(f)=H(f)P_X(f)}\$$ asm $$X[n]\ra\boxed{\t{LSI}}\ra Y[n]$$ [LSI coherence] $$\boxed{\abs{\gamma_{X,Y}(f)}=1}$$ $$\boxed{\gamma_{X,Y}(f)=\t{exp}(j\phi (f))}$$, $$\phi (f)=\angle H(f)\$$ asm $$X[n]\ra\boxed{\t{LSI}}\ra Y[n]$$ implies that $$Y[n]$$ is perfectly predictable from $$X[n]$$, and vice versa (since $$\rho_{X,Y}=\rho_{Y,X}$$) [LSI input-output] $$\boxed{Y[n]=\iisum{k}h[k]X[n-k]}\$$ asm $$X[n]\ra\boxed{\t{LSI}}\ra Y[n]$$