Bayesian posterior distribution 20101125-1

November 25, 2010

In Bayesian hierarchical setting:
$\left.\boldsymbol x_i\right|\boldsymbol\mu, \mathbf\Sigma\sim\mathscr N(\boldsymbol\mu, \mathbf\Sigma)$
$i=1,2,\ldots,n$, where
$\left.\boldsymbol\mu\right|\mathbf\Sigma\sim\mathscr N\left(\boldsymbol\mu_0, \frac{1}{\kappa_0}\mathbf\Sigma\right)$
and
$\mathbf\Sigma\sim\mathscr{IW}(\nu_0, \mathbf\Sigma_0)$
and, most statisticians can tell me that
$\left.\boldsymbol\mu\right|\boldsymbol x_1,\boldsymbol x_2,\ldots, \boldsymbol x_n,\mathbf\Sigma\sim\mathscr N\left(\frac{\kappa_0}{n+\kappa_0}\boldsymbol\mu_0+\frac{n}{n+\kappa_0}\overline{\boldsymbol x}, \frac{1}{\kappa_0}\mathbf\Sigma\right)$
where $\frac{1}{n}\sum_{i=1}^n\boldsymbol x_i$ and
$\left.\mathbf\Sigma\right|\boldsymbol x_1,\boldsymbol x_2,\ldots, \boldsymbol x_n\sim\mathscr{IW}\left(n+\nu_0, \mathbf\Sigma_0 + \sum_{i=1}^n(\boldsymbol x_i-\overline{\boldsymbol x})(\boldsymbol x_i-\overline{\boldsymbol x})^\top + \frac{\kappa_0n}{\kappa_0+n}(\boldsymbol x_i-\boldsymbol\mu_0)(\boldsymbol x_i-\boldsymbol\mu_0)^\top\right)$
right away, but, if we set
$\left.\boldsymbol\mu\right|\mathbf\Sigma\sim\mathscr N\left(\boldsymbol\mu_0,\mathbf{A\Sigma A}^\top\right)$
can we get such a beautiful posterior form?

E
(

X
k

)
=

R

1

Γ

ν
+
1

2

Γ

ν
+
1

2

π
ν

x
k

ν

ν
+

x
2

ν
+
1

2

d
x

Nothing selected

Probabilty note 2010 Oct 13 (1)

October 13, 2010

Consider a collection of random variables $\boldsymbol x = (x_1, x_2, \ldots, x_k)^T$, if $\boldsymbol x$ have a joint characteristic function
$\phi_{\boldsymbol x} (\boldsymbol t)$, where $\boldsymbol t = (t_1,t_2,\ldots, t_k)^T$ are all distinct, then we can conclude that $x_i, x_j$ are independent for all $i\neq j$ if and only if
$\phi_{\boldsymbol x} (\boldsymbol t) = \prod_{i=1}^k\phi_{X_i}(t_i)$

Note that the above statement does not hold if the distinction property among the $t_i$’s does not exist, that means, the condition that
$\phi_{\boldsymbol x} (t\boldsymbol 1) = \prod_{i=1}^k\phi_{X_i}(t) \quad \text{ does not imply that}\quad X_i’s\quad\text{ are all independent}$
The reason is apparent, think about the case $t=0$.

Probability Note 2010 Oct 10 (1)

October 10, 2010

For $X\sim t(\nu)$,
$E(X^k) = \int_{\mathbb R^1}\frac{\Gamma\left(\frac{\nu+1}{2}\right)}{\Gamma\left(\frac{\nu+1}{2}\right)\sqrt{\pi\nu}}x^k\left(\frac{\nu}{\nu+x^2}\right)^{\frac{\nu+1}{2}} dx$
$= \int_{\mathbb R^1}\frac{\Gamma\left(\frac{\nu+1}{2}\right)}{\Gamma\left(\frac{\nu+1}{2}\right)\sqrt{\pi\nu}}\left(\frac{x^2}{\nu+x^2}\right)^{\frac{k}{2}} \left(\frac{\nu}{\nu+x^2}\right)^{\frac{\nu-k+1}{2}} v^{\frac{k}{2}}dx$
$= \int_0^1\frac{\Gamma\left(\frac{\nu+1}{2}\right)\nu^{\frac{k}{2}}}{\Gamma\left(\frac{\nu+1}{2}\right)\sqrt{\pi\nu}}\left[\frac{1}{2}\nu^{\frac{1}{2}}y^{-\frac{1}{2}}(1-y)^{-\frac{1}{2}} + \frac{1}{2}\nu^{\frac{1}{2}}y^{\frac{1}{2}}(1-y)^{-\frac{3}{2}} \right]y^{\frac{k}{2}}(1-y)^{\frac{\nu+1-k}{2}}dy$
$= \frac{\Gamma\left(\frac{\nu+1}{2}\right)\nu^{\frac{k}{2}}}{\Gamma\left(\frac{\nu+1}{2}\right)2\sqrt{\pi}}\left[\int_0^1y^{\frac{k+1}{2}-1}(1-y)^{\frac{\nu-k+2}{2}-1} dy + \int_0^1 y^{\frac{k+3}{2}-1}(1-y)^{\frac{\nu-k}{2}-1}dy\right]$
$= \frac{\Gamma\left(\frac{\nu+1}{2}\right)\nu^{\frac{k}{2}}}{\Gamma\left(\frac{\nu+1}{2}\right)\Gamma\left(\frac{\nu+3}{2}\right)2\sqrt{\pi}}\left[\Gamma\left(\frac{k+1}{2}\right)\Gamma\left(\frac{\nu-k+2}{2}\right) + \Gamma\left(\frac{k+3}{2}\right)\Gamma\left(\frac{\nu-k}{2}\right)\right]$

Thus, for $k > \nu$, the moment does not exist.

Probability note 2010 Oct 09 -(3)

October 10, 2010

If $X_n\overset{\mathcal D}{\rightarrow}X$, and $Y_n\overset{\mathcal D}{\rightarrow}Y$, where $X_n\amalg X_n$ for all $n\in\mathbb N$, $X\amalg Y$, then

$\lim_{n\rightarrow\infty}\phi_{X_n+Y_n}(t) = \lim_{n\rightarrow\infty}\phi_{X_n}(t)\phi_{Y_n}(t) = \lim_{n\rightarrow\infty}\phi_{X_n}(t)\cdot\lim_{n\rightarrow\infty}\phi_{Y_n}(t) = \phi_{X}(t)\phi_{Y}(t) = \phi_{X+Y}(t) ,$
thus, $X_n+Y_n\overset{\mathcal D}{\rightarrow}X+Y$.

Probability note 2010 Oct 09 -(2)

October 10, 2010

Flip over the setting of the previous one post: if $X_n\overset{\mathcal D}{\rightarrow}X$, we want to show that $\mu_n\rightarrow\mu$ and $\sigma_n^2\rightarrow\sigma^2$.

Again, play with Lévy’s theorem, since we have $\lim_{n\rightarrow\infty}\phi_n(t) = \phi(t)$, thus, $\lim_{n\rightarrow\infty}\exp(\mu_n t -\frac{\sigma_n^2 t^2}{2}) = \exp(\mu t -\frac{\sigma^2 t^2}{2})$ for all $t$. Thus, we have $\lim_{n\rightarrow\infty}\mu_n = \mu$ and $\lim_{n\rightarrow\infty}\sigma_n^2 = \sigma^2$.

Probability note 2010 Oct 09 -(1)

October 10, 2010

If $(X_n)_{n\geq 1}\sim\mathscr N (\mu_n, \sigma_n^2)$ and $\mu_n\rightarrow\mu\in\mathbb R$, $\sigma_n^2\rightarrow\sigma^2\geq 0$, then by Lévy’s theorem, then we have the probability measure of $X_n$ convergent to $\mathscr N(\mu, \sigma^2)$, thus, $\phi_n(t)=\exp(\mu_nt-\frac{\sigma_n^2t^2}{2})\rightarrow \phi (t)=\exp(\mu t-\frac{\sigma^2t^2}{2})$