## Bayesian posterior distribution 20101125-1

### November 25, 2010

In Bayesian hierarchical setting:

\[

\left.\boldsymbol x_i\right|\boldsymbol\mu, \mathbf\Sigma\sim\mathscr N(\boldsymbol\mu, \mathbf\Sigma)

\]

$i=1,2,\ldots,n$, where

\[

\left.\boldsymbol\mu\right|\mathbf\Sigma\sim\mathscr N\left(\boldsymbol\mu_0, \frac{1}{\kappa_0}\mathbf\Sigma\right)

\]

and

\[

\mathbf\Sigma\sim\mathscr{IW}(\nu_0, \mathbf\Sigma_0)

\]

and, most statisticians can tell me that

\[

\left.\boldsymbol\mu\right|\boldsymbol x_1,\boldsymbol x_2,\ldots, \boldsymbol x_n,\mathbf\Sigma\sim\mathscr N\left(\frac{\kappa_0}{n+\kappa_0}\boldsymbol\mu_0+\frac{n}{n+\kappa_0}\overline{\boldsymbol x}, \frac{1}{\kappa_0}\mathbf\Sigma\right)

\]

where $\frac{1}{n}\sum_{i=1}^n\boldsymbol x_i$ and

\[

\left.\mathbf\Sigma\right|\boldsymbol x_1,\boldsymbol x_2,\ldots, \boldsymbol x_n\sim\mathscr{IW}\left(n+\nu_0, \mathbf\Sigma_0 + \sum_{i=1}^n(\boldsymbol x_i-\overline{\boldsymbol x})(\boldsymbol x_i-\overline{\boldsymbol x})^\top + \frac{\kappa_0n}{\kappa_0+n}(\boldsymbol x_i-\boldsymbol\mu_0)(\boldsymbol x_i-\boldsymbol\mu_0)^\top\right)

\]

right away, but, if we set

\[

\left.\boldsymbol\mu\right|\mathbf\Sigma\sim\mathscr N\left(\boldsymbol\mu_0,\mathbf{A\Sigma A}^\top\right)

\]

can we get such a beautiful posterior form?

## MathML stretch symbol Test

### October 25, 2010

E

(

X

k

)

=

∫

R

1

Γ

ν

+

1

2

Γ

ν

+

1

2

π

ν

x

k

ν

ν

+

x

2

ν

+

1

2

d

x

Nothing selected

## Probabilty note 2010 Oct 13 (1)

### October 13, 2010

Consider a collection of random variables $\boldsymbol x = (x_1, x_2, \ldots, x_k)^T$, if $\boldsymbol x$ have a joint characteristic function

$\phi_{\boldsymbol x} (\boldsymbol t)$, where $\boldsymbol t = (t_1,t_2,\ldots, t_k)^T$ are all distinct, then we can conclude that $x_i, x_j$ are independent for all $i\neq j$ if and only if

\[

\phi_{\boldsymbol x} (\boldsymbol t) = \prod_{i=1}^k\phi_{X_i}(t_i)

\]

Note that the above statement does not hold if the distinction property among the $t_i$’s does not exist, that means, the condition that

\[

\phi_{\boldsymbol x} (t\boldsymbol 1) = \prod_{i=1}^k\phi_{X_i}(t)

\quad

\text{ does not imply that}\quad X_i’s\quad\text{ are all independent}

\]

The reason is apparent, think about the case $t=0$.

## Probability Note 2010 Oct 10 (1)

### October 10, 2010

For $X\sim t(\nu)$,

\[

E(X^k) = \int_{\mathbb R^1}\frac{\Gamma\left(\frac{\nu+1}{2}\right)}{\Gamma\left(\frac{\nu+1}{2}\right)\sqrt{\pi\nu}}x^k\left(\frac{\nu}{\nu+x^2}\right)^{\frac{\nu+1}{2}}

dx

\]

\[

= \int_{\mathbb R^1}\frac{\Gamma\left(\frac{\nu+1}{2}\right)}{\Gamma\left(\frac{\nu+1}{2}\right)\sqrt{\pi\nu}}\left(\frac{x^2}{\nu+x^2}\right)^{\frac{k}{2}} \left(\frac{\nu}{\nu+x^2}\right)^{\frac{\nu-k+1}{2}}

v^{\frac{k}{2}}dx

\]

\[

= \int_0^1\frac{\Gamma\left(\frac{\nu+1}{2}\right)\nu^{\frac{k}{2}}}{\Gamma\left(\frac{\nu+1}{2}\right)\sqrt{\pi\nu}}\left[\frac{1}{2}\nu^{\frac{1}{2}}y^{-\frac{1}{2}}(1-y)^{-\frac{1}{2}} + \frac{1}{2}\nu^{\frac{1}{2}}y^{\frac{1}{2}}(1-y)^{-\frac{3}{2}} \right]y^{\frac{k}{2}}(1-y)^{\frac{\nu+1-k}{2}}dy

\]

\[

= \frac{\Gamma\left(\frac{\nu+1}{2}\right)\nu^{\frac{k}{2}}}{\Gamma\left(\frac{\nu+1}{2}\right)2\sqrt{\pi}}\left[\int_0^1y^{\frac{k+1}{2}-1}(1-y)^{\frac{\nu-k+2}{2}-1} dy + \int_0^1 y^{\frac{k+3}{2}-1}(1-y)^{\frac{\nu-k}{2}-1}dy\right]

\]

\[

= \frac{\Gamma\left(\frac{\nu+1}{2}\right)\nu^{\frac{k}{2}}}{\Gamma\left(\frac{\nu+1}{2}\right)\Gamma\left(\frac{\nu+3}{2}\right)2\sqrt{\pi}}\left[\Gamma\left(\frac{k+1}{2}\right)\Gamma\left(\frac{\nu-k+2}{2}\right) + \Gamma\left(\frac{k+3}{2}\right)\Gamma\left(\frac{\nu-k}{2}\right)\right]

\]

Thus, for $k > \nu$, the moment does not exist.

## Probability note 2010 Oct 09 -(3)

### October 10, 2010

If $X_n\overset{\mathcal D}{\rightarrow}X$, and $Y_n\overset{\mathcal D}{\rightarrow}Y$, where $X_n\amalg X_n$ for all $n\in\mathbb N$, $X\amalg Y$, then

\[

\lim_{n\rightarrow\infty}\phi_{X_n+Y_n}(t) = \lim_{n\rightarrow\infty}\phi_{X_n}(t)\phi_{Y_n}(t) = \lim_{n\rightarrow\infty}\phi_{X_n}(t)\cdot\lim_{n\rightarrow\infty}\phi_{Y_n}(t) = \phi_{X}(t)\phi_{Y}(t) = \phi_{X+Y}(t)

,\]

thus, $X_n+Y_n\overset{\mathcal D}{\rightarrow}X+Y$.

## Probability note 2010 Oct 09 -(2)

### October 10, 2010

Flip over the setting of the previous one post: if $X_n\overset{\mathcal D}{\rightarrow}X$, we want to show that $\mu_n\rightarrow\mu$ and $\sigma_n^2\rightarrow\sigma^2$.

Again, play with Lévy’s theorem, since we have $\lim_{n\rightarrow\infty}\phi_n(t) = \phi(t)$, thus, $\lim_{n\rightarrow\infty}\exp(\mu_n t -\frac{\sigma_n^2 t^2}{2}) = \exp(\mu t -\frac{\sigma^2 t^2}{2})$ for all $t$. Thus, we have $\lim_{n\rightarrow\infty}\mu_n = \mu$ and $\lim_{n\rightarrow\infty}\sigma_n^2 = \sigma^2$.

## Probability note 2010 Oct 09 -(1)

### October 10, 2010

If $(X_n)_{n\geq 1}\sim\mathscr N (\mu_n, \sigma_n^2)$ and $\mu_n\rightarrow\mu\in\mathbb R$, $\sigma_n^2\rightarrow\sigma^2\geq 0$, then by Lévy’s theorem, then we have the probability measure of $X_n$ convergent to $\mathscr N(\mu, \sigma^2)$, thus, $\phi_n(t)=\exp(\mu_nt-\frac{\sigma_n^2t^2}{2})\rightarrow \phi (t)=\exp(\mu t-\frac{\sigma^2t^2}{2})$