How to calculate the expected value of the logarithm of Gamma distribution?

Posted on Edited on In

Note: This section contains notes on the derivations made while authoring my book. The derivation details originate from public Q&A platforms and will naturally differ from those in the book chapters.

The goal is to calculate the expected value of the logarithm of gamma distribution, which is \[ E[lnX] \] Where \(X\sim Gamma(\alpha,\beta)\)

Part I

To deal with the problem, we need to first figure out how to calculate the density of the transformed variable, which is the density \(f_Y(y)\) when \(Y=g(X)\).

Suppose \(Y=g(X)\), \(X\) is a random variable on probability space \((\Omega,\mathcal{F},\mathcal{P})\), and has continuous density \(f\).

\(\Rightarrow Y=g(X)\) has the distribution function \(F(g^{-1}(y))\).

Taking derivatives via the chain rule, we have \[ \begin{gather*} f_Y(y)=\frac{dF(g^{-1}(y))}{dy}\\=\frac{dF(g^{-1}(y))}{dx}\frac{dx}{dy}\\ =\frac{dF(g^{-1}(y))}{dx}\frac{d[g^{-1}(y)]}{dy}\\ =f(g^{-1}(y))\frac{d[g^{-1}(y)]}{dy} \end{gather*} \] Recall that the derivative of an inverse function is \[ \begin{gather*} \frac{d}{dy}[g^{-1}(y)]\\ =\frac{dx}{d[g(g^{-1}(y))]}\\ =\frac{1}{g'(g^{-1}(y))} \end{gather*} \] The density function of \(g(X)\) is then expressed as \[ \begin{gather*} f_Y(y)=f(g^{-1}(y))\frac{1}{g'(g^{-1}(y))}\\ =\frac{f(g^{-1}(y))}{g'(g^{-1}(y))} \end{gather*} \]

Part II

Now that we have figured out how to calculate the density of the transformed variable, we can proceed to obtain the density of \(Y=lnX\).

Since \(X\sim \Gamma(\alpha,\beta)\), \(Y=g(X)=ln(X)\), we have \(X=e^y\). Because \(\beta\) is a scale parameter, its effect will be to shift the logarithm by \(log\beta\). Specifically, it will shift the logarithm by \(-log\beta\). So we can readily start with the case when \(\beta=1\).

That gives us the distribution of \(X\) as \[ f_X(x)=\frac{1}{\Gamma(\alpha)}x^{\alpha}e^{-x}\frac{1}{x} \] Where \(\Gamma(\alpha)=\int_0^{\infty}x^{\alpha}e^{-x}\frac{dx}{x}\) is the normalizing constant.

Substituting \(x=e^y\) entails \(dy=\frac{dx}{x}\) as \(\frac{de^y}{dy}=e^y=x\), thus we have \[ \begin{gather*} f_Y(y)=\frac{f(g^{-1}(y))}{g'(g^{-1}(y))}\\ =\frac{1}{\Gamma(\alpha)}e^{\alpha y}e^{-e^y}\frac{1}{e^y}/(\frac{1}{e^y})\\ =\frac{1}{\Gamma(\alpha)}e^{\alpha y-e^y} \end{gather*} \] The possible values of \(Y\) now range over all the real numbers \(\mathbb{R}\).

Part III

As \(f_Y\) must integrate to unity, as \[ \begin{gather*} \int_{\mathbb{R}}f_Y(y)dy=\int_{\mathbb{R}}\frac{1}{\Gamma(\alpha)}e^{\alpha y-e^y}dy\\ =\frac{1}{\Gamma(\alpha)}\int_{\mathbb{R}}e^{\alpha y-e^y}dy\\ =1 \end{gather*} \] we can obtain \[ \Gamma(\alpha)=\int_{\mathbb{R}}e^{\alpha y-e^y}dy \] To calculate \(E(Y)=\int_{\mathbb{R}}yf_Y(y)dy\), we need to replace \(y\) with \(\alpha\) in order to eliminate the uncertainty.

To do so, we notice that \(f_Y(y)\) is a differentiable function of \(\alpha\), as \[ \frac{d}{d\alpha}e^{\alpha y-e^y}=ye^{\alpha y-e^y}\\ =y\Gamma(\alpha)f_Y(y)\\ \] Since we can obtain \(e^{\alpha y-e^y}=\Gamma(\alpha)f_Y(y)\) from \[ f_Y(y)=\frac{1}{\Gamma(\alpha)}e^{\alpha y-e^y} \] Now that we can calculate the \(E(Y)\), as \[ \begin{gather*} E(Y)=\int_{\mathbb{R}}yf_Y(y)dy\\ =\frac{1}{\Gamma(\alpha)}\int_{\mathbb{R}}\Gamma(\alpha)yf_Y(y)dy\\ =\frac{1}{\Gamma(\alpha)}\int_{\mathbb{R}}\frac{d}{d\alpha}e^{\alpha y-e^y}dy\\ =\frac{1}{\Gamma(\alpha)}\frac{d}{d\alpha}\int_{\mathbb{R}}e^{\alpha y-e^y}dy\\ =\frac{1}{\Gamma(\alpha)}\frac{d}{d\alpha}\Gamma(\alpha)\\ =\frac{d}{d\alpha}ln\Gamma(\alpha)\\ =\Psi(\alpha) \end{gather*} \] \(\Psi(\cdot)\) is the digamma function, which is the logarithmic derivative of the gamma function.

By restoring the scale parameter \(\beta\) value, we can obtain the general result of \(E[lnX]\) as \[ E[lnX]=\Psi(\alpha)-ln\beta \]

Reference

How to calculate the density function of \(g(X)\)?

What is the expected value of the logarithm of Gamma distribution?