Sensitivity of Posterior Distribution to Hyperparameters
Problem Setup
Lets say that we would like to estimate $\mathbf{\theta} \in \mathbb{R}^i$ given ${\mathbf{X}}_{j=1}^J$. When conducting this analysis within a Bayesian framework, we generally inject subjective into the analysis through the definition of prior and likelihood functions, which are simplifications of the true model. This subjectivity takes the form of hyperparameters, $\alpha \in \mathbf{A} \subseteq \mathbb{R}^k$, which affects the posterior distribution as defined below.
Utilizing the provided posterior distribution, we’re generally interested in calculating the expectation of some function ( $g(\theta)$ ,e.g., the mean ), $\mathbb{E}_{p\alpha}[g(\theta)]$. When conducting this calculation, we would to have a measure of the amount of influence that a permutation of $\alpha$ has on the expectation (i.e., how robust is the posterior distribution to changes in the provided hyperparamters).
Defining Sensitivity
An intuitive way to calculate the global sensitivity of our estimator to variation in $\alpha$ would be to calculate the extrema of $\mathbb{E}_{p\alpha}[g(\theta)]$ over all $\alpha \in \mathbf{A}$; however, this is intractable in general. So, instead, we can calculate the local sensitivity around an origin, $\alpha_o \in \mathbf{A}$ to small variations in $\alpha$, as defined below.
This local measure of sensitivity can be extended to an approximate global measure (to the first order) as provided below.
Next Steps
- Start working on software implementation – Will conduct analysis on sensitivity of clustering to Wishart hyperparamters.