Final write-ups due date

Please turn in your final write-ups by noon this Friday, May 4th. Thanks to everyone for a great semester.

Final project hint: Kronecker product

At one point in the regression + factor model sampler, you will need to perform an update step corresponding to a linear regression model, except the left hand side response variable will be in matrix form. In order to use the existing "machinery" to arrive at your update, you will need to reshape the matrix…

Stochastic search variable selection

The topic of today's post is Bayesian "variable selection" using point-mass mixture priors. This builds of off the previous post concretely, adapting the ideas to the linear regression setting. The key reference for this approach to variable selection is George and McCulloch; see also the literature review of Hahn and Carvalho. The model is simply…

Mixtures of conjugate priors

Conjugate models (likelihood-prior pairs) refer to parametric Bayesian models where the posterior distribution is expressible in the same parametric form as the prior. Conjugate models can be given a deep theoretical characterizations; see, for example, here. The origins of the work, however, can be found in the textbook of Raiffa and Schlaifer, who invented it as a…

Final projects

1) Install and load the bayesm package in R. Load the Scotch data using the command data(Scotch). This data consists of Yes/No survey answers from 2,218 individuals reporting which of 20 Scotch whiskey brands (and one "Other" category) they have bought in the past year. To analyze this data, first discard the variable corresponding to…

Bayesian probit regression

Regression analysis for dichotomous (binary) data usually proceeds by specifying a link function that maps a linear model on the real line back to the unit interval. The most common link function is the logit link, leading to logistic regression. The likelihood function in this case is simply a Bernoulli likelihood for each data point.…

Gaussian factor models

By a Gaussian factor model, I refer to the following specification: $latex X_i = \mathbf{B}f_i + \epsilon_i; \epsilon_i \sim N(0, \Psi); f_i \sim N(0, I_k).&s=1$ Each observation $latex X_i$ is a p-dimensional column vector, $latex \mathbf{B}$ is a p-by-k real-valued matrix of "factor loadings", and the "factor scores" $latex f_i$ are k-dimensional column vectors. The…

Monte Carlo and MCMC slides

I believe that one of these was never officially posted. Here are the two slide decks covering the core computational material. Monte Carlo slides MCMC slides

Problem set

Here is a problem set, to be completed in one sitting like an exam, to help you judge your understanding of some key topics we've covered. We will go over the answers in class next week.

Protecting against underflow when using Bayes rule

If we have data that we assume has arisen via an i.i.d. parametric model and a discrete set of $latex K&s=1$ values that parameter can take, then Bayes rule has the form $latex \pi(\theta \mid y_{1:n}) = c^{-1} \pi(\theta) \prod_{i = 1}^n f(y_i \mid \theta),&s=1$ where $latex c = \sum_{k = 1}^K \pi(\theta_k) \prod_{i =…