Faithfulness and causal discovery

Causal discovery refers to the process of inferring an underlying causal graph from data. To do this, we need to make an assumption called "faithfulness". From Shalizi's book: The joint distribution has all of the conditional independence relations implied by the causal Markov property, and only those conditional independence relations. The point of the faithfulness…

Unmeasured confounder bias

Today we take a look at the classic linear regression model and observe the well-known phenomenon that regression coefficient estimates can be biased if relevant "confounding" variables are not included in the regression. We will revisit this leading example many times during the course of the semester, both to reinforce ideas and to critique the…

Final write-ups due date

Please turn in your final write-ups by noon this Friday, May 4th. Thanks to everyone for a great semester.

Final project hint: Kronecker product

At one point in the regression + factor model sampler, you will need to perform an update step corresponding to a linear regression model, except the left hand side response variable will be in matrix form. In order to use the existing "machinery" to arrive at your update, you will need to reshape the matrix…

Stochastic search variable selection

The topic of today's post is Bayesian "variable selection" using point-mass mixture priors. This builds of off the previous post concretely, adapting the ideas to the linear regression setting. The key reference for this approach to variable selection is George and McCulloch; see also the literature review of Hahn and Carvalho. The model is simply…

Mixtures of conjugate priors

Conjugate models (likelihood-prior pairs) refer to parametric Bayesian models where the posterior distribution is expressible in the same parametric form as the prior. Conjugate models can be given a deep theoretical characterizations; see, for example, here. The origins of the work, however, can be found in the textbook of Raiffa and Schlaifer, who invented it as a…

Final projects

1) Install and load the bayesm package in R. Load the Scotch data using the command data(Scotch). This data consists of Yes/No survey answers from 2,218 individuals reporting which of 20 Scotch whiskey brands (and one "Other" category) they have bought in the past year. To analyze this data, first discard the variable corresponding to…

Bayesian probit regression

Regression analysis for dichotomous (binary) data usually proceeds by specifying a link function that maps a linear model on the real line back to the unit interval. The most common link function is the logit link, leading to logistic regression. The likelihood function in this case is simply a Bernoulli likelihood for each data point.…

Gaussian factor models

By a Gaussian factor model, I refer to the following specification: $latex X_i = \mathbf{B}f_i + \epsilon_i; \epsilon_i \sim N(0, \Psi); f_i \sim N(0, I_k).&s=1$ Each observation $latex X_i$ is a p-dimensional column vector, $latex \mathbf{B}$ is a p-by-k real-valued matrix of "factor loadings", and the "factor scores" $latex f_i$ are k-dimensional column vectors. The…

Final presentations

Here is a list of possible papers that one can present for the final presentation. Groups of 2 to 3 people are allowed. Presentations should be approximately 20 minutes long. We will have presentations on the 18th, 23rd and 25th (the final three days of class). Treatment effect estimation with imperfect instruments Rules for determining…