Bernoulli and Poisson models

Posted on by rhohahn

Today in class we covered the Beta-Binomial model. That is, we considered a model where for i = 1, \dots, n, Y_i \sim \mbox{Bernoulli}(\theta) independently and \theta \sim \mbox{Beta}(a,b). For this model, the total number of successes, or S = \sum_i Y_i, is a sufficient statistic for the parameter \theta and S \sim \mbox{Binomial}(n, \theta).

The choice of a beta distribution for prior \pi(\theta) has the nice property that it is conjugate to the Bernoulli sampling model, meaning that the posterior will again be in the beta family of distributions. Specifically, we showed that the posterior distribution will be \theta \mid y_{1:n} \sim \mbox{Beta}(a + s, b + n - s). We will explore several other conjugate model-prior pairs in the coming weeks. See here for a useful reference on conjugate priors. The idea of conjugate priors for Bayesian analysis was developed by Raiffa and Schlaifer to make Bayesian inference more accessible to their business students at Harvard. To foreshadow, the theory behind conjugacy is intimately related to the idea of an exponential family.

 

Leave a comment