Bạn đang xem: Dirichlet distribution: simple definition, pdf, mean
Since the likelihood function is usually defined from the data generating process, the difference choices of prior can make the integral more or less difficult to calculate. If the prior has the same algebraic khung as the likelihood, then often we can obtain a closed-form expression for the posterior, avoiding the need of numerical integration.
Motivating the Dirichlet Distribution
Here’s how the Dirichlet distribution can be used khổng lồ characterize the random variability of a multinomial distribution. I’ve borrowed this example from a great blog post on visualizing the Dirichlet distribution.
Suppose we’re going khổng lồ manufacture six-sided dice but allow the outcomes of a toss khổng lồ be only one, two or three (so the later visualization is easier). If the die is fair then the probabilities of the three outcomes will be the same và equal to lớn 1/3. We can represent the probabilities for the outcomes as a vector θ =(θ₁, θ₂, θ₃).
θ has two important properties: First, the sum of the probabilities for each entry must equal one, and none of the probabilities can be negative. When these conditions hold, we can use a multinomial distribution to describe the results associated with rolling the die.
In other words, if we observe n dice rolls, D=x₁,…,x_k, then the likelihood function has the form:
Where N_k is the number of times the valuek∈1, 2, 3 has occurred.
We expect there will be some variability in the characteristics of the dice we produce, so even if we try khổng lồ produce fair dice, we won’t expect the probabilities of each outcome for a particular die will be exactly 1/3, due to variability in the production process. Khổng lồ characterize this variability mathematically, we would like to know the probability density of every possible value of θ for a given manufacturing process. To bởi vì this, let’s consider each element of θ as being an independent variable.
That is, for θ =(θ₁, θ₂, θ₃), we can treat θ₁, θ₂ & θ₃ each as an independent variable. Since the multinomial distribution requires that these three variables sum lớn one, we know that the allowable values of θ are confined lớn a plane. Furthermore, since each value θᵢ must be greater than or equal lớn zero, the set of all allowable values of θ is confined to lớn a triangle.
What we want khổng lồ know is the probability mật độ trùng lặp từ khóa at each point on this triangle. This is where the Dirichlet distribution can help us: We can use it as the prior for the multinomial distribution.
The Dirichlet distribution defines a probability mật độ trùng lặp từ khóa for a vector valued input đầu vào having the same characteristics as our multinomial parameter θ. It has tư vấn (the mix of points where it has non-zero values) over
K is the number of variables. Its probability density function has the following form:
The Dirichlet distribution is parameterized by the vector α, which has the same number of elements K as our multinomial parameter θ. So you can interpret p(θ|α) as the answer to the question “what is the probability density associated with multinomial distribution θ, given that our Dirichlet distribution has parameter α?”
Get More From Built In’s Data Science ExpertsThe Poisson Process và Poisson Distribution, Explained (With Meteors!)
Visualizing the Dirichlet Distribution
We see the Dirichlet distribution indeed has the same form as the multinomial likelihood distribution. But what does it actually look like?
To see this, we need to chú ý that the Dirichlet distribution is the multivariate generalization of the beta distribution. The beta distribution is defined on the interval <0, 1> parameterized by two positive shape parameters α & β. As you might expect, it is the conjugate prior of the binomial (including Bernoulli) distribution. The figure shows the probability density function for the Beta distribution with a few α & β values.
As we can see, the beta density function can take a wide variety of different shapes depending on α and β. When both α & β are less than one, the distribution is U-shaped. In the limit of α = β → 0, it is a two point Bernoulli distribution with equal probability 1/2 at each Dirac delta function ends x=0 và x=1, and zero probability everywhere else. When α=β=1 we have the uniform <0, 1> distribution, which is the distribution with the largest entropy. When both α và β are greater than one the distribution is unimodal. This diversity of shapes by varying only two parameters makes it particularly useful for modeling actual measurements.
For the Dirichlet distribution Dir(α) we generalize these shapes lớn a K simplex. For K=3, visualizing the distribution requires us to bởi vì the following:Generate a phối of x-y coordinates over our triangle Map the x-y coordinates lớn the two-simplex coordinate spaceCompute Dir(α) for each point
Below are some examples, you can find the code in my Github repository.
We see it’s now the parameter α that governs the shapes of the distribution. In particular, the sum α₀=∑αᵢ controls the strength of the distribution (how peaked it is). If αᵢ for all i, we get spikes at the corners of the simplex. For values of αᵢ > 1, the distribution tends toward the centre of the simplex. As α₀ increases, the distribution becomes more tightly concentrated around the centre of the simplex.
Xem thêm: Gọi Điện Thoại Miễn Phí Từ Máy Tính Đến Mobile Tiện Ích Nhất Hiện Nay
In the context of our original dice experiment, we would produce consistently fair dice as αᵢ → ∞. For a symmetric Dirichlet distribution with αᵢ > 1, we will produce fair dice, on average. If the goal is khổng lồ produce loaded dice (e.g., with a higher probability of rolling a three), we would want an asymmetric Dirichlet distribution with a higher value for α₃.
Built In Machine Learning TutorialsA Step-by-Step NLP Machine Learning Classifier Tutorial
Now you"ve seen what the Dirichlet distribution looks like, & the implications of using it as a prior for a multinomial likelihood function in the context of dice manufacturing.