distributions | Chen Xing

Beta Distribution — Intuition, Derivation, and Examples

Wed, 03 Aug 2022 00:00:00 +0000

Motivation

Model probabilities

The Beta distribution is a probability distribution on probabilities.

The Beta distribution can be understood as representing a distribution of probabilities, that is, it represents all the possible values of a probability when we don’t know what that probability is. For example,

the Click-Through Rate of your advertisement
the conversion rate of customers actually purchasing in your store
how likely the customer will become “inactive”

Because the Beta distribution models a probability, its domain is bounded between 0 and 1.

Generalization of Uniform Distribution

Give me a continuous and bounded random variable except the Uniform Distribution. This is another way to look at beta distribution, continuous and bounded between 0 and 1; also the density is not flat.

$$ X \sim Beta(a, b), \text{ where } a>0, \ b>0. $$ $$ f_X(x) = c \cdot x ^{a-1}(1-x)^{b-1}, \text{ where } x>0. $$

What is $c$ ? Just a normalization constant! We’ll find the value of $c$ later.

Conjugate Prior

The Beta distribution is the conjugate prior for the Bernoulli, binomial, negative binomial and geometric distributions (seems like those are the distributions that involve success & failure) in Bayesian inference.

Computing a posterior using a conjugate prior is very convenient, because you can avoid expensive numerical computation involved in Bayesian Inference.

Conjugate prior = Convenient prior

For example, the beta distribution is a conjugate prior to the binomial. If we choose to use the beta distribution Beta(α, β) as a prior, during the modeling phase, we already know the posterior will also be a beta distribution. Therefore, after carrying out more experiments, you can compute the posterior simply by adding the number of successes (x), and failures (n-x) to the existing parameters α, β respectively, instead of multiplying the likelihood with the prior distribution. The posterior also becomes a Beta distribution with parameters (x+α, n-x+β).

What is the Intuition?

The intuition for the beta distribution comes into play when we look at it from the lens of the binomial distribution.

The difference between the binomial and the beta is that the former models the number of successes (x), while the latter models the probability (p) of success.

In other words, the probability is a parameter in binomial; In the Beta, the probability is a random variable.

Interpretation of α, β

You can think of α-1 as the number of successes and β-1 as the number of failures, just like n & n-x terms in binomial.

You can choose the α and β parameters however you think they are supposed to be.

If you think the probability of success is very high, let’s say 90%, set 90 for α and 10 for β.
If you think otherwise, 90 for β and 10 for α.

As α becomes larger (more successful events), the bulk of the probability distribution will shift towards the right, whereas an increase in β moves the distribution towards the left (more failures).

Also, the distribution will narrow if both α and β increase, for we are more certain.

Dr. Bognar at the University of Iowa built the calculator for Beta distribution, which I found useful and beautiful. You can experiment with different values of α and β and visualize how the shape changes.

Derivation

In this section, we’ll derive Beta distribution using the Beta-Gamma Connections.

We can regard Beta distribution as the fraction of waiting time.

Fraction of Waiting Time

Let $X$ be the waiting time at Bank,

$$ X \sim Gamma(n_1, \lambda) $$

Let $Y$ be the waiting time at Post Office,

$$ Y \sim Gamma(n_2, \lambda) $$

Assume $X$ and $Y$ are independent. What is the distribution of the proportion $\frac{X}{X+Y}$ ?

Solution:

Let $T := X+Y$ be the total waiting time. Clearly, $T \sim Gamma(n_1+n_2, \lambda)$, you can prove it by MGF.

Let $W =: \frac{X}{X+Y}$ be the proportion of waiting time at Bank to the total waiting time. We need to find the PDF of $W$.

The idea is to find the joint PDF $f_{T,W}(t,w)$ at first, and then get the marginal distribution.

$$ \begin{aligned}f_{T,W}(t,w) &= f_{X,Y}(x,y) \left | \frac{\partial(x,y)}{\partial(t,w)} \right|\\ &= \frac{1}{\Gamma(n_1)}\lambda^{n_1}x^{n_1 - 1}e^{-\lambda x} \frac{1}{\Gamma(n_2)}\lambda^{n_2}x^{n_2 - 1}e^{-\lambda y} \left|-t\right|\\ &= \lambda^{n_1+n_2}t^{n_1+n_2-1}e^{-\lambda t} \frac{1}{\Gamma(n_1)\Gamma(n_2)}w^{n_1 - 1}(1-w)^{n_2-1}\\ &= \frac{\lambda^{n_1+n_2}t^{n_1+n_2-1}e^{-\lambda t}}{\Gamma(n_1+n_2)} \frac{\Gamma(n_1+n_2)}{\Gamma(n_1)\Gamma(n_2)}w^{n_1 - 1}(1-w)^{n_2-1}\\ &= f_T(t) \frac{\Gamma(n_1+n_2)}{\Gamma(n_1)\Gamma(n_2)}w^{n_1 - 1}(1-w)^{n_2-1}\end{aligned} $$

Integrating $t$ out to get the marginal:

$$ \begin{aligned}f_W(w) &= \int_0^\infty f_{T,W}(t,w) dt \\&= \frac{\Gamma(n_1+n_2)}{\Gamma(n_1)\Gamma(n_2)}w^{n_1 - 1}(1-w)^{n_2-1} \cdot\int_0^\infty f_T(t)dt \\&= \frac{\Gamma(n_1+n_2)}{\Gamma(n_1)\Gamma(n_2)}w^{n_1 - 1}(1-w)^{n_2-1} \end{aligned} $$

Here we have Beta,

$$W \sim Beta(n_1, n_2)$$ REMARK: the above result also proves W and T and independent!

Beta Function as a normalizing constant

Note that, $f_W(w)$ is a PDF needed to be integrated to 1,

$$ \int_0^1\frac{\Gamma(n_1+n_2)}{\Gamma(n_1)\Gamma(n_2)}w^{n_1 - 1}(1-w)^{n_2-1} dw \equiv 1 $$

So the normalization constant should be,

$$ c = \frac{\Gamma(n_1+n_2)}{\Gamma(n_1)\Gamma(n_2)} := \frac{1}{B(n_1, n_2)} $$

Mean of Beta distribution

As a byproduct in the above derivation, we get that fact that W and T are independent. Then, we can use this to derive the mean of Beta distribution,

$$E(WT) = E(W)E(T) \implies E(X) =E\left(\frac{X}{X+Y}\right)E(X+Y),$$

rearrange to get,

$$E\left(\frac{X}{X+Y}\right) = \frac{E(X)}{E(X+Y)}$$

This result is clear Not True in general, but under our setting, we have this interesting result.

We can use this result to find the mean of $W \sim Beta(a, b)$ without the slightest trace of calculus.

$$E(W)=E\left(\frac{X}{X+Y}\right)=\frac{E(X)}{E(X+Y)}=\frac{a / \lambda}{a / \lambda+b / \lambda}=\frac{a}{a+b}$$

Getting Beta parameters in practice

The Beta distribution is the conjugate prior for many common distributions. We use it a lot. But in practice, how to figure out its parameters? It is sometimes useful to estimate quickly the parameters of the Beta distribution using the method of moments:

$$X \sim Beta(\alpha, \beta),$$ $$\alpha + \beta = \frac{E(X)(1-E(X))}{Var(X)} - 1,$$ $$\alpha = (\alpha + \beta)E(X),$$ $$\beta = (\alpha + \beta)(1 - E(X))$$

Here is the R code:

# calculate beta params using method of moments
cal_beta_params <- function(meanX, sdX) {
  varX <- sdX^2
  sum_ab <- meanX * (1 - meanX) / varX - 1
  a <- sum_ab * meanX
  b <- sum_ab * (1 - meanX)
  # return
  c("shape" = a, "scale" = b)
}

# example
cal_beta_params(meanX = 0.136, sdX = 0.103)

##    shape    scale 
## 1.370320 8.705559

Summary

To summarize, the bank–post office story tells us that: when we add independent Gamma r.v.s $X$ and $Y$ with the same rate $\lambda$ ,

the total $X+Y$ has a Gamma distribution;
the fraction $\frac{X}{X+Y}$ has a Beta distribution;
the total is independent of the fraction.

Examples

The PDF of Beta distribution can be U-shaped with asymptotic ends, bell-shaped, strictly increasing/decreasing or even straight lines. As you change α or β, the shape of the distribution changes.

I. Bell-Shape

The PDF of a beta distribution is approximately normal if α + β is large enough and α & β are approximately equal.

Intuition behind Bell-Shape

Why would Beta(2,2) be bell-shaped?

If you think:

α-1 as the number of successes
β-1 as the number of failures
Beta(2,2) means you got 1 success and 1 failure

So it makes sense that the probability of the success is highest at 0.5.

Also, Beta(1,1) would mean you got zero for the head and zero for the tail. Then, your guess about the probability of success should be the same throughout [0,1]. The horizontal straight line confirms it.

II. Straight Lines

α = 1 or β = 1, the beta PDF can be a straight line.

III. U-Shape

When $\alpha < 1, \beta<1$ the PDF of the Beta is U-shaped.

Reference

Beta distribution as a prior

Mon, 25 Jul 2022 00:00:00 +0000

Introduction

The Beta distribution is a useful probability distribution when you want model uncertainty over a parameter bounded between 0 and 1.

In this post, we’ll explore how the two parameters of the Beta distribution determine its shape.

One way to see how the shape parameters of the Beta distribution affect its shape is to generate a large number of random draws using the rbeta(n, shape1, shape2) function and visualize these as a histogram.

Beta(1,1)

# Explore using the rbeta function
beta_sample <- rbeta(n = 1000000, shape1 = 1, shape2 = 1)

# Visualize the results
hist(beta_sample)

A Beta(1,1) distribution is the same as a uniform distribution between 0 and 1. It is useful as a so-called non-informative prior as it expresses than any value from 0 to 1 is equally likely.

Discover shape parameters

What happened if you set shape parameters to negative numbers?

# Modify the parameters
beta_sample <- rbeta(n = 1000000, shape1 = -1, shape2 = 1)

# Explore the results
head(beta_sample)

## [1] NaN NaN NaN NaN NaN NaN

Yes, NaN stands for not a number and the reason you got a lot of NaNs is that the Beta distribution is only defined when its shape parameters are positive.

# Modify the parameters
beta_sample <- rbeta(n = 1000000, shape1 = 100, shape2 = 100)

# Visualize the results
hist(beta_sample)

So the larger the shape parameters are, the more concentrated the beta distribution becomes. When used as a prior, this Beta distribution encodes the information that the parameter is most likely close to 0.5 .

# Modify the parameters
beta_sample <- rbeta(n = 1000000, shape1 = 100, shape2 = 20)

# Visualize the results
hist(beta_sample)

So the larger the shape1 parameter is the closer the resulting distribution is to 1.0 and the larger the shape2 the closer it is to 0.

Some Note for Pareto Distribution

Sat, 16 Jul 2022 00:00:00 +0000

Power Law Distribution

log-log-scale of cCDF showing you a straight line ?

This is the signature of the Power Law distribution.

R Code

library(zetaEDA)
library(zetaclv)
enable_zeta_ggplot_theme()

# transactional data for cohort 2019
cohort19 <- eg_trans_data %>%
  with_groups(
    cust,
    mutate,
    fp_yr = min(lubridate::year(date))
  ) %>%
  filter(fp_yr == 2019) %>%
  select(-fp_yr)

# build cbs data
dcbs <- generate_cbs(cohort19, timeUnit = "weeks")

## Note that: time unit is in < weeks >

head(dcbs)

##      cust x      t.x     litt sales sales.x      first     T.cal
## 1 uid0001 1 20.00000 2.995732  4644    1174 2019-12-02  79.00000
## 2 uid0005 0  0.00000 0.000000  1169       0 2019-08-08  95.57143
## 3 uid0006 1 50.71429 3.926208  1430     922 2019-04-20 111.28571
## 4 uid0010 0  0.00000 0.000000  2820       0 2019-02-15 120.42857
## 5 uid0011 0  0.00000 0.000000  6460       0 2019-01-15 124.85714
## 6 uid0012 0  0.00000 0.000000   473       0 2019-10-07  87.00000

Note that t.x is the Time between first and last transactions. This is the “observed” part of lifetime. Let’s look at the distribution of t.x.

dtmp <- dcbs %>%
  # remove single purchase customers
  filter(t.x > 0) %>%
  # get value of cdf, P(X <= x)
  mutate(cdf = ecdf(t.x)(t.x)) %>%
  # get ccef, P(X > x)
  mutate(ccdf = 1 - cdf)


dtmp %>%
  ggplot(aes(x = t.x, y = ccdf)) +
  geom_point(color = "red") +
  geom_line()

References

Youtube video: Network Analysis. Lecture 2. Power laws.

Theory Behind Pareto/NBD Part 1

Sun, 31 Jan 2021 00:00:00 +0000

1 Introduction

The Pareto/NBD model was developed by Schmittlein et al. (1987) to describe repeat-buying behavior in a noncontractual setting.

There are 4 key questions:

How many “alive” customers does the firm now have?
How has this customer base grown over the past year?
Which individuals on this list most likely represent active customers? Inactive customers?
What level of transactions should be expected next year by those on the list, both individually and collectively?

In order to answer these questions, we need to build up the model(s) to estimate:

What is the $\mathbb{P}(alive|\text{her trans infor})$?
What is the $\mathbb{E}(\text{# of trans}|\text{her trans infor})$?

2 PNBD Model Assumptions

2.1 Two stages in the lifetime

Customers go through 2 stages in their “lifetime”: they are “alive” for some period of time, then become permanently inactive.

2.2 Poisson Purchase

Given a customer while alive, the number of transactions follows Poisson distribution with parameter $\lambda$, called transaction rate. The probability of observing $x$ transactions in the time interval $(0,t]$ is given by:

\[P(X(t) = x | \lambda ) = e^{-\lambda t}\frac{(\lambda t)^x}{x!}, \ x = 0,1,2,...\]

This is equivalent to assuming that the time between transactions is $Exp(\lambda)$,

\[f(t_j - t_{j-1} | \lambda ) = \lambda e^{(t_j - t_{j-1})}, \ t_j > t_{j-1}> 0, \]

where $t_j$ is the time of the jth purchase.

2.3 Exponential Lifetime

A customer’s unobserved “lifetime” of length $\tau$, \[\tau \sim Exp(\mu), \ \ f(\tau | \mu) = \mu e^{-\mu\tau},\] where $\mu$ is called dropout rate.

2.4 Gamma transaction rate

Heterogeneity in transaction rates across customers follows a gamma distribution with shape parameter $r$ and scale parameter $\alpha$:

\[ \lambda \sim Gamma(r, \alpha ), \ \ g(\lambda|r, \alpha ) = \frac{\alpha ^r \lambda^{r-1}e^{-\lambda \alpha }}{\Gamma (r)}. \]

2.5 Gamma dropout rate

Heterogeneity in dropout rates across customers follows a gamma distribution with shape parameter $s$ and scale parameter $\beta$:

\[ \mu \sim Gamma(s, \beta), \ \ g(\mu | s, \beta ) = \frac{\beta^s\mu^{s-1}e^{-\mu\beta}}{\Gamma(s)}. \]

2.6 Two processes are Independent

The transaction rate $\lambda$ and the dropout rate $\mu$ vary independently across customers,

\[ \lambda \perp \mu. \]

3 Why named Pareto/NBD?

Short answer:

\[ \text{Poisson Purchase} + \text{Gamma transaction rate} \implies \text{NegBin} \]

\[ \text{Exponential lifetime} + \text{Gamma dropout rate} \implies \text{Pareto} \]

3.1 Poisson Gamma Mixture

Theorem 3.1 If we assume the Poisson purchase and the Gamma transaction rate, then the distribution of the number of transactions while the customer is alive is Negative Binomial (NBD).

Proof. \[\begin{align} P(X(t) = x | r, \alpha ) &= \int_{0}^{\infty}P(X(t) = x | \lambda )g(\lambda |r, \alpha ) d\lambda \\ & = \int_{0}^{\infty} e^{-\lambda t}\frac{(\lambda t)^x}{x!}\frac{\lambda^{r-1}\alpha^re^{-\lambda \alpha }}{\Gamma(r)} d\lambda\\ & = \frac{\alpha ^r}{\Gamma(r)}\frac{t^x}{x!}\int_{0}^{\infty }\lambda ^{x+r-1}e^{-\lambda (t+\alpha )} d\lambda, \text{ let } u = (t+\alpha )\lambda ,\\ & = \frac{\alpha ^r}{\Gamma(r)}\frac{t^x}{x!}\frac{1}{(t+\alpha)^{x+r} }\int_{0}^{\infty }u^{x+r-1}e^{-u}du, \text{ note the form of } \Gamma(.),\\ & = \frac{\alpha ^r}{\Gamma(r)}\frac{t^x}{x!}\frac{1}{(t+\alpha)^{x+r} }\Gamma(x+r) \\ & = \frac{\Gamma(r+x)}{\Gamma(r)x!}\left ( \frac{\alpha }{\alpha +t} \right )^{r}\left ( \frac{t}{\alpha +t} \right )^x \end{align}\]

Note that the last line is the density of negative binomial. It looks a little bit different from our familiar version of NegBin, and the parameter $r$ extends to the $\mathbb{R}^{+}$. In this case, it is called Polya distribution which is a special case of negative binomial.

3.2 Exponential Gamma Mixture

Theorem 3.2 If we assume the Exponential lifetime and the Gamma dropout rate, then the distribution of the “lifetime” is “Pareto distribution of the second kind”.

Proof. \[\begin{align} f(\tau|s, \beta ) &= \int_{0}^{\infty }f(\tau|\mu)g(\mu|s, \beta)d\mu \\ &= \int_{0}^{\infty }\mu e^{-\mu\tau}\frac{\beta e^{-\beta\mu}(\beta\mu)^{s-1}}{\Gamma(s)}d\mu\\ &= \frac{\beta^s}{\Gamma(s)} \int_{0}^{\infty }\mu^{s} e^{-\mu(\tau+\beta)}d\mu, \text{ let } \ \ u = \mu(\tau+\beta) \\ &= \frac{\beta^s}{\Gamma(s)} \int_{0}^{\infty }\frac{1}{(\tau+\beta)^s}u^s e^{-u}\frac{1}{\tau+\beta}du\\ &= \frac{\beta^s}{\Gamma(s)}\frac{1}{(\tau+\beta)^{s+1}}\int_{0}^{\infty }u^s e^{-u}du\\ &= \frac{\beta^s}{\Gamma(s)}\frac{1}{(\tau+\beta)^{s+1}}\Gamma(s+1)\\ &= \frac{\beta^s}{\Gamma(s)}\frac{1}{(\tau+\beta)^{s+1}}s\Gamma(s)\\ &= \frac{s}{\beta}\left ( \frac{\beta}{\tau+\beta} \right )^{s+1} \end{align}\]

Note that

\[ \begin{align} F(\tau|s, \beta ) &= \int_{0}^{\infty }F(\tau|\mu)g(\mu|s, \beta)d\mu \\ &= 1 - \left ( \frac{\beta}{\beta + \tau} \right ) ^{s} \end{align} \]

Therefore, if we assume Exponential lifetime and Gamma dropout rate, we have ended with Pareto Type II distribution, or more specifically, Lomax distribution.

In conclusion, the NBD and Pareto labels for each of the sub-models naturally leads to the name of the integrated model.

In the next post we will talk about the likelihood, the mean of the Pareto/NBD model, and other related derivations, eg. probability of the customer being alive.

4 Reference

Schmittlein DC, Morrison DG, Colombo R (1987). “Counting Your Customers: Who-Are They and What Will They Do Next?” Management Science, 33(1), 1-24.
Fader PS, Hardie BGS (2005). “A Note on Deriving the Pareto/NBD Model and Related Expressions.” URL
Fader PS, Hardie BGS (2007). “Incorporating time-invariant covariates into the Pareto/NBD and BG/NBD models.” URL.
Fader PS, Hardie BGS (2020). “Deriving an Expression for P(X(t)=x) Under the Pareto/NBD Model.” URL

Theory Behind Pareto/NBD Part 2

Sun, 31 Jan 2021 00:00:00 +0000

1 Deriving the Likelihood Function

Last time we talked about the ParetoNBD Model. Today we’ll derive the model likelihood function.

1.1 Some notation

For an customer,

Define:

\[ x = \text{the number of purchase,} \]

\[ t_i = \text{the time of ith purchase}, \ \text{ where } 1 \le t_i \le t_x, \]

\[ t_x = \text{the time of last purchase in the history,} \]

\[ T = \text{total time being observed} \]

Next, we’ll show that it is sufficient to use individual’s $(x, t_x, T)$ to describe his/her purchase behavior in Pareto/NBD model.

1.2 Conditional on $\lambda$ and $\mu$

Assume a customer’s $x$ transactions occurred during the period $(0,T]$; we denote these times by $t_1, t_2, t_3, \cdots, t_x$.

There are two possible ways this pattern of transactions could arise:

The customer is still alive at the end of the observation period (i.e., $\tau > T$ ), the individual-level likelihood function is simply the product of the (inter-transaction-time) exponential pdf and the associated survivor function:

\[\begin{aligned} L\left(\lambda \mid t_{1}, \ldots, t_{x}, T, \tau>T\right) &= \lambda e^{-\lambda t_{1}} \lambda e^{-\lambda\left(t_{2}-t_{1}\right)} \cdots \lambda e^{-\lambda\left(t_{x}-t_{x-1}\right)} e^{-\lambda\left(T-t_{x}\right)} \\ &=\lambda^{x} e^{-\lambda T} \end{aligned}\]

The customer became inactive at some time $\tau$ in the interval $(t_x, T]$ (i.e. $\tau \in (t_x, T]$), in which case the individual-level likelihood function is

\[\begin{aligned} & L\left(\lambda \mid t_{1}, \ldots, t_{x}, T, \text { inactive at } \tau \in\left(t_{x}, T\right]\right) \\ &=\lambda e^{-\lambda t_{1}} \lambda e^{-\lambda\left(t_{2}-t_{1}\right)} \cdots \lambda e^{-\lambda\left(t_{x}-t_{x-1}\right)} e^{-\lambda\left(\tau-t_{x}\right)} \\ &=\lambda^{x} e^{-\lambda \tau} \end{aligned}\]

Note that in both cases, information on when each of the x transactions occurred is not required.

We can replace $t_1, ...t_x$ , $T$ with $(x, t_x , T)$ where, by definition, $t_x = 0$ when $x = 0$. In other words, $t_x$, $x$ and $T$ are sufficient summaries of a customer’s transaction history. Using direct marketing terminology, $t_x$ is recency and $x$ is frequency.

由以上两个事实可知，无需知晓客户每次的购买时间，第一次消费时间、最后一次消费时间、消费频次 作为充分统计量，已经足够我们导出似然函数了！

Removing the conditioning on $\tau$ gives us the following expression for the individual-level likelihood function:

\[\begin{aligned} L\left(\lambda, \mu \mid x, t_{x}, T\right)=& L(\lambda \mid x, T, \tau>T) P(\tau>T \mid \mu) + \\ &\int_{t_{x}}^{T} L\left(\lambda \mid x, T, \text { inactive at } \tau \in\left(t_{x}, T\right]\right) f(\tau \mid \mu) d \tau \\ &=\lambda^{x} e^{-\lambda T} e^{-\mu T}+\lambda^{x} \int_{t_{x}}^{T} e^{-\lambda \tau} \mu e^{-\mu \tau} d \tau \\ &=\lambda^{x} e^{-(\lambda+\mu) T}+\frac{\lambda^{x} \mu}{\lambda+\mu} e^{-(\lambda+\mu) t_{x}}-\frac{\lambda^{x} \mu}{\lambda+\mu} e^{-(\lambda+\mu) T} \\ &=\frac{\lambda^{x} \mu}{\lambda+\mu} e^{-(\lambda+\mu) t_{x}}+\frac{\lambda^{x+1}}{\lambda+\mu} e^{-(\lambda+\mu) T} \end{aligned}\]

1.3 Removing the Conditioning on $\lambda$ and $\mu$

We remove the conditioning on $\lambda$ and $\mu$ by taking the expectation of $L(\lambda, \mu | x, t_x , T)$ over the distributions of $\lambda$ and $\mu$ :

\[ L\left(r, \alpha, s, \beta \mid x, t_{x}, T\right)=\int_{0}^{\infty} \int_{0}^{\infty} L\left(\lambda, \mu \mid x, t_{x}, T\right) g(\lambda \mid r, \alpha) g(\mu \mid s, \beta) d \lambda d \mu \]

The computation is tedious, check the paper “A Note on Deriving the Pareto/NBD Model and Related Expressions” to know the details.

1.4 MLE for $r, \alpha, s, \beta$

Since we have derived the likelihood function $L\left(r, \alpha, s, \beta \mid x, t_{x}, T\right)$, the 4 Pareto/NBD model parameters $(r, \alpha, s, \beta)$ can be estimated via the method of MLE. Specifically, suppose we have a sample of $N$ customers, where customer $i$ had $x_i$ transactions in the period $(0, T_i ]$, with the last transaction occurring at $t_{x_i}$ . The sample log-likelihood function is given by

\[ L L(r, \alpha, s, \beta)=\sum_{i=1}^{N} \ln \left[L\left(r, \alpha, s, \beta \mid x_{i}, t_{x_{i}}, T_{i}\right)\right]. \]

This can be maximized using standard numerical optimization routines. Therefore, we will obtain 4 maximum likelihood estimators $(\widehat{r} \ , \ \widehat{\alpha} \ , \ \widehat{s} \ , \ \widehat{\beta})$

2 Derivations

2.1 Mean of the Pareto/NBD Model

Given that the number of transactions follows a Poisson process while the customer is alive,

if $\tau > t$, the expected number of transactions is simply $\lambda t$.
if $\tau \le t$, the expected number of transactions in the interval (0, τ] is $\lambda \tau$.

Removing the conditioning on the time at which the customer becomes inactive, it follows that the expected number of transactions in the time interval $(0, t]$, conditional on $\lambda$ and $\mu$, is

\[ \begin{aligned} E[X(t) \mid \lambda, \mu] &=\lambda t P(\tau>t \mid \mu)+\int_{0}^{t} \lambda \tau f(\tau \mid \mu) d \tau \\ &=\lambda t e^{-\mu t}+\lambda \int_{0}^{t} \mu \tau e^{-\mu \tau} d \tau \\ &=\lambda t e^{-\mu t}+\frac{\lambda}{\mu} \int_{0}^{t} \mu^{2} \tau e^{-\mu \tau} d \tau, \text{where integrand is an Erlang-2} \\ &=\lambda t e^{-\mu t}+\frac{\lambda}{\mu}\left\{1-e^{-\mu t}-\mu t e^{-\mu t}\right\} \\ &=\frac{\lambda}{\mu}-\frac{\lambda}{\mu} e^{-\mu t} \end{aligned} \]

Now removing the Conditioning on $\lambda$ and $\mu$,

\[\begin{align} E[X(t) \mid r, \alpha, s, \beta] &=\int_{0}^{\infty} \int_{0}^{\infty} E[X(t) \mid \lambda, \mu] g(\lambda \mid r, \alpha) g(\mu \mid s, \beta) d \lambda d \mu \\ &=\frac{r \beta}{\alpha(s-1)}-\frac{r \beta^{s}}{\alpha(s-1)(\beta+t)^{s-1}} \\ &=\frac{r \beta}{\alpha(s-1)}\left[1-\left(\frac{\beta}{\beta+t}\right)^{s-1}\right] \tag{2.1} \end{align}\]

2.2 Derivation of PAlive

The probability that a customer with purchase history $(x, t_x , T)$ is “alive” at time $T$ is $P(\tau > T)$.

\[\begin{aligned} P\left(\tau>T \mid \lambda, \mu, x, t_{x}, T\right) &=\frac{L(\lambda \mid x, T, \tau>T) P(\tau>T \mid \mu)}{L\left(\lambda, \mu \mid x, t_{x}, T\right)} \\ &=\frac{\lambda^{x} e^{-(\lambda+\mu) T}}{L\left(\lambda, \mu \mid x, t_{x}, T\right)} \end{aligned}\]

As the $\lambda$ and $\mu$ are unobserved, we compute $P(alive | x, t_x , T)$ for a randomly-chosen individual by taking the expectation of the above result over the distribution of $\lambda$ and $\mu$ , updated to take account of the information $(x, t_x , T)$:

\[\begin{array}{l} P\left(\text { alive } \mid r, \alpha, s, \beta, x, t_{x}, T\right) \\ \qquad=\int_{0}^{\infty} \int_{0}^{\infty} P\left(\tau>T \mid \lambda, \mu, x, t_{x}, T\right) g\left(\lambda, \mu \mid r, \alpha, s, \beta, x, t_{x}, T\right) d \lambda d \mu \end{array}\]

By Bayes’ theorem, the joint posterior distribution of $\lambda$ and $\mu$ is

\[ g\left(\lambda, \mu \mid r, \alpha, s, \beta, x, t_{x}, T\right)=\frac{L\left(\lambda, \mu \mid x, t_{x}, T\right) g(\lambda \mid r, \alpha) g(\mu \mid s, \beta)}{L\left(r, \alpha, s, \beta \mid x, t_{x}, T\right)} \]

Thus,

\[\begin{array}{l} P\left(\text { alive } \mid r, \alpha, s, \beta, x, t_{x}, T\right) \\ \quad=\int_{0}^{\infty} \int_{0}^{\infty} \lambda^{x} e^{-(\lambda+\mu) T} g(\lambda \mid r, \alpha) g(\mu \mid s, \beta) d \lambda d \mu / L\left(r, \alpha, s, \beta \mid x, t_{x}, T\right) \\ \quad=\frac{\Gamma(r+x) \alpha^{r} \beta^{s}}{\Gamma(r)(\alpha+T)^{r+x}(\beta+T)^{s}} / L\left(r, \alpha, s, \beta \mid x, t_{x}, T\right)\\ \quad=\left\{1+\left(\frac{s}{r+s+x}\right)(\alpha+T)^{r+x}(\beta+T)^{s} \mathrm{~A}_{0}\right\}^{-1} \end{array}\]

For details check the reference paper. Note that, the above result is the formula to calculate PAlive used in BTYD 📦 implemented in R.

2.3 Conditional Expectation of Transactions

Let random variable $Y(t) = \text{num of purchase made in } (T, T+t]$. We are interested in computing $E(Y(t)|x, t_x, T)$, the expected number of purchase in the period $(T, T+t]$ for a customer with purchase history $(x, t_x, T)$.

If the customer is active at $T$,

\[ \begin{array}{l} &E[Y(t) \mid \lambda, \mu, \text { alive at } T]\\ &=\lambda t P(\tau>T+t \mid \mu, \tau>T)+\int_{T}^{T+t} \lambda \tau f(\tau \mid \mu, \tau>T) d \tau\\ &=\lambda t e^{-\mu t}+\lambda \int_{0}^{t} \mu \tau e^{-\mu \tau} d \tau \\ &=\frac{\lambda}{\mu}-\frac{\lambda}{\mu} e^{-\mu t} \end{array} \]

Of course we don’t know whether a customer is alive at $T$; therefore

\[ E\left[Y(t) \mid \lambda, \mu, x, t_{x}, T\right]=E[Y(t) \mid \lambda, \mu, \text { alive at } T] P\left(\tau>T \mid \lambda, \mu, x, t_{x}, T\right) \]

Also, since $\lambda$ and $\mu$ are unobserved, we need to integrate them out:

\[ \begin{array}{c} E\left[Y(t) \mid r, \alpha, s, \beta, x, t_{x}, T\right]=\int_{0}^{\infty} \int_{0}^{\infty}\left\{E[Y(t) \mid \lambda, \mu, \text { alive at } T] P\left(\tau>T \mid \lambda, \mu, x, t_{x}, T\right)\right. \\ \left.g\left(\lambda, \mu \mid r, \alpha, s, \beta, x, t_{x}, T\right)\right\} d \lambda d \mu \end{array} \]

After the tedious computation, we will get

\[\begin{aligned} &E\left[Y(t) \mid r, \alpha, s, \beta, x, t_{x}, T\right]\\ &=\{\frac{\Gamma(r+x) \alpha^{r} \beta^{s}}{\Gamma(r)(\alpha+T)^{r+x}(\beta+T)^{s}} / L\left(r, \alpha, s, \beta \mid x, t_{x}, T\right)\} \\ &\times \frac{(r+x)(\beta+T)}{(\alpha+T)(s-1)}\left[1-\left(\frac{\beta+T}{\beta+T+t}\right)^{s-1}\right]\\ &= \{P(\text{alive}|x, t_x, T)\} \times \text{updated mean of Pareto/NBD} \end{aligned}\]

Note that:

The first part, the bracketed term, is out expression for $P(\text{alive}|x, t_x, T)$.
The rest part is mean of the Pareto/NBD (2.1), with “updated” parameters that reflect the individual’s behavior up to time $T$ (assuming no “death” in $(0,T])$).

Next time, we’ll finally take about the prediction of CLV.

3 Reference

Schmittlein DC, Morrison DG, Colombo R (1987). “Counting Your Customers: Who-Are They and What Will They Do Next?” Management Science, 33(1), 1-24.
Fader PS, Hardie BGS (2005). “A Note on Deriving the Pareto/NBD Model and Related Expressions.” URL
Fader PS, Hardie BGS (2007). “Incorporating time-invariant covariates into the Pareto/NBD and BG/NBD models.” URL.
Fader PS, Hardie BGS (2020). “Deriving an Expression for P(X(t)=x) Under the Pareto/NBD Model.” URL

Gamma-Gamma Spend Model

Sat, 30 Jan 2021 00:00:00 +0000

How to predict a customer’s mean spending in the future?

Answer: You can use gamma-gamma model.

1 Model Assumption

There are 3 general assumptions for this model:

The monetary value (e.g. $, ¥) of a customer’s given transaction varies randomly around their average transaction value.
Average transaction values vary across customers but do not vary over time for any given individual.
The distribution of average transaction values across customers is independent of the transaction process.

For a customer with $x$ transactions:

let $\{z_1, z_2, \cdots, z_x\}$ denote the value of each transaction.
$z_i$’s are samples from the distribution of R.V. $Z$.

The customer’s observed average transaction value is $\bar{z} = \sum_{i = 1}^{x}\frac{z_i}{x}$.

However, $\bar{z}$ is an imperfect estimate of the unobserved mean transaction value.

Why? Consider when a customer only had very limited transactions, say 1 or 2 purchases, then it is questionable to use his average spending to estimate the spending power. At least we should use the population mean as the standard criteria to help. On the other hand, if the customer had enough purchase history, then we want to emphasize more on his own average spending while put relatively less weight on the population mean.

The goal is to make inference about

\[ E(Z| \bar{z}, x) \]

What is distribution for $Z$?

Maybe log-normal or gamma, since spend data tend to be right skewed.

Here we assume the gamma distribution. More formally, we assume that:

$Z \sim gamma(p, \nu)$
$\nu \sim gamma(q, \gamma)$

From above setting, we know

\[ E(Z|p, \nu) := \zeta = \frac{p}{\nu} \]

\[ \text{total spend} = \sum_{i = 1}^{x}z_i \sim gamma(px, \nu) \]

\[ \text{average spend} = \bar{z} = \sum_{i = 1}^{x}\frac{z_i}{x} \sim gamma(px, \nu x) \]

They can be easily proved using MGF.

This results in what we call the gamma-gamma model of spend.

2 Compute $E(Z \mid \bar{z}, x)$

We wish to make inferences about an individual customer’s mean spending given $\bar{z}$, which we denote as $E(Z|\bar{z}, x)$.

Note that,

\[ \begin{equation} E(Z \mid \bar{z}, x) = E(Z \mid p, q, \gamma ; \bar{z}, x) =\int_{0}^{\infty }E(Z\mid p, \nu)g(\nu \mid p, q, \gamma, \bar{z},x )d\nu \tag{2.1} \end{equation} \]

\[ Z \sim gamma(p, v) \implies E(Z \mid p, \nu) = \frac{p}{\nu} \]

2.2 Get the result

Now we have everything we need to derive (2.1). Before plug in, let’s review the inverse gamma distribution which will be used later.

If $X \sim Gamma(\alpha, \beta)$ then $Y :=\frac{1}{x} \sim \text{Inv-Gamma}(\alpha, \beta)$, with

$E(Y) = \frac {\beta }{\alpha -1}$, for $\alpha > 1$

$Var(Y) = \frac {\beta ^{2}}{(\alpha -1)^{2}(\alpha -2)}$, for $\alpha > 2$.

Plug into (2.1),

\[\begin{align} E(Z \mid p, q, \gamma ; \bar{z}, x) &=\int_{0}^{\infty }E(Z|p, \nu)g(\nu |p, q, \gamma, \bar{z},x )d\nu \\ &= \int_{0}^{\infty } \frac{p}{\nu} g(\nu |p, q, \gamma, \bar{z},x )d\nu \\ &= p \cdot E(\frac{1}{\nu}|p, q, \gamma, \bar{z},x ), \\ &\text{note: }\\ &\frac{1}{\nu} \sim \text{Inv-Gamma}(px+q, \gamma+x \bar{z}) \\ &E(\text{Inv-Gamma}) = \frac{\text{scale}}{\text{shape - 1}} \\ &=\frac{p(\gamma+x \bar{z})}{p x+q-1} \tag{2.6} \end{align}\]

3 Understand the result

How to understand $E(Z \mid p, q, \gamma ; \bar{z}, x)$? What can we say about the estimator of the individual’s future spend per transaction given his/her historical average spend $\bar{z}$?

First, let’s derive $E(Z \mid p, q, \gamma)$, which is the population mean.

\[ \begin{aligned} E(Z \mid p, q, \gamma) &= \int_{0}^{\infty }E(Z\mid p, \nu)g(\nu \mid q, \gamma)d\nu \\ &= \int_{0}^{\infty } \frac{p}{\nu}g(\nu \mid q, \gamma)d\nu\\ &= pE(\frac{1}{\nu}), \text{ where } \frac{1}{\nu} \sim \text{Inv-Gamma}(q, \gamma)\\ &= \frac{p\gamma}{q-1} \end{aligned} \]

Now let’s rearrange the result of (2.6),

\[\begin{align} E(Z \mid p, q, \gamma ; \bar{z}, x) &= \frac{p(\gamma+x \bar{z})}{px+q-1} \\ &=\left(\frac{q-1}{p x+q-1}\right) \underbrace{\frac{p \gamma}{q-1}}_{\text{population mean}} + \left(\frac{p x}{p x+q-1}\right) \underbrace{\bar{z} \vphantom{\frac{p \gamma}{q-1}}}_{\text{observed average}} \end{align}\]

3.1 Key point

We note that this is the weighted average of the population mean, $E(Z|p, q, \gamma)$, and the given individual’s observed average transaction value, $\bar{z}$.

\[ E(Z \mid \bar{z}, x) = w \cdot \{\text{population mean}\} + (1-w) \cdot \{\text{observed average}\} \]

As the number of observations $(x)$ used to compute $\bar{z}$ increases, less weight is placed on the population mean and more weight is placed on the customer’s observed average.

4 References

[1] Fader PS, Hardie BG (2013). “The Gamma-Gamma Model of Monetary Value.” URL http://www.brucehardie.com/notes/025/gamma_gamma.pdf.

[2] Colombo R, Jiang W (1999). “A stochastic RFM model.” Journal of Interactive Marketing, 13(3), 2-12.

Note for Beta Distribution

Sat, 30 Jan 2021 00:00:00 +0000

This post is out of date, please check the new post named “Beta Distribution — Intuition, Derivation, and Examples”.

1 Why Beta Distribution?

1.1 Model probabilities

The short story is that the Beta distribution can be understood as representing a distribution of probabilities, that is, it represents all the possible values of a probability when we don’t know what that probability is.

1.2 Generalization of uniform

Give me a continuous and bounded random variable, em, except the Uniform distribution. That is another way to look at beta distribution, continuous and bounded between 0, 1; also the density is not flat. \[ X \sim Beta(a, b), \text{ where } a>0, \ b>0. \]

\[ f_X(x) = c \cdot x ^{a-1}(1-x)^{b-1}, \text{ where } x>0. \]

What is $c$? Just a normalization constant. We’ll get the value of $c$ later.

2 Construction

2.1 Bank and Post Office Story

Let $X$ be the waiting time at Bank,

\[ X \sim Gamma(n_1, \lambda) \]

Let $Y$ be the waiting time at Post Office,

\[ Y \sim Gamma(n_2, \lambda) \]

Assume $X$ and $Y$ are independent.

Then, what is the distribution of the proportion $\frac{X}{X+Y}$?

Define $T = X+Y$ be the total waiting time.

Clearly, $T \sim Gamma(n_1+n_2, \lambda)$, proved by MGF.

Define $W = \frac{X}{X+Y}$ , the proportion of waiting time at Bank to the total waiting time.

What is the distribution of $W$?

We need to derive $f_W(w)$, but first let’s find the joint pdf $f_{T,W}(t,w)$ \[ \begin{align} f_{T,W}(t,w) &= f_{X,Y}(x,y) \left | \frac{\partial(x,y)}{\partial(t,w)} \right|\\ &= \frac{1}{\Gamma(n_1)}\lambda^{n_1}x^{n_1 - 1}e^{-\lambda x} \frac{1}{\Gamma(n_2)}\lambda^{n_2}x^{n_2 - 1}e^{-\lambda y} \left|-t\right| \\ &= \lambda^{n_1+n_2}t^{n_1+n_2-1}e^{\lambda t} \frac{1}{\Gamma(n_1)\Gamma(n_2)}w^{n_1 - 1}(1-w)^{n_2-1} \\ &= \frac{\lambda^{n_1+n_2}t^{n_1+n_2-1}e^{\lambda t}}{\Gamma(n_1+n_2)} \frac{\Gamma(n_1+n_2)}{\Gamma(n_1)\Gamma(n_2)}w^{n_1 - 1}(1-w)^{n_2-1}\\ &= f_T(t) \frac{\Gamma(n_1+n_2)}{\Gamma(n_1)\Gamma(n_2)}w^{n_1 - 1}(1-w)^{n_2-1} \end{align} \]

Then we find the marginal,

\[\begin{aligned} f_W(w) &= \int_0^\infty f_{T,W}(t,w) dt \\ &= \frac{\Gamma(n_1+n_2)}{\Gamma(n_1)\Gamma(n_2)}w^{n_1 - 1}(1-w)^{n_2-1} \cdot\int_0^\infty f_T(t)dt \\ &= \frac{\Gamma(n_1+n_2)}{\Gamma(n_1)\Gamma(n_2)}w^{n_1 - 1}(1-w)^{n_2-1} \end{aligned}\]

Since $f_W(w)$ is the pdf needed to be integrated to 1,

\[ \int_0^1\frac{\Gamma(n_1+n_2)}{\Gamma(n_1)\Gamma(n_2)}w^{n_1 - 1}(1-w)^{n_2-1} dw \equiv 1 \]

so the normalization constant should be

\[ c = \frac{\Gamma(n_1+n_2)}{\Gamma(n_1)\Gamma(n_2)} := \frac{1}{B(n_1, n_2)} \]

2.1.1 Summary

The connection between Gamma and Beta distribution helps us to find the normalization constant in Beta. In summary,

If $X \sim Gamma(\alpha, \lambda)$ and $Y \sim Gamma(\beta, \lambda)$ are independent, then $\frac{X}{X+Y} \sim Beta(\alpha, \beta)$.

2.2 plots

library(zetaEDA)
library(ggfortify)
enable_zeta_ggplot_theme()

Let’s check Beta density for some different parameters value.

How about $a = b = 1$? The Beta(1,1) is just the Unif(0,1).

ggdistribution(func = dbeta, x = seq(0, 1, .01), shape1 = 1, shape2 = 1) +
  labs(title = "Beta Density with a = 1, b = 1")

How about $a = b = \frac{1}{2}$?

How about $a = b = 2$?

How about $a= 2, \ b = 1$?

One more,

p <- ggdistribution(func = dbeta, x = seq(0, 1, .01), shape1 = 1.5, shape2 = 5, colour = "tomato", linetype = "dashed")

ggdistribution(func = dbeta, x = seq(0, 1, .01), shape1 = 5, shape2 = 1.5, colour = "blue", p = p) +
  labs(title = "Red: a = 1.5, b = 5\n Blue: a = 5, b = 1.5")

For more checking, click this link and try some parameters to check the density curve.

Have fun!

3 Reference

Note for Gamma Distribution

Sat, 30 Jan 2021 00:00:00 +0000

Motivation for Gamma Function

We all know how to compute the factorial of integer. BUT what is the factorial of 1/2?

In other words, how to interpolate the factorial function?

The gamma function can be seen as a solution to the following interpolation problem:

“Find a smooth curve that connects the points (x, y) given by y = (x − 1)! at the positive integer values for x.”

More details, check the wiki page.

Definition of Gamma Function

Definition 1 (Gamma Function) \[\begin{align} \Gamma(z) = \int_{0}^{\infty}x^{z-1}e^{-x}dx, \ \ z \in \mathbb{R}^+ \end{align}\]

For the the Gamma function, it is enough to know the following properties for now.

Lemma 1 \[\begin{align} & \Gamma(z+1) = z\Gamma(z), \ \ z \in \mathbb{R}^+ \\ & \Gamma(n) = (n-1)!, \ \ n = 1,2,3,... \end{align}\]

Easy to prove using integration by parts.

Now, what is $\Gamma(\frac{1}{2})$?

\[ \Gamma(\frac{1}{2}) = \int_{0}^{\infty}x^{-1/2}e^{-x}dx = ? \]

Recall that, $\int_{0}^{\infty} e^{-x^2} = \frac{1}{2}\sqrt{\pi}$, let $u = x^2$ and will get the result $\Gamma(\frac{1}{2} )= \sqrt{\pi}$.

Gamma Distribution

From the Gamma function, it is pretty natural to get Gamma pdf. JUST normalizing!

Clearly, \[ 1 = \int_0^\infty \frac{x^{r-1}e^{-x}}{\Gamma(r)}dx = \int_0^\infty f_X(x)dx, \ \ \ X := Gamma(r, 1) \]

What is the pdf for the general $Gamma(r, \lambda)$ ? Let

\[ Y = \frac{X}{\lambda}, \ Y \sim Gamma(r, \lambda) \] \[ f_Y(y) = f_X(x)\frac{dx}{dy} \]

We’ll get

\[ f(y; r, \lambda ) = \frac{\lambda ^{r}y^{r-1}e^{-\lambda y}}{\Gamma(r)} \]

Here, $r$ is called the shape parameter and $\lambda$ is called the rate parameter.

How to remember the Gamma pdf?

That’s my trick: exponential density times the power rise to (shape-1), then divided by normalizer.

Exponential density (very familiar): $\lambda e^{-\lambda x}$
power rise to (shape-1): $(\lambda x)^{r-1}$
normalizing constant (using shape): $\Gamma(r)$

\[ \begin{align} f(x; r, \lambda ) &= \frac{\text{exp density} \cdot \text{power}^\text{shape-1} }{normalizer} \\ & = \frac{\lambda e^{-\lambda x}(\lambda x)^{r-1}}{\Gamma(r)} \\ & = \frac{\lambda ^{r}x^{r-1}e^{-\lambda x}}{\Gamma(r)} \end{align} \]

Another way to remember is this:

Exponential key part: \[ e^{-\lambda x} \]
Add Power part: \[ \lambda^{\square} x^{\square} e^{- \lambda x} \]
- multiply the power part in exponential
- rate rises to shape
- variable rises to shape-1
\[ \lambda^{r} x^{r-1} e^{- \lambda x} \]
Add Normalizing part: \[ \frac{\lambda^{r} x^{r-1} e^{- \lambda x}}{\Gamma(r)} \]

Gamma & Exponential Connection

Let’s recall the Poisson Process, \[ N_t = \text{number of arrials up to time t} \sim Pois(\lambda t) \] The number of arrivals in the disjoint intervals are independent.

Let $T_1$ be the time of 1st arrival, \[ P(T_1 > t) = P(N_t = 0) = e^{-\lambda t} \ \implies T_1 \sim Exp(\lambda) \] Now, let $T_n$ be the time of nth arrival, that is, $T_n = \sum_{i=1}^n X_i$ , where $X_i \overset{\text{iid}}{\sim}Exp(\lambda)$

What is the pdf of $T_n$? Answer is Gamma!

Proposition 1 Gamma is the sum of iid Exponentials.

Proof:

Since $M_X(t) = \frac{\lambda}{\lambda-t}$, where $t < \lambda$,

\[ M_{\sum_{i=1}^n X_i}(t) = (M_X(t))^n = (\frac{\lambda}{\lambda-t})^n \]

It is enough to show the MGF of Gamma equals the above value.

Let $Y \sim Gamma(n, \lambda)$,

\[ \begin{align} M_Y(t) = E(e^{ty}) &= \int_0^\infty e^{ty} \frac{1}{\Gamma(n)} \lambda^{n}e^{-\lambda y}y^{n-1} dy \\ &= \frac{\lambda^n}{\Gamma(n)} \int_0^\infty e^{-(\lambda - t)y}y^{n-1}dy, \text{ let } u = (\lambda - t)y \\ &= \frac{\lambda^n}{\Gamma(n)} (\frac{1}{\lambda-t})^n \int_0^\infty e^{-u}u^{n-1}du \\ &= \frac{\lambda^n}{\Gamma(n)} (\frac{1}{\lambda-t})^n\Gamma(n) \\ &= (\frac{\lambda}{\lambda-t})^n \end{align} \]

Proved!

Remark. The Exponential is the continuous analog of the Geometric. Similarly, the Gamma is the continuous analog of the Negative Binomial.

Deriving the conditional distributions of a multivariate normal distribution

Fri, 31 Mar 2017 00:00:00 +0000

Overall, the intuition behind the conditional distribution of a bivariate normal is that even though the two variables are correlated in the joint distribution, they can still be treated as independent when you’re looking at the distribution of one variable, given a fixed value of the other variable. This allows you to use the normal distribution, which is a well-understood and widely-used probability distribution, to model the conditional distribution of a bivariate normal.

More generally,

Assume we know there’s a theorem that says all conditional distributions of a multivariate normal distribution are normal.

distributions | Chen Xing

Beta Distribution — Intuition, Derivation, and Examples

Motivation

Model probabilities

Generalization of Uniform Distribution

Conjugate Prior

What is the Intuition?

Interpretation of α, β

Derivation

Fraction of Waiting Time

Beta Function as a normalizing constant

Mean of Beta distribution

Getting Beta parameters in practice

Summary

Examples

I. Bell-Shape

Intuition behind Bell-Shape

II. Straight Lines

III. U-Shape

Reference

Beta distribution as a prior

Introduction

Beta(1,1)

Discover shape parameters

Some Note for Pareto Distribution

Power Law Distribution

R Code

References

Theory Behind Pareto/NBD Part 1

1 Introduction

2 PNBD Model Assumptions

2.1 Two stages in the lifetime

2.2 Poisson Purchase

2.3 Exponential Lifetime

2.4 Gamma transaction rate

2.5 Gamma dropout rate

2.6 Two processes are Independent

3 Why named Pareto/NBD?

3.1 Poisson Gamma Mixture

3.2 Exponential Gamma Mixture

4 Reference

Theory Behind Pareto/NBD Part 2

1 Deriving the Likelihood Function

1.1 Some notation

1.2 Conditional on \(\lambda\) and \(\mu\)

1.3 Removing the Conditioning on \(\lambda\) and \(\mu\)

1.4 MLE for \(r, \alpha, s, \beta\)

2 Derivations

2.1 Mean of the Pareto/NBD Model

2.2 Derivation of PAlive

2.3 Conditional Expectation of Transactions

3 Reference

Gamma-Gamma Spend Model

1 Model Assumption

2 Compute \(E(Z \mid \bar{z}, x)\)

2.1 Derive related conditional density

2.2 Get the result

3 Understand the result

3.1 Key point

4 References

Note for Beta Distribution

1 Why Beta Distribution?

1.1 Model probabilities

1.2 Generalization of uniform

2 Construction

2.1 Bank and Post Office Story

2.1.1 Summary

2.2 plots

3 Reference

Note for Gamma Distribution

Motivation for Gamma Function

Definition of Gamma Function

Gamma Distribution

How to remember the Gamma pdf?

Gamma & Exponential Connection

Deriving the conditional distributions of a multivariate normal distribution