Probability & Bayesian Statistics

Notes for Variational Inference
Introduction In modern Bayesian statistics, we often face posterior distributions that are difficult to compute. Let $p(z)$ be prior density and $p(x \mid z)$ be likelihood. The standard approach to compute posterior $p(z \mid x)$ is to use MCMC (like Metropolis-Hastings, Gibbs sampling and HMC).
Apr 27, 2025 4 min read Probability & Bayesian Statistics
Using Stan to do MCMC
What is Stan? Stan is an intuitive, yet sophisticated, probabilistic programming language that provides an interface to a recently proposed extension to Hamiltonian Monte Carlo (HMC), known as No U-Turn Sampler (NUTS).
Apr 20, 2023 1 min read Probability & Bayesian Statistics
Bayesian Statistics Self-Study Resources
Here are some good resources for Bayesian Statistics self-learning. Textbooks “A Student’s Guide to Bayesian Statistics”: Easy to understand, also with problem and solution available on the author’s personal blog. There are also lots of other useful Bayesian resources on his blog, for example, the Bayesian lecture slides, definitely deserve to check!
Mar 11, 2023 2 min read Probability & Bayesian Statistics
A Story Behind Maximum Likelihood
TL;DR What are we actually doing on MLE? What is the motivation for the Maximum Likelihood Estimation? In this post, we will go over the story behind the maximum likelihood.
Aug 31, 2022 3 min read Probability & Bayesian Statistics
Beta Distribution — Intuition, Derivation, and Examples
Motivation Model probabilities The Beta distribution is a probability distribution on probabilities. The Beta distribution can be understood as representing a distribution of probabilities, that is, it represents all the
Aug 3, 2022 3 min read Probability & Bayesian Statistics
Tutorial Notes for Bayesian Statistics
Bayesian Inference Posterior Distribution The intuition behind the posterior distribution: “It is a weighted version of likelihood! … just weighting the likelihood using my prior belief on theta …” Useful
Jul 26, 2022 1 min read Probability & Bayesian Statistics
Beta distribution as a prior
Introduction The Beta distribution is a useful probability distribution when you want model uncertainty over a parameter bounded between 0 and 1. In this post, we’ll explore how the two parameters of the Beta distribution determine its shape.
Jul 25, 2022 2 min read Probability & Bayesian Statistics
Some Note for Pareto Distribution
Jul 16, 2022 2 min read Probability & Bayesian Statistics
Note for Beta Distribution
Why Beta Distribution?
Jan 30, 2021 3 min read Probability & Bayesian Statistics
Note for Gamma Distribution
Motivation for Gamma Function Definition of Gamma Function Gamma Distribution How to remember the Gamma pdf? Gamma & Exponential Connection Motivation for Gamma Function We all know how to compute the factorial of integer.
Jan 30, 2021 3 min read Probability & Bayesian Statistics
Probability Theory Study Notes
Online Courses Probability Theory and Stochastic Processes (video lecture). Description: These lectures encompass a full-year course in probability theory and stochastic processes, as taught at the University of California, San Diego (as Math 280).
Dec 17, 2020 1 min read Probability & Bayesian Statistics
Notes for LP Inequality
Here, I summarize some useful LP Inequalities with proofs. Download PDF file here.
May 24, 2017 1 min read Probability & Bayesian Statistics
Deriving the conditional distributions of a multivariate normal distribution
Overall, the intuition behind the conditional distribution of a bivariate normal is that even though the two variables are correlated in the joint distribution, they can still be treated as independent when you’re looking at the distribution of one variable, given a fixed value of the other variable.
Mar 31, 2017 1 min read Probability & Bayesian Statistics