Derivation of Bayes' Theorem

Bayes' Theorem is derived from the definition of conditional probability.

Definition of Conditional Probability:

The probability of $A$ given $B$ is:

$$P(A|B) = \frac{P(A \cap B)}{P(B)}, \quad \text{where } P(B) > 0$$

Similarly, the probability of $B$ given $A$ is:

$$P(B|A) = \frac{P(A \cap B)}{P(A)}, \quad \text{where } P(A) > 0$$

Where $\cap$ means intersection, or joint probability, or the probability that $A$ and $B$ occur. Rearrange both equations:

From $P(A|B)$:

$$P(A \cap B) = P(A|B) \cdot P(B)$$

From $P(B|A)$:

$$P(A \cap B) = P(B|A) \cdot P(A)$$

Equate the two expressions for $P(A \cap B)$ and since both equations describe $P(A \cap B)$, set them equal:

$$P(A|B) \cdot P(B) = P(B|A) \cdot P(A)$$

Then we solve for $P(A|B)$, divide through by $P(B)$ (assuming $P(B) > 0$) to get the final form of Bayes' Theorem:

$$P(A|B) = \frac{P(B|A) \cdot P(A)}{P(B)}$$

where

• $P(A|B)$: Probability of $A$ given $B$.
• $P(B|A)$: Probability of $B$ given $A$.
• $P(A)$: Prior probability of $A$.
• $P(B)$: Probability of $B$, the normalizing constant.