Bayes’ Theorem

Here's a rough note I have prepared for a friend during last semester's exams.

Bayes theorem can be seen as a way of understanding how the probability that a theory is true is affected by a new piece of evidence.

Prob(M|D) = Prob(D|M) * Prob(M) / Prob(D)

M: Theory of hypothesis that we are interested in testing.

D: New piece of evidence that seems to confirm or dis-confirm the theory.

Prob(M|D): Known as posterior probability. This is the new theory or hypothesis after accepting the new evidence or data.

P(D|M): Known as the likelihood. This is the probability that a piece of evidence could be observed in a given dataset.

Intention is to discover the probability that M is true by considering that new evidence is true. This is a conditional probability, the probability that one proposition is true provided that another proposition is true.

A theory is basically a property or a pattern described by the given dataset. For example sales patterns identified using credit card transactions history. In this case, a theory may describe Bank A’s credit card holders are tend to be deal with Company B. Therefore Company B can offer sales price discounts to Bank A’s card holders, which will attract more customers.