Chapter 2 Probability
Imagine a game where you flip a single coin once. The possible outcomes are then head (H) and tail (T). The set of all possible outcomes of the game is called the sample space of the experiment. We will denote this set with \(\Omega\). For a single coin toss game the sample space is \(\Omega = \{\text{heads}, \text{tails}\}\).
Definition 2.1 (Probability axioms) Let \(\Omega\) denote the sample space of a random experiment. Let \(A \subseteq \Omega\) and \(B \subset \Omega\) be two disjoint events (i.e. \(A \cap B = \varnothing\)).
\[\begin{align} P(A) & \geq 0 \\ P(\Omega) & = 1 \quad \text{unit measure} \tag{2.1} \\ P(A \cup B) & = P(A) + P(B) \quad \text{additivity} \tag{2.2} \end{align}\]See Bertsekas and Tsitsiklis (2008) for a more thorough treatment of the subject.
Proof. For the proof note that \(A\) and \(\bar{A}\) are disjoint by definition (\(A \cap \bar{A} = \varnothing\)). Using (2.2) and (2.1) it follows that: \[\begin{align} P(A \cup \bar{A}) = P(A) + P(\bar{A}) \end{align}\]
\[\begin{align} P(A \cup \bar{A}) & = P(\Omega) \\ P(A) + P(\bar{A}) & = 1 \\ \implies P(A) & = 1 - P(\bar{A}). \end{align}\]
Example 2.1 Looking at the weather forecast for the next day you see that it will be raining (\(A\)) with probability \(P(A) = 0.3\). The sample space is \(\Omega = \{A, \bar{A}\}\) and the probability of not raining (\(\bar{A}\)) is then \(P(\bar{A}) = 1 - 0.3 = 0.7\).
Example 2.2 In a game where you roll a (6-sided) dice once the sample space is \(\Omega = \{1, 2, 3, 4, 5, 6\}\). Denote the outcome of a roll with \(X\) and assume that the probability of each outcome is equal: \(P(X = i) = 1 / 6, i = 1,2,\ldots,6\). The probability of the event \(X = 1\) is then \(P(X = 1) = 1/6\). The probability of the event “outcome is not one” is \(P(X \neq 1) = P(X > 1) = 5 / 6.\)
References
Bertsekas, Dimitri P., and John N. Tsitsiklis. 2008. Introduction to Probability. Belmont, Mass: Athena Scientific.