Basic Concepts of Probability

Concepts and Definition

Statistical experiment

Any procedure that produces data or observations.

Sample space S

All the possible outcomes

The sample space is itself an event and called a sure event. The null space {} is itself an event and called a null event.

Sample points

A possible outcome (element) in sample space

Event

A subset of sample space

Probability Theory

Event Operations

• Union
• Intersection
• Complement and

Event Relationships

• Mutually Exclusive/Disjoint: if
• Contained: is contained in if
• Equivalent: if and , .

De Morgan's Law

Event Operations:

• Distributive Law
• Distributive Law

Multiplication Principle If different experiments are to be performed sequentially with outcomes, there are number of possible outcomes.

Addition Principles If an experiment can be done in different procedures that do not overlap, there are

Permutation The selection and arrangement of objects out of . Order is taken into consideration

Combination The selection of objects out of where order is not taken into consideration

Probability

Probability

Understood as the chance or how likely a certain event may occur.

Can be interpreted as relative frequency:

Can be defined on the probability space with the axioms:

• • Probability of sample space (sample event) is 1
• • Probability of two mutually exclusive events happening (using addition rule)

Basic Properties of Probability

1. Proposition 2 Probability of the empty set is
2. Proposition 3 If are mutually exclusive events, that is for any , then
3. Proposition 4 For any event A,
4. Proposition 5 For any events ,
5. Proposition 6 For any two events , the inclusion exclusion principle holds:

Inclusion Exclusion Principle

6. Proposition 7 If , then .

Finite Sample Space with Equally Likely Outcomes

If there is a sample space with events (e.g ) where all outcomes are equally likely to occur, for any event , .

Conditional Probability

Useful for computing the probability of some events when some partial information is available. Specifically useful for computing the probability of event given an event has occured.

Conditional Probability

The conditional probability is the conditional probability of the event given that event has occurred.

For any two events with , the conditional probability of given that has occurred is

can then be considered here as a reduced sample space.

Using the multiplication rule, we can get:

or

The inverse probability formula also states:

Independence

Independence refers to whether an event affects the other event. For example, is independent from , if learning that occurred does not affect the probability of occurring.

Independence

Two events and are independent if and only if

This can also be written as:

If A and B are not independent, they are dependent, noted as

Thus, from this, if ,

Conditional probability

If are independent,

Law Of Total Probability

Partition

If are mutually exclusive events, and , the set is a partition of .

Thus, we get the theorem:

Theorem 11: Law Of Total Probability

If the set is a partition of , then for any event ,

Special Case A,B,

Bayes Theorem

Bayes' Theorem

Relates to .

Let the set be a partition of , then for any event and

Special Case n = 2, {A, A'} is a partition of S

The Bayes’ Theorem is derived from Conditional Probability, multiplication rule, and the Law Of Total Probability.