Leonard Mlodinow 14 Euclid’s Window: the Story of Geometry from Parallel Lines to Hyperspace Feynman’s Rainbow: a Search for Beauty in Physics and in ...

0 downloads 27 Views 2MB Size

Dr. Prapun Suksompong [email protected]

Lecture 1 Office Hours: BKD 3601-7 Monday 14:40-16:00 Friday 14:00-16:00 1

Probability and Random Processes ECS 315

Dr. Prapun Suksompong [email protected]

Introduction Office Hours: BKD 3601-7 Monday 14:40-16:00 Friday 14:00-16:00 2

Course Organization Course Website:

http://www2.siit.tu.ac.th/prapun/ecs315/ Lectures: Tuesday 13:00-14:20 BKD 3510 Thursday 10:40-12:00 BKD 3510

Tutorial sessions: Tuesday 14.40-16.00 BKD 3510

Textbook: Probability and stochastic processes: a friendly introduction for

electrical and computer engineers

By Roy D.Yates and David J. Goodman

2nd Edition ISBN 978-0-471-27214-4 Library Call No. QA273 Y384 2005 Student Companion Site: 3

http://bcs.wiley.com/he-bcs/Books?action=index&itemId=0471272140&bcsId=1991

Course Web Site Please check the course website

regularly. Announcements References Handouts/Slides Calendar Exams HW due dates

www2.siit.tu.ac.th/prapun/ecs315/ 4

Course Website: Notes & Slides Notes will be posted before the corresponding lectures. Hard copies can also be purchased from the copy center.

In lectures… Notes will be highlighted and updated with examples /

comments. Some lectures may use slides. The slides and updated notes will be posted after the corresponding lectures. I also frequently use Microsoft OneNote on my tablet instead of the whiteboard. The files will be exported as pdf and posted after the corresponding lectures. Remind me the day after the lecture if the notes/slides from the day before are still not posted on the web. 5

“Les questions les plus importantes de la vie ne sont en effet, pour la plupart, que des problèmes de probabilité.”

“The most important questions of life are, for the most part, really only problems of probability.”

Pierre Simon Laplace (1749 - 1827)

6

“On voit, par cet Essai, que la théorie des probabilités n'est, au fond, que le bon sens réduit au calcul; elle fait apprécier avec exactitude ce que les esprits justes sentent par une sorte d'instinct, sans qu'ils puissent souvent s'en rendre compte.”

“One sees, from this Essay, that the theory of probabilities is basically just common sense reduced to calculus; it enables us to appreciate with exactness that which accurate minds feel with a sort of instinct, often without being able to account for it.”

Pierre Simon Laplace (1749 - 1827)

7

Levels of Study in Probability Theory Probability theory is the branch of mathematics

devoted to analyzing problems of chance.

11

Art of Guessing

1.

High School: classical

2.

Undergraduate: calculus

3.

Graduate: measure-theoretic

We are here!

More references Use ones that say probability

and random (or stochastic) processes If it has the word “statistics” in the title, it may not be rigorous enough for this class If it has the word “measure” or “ergodic” in there, it is probably too advanced. 12

The Drunkard's Walk The Drunkard's Walk: How

13

Randomness Rules Our Lives By Leonard Mlodinow Deals with randomness and people's inability to take it into account in their daily lives. A bestseller, and a “NY Times notable book of the year” Named “one of the 10 best science books of 2008” on Amazon.com. [Thai Translation: ชีวิตนี้ ฟ้ าลิขิต: การสุ่มเลือก ควบคุมบัญชา ทุกเรื่องราวในชีวิตของเรา]

Leonard Mlodinow Euclid’s Window: the Story of Geometry

from Parallel Lines to Hyperspace Feynman’s Rainbow: a Search for Beauty in Physics and in Life A Briefer History of Time

with Stephen Hawking an international best-seller that has appeared in 25 languages.

The Drunkard's Walk: How Randomness Rules our Lives Apart from books on popular science, he also has been a

screenwriter for television series, including Star Trek: The Next Generation and MacGyver.

14

Watch Mlodinow’s talk Delivered to Google employees About his book (“The Drunkard's Walk”)

http://www.youtube.com/watch?v=F0sLuRsu1Do 15

The Seven Card Hustle Take five red cards and two black cards from a pack. Ask your friend to shuffle them and then, without looking at the

faces, lay them out in a row.

Bet that they can’t turn over three red cards. Explain how the bet is in their favor. The first draw is 5 to 2 (five red cards and two black cards) in their

favor. The second draw is 4 to 2 (or 2 to 1 if you like) because there will be four red cards and two black cards left. The last draw is still in their favor by 3 to 2 (three reds and two blacks). The game seems heavily in their favor, but YOU, are willing to 16

offer them even money that they can’t do it!

The Seven Card Hustle Take five red cards and two black cards from a pack. Ask your friend to shuffle them and then, without looking at the

faces, lay them out in a row.

Bet that them can’t turn over three red cards. Explain how the bet is in their favor. The game seems heavily in their favor,

but YOU, are willing to offer them even money that they can’t do it!

17

[Lovell, 2006]

The Seven Card Hustle: Sol 5 43 2 7 6 5 7 5 3 5! 3! 4! 7 3! 2! 7! 3 1 5 4 3 765 2 7 18

[Lovell, 2006]

Monty Hall Problem (MHP): Origin Problem, paradox, illusion Loosely based on the American television game show

Let’s Make a Deal. The name comes from the show’s original host, Monty Hall. One of the most interesting mathematical brain teasers of recent times.

19

Monty Hall Problem: Math Version Originally posed in a letter by Steve Selvin to the American

Statistician in 1975. A well-known statement of the problem was published in Marilyn vos Savant’s “Ask Marilyn” column in Parade magazine in 1990: “Suppose you're on a game show, and

20

you're given the choice of three doors: Behind one door is a car; behind the others, goats.You pick a door, say No. 1, and the host, who knows what's behind the doors, opens another door, say No. 3, which has a goat. He then says to you, "Do you want to pick door No. 2?" Is it to your advantage to switch your choice?”

Marilyn vos Savant Vos Savant was listed in each edition of the Guinness Book

of World Records from 1986 to 1989 as having the “Highest IQ.” Since 1986 she has written “Ask Marilyn” Sunday column in Parade magazine

Solve puzzles and answer questions from readers

21

[ http://www.marilynvossavant.com ]

MHP: Step 0 There are three closed doors. They look identical.

22

MHP: Step 0 Behind one of the doors is the star prize - a car. The car is initially equally likely to be behind each door.

Behind each of the other two doors is just a goat.

23

MHP: Step 1 Obviously we want to win the car, but do not

know which door conceals the car. We are asked to choose a door. That door remains closed for the time being.

24

“Pick one of these doors”

MHP: Step 2 The host of the show (Monty Hall), who knows what is behind

the doors, now opens a door different from our initial choice. He carefully picks the door that conceals a goat.

25

We stipulate that if Monty has a choice of doors to open, then he chooses randomly from among his options.

MHP: Step 3

“Do you want to switch doors?”

Monty now gives us the options of either 1. 2.

sticking with our original choice or switching to the one other unopened door.

After making our decision, we win whatever is behind our door.

26

Monty Hall Problem: Question Assuming that our goal is to maximize our chances of winning the car, what decision should we make? Will you do better by

sticking with your first choice, or

by switching to the other remaining door? Make no difference?

27

Let’s play! 28

Interactive Monty Hall

29

http://montyhallgame.shawnolson.net/ http://www.shodor.org/interactivate/activities/SimpleMontyHall/ http://www.math.uah.edu/stat/applets/MontyHallGame.xhtml http://scratch.mit.edu/projects/nadja/484178 http://www.math.ucsd.edu/~crypto/Monty/monty.html

Interactive Monty Hall The New York Times’sVersion

http://www.nytimes.com/2008/04/08/science/08monty.html 30

Back to the boring administrative stuff! 31

Calendar (Google)

32

Calendar M 13-Jun-11 20-Jun-11 27-Jun-11 4-Jul-11 11-Jul-11 18-Jul-11 25-Jul-11 1-Aug-11 8-Aug-11 15-Aug-11 22-Aug-11 29-Aug-11 5-Sep-11 12-Sep-11 19-Sep-11 26-Sep-11 3-Oct-11 10-Oct-11 33

T 14-Jun-11 21-Jun-11 28-Jun-11 5-Jul-11 12-Jul-11 19-Jul-11 26-Jul-11 2-Aug-11 9-Aug-11 16-Aug-11 23-Aug-11 30-Aug-11 6-Sep-11 13-Sep-11 20-Sep-11 27-Sep-11 4-Oct-11 11-Oct-11

Lecture W 15-Jun-11 22-Jun-11 29-Jun-11 6-Jul-11 13-Jul-11 20-Jul-11 27-Jul-11 3-Aug-11 10-Aug-11 17-Aug-11 24-Aug-11 31-Aug-11 7-Sep-11 14-Sep-11 21-Sep-11 28-Sep-11 5-Oct-11 12-Oct-11

R 16-Jun-11 23-Jun-11 30-Jun-11 7-Jul-11 14-Jul-11 21-Jul-11 28-Jul-11 4-Aug-11 11-Aug-11 18-Aug-11 25-Aug-11 1-Sep-11 8-Sep-11 15-Sep-11 22-Sep-11 29-Sep-11 6-Oct-11 13-Oct-11

F 17-Jun-11 24-Jun-11 1-Jul-11 8-Jul-11 15-Jul-11 22-Jul-11 29-Jul-11 5-Aug-11 12-Aug-11 19-Aug-11 26-Aug-11 2-Sep-11 9-Sep-11 16-Sep-11 23-Sep-11 30-Sep-11 7-Oct-11 14-Oct-11

Exam

Please Double-Check Exam Dates!

Grading System Coursework will be weighted as follows:

Assignments Class Participation and Quizzes Midterm Examination

5% 15% 40%

•4 Aug 2011 TIME 09:00 - 12:00

Final Examination (comprehensive) •11 Oct 2011 TIME 09:00 - 12:00

Mark your calendars now! Late HW submission will be rejected. 34

40%

Class Participation NOT the same as class attendance! If you come only to receive, you will fall asleep. Do not simply sit quietly in the class.

Need interaction between lecturer and students. Ask question when there is something that you don’t

understand. Don’t be shy! It is very likely that your friends don’t understand it as well.

If you already understand what I’m presenting, SHOW ME! Point out the errors/typos. I will raise many issues/questions in class. Try to comment on them. 35

Class Participation (2) Record what you have done. Submitted before the midterm and before the final.

36

Policy We will start the class on time and will finish on time. Raise your hand and tell me immediately if I go over the time

limit. Does NOT mean that I will leave the room immediately after lecture. I will stay and answer questions.

Mobile phones must be turned off or set in silent mode. We may have some pop quizzes (without prior warning or

announcement) and in-class activities. Attendance and pop quizzes will be taken/given irregularly and randomly. Cheating will not be tolerated. 37

Policy (con’t) Feel free to stop me when I talk too fast or too slow. I will surely make some mistakes in lectures / HWs /

exams. Some amount of class participation scores will be reserved to

reward the first student who inform me about each of these mistakes. Points on quizzes/ exercises/ exams are generally based on

your entire solution, not your final answer. You can get full credit even when you have the wrong final

answer. You may get zero even when you write down a right answer without justification. 38

Help and Office Hours Get some help! Do not wait until the final exam time or after the grade is out. Right after lecture is always a good time to ask question.

Office Hours (BKD-3601) Time: Monday 14:40-16:00, Friday 14:00-16:00

Appointment can be made. Tutorial session can be arranged. Feel free to come to my office and chat! Don’t be shy.

39

Warning This class is difficult. Keep up with the lectures. Make sure that you understand the concepts presented in the

lecture before you go home. I will evaluate your understanding of the course regularly

through In class problems/activities Quizzes Exams

40

ECS315: Difficulty Combinatorics (counting) Solving word problems Not the main focus of this class but unavoidable if you want to

solve/consider interesting questions.

Calculus Can be messy

Concept of probability Most students do not learn probability until two or three

exposures to it.

Large number of definitions, formulas and equations No need to remember a lot of formulas if you understand them. 41

Prerequisite Working knowledge of calculus Some MATLAB skills for doing HWs and understanding in-

class demo Frequency domain analysis (Fourier transform) Reviewed in ECS 332.

Bell curve

Soon, we will need to find

42

1 2

e

x2 2

dx ?

1 x e 2 2

2

?

x

Course Outline

43

1.

Introduction, Classical probability, Mathematical background

2.

Counting Methods (Combinatorics)

3.

Probability Foundation

4.

Random variable (real-valued)

5.

Real-valued functions of a random variable

6.

Expectation, moment, variance, standard deviation, central moment

7.

MIDTERM: 4 Aug 2011 TIME 09:00 - 12:00

8.

Transform method: Characteristic function

9.

Random vectors

10.

Function of random variables/vectors

11.

Real-valued jointly Gaussian random vectors

12.

Summation of i.i.d. random variable and laws of large numbers

13.

Central limit theorem

14.

Random processes

15.

Cross-correlation and Power spectral density

16.

FINAL: 11 Oct 2011 TIME 09:00 - 12:00

Save the Trees! PowerPoint slides will be posted on web soon after class. Try to minimize the amount of printing! I don’t expect you to read the material that haven’t been presented. If you want, you could read the textbook.

Anything that I have written on my tablet will be saved and posted on

web soon after class as well.

Sometimes, I write on the whiteboard because I want to walk around. It is

also possible that something goes wrong with my tablet. In which case, someone should take a good note.

No need to take detailed lecture notes (if you don’t want to). Put all of your energy into understanding the material. Of course, there is always someone who will take good notes anyway and

you can (potentially) borrow or make a copy of the notes from them.

Have fun with the materials presented in class.

44

Need More Examples or Practice? Textbook in the library: Schaum’s

outline of theory and problems of probability, random variables, and random processes / Hwei P. Hsu. Call No. QA273.25 H78 1997 Free pdf textbook: Introduction to Probability by Grinstead and Snell http://www.dartmouth.edu/~chance /teaching_aids/books_articles/proba bility_book/book.html 45

Concluding remarks Get as much legitimate help as you can Participate actively in class and outside of class Record what you have done.

If you feel that the class is very easy, you might overlook

something. If you feel that the class is very difficult, you are probably not the only one who feel that way. Don’t give up. Chat with me. It takes me a long time to feel comfortable with these materials; yet, I

still make mistakes.

My notation can be different from the textbook. Every notation has some advantages and disadvantages. 46

Monty Hall Problem: a first revisit Assuming that our goal is to maximize our chances of winning the car, what decision should we make? Will you do better by

sticking with your first choice, or

by switching to the other remaining door? Make no difference?

47

Monty Hall Problem: vos Savant’s Answer

“You double your chances of winning by switching doors.”

48

Monty Hall Problem: Controversy Approximately 10,000 readers, including nearly 1,000 with PhDs (many of them math

professors),

wrote to the magazine

claiming the published solution was wrong. “You blew it,” wrote a mathematician from George Mason

University. From Dickinson State University came this: “I am in shock that after being corrected by at least three mathematicians, you still do not see your mistake.” 49

[Mlodinow, 2008, p 42-45]

Controversy (2) From Georgetown: "How many irate mathematicians are

needed to change your mind?" And someone from the U.S. Army Research Institute remarked, "If all those Ph.D.s are wrong the country would be in serious trouble." When told of this, Paul Erdos, one of the leading

mathematicians of the 20th century, said, "That's impossible." Then, when presented with a formal mathematical proof of the

correct answer, he still didn't believe it and grew angry. 50

Keep this problem in mind We will return to the Monty Hall problem another time later

in the semester. Here, it serves as an interesting problem that demonstrates how confusing probability theory can be.

51

Let’s learn some concepts so that we can analyze interesting examples!

52