Statistical and Mathematical Methods for Data Science

Statistical and Mathematical Methods for Data Science

Credit Hours: 3

Prerequisites: None

Date : 7 June 2018

Course Contents:

Probability:

Probability basics (axioms of probability,

https://www.thoughtco.com/what-are-probability-axioms-3126567

https://en.wikipedia.org/wiki/Probability_axioms

https://math.tutorvista.com/statistics/probability-axioms.html

http://pages.cs.wisc.edu/~dsmyers/cs547/lecture_6_probability.pdf

conditional probability,

http://sphweb.bumc.bu.edu/otlt/MPH-Modules/BS/BS704_Probability/BS704_Probability3.html

http://onlinestatbook.com/2/probability/basic.html

random variables,

https://projects.ncsu.edu/crsc/events/ugw06/presentations/scheywar/finalundergrad.pdf

https://www.linkedin.com/learning/statistics-foundations-2/probability-and-random-variables

expectation, independence,

https://en.wikipedia.org/wiki/Independence_(probability_theory)

https://www.wyzant.com/resources/lessons/math/statistics_and_probability/probability/further_concepts_in_probability

https://www.cse.iitk.ac.in/users/piyush/courses/pml_fall17/material/probabilty_tutorial.pdf

etc.), (Ignored)

https://www.wikihow.com/Use-%22Etc.%22-Correctly

multivariate distributions,

http://idiom.ucsd.edu/~rlevy/pmsl_textbook/chapters/pmsl_3.pdf

Maximum a posteriori and maximum likelihood estimation;

https://www.quora.com/What-is-the-difference-between-Maximum-Likelihood-ML-and-Maximum-a-Posteriori-MAP-estimation

https://en.wikipedia.org/wiki/Maximum_a_posteriori_estimation

https://wiseodd.github.io/techblog/2017/01/01/mle-vs-map/

http://blog.ivansiu.com/blog/2014/09/24/notes-on-maximum-likelihood/

https://www.probabilitycourse.com/chapter9/9_1_2_MAP_estimation.php

https://en.wikipedia.org/wiki/Maximum_a_posteriori_estimation

Statistics:

introduction to concentration bounds,

https://www.cs.utah.edu/~jeffp/teaching/cs7931-S15/cs7931/6-matrix-concentration-bounds.pdf

https://www.cs.cornell.edu/~sridharan/concentration.pdf

https://www.stat.berkeley.edu/~mjwain/stat210b/Chap2_TailBounds_Jan22_2015.pdf

laws of large numbers,

https://www.khanacademy.org/math/statistics-probability/random-variables-stats-library/expected-value-lib/v/law-of-large-numbers

https://www.britannica.com/science/law-of-large-numbers

https://en.wikipedia.org/wiki/Law_of_large_numbers

central limit theorem,

http://stattrek.com/statistics/dictionary.aspx?definition=Central_limit_theorem

https://en.wikipedia.org/wiki/Central_limit_theorem

https://www.khanacademy.org/math/ap-statistics/sampling-distribution-ap/sampling-distribution-mean/v/central-limit-theorem

https://www.youtube.com/watch?v=JNm3M9cqWyc

minimum mean - squared error estimation,

https://en.wikipedia.org/wiki/Minimum_mean_square_error

https://ocw.mit.edu/courses/electrical-engineering-and-computer-science/6-011-introduction-to-communication-control-and-signal-processing-spring-2010/readings/MIT6_011S10_chap08.pdf

https://math.tutorvista.com/statistics/mean-squared-error.html

confidence intervals;

http://stattrek.com/estimation/confidence-interval.aspx

https://www.khanacademy.org/math/statistics-probability/confidence-intervals-one-sample

https://www.youtube.com/watch?v=lwpobQmUTd8

Linear algebra:

Vector spaces,

https://www.khanacademy.org/math/linear-algebra/vectors-and-spaces

https://www.youtube.com/watch?v=XDvSsDsLVLs

Projections (will also cover the least regression),

https://en.wikipedia.org/wiki/Projection_(linear_algebra)

http://math.mit.edu/~gs/linearalgebra/ila0403.pdf

https://ww2.amstat.org/publications/jse/v18n1/kachapova.pdf

linear transformations,

https://www.khanacademy.org/math/linear-algebra/matrix-transformations/linear-transformations/v/linear-transformations

http://linear.ups.edu/html/section-LT.html

https://en.wikibooks.org/wiki/Linear_Algebra/Linear_Transformations

singular value decomposition (this substitute for PCA),

https://jeremykun.com/2016/04/18/singular-value-decomposition-part-1-perspectives-on-linear-algebra/

http://andrew.gibiansky.com/blog/mathematics/cool-linear-algebra-singular-value-decomposition/

https://ocw.mit.edu/courses/mathematics/18-06sc-linear-algebra-fall-2011/positive-definite-matrices-and-applications/singular-value-decomposition/

https://en.wikipedia.org/wiki/Singular-value_decomposition

eigen decomposition,

https://eigen.tuxfamily.org/dox/group__TutorialLinearAlgebra.html

https://www.youtube.com/watch?v=HWnCv4iHCDc

https://cseweb.ucsd.edu/~dasgupta/291-unsup/lec7.pdf

power method;

https://math.stackexchange.com/questions/89572/understanding-power-method-inverse-iteration-in-linear-algebra

https://en.wikibooks.org/wiki/Linear_Algebra/Topic:_The_Method_of_Powers

https://en.wikipedia.org/wiki/Power_iteration

Optimization:

Matrix calculus with Lagrange Multipliers,

https://www.khanacademy.org/math/multivariable-calculus/applications-of-multivariable-

https://math.stackexchange.com/questions/2116137/explicit-derivation-of-lagrange-multipliers

https://math.stackexchange.com/questions/2019471/lagrange-multipliers-on-matrix-function

http://www.cs.cmu.edu/~ggordon/lp.pdf

http://tutorial.math.lamar.edu/Classes/CalcIII/LagrangeMultipliers.aspx

derivatives/constrained-optimization/a/lagrange-multipliers-examples

http://people.duke.edu/~hpgavin/cee201/LagrangeMultipliers.pdf

https://www.khanacademy.org/math/multivariable-calculus/applications-of-multivariable-derivatives/constrained-optimization/a/lagrange-multipliers-single-constraint

gradient descent,

https://machinelearningmastery.com/linear-regression-tutorial-using-gradient-descent-for-machine-learning/

https://math.stackexchange.com/questions/1652661/gradient-descent-l2-norm-regularization

coordinate descent,

https://www.youtube.com/watch?v=uvgru6FcyxU

http://www.math.tamu.edu/~yvorobet/MATH304-2011A/Lect2-07web.pdf

https://en.wikipedia.org/wiki/Linear_algebra

http://www.math.ucla.edu/~wotaoyin/papers/coordinate_descent.html

introduction to convex optimization.

https://ocw.mit.edu/courses/electrical-engineering-and-computer-science/6-079-introduction-to-convex-optimization-fall-2009/lecture-notes/MIT6_079F09_lec15.pdf

https://web.stanford.edu/~boyd/cvxbook/bv_cvxbook.pdf

https://ocw.mit.edu/courses/electrical-engineering-and-computer-science/6-079-introduction-to-convex-optimization-fall-2009/lecture-notes/

Teaching Methodology: Lectures, Problem based learning

Course Assessment:

Sessional Exam, Home Assignments, Quizzes, Project, Presentations, Final Exam

Reference Materials

Books:

1. Probability and Statistics for Computer Scientists, 2^nd Edition, Michael Baron

2. Linear Algebra and Its Applications, 5^th Edition, David C. Lay and Steven R. Lay

3. Introduction to Linear Algebra, 5^th Edition, Gilbert Strang

4. Probability for Computer Scientists, online Edition, David Forsyth.