The course will introduce computer intensive methods
of statistical analysis and their application to statistical modeling.
Computer-intensive methods use resampling and repeated
simulations to calculate standard errors, confidence intervals and
significance tests or, more in general, to assess the quality of a
statistical models. These methods are not only in general use by statisticians
but are also applied by quantitative researchers in the life sciences,
social sciences and business. The methods apply for any level of modeling
and are well reputed for their easy understanding and implementation.
The course will emphasize the practical side of resampling methods
by illustrating the theoretical issues with practical applications to
data analysis, statistical modeling and machine learning.
The course will be supplemented by a set of computer based
examples (using the Matlab and the R language).
Prerequisites: basic notions of probability and statistics.
- Thursday from 10 to 12 at room A2.222 (Campus La Plaine)
- First lesson (6th of March)
Re-sampling methods for
- Basic concepts of statistical inference
Modeling and Data analysis.
- Parametric bootstrap
- Nonparametric bootstrap
- Permutation tests
- Randomization tests
- Bootstrap tests
for model assessment and selection.
- Regression modeling
- Multiple linear regression
- Empirical error and Final Prediction Error
- Nonlinear models (Supervised learning)
- Examples of nonlinear models
- Parametric identification
- Structural identification
- Model assessment and selection
- The bias/variance trade-off
Model selection and model averaging
- PRESS statistic
- The .632 bootstrap estimator
Matlab and R scripts (see slides for
- B. Efron, R. Tibshirani (1993) An Introduction
to the bootstrap Chapman and Hall.
- T. Hastie, R. Tibshirani, J. Friedman (2001) The
Elements of Statistical Learning. Springer
- M. R. Chernick (1999) Bootstrap Methods: a practitioner's
- Duda,Hart,Stork (2001) Pattern Classification
(2nd ed). Wiley
- A.C. Davison, D.V. Hinkley (1997) Bootstrap Methods
and their Applications Cambridge University Press.
- J. Hjorth (1994) Computer Intensive Statistical
Methods Chapman & Hall.
Bootstrap Methods: Another Look at the Jackknife
Annals of Statistics, Vol. 7, No. 1. (Jan., 1979), pp.
A Leisurely Look at the Bootstrap, the Jackknife, and Cross-Validation
Bradley Efron; Gail Gong
The American Statistician, Vol. 37, No. 1. (Feb., 1983),
Bootstrap Methods for Standard Errors, Confidence Intervals,
and Other Measures of Statistical Accuracy
B. Efron; R. Tibshirani
Statistical Science, Vol. 1, No. 1. (Feb., 1986), pp. 54-75.
Jackknife, Bootstrap and Other Resampling Methods in Regression
(in Invited Paper)
C. F. J. Wu
Annals of Statistics, Vol. 14, No. 4. (Dec., 1986), pp. 1261-1295.
Discussion: Jackknife, Bootstrap and Other Resampling Methods
in Regression Analysis
(in Invited Paper)
Annals of Statistics, Vol. 14, No. 4. (Dec., 1986), pp. 1301-1304.
Combining Estimates in Regression and Classification
Michael LeBlanc; Robert Tibshirani
Journal of the American Statistical Association, Vol. 91, No. 436.
(Dec., 1996), pp. 1641-1650.
Heuristics of Instability and Stabilization in Model Selection
The Annals of Statistics, Vol. 24, No. 6. (Dec., 1996), pp. 2350-2383.
and Arcing Classifiers
Leo Breiman (1996)
Technical Report 460, Statistics Department, University of California
Leo Breiman (1996)
Resampling Stats homepage
An Annotated Bibliography for Bootstrap Resampling
- Tibshirani's home
Boosting Research Site: Boosting.org
Combination of estimators: Volker Tresp's Home Page
Intelligent data analysis
Data mining glossary
Data Mining: a short tutorial
The Comprehensive R Archive Network
R mailing lists archive
CRAN: R News
ESS -- Emacs Speaks Statistics
R: Package Index
CRAN - Package Sources
CRAN: books Contributed Documentation
MATLAB: The MathWorks
The MathWorks -page francaise
Octave, un clone gratuit de Matlab
US NAVY MATLAB tutorial
A practical introduction to MATLAB
Books and Tutorials on MATLAB
Open source project MatLinks/Chorus on Matlab/Octave