StatsWAP2009Aug07

Nonlinear Regression Models

• Brian Caffo's websiite: http://www.biostat.jhsph.edu/~bcaffo/
• Series Home: http://putter.ece.jhu.edu/StatsWAP

Resources

• Slides will be available here
• R-code will be available here

Notes

• Not covered: kernel smoothing, local weighting, moving averages, binning, loess (local estimation) etc.
• Non-parametric regression -
  • can factor in <math>y=f(x)+other stuff </math>
  • confounding effects
  • interactions
  • can generalize to discrete and/or multivariate responses (logistic regression, etc.)
• Example bases
  • linear
  • polynomial (Taylor series expansion)
    • why not?
    • it works... sort of
    • not good for smoothing: not "localized", not "parsimonious" ==> takes a lot of terms to get non-exactly polynomial
  • See slide on general functions for tips on selected basis sets.
    • wavelet bases - smooth trends and spikes
      • can be "same" as wavelet transform, slowly
    • trigonometric (Fourier) - "frequency concept"
      • can be "same" as Fourier transform, slowly
    • Spline bases - general smoothing
      • We'll talk about these today. Good for general smoothing. General purpose, but do not preserve spikes.
• Pick the basis for the eventual goal.

Spline Bases

• Highly controversial topic on spline fitting: fun reading [1]
• Gauss and the "invention" of least squares: [2]
• Big question: where to put the knots
  • known inflection points
  • quantiles
• Only a few knot points can be fit with LS
• If # parameters ~ # of data points -> fitting problems.
• B-spline bases are equivalent, but the bases are much closer to orthogonal. Much more efficient and stable for solution than truncated polynomial terms. Slight change of bases, but fitted functions are the same.