Goodness of

A model should provide a good quantitative fit to the data. A failure to explain a high percentage of the variance may be due to variability of the data or a mismatch between the true model and the fitted model. Even if the percentage of variance accounted for is very high, and not due to fitting the noise, the fitted model may not be the true model. This can most readily be determined by examination of the pattern of differences between the data and the model (the residuals). Ideally, the pattern of residuals will be random. If the residuals are systematic, then there is a failure to identify some aspects of the true model. Of course, whether small systematic deviations are important for revealing the underlying mechanism or are due to some disturbance unrelated to the major problem is a matter for scientific judgment.

However, a good quantitative fit of a model to the data is not a sufficient criterion for acceptance of a theory. There is the important concern that a model with a large number of parameters may be fitting random variability in the particular sample of data being examined rather than fitting the true model (Roberts and Pashler, 2000; Zucchini, 2000). This is called overfitting, and various approaches have been used to correct for the number of free parameters. Probably the simplest approach is to estimate the parameters of the model based on some of the data and to apply these estimates to other data. This is called cross-validation.

If the data are reasonably regular, scalar timing theory often accounts for more than 95% of the variance and, in some cases, even more.

0 0

Post a comment