To describe the movements of all known celestial bodies, Ptolemy used abstract mathematical functions (cycles and epicycles). He obtained a remarkably good fit.
However, the good fit did not help in predicting future observations.
Aristarchus of Samos (~310-230 BC), and later Copernicus (AD 1473-1545), based their computations on a physical basis, the heliocentric hypothesis. However, their fit was not better than Ptolemy’s.
Kinetic validation is done to find out whether the model is quantitatively consistent with the experimental tracer kinetic data. Inconsistent models are rejected; usually, the kinetic and biochemical information gained in the validation studies is useful for reformulating the model. Mismatches are revealed by the existence of nonrandom residuals; the difference between the experimental data and the response function fitted to the data has some time-dependent pattern (Motulsky and Ransnas, 1987).
The standard errors of the estimates also needs to be considered to determine the reliability of the estimated parameters. Large standard errors of the estimates usually mean that the data are very noisy or that the number of variable parameters in the model is more than necessary (over-parameterization). Statistical methods (Schwartz criterion, Akaike Criterion, F test; see Landaw and DiStefano 1984 or Giatting et al. 2007) that consider the goodness of fit versus the number of parameters in the model should be used to guide the selection among alternative models. The noise level in the data affects the number of parameters that may be estimated from the data and usually is the primary determinant of the precision in the estimated parameters. To determine the range of optimized parameter values, Monte Carlo simulations are recommended. The alternative approach, using covariance matrices, requires the assumption that the process being modelled is linear. Since most of the kinetic models of metabolism are nonlinear, this assumption is invalid (Graham 1985).
The choice of the model should not be done solely according to identifiability criteria (e.g. F test). A non-identifiable model may permit the determination of the range of some meaningful parameters, while with an identifiable model one may compute the exact value of some parameters deprived of any physical or physiological meaning.
If the correlation between two estimated parameters is very high, then the likelihood of determining a unique set of values for the parameters is low (Budinger et al. 1985); this suggests that either of the parameters should be fixed or the model should be reduced.
A critical test of the validity of a model is to compare the biochemical results predicted from the kinetic data by the model with those measured directly by chemical means (e.g. microdialysis or biopsy samples). These studies not only provide the time course of total radioactivity in plasma and in the target organ, but also differentiate between the original tracer and its metabolites.
Intervention studies can be performed to estimate the sensitivity of the kinetic data and estimates to the physiologic parameter of interest. Model parameters of interest should change in the proper direction and by an appropriate magnitude in response to a variety of biologic stimuli. In addition, it is useful to test whether the parameters of interest do not change in response to a perturbation in a different factor, e.g. does an estimate of receptor density remain unchanged when blood flow is increased (Carson 1991).
The assumptions made in the reduction of model should be tested, either by explicit experimentation or at least by computer simulation. Simulations provide the magnitude of bias in the parameter estimates due to errors in various assumptions.
The absolute accuracy of model parameters should be tested with a "gold standard", if one is available for the measurement of interest (e.g., the microsphere technique for blood flow measurements). It should be noted that e.g. scanner resolution and change in the physical meaning of parameters due to model reduction can make the comparison of results difficult.
The validation of a PET method for a neuroreceptor assay can be done by comparing the results with those obtained from in vitro binding experiments. However, caution must be exercised in these comparisons. Binding discrepansies between in vitro and in vivo conditions have been observed. In in vitro receptor binding experiments, the concentration of the unbound ligand is uniform, while the uniformity of the unbound ligand concentration in tissue in vivo is less likely to be maintained owing to a multiplicity of factors such as the fast binding-rebinding phenomenon in the synaptic zone and diffusional restrictions. In addition, the ligand binding characteristics of receptor systems have been shown to have large species-to-species differences in both KD and Bmax values (Huang et al. 1986).
In some cases, PET method can be compared to traditional Fick's method, if venous blood sampling from the organ of interest is possible.
Test/retest studies provide an estimate of the within-subject variability (lack of reproducibility) of the method. However, reproducibility is only one aspect of the problem. A simple ratio provides more reproducible results than a model based method. Yet, the model based method might reveal true between-subject or treatment differences that are ignored or "normalized" by empirical methods.
Repeatability is estimated by calculating the mean and standard deviation of the absolute values of the difference between test and retest values. Repeatability coefficient (RC) is recommended by the British Standards Institution (1976), and when applied to PET test-retest studies, it is defined as RC = 2 * SD(scan1 - scan2). Assuming that the data is normally distributed, in 95% of the cases the difference between the two measurements will be less than the repeatability coefficient (Bland and Altman, 1986). To facilitate comparisons across regions of interest, test-retest variability (TRV)can be calculated as the absolute value of the difference between test and retest values, divided by the mean of both measurements (Parsey et al., 2000). The mean TRV percentage ± SD should be reported.
The intraclass correlation coefficient (ICC) (Shrout and Fleiss, 1979) is a useful measure of reliability, as this statistical parameter compares the within-subject (WS) variability to the between-subject (BS) variability in repeated observations.
In the one-way ANOVA model, BSMSS is the mean sum of squares between subjects, WSMSS is the mean sum of squares within subjects, K is the number of repeated observations (K=2 in test-retest study) and N is the number of subjects.
Negative ICC value indicates that more differences are observed within than between subjects. ICC ranges between -1 (no reliability, i.e. BSMSS=0) to 1 (maximum reliability, achieved in the case of identity between test and retest, i.e. WSMSS=0) (Parsey et al. 2000). As a rule, the method with the highest reliability (higher ICC) should be chosen (Laruelle 1999).
Ludbrook (2010) has reviewed linear regression methods for comparing two methods or to measurers.
Good parameter should also have low interobserver variability. Interobserver variability can be assessed by using Bland-Altman analysis (Bland & Altman, 1986) and Lin's concordance correlation coefficient (CCC) (Lin, 1989; Barnhart et al., 2002).
Parameters for assessing the effectiveness of therapy should, in addition to low interobserver variability, also have the ability to differentiate between treatment responders and nonresponders. These qualities can be combined in the variability effect coefficient (VEC) (Benz et al., 2008).
Barnhart HX, Haber M, Song J. Overall concordance correlation coefficient for evaluating agreement among multiple observers. Biometrics 2002; 58:1020-1027.
Benz MR, Evilevitch V, Allen-Auerbach MS, Eilber FC, Phelps ME, Czernin J, Weber WA. Treatment monitoring by 18F-FDG PET/CT in patients with sarcomas: interobserver variability of quantitative parameters in treatment-induced changes in histopathologically responding and nonresponding tumors. J Nucl Med. 2008; 49(7): 1038-1046.
Bland JM, Altman DG. Statistical methods for assessing agreement between two methods of clinical measurement. Lancet 1986; 2: 307-310.
British Standards Institution. Precision of test methods I: Guide for the determination and reproducibility for a standard test method. British Standard 5497; 1979.
Budinger TF, Huesman RH, Knittel B, Friedland RP, Derenzo SE (1985): Physiological modeling of dynamic measurements of metabolism using positron emission tomography. In: The Metabolism of the Human Brain Studied with Positron Emission Tomography. (Eds: Greitz T et al.) Raven Press, New York, 165-183.
Garfinkel D. Computer modeling, complex biological systems, and their simplifications. Am J Physiol 1980; 239: R1-R6.
Glatting G, Kletting P, Reske SN, Hohl K, Ring C. Choosing the optimal fit function: Comparison of the Akaike information criterion and the F-test. Med Phys. 2007; 34(11): 4285-4292.
Huang SC, Phelps ME (1986): Principles of tracer kinetic modeling in positron emission tomography and autoradiography. In: Positron Emission Tomography and Autoradiography: Principles and Applications for the Brain and Heart. (Eds: Phelps M, Mazziotta J, Schelbert H) Raven Press, New York, 287-346.
Johnson M, Karanikolas BDW, Priceman SJ, Powell R, Black ME, Wu H-M, Czernin J, Huang S-C, Wu L. Titration of variant HSV1-tk gene expression to determine the sensitivity of 18F-FHBG PET imaging in a prostate tumor. J Nucl Med. 2009; 50(5): 757-764.
Laruelle M. Modelling: when and why? Eur J Nucl Med 1999; 26, 571-572.
Lin LI. A concordance correlation coefficient to evaluate reproducibility. Biometrics. 1989; 45: 255-268.
Ludbrook J. Linear regression analysis for comparing two measurerers or methods of measurements: But which regression? Clin Exp Pharmacol Physiol. 2010; 37: 692-699.
Ogden RT, Ojha A, Erlandsson K, Oquendo MA, Mann JJ, Parsey RV. In vivo quantification of serotonin transporters using [11C]DASB and positron emission tomography in humans: modeling considerations. J Cereb Blood Flow Metab 2007; 27: 205-217.
Parsey RV, Slifstein M, Hwang D-R, Abi-Dargham A, Simpson N, Mawlawi O, Guo N-N, Van Heertum R, Mann JJ, Laruelle M. Validation and reproducibility of measurement of 5-HT1A receptor parameters with [carbonyl-11C]WAY-100635 in humans: comparison of arterial and reference tissue input functions. J Cereb Blood Flow Metab 2000; 20: 1111-1133.
Phair RD. Development of kinetic models in the nonlinear world of molecular cell biology. Metabolism 1997; 46:1489-1495.
Shrout PE, Fleiss JL. Intraclass correlations: uses in assessing rater reliability. Psychol Bull 1979; 86: 420-428.