SAMPLING AND ANALYSIS OF COMMERCIAL FATS AND OILS

APPARATUS

NIR measurements are performed with reflectance or transmittance instruments. These guidelines apply equally to both configurations.

1. In reflectance, one side of the sample is illuminated and the reflected light is detected. The light generally penetrates in the first few millimeters of samples. This requires samples to be homogeneous to limit scattering differences.

2. In transmittance, the light is sent through the sample contained in a cup or cuvette with a fixed path length.

3. Several measurements of the same sample are often averaged to increase the signal to noise ratio (S/N). S/N is proportional to the square root of the number of subsamples.

INSTRUMENT PERFORMANCE

1. Warm-up Time—Follow manufacturer recommendations to perform checks on the internal functioning of the spectrometer.

2. Environmental conditions—The instrument must be stored and operated within the conditions of temperature and humidity specified by the manufacturer.

3. Instrument Accuracy—Accuracy defines how close each measurements of the same sample are from each other. It should be tested on a minimum of 20 samples and 3 replicates for each sample. Instrument Accuracy is measured by the standard error of instrumental performance (SEIP) and comparisons are made with the reference measurement for each sample with the first replicate for each sample.

(1)

where

y_i = x_i – r_i represents the difference between the prediction for the first replicate and the reference value for sample i

y represents the average of y_i

n represents the number of sample

 

4. Repeatability—Also called precision, measures how close a measurement is from the accepted value. It should be tested on a minimum of 20 representative samples and 3 replicates for each sample. Repeatability is determined by calculating the standard deviation (SD) of the replicates across the entire sample population.

(2)

where

P_ij represents the predicted value for sample i and replicate j

P_i represents the average of the three predicted values for sample i

n represents the number of sample

 

5. Daily check—In addition to instrument self-checks and in substitution to Accuracy and Repeatability determination, it is possible to implement daily checks on samples that are usually analyzed by the lab (usually in duplicate). This allows a daily evaluation of instrument performances and the evaluation of instrumental variations due to environmental changes and hardware evolution

CALIBRATION SAMPLES

1. Representativity—Samples selected for calibration should be representative of the population of samples that the prediction model will encounter. The distribution of the sample should be as nearly uniform as possible. Including a type of samples that for one reason or another will not be predicted in the future may add extraneous noise and reduce the calibration performances. Algorithms have been developed to select the most appropriate samples to include in the calibration set. Depending on the application and calibration algorithm, the minimum number of samples will vary. For a grain calibration, a minimum of 100 samples over the entire range of the constituent is usually a standard.

2. Sample presentation—The sample presentation will vary by instrument and application. However, it is necessary to maintain a standardized process for an application in the presentation of the calibration, validation, and routine samples.

3. Temperature—Sample temperature causes wavelength shifts in the NIR spectrum. If various sample temperature conditions will be encountered, include samples over the range of the calibration whose temperature has been adjusted to 3 to 5 temperatures over the future range of use.

4. Timing—Reference measurements should be performed, on the samples kept in the calibration set, as soon as possible after the spectra were collected. Some elements such as moisture and other volatile constituents can vary over time. For other elements not sensible to environmental conditions, analysis can be done later if needed.

5. Approximation of the number of samples

REFERENCE MEASUREMENTS

1. NIR spectroscopy is an indirect method and its prediction error relies on the error of the laboratory. To measure this error, the standard error of the laboratory can be evaluated.

(3)

where

y_ij represents the concentration measured for sample i and replicate j

y_i represents the average values of the different replicates for sample i

n represents the number of samples

r represents the number of replicates

A representative block of samples (n > 20) should be assayed repeatedly to establish and update SEL.

2. The error of the prediction model will be a function of the SEL. A calibration model can be, at most, as good as the reference method.

DATA PROCESSING

1. Preprocessing or pretreatment methods are techniques used to enhance, smooth, correct, and scale raw spectra.

2. The most common preprocessing methods are:

3. Consult manufacturer for the list with the various preprocessing methods that are compatible with their instrument software.

CALIBRATION DEVELOPMENT

1. For cases where the relationship is linear and the collinearity among wavelength is low, multiple linear regression (MLR) can be used.

2. For highly correlated spectral data, principal component regression (PCR) and partial least squares (PLS) regression methods have been developed.

3. When dealing with non-linear relationships, learning-based methods such as artificial neural networks (ANN) and support vector regression (SVR) methods can be employed.

4. Most instruments can easily handle calibration models developed using MLR, PCR, and PLS. ANN and SVR require an additional prediction engine because they are not included in most chemometrics suite.

VALIDATION OF THE CALIBRATION MODEL

1. Any calibration model must be validated on samples not included in the calibration set. If a totally independent validation set is available (next batch samples, next year samples), the validation parameters will estimate the true model performances. When independent samples are not available, it is possible to keep a part of the calibration set and use it for validation or to employ cross-validation methods. These methods overstate performances and are normally used only for initial concept studies.

2. If sample temperature is of concern, samples at various temperatures should be included in the validation set.

3. Prediction precision—It can be measured either:

• During calibration—The standard error of calibration (SEC) will provide a first evaluation of the predictive ability of the model. It is calculated by predicting all calibration samples with the model. It generally provides over-optimistic statistics.

• During cross-validation—The standard error of cross validation (SECV) is calculated by withholding predetermined groups of samples from the calibration set, developing new models from the remaining samples, and finally predicting left apart samples. The operations is repeated by including withholded samples in the calibration set and predicting a second set of samples isolated for validation. Prediction statistics are combined to output cross-validation error. SECV is usually a better indicator of method precision than SEC. Different cross-validation methods can be used based on the nature of the data.

• During operation—The standard error of prediction (SEP) is calculated by predicting “unknown” samples. SEP is the best estimate of model precision of a model given the variability encountered in validation is present in calibration. SEC/SECV/SEP are calculated as follows:

(4)

 

where

ŷ_ip represents the predicted value of the sample i in prediction scenario

y_i represents the reference value for the sample i

n represents the number of samples

 

Note that this is a bias adjusted formula. The root mean square error, including the bias is presented later.

 

Relative predictive determinant (RPD) is another common statistics for precision. RPD is the ratio of the standard deviation of the reference values of the set to predict to the SEC/SECV/SEP, when applied to this same set. RPD values can be interpreted using the following scale.

4. Prediction accuracy—Bias is often used to characterize the prediction model accuracy in validation situations along with SEP.

(5)

where

ŷ_i represents the predicted value of the sample i

y_i represents the reference value for the sample i

n represents the number of samples

 

Provided the primary (destructive) method is used to calibrate the secondary (non-destructive) method, the bias generally should be very close to zero. If not, of course, it’s time to recalibrate.

Another statistics combining both precision and accuracy is the root mean scare error of calibration / cross-validation / prediction and is calculated as follows:

(6)

Of course, for all of these calculations, it is important to have a sufficiently large data set (replicates over multiple days, multiple analysts, etc.), and when performing calibration and cross-validation, it is important that the sample set be diverse enough to represent the range of values to be expected in practice.

5. Impact of other sources of error. As mentioned earlier, the final error of a calibration model is function of many parameters, the most common being the error of the laboratory, the sampling error, and the instrumental error.

CALIBRATION TRANSFER

1. It is often necessary to use a calibration model, developed on a master unit, on a secondary unit. This is cheaper than developing a model on the secondary unit. Differences in instrumental properties (absorption, S/N, wavelength alignment) can exist and it is necessary to correct for them.

2. Most of the standardization methods require standardization samples run on both master and secondary units Standardization samples must be representative of the constituent range and be well predicted by the calibration. It is possible that a different standardization set exist for different parameters on the same product. A minimum of 20 samples should be used.

3. Optical methods—They are based on the modification of the spectra collected on the secondary unit to match the spectra on the master unit. Methods such as single wavelength standardization, direct standardization, and piecewise direct standardization can be used. They calculate for each wavelength a slope and a bias that are applied to the spectra of the secondary units.

4. Post-regression correction—A slope and a bias are calculated to match predictions of the standardization set on both master and secondary units.

5. Robust models—It is possible to include in the calibration model, spectra collected on the secondary unit. The variability is modeled by the regression algorithm and no further standardization is required. This can be complex for large numbers of secondary units.

6. A post-standardization set of a minimum of 20 samples (independent from the calibration, validation, and standardization sets) should be used to validate the standardization method.

ROUTINE TEST

1. Follow manufacturer specifications for warm-up time and self-check procedure. Use a daily check to develop control charts and manage the evolution of the instrument and sample over time.

2. Perform frequent instrument check performances of accuracy and repeatability to ensure proper functioning.

3. Prepare samples in the same way they were prepared during the calibration process.

OUTLIER DETECTION

1. Outliers are samples that are statistically different from the population of interest. This difference can come from hardware failures, from sample degradation, or from poor reference values.

2. The detection of such samples is typically done with a Hotelling’s T² test—a multivariate method that tests the membership of an observation to a group—at a given confidence interval, or by detecting samples in the calibration process that present an abnormally large residual value.

3. Obvious outliers must be removed. However, “abnormal” samples can be representative of a new variability that should be included in the calibration set. There is no substitute for an understanding of the physical or biological system.

MOISTURE BASIS

1. Moisture basis—It is the moisture basis at which the constituent concentration was measured in. The moisture basis is often converted to trade standards such as 12% for wheat, 13% for soybean, and 15% for corn. It is also possible to express predictions in dry basis (0%) or “as-is”.

2. Native moisture basis—Calibration models will often be developed with constituents converted to a specific moisture basis (direct moisture basis models) or “as-is”. As-is predictions will often require conversion to a constant moisture basis based on the measure of the moisture at the time of the NIR analysis. Most instruments can perform this conversion. Direct moisture basis calibrations will provide unacceptable results if the moisture of the sample is not in the range of samples present in the calibration set.

CALIBRATION OF CORRELATED ELEMENTS

1. Often, the parameter to measure will be highly correlated with other constituents that are commonly predicted by NIR.

2. In some cases, using a simple model, based on common predicted parameters to predict the correlated elements, will provide accurate results and be more economically viable than developing an NIR calibration. However, using indirect measurements such as NIR predictions to predict another element with a simple model can quickly become unstable because of the multiplication of error sources.

3. It is possible that some samples will not follow a general pattern of correlation with one or more easily predicted constituents and will be miss-predicted. A strong validation strategy must be employed to ensure the validity of these third level models.

REFERENCE

AACC International. 2000. Approved Methods of the American Association of Cereal Chemists, 10th Ed. Methods 39-00 and 39-01. The Association: St. Paul, MN.

ASTM International. 2005. Standard Practice for Infrared Multivariate Quantitative Analysis, E 1655—05.

National Conference on Weight and Measures. 2002. Grain Moisture Meters and Near Infrared Grain Analyzers, NCWM Publication 14.

U.S.Pharmacopeia. 2006. Near-Infrared Spectrophotometry, method 1119.

Williams P., 2001. Implementation of Near-Infrared Technology. In Near-Infrared Technology in the Agricultural and Food Industries, Williams P. and Norris K. (editors), 2nd ed., AACC Inc., St. Paul, MN, USA.