April 22, 2002

Dear Mr. Hailer:

The United States Environmental Protection Agency (U.S. EPA) is in receipt of your March 6, 2002 and March 20, 2002 correspondence regarding laboratory and quality assurance issues associated with the Bloomington PCB sites. The U.S. EPA has been working with its contractors Tetra Tech and Earth Tech investigating the issues described in your correspondence and below is a point by point response.

PSARA's Results for the Split Samples Identified in Tetra Tech's Letter of April 28, 2000

• Determine whether the results were statistically different at a specified confidence level (for example, 95 percent or 99 percent) (see "Comparison of Split Sample Results" for details on this approach)
• Calculate the ratio of PSARA's laboratory result to Tetra Tech's laboratory result in order to determine whether the ratio was greater than 5.6 for the split soil samples (see "Comparison of Split Sample Results" for details on this approach)

Of the two approaches presented above, we could not use the first approach to compare the split sample results because the data set contained less than three pairs of split sample results that were above the laboratory reporting limits. Using the second approach, the EPA and Tetra Tech have concluded that the split sample results were not significantly different because the ratio of PSARA's laboratory result to Tetra Tech's laboratory result was not greater than 5.6 for either of the two samples; specifically, the ratios were 4.1 and 3.5 for Samples No. LL-SB-08-06 and LL-SB-E18-06, respectively.

Table 1. Split Sample Analysis Results for Perimeter Soil, Lemon Lane Landfill Site

Comparison of Split Sample Results for the Entire Project Duration

Regarding the comparison of split sample results, the QAPPstates that "Tetra Tech will notify the EPA Work Assignment Manager (WAM) of any differences between split sample results and CBS sample results. If the differences are determined to be significant, further action .... such as resampling or data evaluation may be required." However, as stated above, the QAPP does not define the term "significant difference." For this reason, the EPA and Tetra Tech used both statistical and nonstatistical approaches to compare the results when both laboratory results for a given split sample were above the reporting limits. A quantitative comparison of the results could not be performed when one or both laboratories' results for a given split sample were below the reporting limits. Table 2 lists the number of samples for which (1) both laboratories reported below detection limit (BDL) results, (2) only one laboratory reported a BDL result, and (3) both laboratories reported results other than BDL. The term BDL is used here to be consistent with the data evaluation reports; however, "below reporting limit" is a more accurate description of the actual occurrences.

For statistical comparison of split sample results, the null hypothesis was that the average of the differences between paired observations (Tetra Tech's laboratory and PSARA's laboratory results) was equal to zero. The null hypothesis was tested using either a parametric (Student's t- test) or a nonparametric test (Wilcoxon signed rank test). Before a parametric test was performed, the Wilk-Shapiro test was used to determine whether the Tetra Tech's laboratory results and PSARA's laboratory results or, when appropriate, their differences were normally distributed at the 95 percent confidence level. If the results or their differences were not normally distributed, the Wilk-Shapiro test was performed on transformed results (for example, logarithm and square root transformations) to verify the normality assumption. If the normality assumption was not met, a nonparametric test was performed. Nonparametric tests are not as powerful as parametric tests because the nonparametric tests do not account for the magnitude of the difference between the split sample results. Despite this limitation, when the normality assumption was not met, performing a nonparametric test was considered to be a better alternative than performing no statistical comparison.

Table 2 shows that the split sample results were statistically the same for (1) four of six data sets at the 99 percent confidence level and (2) three of six data sets at the 95 percent confidence level. As stated above, this statistical analysis did not include the sample results that were reported as below reporting limits by both laboratories; such an exclusion increases the probability of rejecting the null hypothesis that the results were statistically the same. However, it was difficult to accurately quantify the additional number of data sets that would have been concluded to be the same, if one were to somehow use the results reported as "BDL" by both laboratories. In summary, if the results reported as "BDL" by both laboratories were to be used in the statistical analysis, the number of data sets that are statistically the same would be greater than four at the 99 percent confidence level and greater than three at the 95 percent confidence level.

For nonstatistical comparison of split sample results, Tetra Tech calculated the ratio of PCB concentration reported by PSARA's laboratory to that of Tetra Tech's laboratory for each split sample, and compared the concentration ratio with the ratio established using the performance acceptance limits (PAL) for performance evaluation (PE) samples. If the concentration ratio for environmental samples was less than or equal to the PE sample PAL ratio, the environmental sample results were considered to be the same. The following example provides additional information on how this approach was applied to compare PSARA's laboratory results with Tetra Tech's laboratory results. According to Environmental Resource Associates (ERA), a reputable commercial provider of PE samples, a soil PE sample with a certified value of 12.1 mg/kg of Aroclor 1242 has PALs of 2.73 and 15.4 mg/kg. These PALs were based on ERA's historical data on percent recoveries and relative standard deviations (RSD) from multiple laboratories that had analyzed split samples prepared under controlled conditions (PE samples). The PALs were determined at the 95 percent confidence level using the following equation.

PALs = Certified Value x (Average Percent Recovery + 2 (Average RSD))

According to SW-846 (EPA 1996b), the 95 percent confidence limits should be treated as warning limits, whereas the 99 percent confidence limits should be treated as control limits. The 99 percent confidence limits are calculated using three times the average RSD in the above equation instead of two times the average RSD; 99 percent confidence limits are broader than the 95 percent confidence limits. ERA updates the PALs generally once a year to make sure that they reflect current performance of the participating laboratories.

From the above data, if a PE sample (split sample prepared under controlled conditions) with a certified value of 12.1 mg/kg was analyzed by two laboratories and the laboratories reported results of 2.73 and 15.4 mg/kg, both laboratories would have met the PALs at the 95 percent confidence level established using industry-wide performance data. Therefore, soil PCB results that are within a factor of 5.6 (the ratio of upper PAL [15.4 mg/kg] to lower PAL [2.73 mg/kg]) cannot be considered different. Using a similar approach for water samples, PCB results that are within a factor of 2.4 cannot be considered different (see EPA's letter dated February 22, 2002, for specific details regarding the factor 2.4). These factors established based on industry-wide analysis of PE samples are considered stringent criteria when applied to environmental split samples (particularly, for split soil samples) because PE samples prepared by commercial providers and provided to participating laboratories are split samples prepared under controlled conditions; compared to environmental split samples, the PE samples are expected to be much more homogenous and should be practically free from matrix interferences. However, the EPA and Tetra Tech have used this approach to evaluate how many of the split field samples would meet the stringent criteria.

Table 2 shows that a small number of environmental split samples did not meet the ratios established based on the PALs for the PE samples. Because the ratios were established based on the warning limits (95 percent confidence limits), Tetra Tech performed further analysis of the results. The analysis included examination of the laboratory data packages and also, an evaluation of the impact on the primary project objective, that is achieving site cleanup goals.

At the request of EPA, Tetra Tech's review of laboratory data packages did not reveal any quality assurance and quality control problems that would have led to the rejection of results reported by either laboratory. In general, four of five times PSARA's laboratory reported results greater than those reported by Tetra Tech's laboratory. There were only three soil samples for which Tetra Tech's laboratory reported results that were above 5.6 times the PSARA's laboratory results; no water sample results reported by Tetra Tech's laboratory were above 2.4 times the PSARA's laboratory results. For this project, EPA has always used higher of the two results reported by the laboratories to determine whether the cleanup goal at a given site was met. Because EPA is using a conservative approach (less risk to the environment) to determine whether samples contain PCB concentrations above the cleanup goal, we believe that the cleanup decisions made on this project are defensible.

Understanding of Quality Assurance and Split Sample Concepts

First, Dr. Kumar Topudurti of Tetra Tech has been involved with the Bloomington sites and understands the concepts of quality assurance and split samples. Because of his expertise in the field of laboratory quality assurance, he serves on the EPA federal advisory committee known as the Environmental Laboratory Advisory Board (ELAB), which was created to provide advice and comments to the EPA Administrator on accreditation of environmental laboratories and offer recommendations concerning the National Environmental Laboratory Accreditation Conference (NELAC). NELAC is the result of a joint effort by EPA, other federal agencies, the States, and the private sector that began in 1990 when EPA's Environmental Monitoring Management Council established an internal workgroup to consider the feasibility and advisability of a national laboratory accreditation program. In addition to serving on the ELAB, for many years, Dr. Topudurti has been assisting the EPA Office of Research and Development (ORD) in designing many quality assurance initiatives including large PE programs. Because of his contributions to the field of quality assurance, EPA ORD has requested Dr. Topudurti to author "A Guide to the Selection and Assessment of Analytical Laboratories."

Below are examples of questionable statements made by you in your correspondence.

• "Split samples are not measures of recoveries as indicated in the 1Tetra Tech letter." As discussed under "Comparison of Split Samples," split samples can be environmental samples or PE samples (split samples obtained from a matrix spiked with known analyte concentrations). As shown in Table 10 of SW-846 Method 8082, which is included as an attachment to your letter dated March 6, 2002, recoveries are calculated for the PE samples. In EPA's letter dated February 22, 2002, Tetra Tech, with EPA concurrence has accurately stated that "The typical acceptance criterion for PCB analysis of aqueous samples, which was derived from an industry-wide analysis of performance evaluation (PE) samples, is a recovery ranging from 53 to 129 percent." You should provide a reference to support your statement, if you believe that recoveries should not be calculated for PE samples.
• "When the answers from the two laboratories differ, we really don't know which result is "correct" (though we might suspect that the higher value might be)." Please provide a reference to support your statement that why the higher value might be suspected to be correct.
• "We can't use concepts like recovery acceptability to determine how correct the results are." Recovery is commonly used to determine how correct (accurate) the sample analysis results are. Specifically, the surrogate recoveries, laboratory control sample recoveries, and matrix spike recoveries provide an indication of how accurate are the environmental sample results. However, matrix spike recovery is a good measure of accuracy only when the spike level is appropriate. For additional information on the use of matrix spike recoveries, you may refer to Provost and Elder (1983), which provides a good discussion on interpretation of percent recovery data. In any case, you should provide a reference to support your statement that recovery cannot be used to determine how correct the results are.
• "Spike samples as matrix spike or laboratory control sample duplicates are the usual measure of precision, not unknown split samples." It is unclear why you are discussing precision for matrix spike duplicates and laboratory control sample duplicates when the objective was to evaluate how well the environmental split sample results provided by two laboratories compare with each other. In addition, you should note that the precision estimates based on matrix spike duplicate analysis assume that the background analyte concentrations in the spiked aliquots are the same as those of unspiked aliquots. This assumption is true only when the field sample is homogenous. You should also note that laboratory control sample duplicate analysis provides an estimate of analytical precision for control samples, not for field (environmental) samples. Finally, contrary to your statement, unknown split samples are typically used to estimate precision in the laboratory practice. For example, laboratory duplicates, which are routinely used to measure intra-laboratory precision, are unknown split samples that are generated in the laboratory. This approach also has limitations because the reliability of such an estimate depends on how the duplicates are obtained in the laboratory.

  Additional discussion on approaches to evaluate precision when the QAPP does not define the term "significant difference" between split sample results is included under "Comparison of Split Sample Results." Please provide a reference to support your statement that unknown split sample results cannot be used to estimate precision.

Misquoting Method Performance Data

Finally, the attachment that you used contains data for soil samples not for water samples discussed in EPA's letter dated February 22, 2002. You should explain why you believe that soil sample recovery data associated with automated Soxhlet extraction are appropriate for water samples extracted using separatory funnel liquid-liquid extraction procedure and should provide a reference to support your explanation.

Finally, contrary to your statement, SW-846 Method 8082 does provide precision data. Specifically, Table 10 of the method (included as an attachment by you) contains several entries identified as "S. D.," which stands for standard deviation (a measure of precision). You should explain why you believe that the method provides only recovery data, but not the precision data.

Discarding of Sediment in Water Samples

To avoid any misunderstanding of how the water samples were handled, Tetra Tech contacted both laboratories and asked the laboratory staff how many water samples contained sediment and whether the laboratories discarded the sediment present. Both laboratories responded that, based on visual observations, no water sample for this project contained sediment; therefore, the water samples were analyzed as received. In addition, Tetra Tech has verified that the laboratories were using the same extraction procedure. In summary, there is no direct basis to support your statement that laboratories mishandled the water samples and followed inconsistent laboratory procedures.

Incorrect Analysis Method and Its Impacts

• Page 2 of SW-846 Method 8082 does not issue a "warning" regarding the use of Aroclor based quantification of PCBs. Section 1.2 of the method simply states that "When samples contain more than one Aroclor, a higher level of analyst expertise is required to attain acceptable levels of qualitative and quantitative analysis." On the contrary, Section 1.4 of the method states that "Analysts should use caution when using the congener method when regulatory requirements are based on Aroclor concentrations."
• Section 1.3 of SW-846 Method 8082 states that "Quantification of PCBs as Aroclors is appropriate for many regulatory compliance determinations, but is particularly difficult when the Aroclors have been weathered by long exposure in the environment." However, the method does not state that the "Aroclor method will likely undermeasure [underestimate] the PCB content." On the contrary, Section 9.4 of the method states that "During method performance studies, the concentrations determined as Aroclors were larger than those obtained using the congener method. In certain soils, interference prevented the measurement of congener 66. Recoveries of congeners from soils spiked with Aroclor 1254 and 1260 were between 80% and 90%. Recoveries of congeners from environmental reference materials ranged from 51-66% of the certified Aroclor values." Because the method performance data presented in SW-846 Method 8082 directly contradicts your statement that the "Aroclor method will likely undermeasure [underestimate] the PCB content," You should provide a credible reference to support your statement.
• According to Sather and others (2001), who conducted a comparison study of Aroclor-and congener-based PCB concentration measurement methods for environmental samples (fish tissue), the "data show a strong correlation between the sum of Arclors and the total PCBs obtained from the full congener determinations.,' The linear model developed by Sather and others (2001), which is presented below, shows that the PCB concentration estimated from the sum of Aroclors will be larger than that estimated from the full congener determination.

  Total PCBs (Aroclor Based) = 1.079 x Total PCBs (Congener Based) +491.6

  In the above model, which was reported to have the square of correlation coefficient 0.9615, nanograms per gram is the concentration unit.

• For this project, EPA has chosen to use the Aroclor method because no Superfund policy currently exists regarding sampling for congeners or dioxin-like PCBs. Until this matter is resolved and a policy delineated, EPA has concluded it would be unproductive to sample for congeners or dioxin-like PCBs and base cleanup decisions upon them. To address some of the citizens concerns, the EPA has completed some congener analysis at the Bloomington sites during verification sampling.
• Based upon the National Academy of Science report for sediment cleanup, they recommend that some congener analysis be completed. The EPA did complete some dioxin-like congener analysis along with Aroclor analysis for fish tissue samples during its August 2001 sampling event in Conard's Branch and Richland Creek (the report is on the COPA web page and is in the information repository). Dr. James Chapman of the EPA has evaluated the ecological risk associated with the fish tissue analysis and has compared both dioxin-like PCB risk to Aroclor risk. The risk results show that in some instances that risk calculated using Aroclor analysis are higher and in some instances the congener risk is higher. Overall, the risk estimates are similar between the two methods. The ecological risk assessment produced by Dr. Chapman will be released shortly and was presented at Citizens Information Committee meeting on February 19, 2002.

Path Forward

We have from the beginning of this project taken many measures to ensure our laboratory data is of good quality and we our of the opinion that the laboratory data is defensible. Enclosed is a copy of the laboratory data from Viacom and EPA for the split samples you have questioned along with the EPA QAPP. If you would like to discuss or clarify any issues with Dr. Topudurti, the EPA can arrange a meeting in Chicago, Illinois or set up a conference call with you. Sincerely,

Thomas Alcamo
Chemical Engineer
U.S. EPA

References

Sather,P. J. and others. 2001. "Similarity of an Aroclor-Based and a Full Congener-Based Method in Determining Total PCBs and a Modeling Approach to Estimate Aroclor Speciation from Congener-Specific PCB Data. Environmental Science & Technology. Vol. 35, No. 24. Pages 4874 to 4880.

Tetra Tech EM Inc. (Tetra Tech). 1998. "Draft Site-Specific Plans for Neal's Landfill, Bennet's Dump, and Lemon Lane Landfill Sites, Monroe County, Indiana." February.

Tetra Tech 1999. "Data Evaluation Report for Neal's Dump Site, Owen County, Indiana." January.

Tetra Tech 2000a. "Data Evaluation Report for Winston-Thomas Wastewater Treatment Plant." January.

Tetra Tech 2000b. "Data Evaluation Report for Split and Nonsplit Sampling Activities, Neal's Landfill Site, Bloomington, Indiana." May.

Tetra Tech 2000c. "Data Evaluation Report for Bennett's Dump Site, Bloomington, Indiana." August.

Tetra Tech 2001a. "Final Data Evaluation Report for Split and Nonsplit Sampling Activities, Lemon Lane Landfill Site, Bloomington, Monroe County, Indiana." April.

Tetra Tech 2001b. "Data Evaluation Summary of 2001 Split Sample Analytical Results for Viacom Multisite Sampling and Analysis Support." November.

U.S. Environmental Protection Agency (EPA). 1996a. "PCBs: Cancer Dose-Response Assessment and Application to Environmental Mixtures." EPA/600/P-96/OOl F. Office of Research and Development. Washington, DC. September.

EPA. 1996b. "Test Methods for Evaluating Solid Waste." Volumes IA through IC. SW-846. Third Edition. Update III. Office of Solid Waste and Emergency Response. Washington, DC. December. 11

Thomas Alcamo
US EPA Region 5
77 West Jackson Blvd.
Chicago, IL 60604-3590