March 6, 2002

Mr. Alcamo:

I have received the EPA's response to my questions on the Tetra Tech/PSARA split sample report (11/16/2001) posted on the COPA website. I appreciate your conclusion that the disparity in results between PSARA and Tetra Tech data is worth investigating and am glad that I brought it to your attention. I look forward to the actual comparison of the two data sets as promised in your letter. In addition to that analysis, another split sample report needs to be addressed. If it is possible, please track down the data for the split samples mentioned in the attached cover page, dated April 28,2000. That data is on a set of Tetra Tech/PSARA soil split samples from Lemon Lane. The essential purpose of a report on split samples requires a comparison of the two data sets. Anything else really does not work very well.

Meanwhile, I would like to point out that the discussion supplied by Tetra Tech does not show an understanding of quality assurance concepts. First, split samples are not measures of recovery as indicated in the letter. Split samples are actual samples that are "hopefully" equivalent, which are divided and sent to two or more laboratories. The exercise is intended to determine if the laboratories are able to provide equal, or "acceptably" similar results. A difference allows the user to detect bias or errors that may indicate that one laboratory is generating questionable data on at least this sample, and possibly on others. The samples must be at least single-blind in nature. Single-blind means that the true concentration is unknown to the laboratory. Double-blind means that the laboratory doesn't even know that the sample is a test. Unfortunately, because the laboratory does often know that the sample is a test sample, the only prevention of undue influence is that no one knows what the "true " answer is for a split sample from the actual site. However, it does not prevent a laboratory from providing an "extra" measure of special attention that is normally not given to other samples.

So, a) when the answers from the two laboratories differ, we really don't know which result is "correct" (though we might suspect that the higher value might be), and b) we can't use concepts like recovery acceptability to determine how correct the results are. Both laboratories may be wrong. What is appropriate is an acceptance range for the quality assurance term called precision, or reproducibility. Reproducibility is set in the quality assurance project plan based on equivalency of laboratory results for a known concentration sample. Such spike samples as matrix spike or laboratory control sample duplicates are the usual measure of precision, not unknown split samples. The test samples have to be known, they have to be equivalent, and they have to be treated the same by the laboratory before the results are considered acceptably precise.

It is currently not possible to determine what the target reproducibility is for the Bloomington Project. The Monroe County Public Library in the past has been unable to locate the quality assurance project plan (QAPP). Another request to find this document was initiated on 3/1/2002, but the librarians could not get the QAPP to surface out of the files. Could the EPA send an updated copy to the Library?

The results, if similar within control limits, indicate that the samples were probably equivalent in their contents, and treated by the laboratory equivalently, and sent to the analytical instrument under equivalent analytical circumstances. Exactly what one hopes for. However, if the results turn out to be different, then the detective work begins. Were the samples different? Was something else different in the laboratory's handling? Was the instrument working correctly that day? With these data, that process has not even begun. No one at the EPA followed through on the process. It's almost hardly worth it now to check. The split samples were supposed to tell whether the process worked. How much data went through the system without these simple quality assurance checks? What actually occurred were the simple steps, that is, taking the samples and calling it quality assurance without follow through. The cynicism, or "oversight", is disturbing.

And, in addition to misunderstanding the split sample concept, your contractor may have misquoted Method recovery figures. First, as noted above, "recovery" applies only to samples of known concentration and addresses accuracy, not precision. The Method does not provide such precision data for split sample analyses, but does give some recovery results for a spiked, homogenized soil sample sent to eight laboratories to illustrate what the Method can accomplish. The results are different from what is cited as the "industry-wide" standard in your letter. (A copy of the actual Method results is attached.) However, wherever they got those numbers doesn't actually matter. Their use is incorrect.

Second, the spiking example provided in the Method is not given to set project control limits based on any matrix, or any laboratory, or the industry, and should not be used to set control limits for any real project (even for the correct use as a measure of accuracy). Accuracy and precision (reproducibility) acceptance limits for any project must be set based on specific project data quality requirements. They can vary from the minimum of a laboratory, and must depend on the use of the data. An assessment of health risk, or the overall accuracy of a laboratory protocol for hundreds of samples such as the Bloomington Project requires a much higher data standard than a "go-no go" decision in a conduit study.

And, while I have your attention, I thought I would bring up another issue that you addressed in your letter to me. The tracer studies done at Lemon Lane. I must apologize for focusing on the 1989 study. I thought that it was the primary basis for EPA's conclusions on Lemon Lane flow dominance to be the Illinois Central Spring. I feel that this was a natural mistake. I did not realize that the EPA was using an even weaker study to make its case. The 1990 study is of significantly poorer usability than the 1989 study. At least the 1989 study can be shown to have recovered about 60% of the injected tracer. The 1990 study recovered only 18%. Also, the plant target spring, Illinois Central Spring was not even in the 1990 study. The highest recovery of dye occurred at two springs hydrologically downgradient from Illinois Central Spring.

Both the 1989 and 1990 studies suffer from serious flaws and should not have been the basis for any decision on treatment of contamination. As the 1990 study states, there is no evidence of connection between the monitoring wells and injection wells. Because of the poor recovery, one also must conclude that there is little connection between the injection points and the other monitoring points at springs. And, even more destructive to any use of the results is the absence of any connection between the injection points and the actual waste itself. The injections are done outside of the landfill with no evidence of being in the contamination flow system. Their connection to the springs is clearly not established, and the 18% dye recovery is marginally useful. And then, EPA locates the plant at another spring!

The only valid basis for assessment of contamination transport via the water is where the contamination is exiting, not where a dye has exited. Those points can only be reliably identified based on contamination (and, not just PCBs) exiting the site during both low and high flow conditions (in springs and groundwater). A proper site characterization requires developing an understanding of precipitation, flow, sedimentation, and contamination over a range of conditions at all potential exits for any of the sites has never been done.

Hopefully, these points help you understand why I feel compelled to question the split sample work and the tracer studies as a basis for water treatment. I am looking forward to your help with these questions.