COPA Home

Data Quality Questions for Bloomington Superfund Sites

Joseph G. Hailer
Bloomington IN

This document summarized some of the questions that I have on the data reports for the Bloomington Superfund sites. Although most of the observations and comments focus on Lemon Lane, these problems were first noticed as similar issues and questions for stream and spring samples at Neal's Landfill, and the field reports for the site remediation at Bennett's Quarry. I am concerned whether these issues are ongoing and pervasive in all of the work. The data sets which I have specifically examined are a) the Admin Record Update 7/29/99 for Water Sampling at Neals 2/24-5/4, 1999, b) the Summary of Lemon Lane Landfill Analytical Results for Split Samples Collected by EarthTech, February, 1997, and c) the 1/19/99 Conduit Investigation for July to October, 1998. Where a problem is described, I also try to give an example from the data sets. I have also stated a specific question for which I would appreciate a response that directly addresses the problem and provides a resolution.

1) Reporting results from the wrong analytical method - The method cited in the Quality Assurance Project Plan (QAPP) for the analysis of PCBs is SW-846 Method 8082, which is correct. However, the laboratory reports for the Lemon Lane data recurringly cite Method 8081 as having been used. Method 8081 is for Organochlorine Pesticides such as chlordane, and not PCBs. Although the two methods are similar, I cannot imagine that the laboratory could make such an error I suspect that this is a typographical error, but it is so obvious a question that a data validalator should spot and question. It is more an indication of poor attention to detail that should have been checked by CBS and caught by EPA.

Could you confirm that analyses done by all the laboratories doing work for the Bloomington Superfund sites were appropriate for polychlorinated biphenyls and explain why Heritage Laboratories was using an incorrect method? Revised reports should be submitted for the administrative record.

2) Consistently using comparable methods - It is unclear whether all analytical programs are using the same methods. For example, at Lemon Lane CBS, EarthTech, and EPA had samples analyzed. The EPA laboratory would probably have used a contract laboratory program (CLP) participant. They typically use different methods than used under RCRA. EarthTech submitted CERCLA analyte lists in its Summary Report (item b), but CBS used Heritage Laboratories and they use RCRA methods. EPA needs to confirm that the methods and quality control used are consistent and directly comparable. I raise this question because the CBS and EarthTech split samples yield widely varying results and it is important to know if it was caused by the analytical methods, quality control prohlems, or analytical errors.

Could you provide evidence that the analytical methods used for the samples at the Bloomington Superfund sites were all consistent with the QAPP (which calls for SW-846 Method 8080) and, if not, provide an analysis of the effect of any differences to ensure accuracy and comparability of the data?

3) Lack of quality control data - The QAPP calls for a number of quality control checks to be done along with the samples to demonstrate that the results are accurate within the project's data quality requirements. The important quality control (QC) reports are the continuing calibration verification (CCV) reports, the results of internal standard checks, method blank runs, laboratory control sample checks, and matrix spike and duplicate results. None of these are presented with the data (items a and c). It is impossible to say whether the data are actually usable.

Could you provide the quality control results for the analytical data for the site characterization, remediation work, and ongoing verification of cleanup effectiveness at the Superfund sites, and direct that all future reports contain this type of information?

4) Inadequate surrogate recovery information - The one piece of QC data that is supplied is the surrogate recovery for each sample (in items a and c). However, the control limits are not given. Control limits are the range of acceptable recoveries that indicate that the extraction and analysis were done correctly.

Some explanations to help you understand the importance of these data. Surrogates are compounds that are similar in chemistry to the target compounds and will be affected the same by all processing steps. The surrogates are added to each sample at the beginning of processing to track the effectiveness of the extraction, concentration, and analysis of the target pollutants. If the result shows that 100% of what was 'spiked' into the sample at the beginning was recovered at the end, then it is very likely that 100% of the target compounds in the sample were also extracted and measured. Anything less, indicates either inefficiencies in extraction or losses during processing or analysis. What affects the surrogates also affects the targets, and low surrogate recovery indicates undermeasurement of the target compounds.

In spite of the attempts by the laboratory to minimize interferences with extraction (noted below as an additional questionable practices surrogate recoveries are frequently quite low (see for example item c, samples LL1447 and LL1455). If they exceed the tolerance limits (high or low) in the method, a fresh portion ofthe sample should be re-extracted and re-analyzed.

Surrogate recoveries exceeding 100% also questionable. It is impossible to actually get more than was put in unless it was initially present, or a serious analytical error has occurred. Because surrogates are selected to be unique, they are very unlikely to be in a sample to provide more than was added. When most of the data suggests recoveries are 50-60%, a recovery of 149% (item c, sample LL1453) suggests that the surrogate was very likely spiked twice into the sample. Errors like this can erroneously quality a result as acceptable. However, none of the ancillary detail is provided to allow determination of aceeptable recoveries of the surrogate.

Could you direct that the Quality Assurance Project Plan that contains the acceptance limits for quality control results be immediately included in the infonnation record at the Monroe County Public Library? Although much of what is addressed in this issue appears to be covered by the request for the quality control reports, in order to assess the quality of the data the acceptance limits are also needed.

5) Discarding a critical portion of the sample - Evident in the reports for water samples in items a and c is the recurrence of a very questionable practice of only using a portion of the supplied sample (numerous examples in items a and c - any sample less than 1,000 ml.). Because the final volume used at the instrument is fixed, using more or less of the original sample in the first extraction step will change what level of contaminant ends up in the final volume. Using less than the method-prescribed or mass will also reduce the detectability of low concentrations.

The method for PCBs in water calls for a 1,000-ml sample at the extraction step. From the field sheets, this volume is what has been supplied from the field, but the laboratory has been using only a portion of the sample. This causes two problems. First, if the water sample is free of sediment, a smaller than required volume causes a decrease in the detectability than that required by the QAPP. This activity can miss low, but significant occurrences of PCBs. The laboratory is not correcting its detection limits to reflect the lower sample volume. Second, for samples with sediment, the laboratory may be letting the sediment settle out, and pouring off a "cleaner" liquid-only portion. Considering the relationship of PCBs and their transport by sorption on the sediment, any practice that reduces or eliminates the most critical portion of the sample results in a serious biasing of the result to lower values.

Could you explain the discrepancies between a) the volume of sample required for analysis in the Method, b) the volume available for analysis as indicated by field records, c) the volume actually used in the analysis which suggests a discarding of a critical component, and d) the relatively low levels of PCBs detected?

6) Unnecessary dilutions - The laboratory also incorrectly uses a practice that is usually acceptable to bring concentrations within the range of calibration, but not to avoid the impact of sediment in the sample. Instead of discarding only 15-20% of the sample, they frequently discard up to 95% for processing (item c samples LL1450 and LL1451 ) and dilute the sample, supposedly to reduce matrix interferences. This elevates detection limits from 0.1 ug/l to 2.0 ug/l. At this level many samples are run without the ability to detect significant concentrations of contaminants.

They ascribe this practice to a need to minimize interference. This practice is actually justifiable only when target concentrations are so great that they exceed the quantitation limit for the calibration standards. It is not effective if other components are present. Many other contaminants such as pesticides and many petroleum hydrocarbons are remowahle through chemical and physical cleanup techniques. The laboratory is not applying due diligence in order to supply accurate results relying on the complacency of the client to accept "analytical difficulties" as the cause.

Further, the laboratory cannot tell that dilution will be necessary until an initial analysis is done. This initial result should be presented in the report to justify the subsequent dilution of the extract. This allows an evaluation of the surrogate recovery in the extraction step to determine if the processing was correctly done and verifies that concentrations exceeded calibration limits. The dilution is done on the extract, not on the original sample. Yet, data are presented on diluted samples faith apparently good surrogate recoveries and low PCB levels indicating that a smaller sample was taken initially. Note: Surrogates go into the original sample volume. Using the smaller extraction volume distorts surrogate recovery data. The surrogates must also go through the extraction process. Spiking the surrogate into a volume lower than required artificially elevates recovery results. This falsely states the recovery and implies a higher quality of performance than was actually achieved.

From the "dilution" problem noted here, the discarding of "dirty water" in #5, and the surrogate spiking errors, it appears that the laboratory does not follow the proper execution of the extraction method. The method requires a total transfer of the supplied sample, plus a solvent rinse of the sample container that is added to the extraction vessel. Any debris carried into the extractant is removed in cleanup steps. To work with only "clean" water expedites sample processing, but falsifies the result. The only acceptable reason for using less than 1,000 ml is that insufficient sample was available (and then, detection levels have to be raised in the report). The field notes do not indicate any problem getting sufficient water. Most samples are taken from wells and streams with sufficient water to take a liter sample. For example, Sample LL1445 was taken from a 55-gallon drum and only 900 ml was used. LL1142 had three 55-gallon drums as a source, yet about 500 ml was used. Note: Taking a water sample from a large volume stored in a drum is an erroneous sampling procedure and invalidates any results.

Could you explain why the laboratory is failing to follow the analytical method with regard to proper sample extraction and surrogate spiking procedures?

7) Inadequate information to assess data problems - An indication of the severity of this problem of discarding a portion of the sample might be available if accurate measures of suspended solids were supplied for each sample, or if field sample data sheets showing the sample appearance were included with the laboratory sample reports, but they are not. Drilling activity field notes have many discrepancies, not the least of which are missing days and samples (item c LL1460-111484).

Could you direct that legible copies of all reports on samples collected at Neal's Landfill, Bennett's Quarry, and Lemon Lane Landfill include sampling records showing purging datat, field measurements, turbidity, specifice donductivity, etc. for each sample be provided to the information repostiory?

8) Improper quantitation technique The laboratory, under the guise of a legitimate confirmation-column check, is using, two analyses and choosing the lower result to report (item a). This biases data to the low side and is an error. If calibration was correct for each method, the results of two analyses should be the same. Therefore the only legitimate use of two analytical results is to confirm results and to report both, and possibly claim that the average of the two is the "better" number. If the laboratory is actually calibrating two methods, that is, two columns and two separate detectors, this is an extraordinary effort that requires a level of time and effort that can only be justified by and compensated with higher per sample charges. If CBS is compensating them tor this type of analyses, I suspect that there is a collusion to bias the results to the lower concentration. This possible "collusive" action is also apparent in the discard of sediment, but may not come at a higher per sample cost because it actually reduces the cost of analyses by reducing the processing steps for the samples. The laboratory's productivity goes up and that is its own reward.
Could you verify that the quantitation technique used for these samples meets the requirement of the QAPP and is consistent with good analytical technique and EPA policy for these analytical methods?

9) Dye tracer tests - I have concerns about the usability of the 1989 Westinghouse High Flow Tracer Results, which appear as a supporting document to the remediaiion plan for Lemon Lane on the water operable unit. The issue involves a recurring contention based on that study that Lemon Lane is predominantly drained by Illinois Central Spring and only trivially, under storm conditions, a few others. In my analysis, this report does not, and cannot be used to support that contention. No accurate quantification of the tracer dye detections was possible with the methods used, nor was the data presented to support that dye or bromide recoveries could support it either.

Could you or CBS provide an explanation of the data in that study which is being used to conclude that 98.9řio of the site drainage is occurring through the Illinois Central, Quarry, and ICG-1 complex of springs?

10) Quality of previous data - Considering the questions on data quality for water and sediment samples raised above, data from past investigations of PCB releases may need review to concern its accuracy and any conclusions resulting from the data. This includes the statements in the proposed plan that contends PEES releases have only occurred in Illinois Central, Quarry, and Slaughterhouse Springs. A significant weakness of the site characterization studies has been that no comprehensive samplings have been undertaken at potential drainage points during storm events that would provide data that supports the contention that these other points are not releasing contamination to the environment.

Could EPA please provide the field notes, the data and quality control reports and any analysis of the usability of this data for public review and conirmationi of its usibility to the totality if environmental releases under all weather conditions?

11) Water treatment plant - Because the EPA has decided that complete removal is not necessary to meet the objectives of the Superfund cleanup of the Bloomington sites, it is critically, important to the community to understand and be convinced that the treatment plant will meet the goal of clean water from the sites. The assumptions and information previously provided for review are appreciated. However. the system design goes beyond these to the actual calculations and sizing of facility equipment. Unless the community can review the actual design details, performance expectations. and monitoring program, it is difficult to assure the residents that the system is what is needed to do the job.

Could EPA please provide the community with the actual design plans for the treatment facility that show the development of equipment, capacities, performance expectations, monitoring, and system controls, which will meet the needs at Illinois Central Spring and the plans to treat the other springs which now, or may show evidence of contaminations

12) Administrative Record - Because the information repository at the Monroe County Public Library does not meet the standards for an Administrative Record under Superfund, a number of difficulties have arisen in accessing information necessary to meet the needs of the community to understand the clean-up process at the Superfund sites. In addition to the difficulty with locating specific documents due to the cataloging system, data is included in a very difficult form (if present at all).

Could the EPA require all existing and future reports and data sets, including maps and drawings in CAD form be made available in electronic format accessible to computer searching?

Joseph G. Hailer
Environmental Geochemistry & Quality Assurance
650 Dittemore Rd.
Bloomington, IN 47404

Index


Joseph G. Hailer
Environmental Geochemistry & Quality Assurance
650 Dittemore Rd.
Bloomington, IN 47404

Home
COPA

For more info, e-mail info@copa.org.
Copyright © 1990-2002 COPA, Inc. All rights reserved.
See legal page for terms of use and disclaimers.
All trademarks belong to their respective owners.