January-February 2008

Development of a Scoring System for Construction-Site BMP Assessment

A tool to rate the quality of implementation and maintenance

A scoring system to evaluate best management practices (BMPs) implementation at construction sites was developed and used in Lincoln and Omaha, NE. A simple metric tool was developed to score the quality of implementation and maintenance of five types of construction-site BMPs at 87 construction sites from the two communities. The scoring system was developed to be simple to use, employing a three-tiered rating system for each BMP type. This simple system requires minimal training and allows for quick assessments that are reproducible between evaluators. The scores were used to assess the relative frequency at which construction-site BMPs were properly or improperly implemented. The system could be used to monitor and document BMP implementation improvement through time and to identify the BMP types that are in need of improved implementation, increased monitoring, or improved training. Results, using the inspection metric, showed that implementation and maintenance of stormwater BMPs followed a uniform distribution from excellent to very poor. That is, at least half of the construction sites had BMP systems that were not implemented or maintained well and were, therefore, likely providing little stormwater protection. Because the effectiveness of various BMPs is typically reported for properly implemented and well-maintained installations, the actual in-the-field performance may be significantly less than the reported effectiveness implies.

Introduction
As a result of the 1987 amendments to the Clean Water Act (Clean Water Act 1977), the USEPA established a comprehensive phased approach to stormwater management under the existing National Pollutant Discharge Elimination System (NPDES) program. NPDES Phase II lowered the threshold on municipal separate storm sewer systems (MS4s) to populations greater than 10,000 and construction activities to 1 acre (NDEQ 2004; USEPA 2000).

Nonpoint-source pollution from stormwater is difficult to monitor and/or treat for a number of reasons, including the nondiscrete location of the sources and transport routes of pollutants. Because of this difficulty, nonpoint-source pollution is not regulated to a numeric limit but is instead addressed through the use of BMPs, which are techniques, measures, or structural controls that are used to manage the quantity or quality of stormwater runoff (USEPA 1999, 2004).

Studies have shown construction activities to be a major source of nonpoint-source pollution (primarily sediments and nutrients) (Willett 1980; Yorke and Herb 1976; Owens et al. 2000; Novotny et al. 1979). Construction-site runoff controls are typically focused on erosion prevention and sediment control, due to the large amounts of uncovered ground present during the construction process.

Since the mid-1990s, when stormwater management became more common, the number of available construction-site controls has significantly increased. Accordingly, so has the research on these controls. Most available research focuses on pollutant removal capabilities of BMPs and not on their maintainability. Recently, many researchers are finding that proper maintenance and inspection is a significant problem for BMP programs. For example, Corish (1995) concluded that 67% of communities encountered major problems with inadequate installation and maintenance of BMPs. Similarly Patterson (1999) concluded that “poor implementation remains a widespread obstacle [to effective BMP use.]” Shaver and Piorko (1996) stated that “most ESC [erosion and sediment control] programs do not have the resources to effectively inspect construction activity.” Shaver and Piorko also noted that of surveyed inspectors, each inspector is responsible for an average of 150 sites, which can significantly limit the ability to properly inspect each site.

In addition, a general lack of guidance on BMP inspection methods and requirements has led to variability in the methods and quality of inspections. This is apparent from a review of the inspection checklists commonly available over the Internet. For example, some communities like Bellevue, WA, have a number of detailed checklists (SCVURPP 2006), while others like Overland Park, KS, have a simpler system (Overland Park 2006). Additionally, in some communities like Lincoln, NE, a majority of the inspections are conducted by a designated inspector who works for the community, while in other communities like Omaha, NE, a majority of the inspections are conducted by contractors for the developers who report their findings to the community.

Lincoln and Omaha, NE, are classified as large MS4s; therefore, they are regulated under the stormwater provisions of the Clean Water Act. One of the six minimum measures required for compliance for each of these communities is construction-site runoff control. This is accomplished in both communities through permitting and inspections of implemented BMPs.

Purpose
A study was conducted to evaluate the implementation and maintenance and, therefore, the effectiveness of construction-site BMPs in two Midwestern communities. During the study, it was found that a quick and reproducible scoring method was needed for construction-site BMP assessment. The purpose of this article is to describe the development and use of this method.

Methods
Selection of Construction Sites
Eighty-seven active construction sites (42 from Lincoln and 45 from Omaha) were selected randomly from recently submitted and approved stormwater pollution prevention plan (SWPPP) permits. Lincoln and Omaha are typical small- to medium-sized Midwestern cities. Lincoln has a population of about 240,000, and Omaha has a population of about 400,000 with an immediate urban area population of about 800,000. In an effort not to bias the site selection, information contained in the permit and associated SWPPPs was not used for the selection or evaluation of the sites. All sites were inspected during the growing season (i.e., April through September). The Lincoln sites were inspected in September 2004 and April/May 2005. The Omaha sites were inspected during the summer of 2005. Inspections were done by two individuals. The Omaha inspector was trained by the Lincoln inspector. The training included consistency checking to ensure that the BMPs were evaluated the same way in both communities. Because of the relatively simple assessment metric, training and consistency checking were completed with only a few visits to each BMP type.

Development of Metric for Assessment
To evaluate construction sites objectively, a simple metric tool was developed to rate the quality of construction-site BMP implementation and maintenance. The metric was based on the “Ohio EPA Construction Site Inspection Checklist” (Ohio EPA 2003). The metric included evaluation of five BMP categories: stabilization, sediment tracking control, ponds, linear controls, and inlet protection. These five BMP types were selected because they are the primary BMP types used to reduce the transport of soil and sediment from construction sites. The evaluated categories of BMPs are similar to the list of the five common BMP types discussed by Lee (2000). Wind erosion controls were not assessed, as typical applications are temporary.

Scoring Metric for BMP Types
The following sections describe the assessment matrix used for the five construction-site BMP types that were evaluated in this study.

Stabilization Assessment
Stabilization refers to the use of cover materials, typically vegetation, used to reduce erosion potential. Most communities have standards for how long an area can remain unstabilized during the construction process. Because one-time visits were used to assess the construction sites, it was not possible to evaluate how long areas remained unstabilized. Therefore, evaluation of the need for stabilization was based on the state of the site at the time of the visit. If construction on the facility or structure (i.e., homes or offices) had begun, stabilization was deemed required. Sites were given a score of 1.0 if stabilization measures (e.g., seeding, sodding, or artificial cover material) were actively being used over two-thirds of the construction site. Only two-thirds of the site was required to be stabilized to receive a score of 1.0 to allow for ongoing construction activities on the other one-third of the site. A score of 0.5 was given where stabilization was present but inadequate (e.g., less than two-thirds of the applicable area was properly stabilized). A score of 0 was given where no stabilization was seen and construction was ongoing. A score of not applicable (NA) was given where stabilization was not present but was deemed not necessary due to site conditions (e.g., where grading had been done and the soil was left bare). Figure 1 shows the metric developed to assess stabilization practices.

Sediment Tracking Control Assessment
The sediment tracking control category was used to assess the use of measures (i.e., rocked construction-site entrances and wash stations) to prevent the tracking of sediments onto streets or adjacent properties. A score of 1.0 was given where construction-site entrances were present and constructed properly. A score of 0.5 was given where a construction entrance was present but improperly designed or operated (e.g., where rock sizes were less than the typically required 2-inch diameter [Ohio EPA 2003]). A score of 0 was given where no sediment tracking control was present. A score of NA was given where the use of construction sediment tracking control was no longer necessary due to stabilization. Figure 2 shows the metric developed to assess sediment tracking practices.

Ponding Practices Assessment
The pond category was used to assess the use of ponding facilities (i.e., sediment basins, retention ponds, detention ponds). A score of 1.0 was given where ponding facilities were necessary and present. Scores of 0.5 were not given, because ponds are generally either present or not. A score of 0 was given where no ponding facility was present and the site was larger than six residential lots (approximately 1 acre). A score of NA was given where the use of ponding facilities was determined unnecessary because the size of the construction site was less than six residential lots. Figure 3 shows the metric developed to assess ponding practices.

Linear Controls Assessment
The linear controls category was used to assess the use of linear sediment control devices (e.g., silt fence). A score of 1.0 was given where controls were present, properly installed (e.g., dug in), and maintained over at least two-thirds of the construction site. The use of two-thirds of the construction site was used to allow for areas of ongoing construction activities that would prevent linear control installation and maintenance on the other one-third of the site. A score of 0.5 was given where controls were present but either not properly installed or not properly maintained. A score of 0 was given where no controls were present but were necessary. A score of NA was given where controls were determined unnecessary because of stabilization. Figure 4 shows the metric developed to assess linear control practices.

Inlet Protection Assessment
The inlet protection category was used to assess the installation and maintenance of storm sewer inlet protection devices (e.g., fabric barriers, inlet blockages). This category did not consider inlet marking or storm sewer stenciling. A score of 1.0 was given where controls were present and properly installed and maintained for two-thirds of the site. The use of two-thirds of the construction site was used to allow for areas of ongoing construction activities that would prevent inlet control installation and maintenance on the other one-third of the site. A score of 0.5 was given where controls were present but not properly installed or maintained (e.g., didn’t cover the entire inlet, silt fence had failed). A score of 0 was given where no controls were present. A score of NA was given where controls were determined unnecessary because of stabilization. Figure 5 shows the metric developed to assess inlet protection practices.

Results and Discussion
The assessment method was applied to 42 construction sites in Lincoln (16 commercial, 12 general, 14 residential) and 45 construction sites in Omaha (15 commercial, 15 general, 15 residential). Figure 6 shows the overall results of construction-site visits according to the type of control and type of construction for the Lincoln sites. Figure 7 shows the overall results of construction-site visits according to the type of control and type of construction for the Omaha sites.

Stabilization
Stabilization was the most difficult BMP category to assess because of the potentially transient nature of this BMP. However, it did receive some of the highest scores. Of the 65 sites (Lincoln and Omaha combined) where stabilization was deemed necessary, 33 (51%) received the highest score of 1.0, while 14 (22%) received a score of 0 (showing no presence of stabilization). Residential construction in Lincoln received the lowest score for stabilization, with typical scores significantly lower than for commercial and general construction. It was not uncommon to see home sites completely finished that had not been stabilized. This may be because, in part, of the perceived ambiguity in the construction-site regulations. Because individual residential lots are typically smaller than 1 acre, some builders may believe that the construction-site regulations do not apply, even though the lot is part of a larger development. In addition, because the lot is small, construction activities typically involve most of the lot; therefore, it is difficult to properly maintain BMPs such as stabilization.

Sediment Tracking Controls
Sediment tracking controls received the lowest scores as a group. Of the 72 sites where tracking controls were deemed necessary, only three (4%) received the highest score of 1, while 38 (53%) received a score of 0 (showing no presence of tracking controls).

Low scoring in this category also came as a result of improper rock size selection on construction entrances. Most guidance stipulates that a construction entrance should be constructed of rock with a mean diameter of 2 inches (Ohio EPA 2003; Fifield 2004). Often entrances were constructed of much smaller stone.

Ponds
Ponding facilities received relatively high scores. This was also the category that produced the best results for residential construction. Of the 46 sites where ponds were deemed necessary, 35 (76%) received the highest score of 1, while the remaining 11 (24%) received a score of 0. Note that only scores of 1.0 or 0.0 were given for this BMP category.

Commercial and general construction likely received lower grades due to ambiguity of the need for a facility, reluctance to construct ponding facilities on high-value land, or difficulties in sequencing construction and removal of a pond at a commercial site. Residential construction is typically of the scale that there is no question of the need for such a facility. Residential developments are typically amenable to the sequencing of construction, and ponding facilities can often be left as green space following construction.

Linear Controls
Linear controls were present at most sites, and results in this category were similar for all types of construction. Silt fence was the most common BMP in this category and on construction sites in general. Of the 83 sites where linear controls were deemed necessary, 38 (46%) received the highest score of 1, 20 (24%) received a score of 0 (showing no linear controls onsite), and 58 (30%) received a score of 0.5 (showing that linear controls were present but not properly installed or maintained). Often, silt fence was located in areas with little function (e.g., as perimeter control or near tops of hills). Maintenance and proper installation of silt fence was a common problem at the visited sites.

Inlet Protection
Inlet protection was fairly common at the visited sites. However, of the 57 sites visited where inlet protection was deemed necessary, only 13 (23%) received the high score of 1, while 30 (53%) received the low score of 0.

Two main types of inlet protection were observed: silt fence around an inlet and preconstructed, commercially available inlet protection devices. In the silt fence inlet protection applications, a number of well-constructed applications were found; often they were backed with chicken wire and supported by a sturdy 2x4 frame. However, many installations using silt fence for inlet protection simply installed the silt fence like standard linear control silt fence. These sites proved to have difficulty being maintained, likely because of the presence of concentrated flows. This shows that although silt fence can be used for inlet protection, it must be designed and installed as inlet protection, not as linear control.

Preconstructed, commercially available devices showed few problems with maintenance. However, these devices often received lower grades due to improper application.

Evaluation of Overall Site Scores
Figures 6 and 7 show that the sediment tracking and inlet protection (except for general construction in Lincoln) BMPs scored the lowest in this study. The results from this type of evaluation could be used to focus inspection efforts and to improve training for construction contractors. In addition, the results could be used to monitor and document improvements in BMP implementation through time.

The data do not follow a normal distribution; therefore, typical statistical analysis cannot be conducted. However, analysis of the data was done using the nonparametric sign test. This test allows the comparison of the mean of a data set to some predetermined value at a specified confidence level. For example, for the BMP assessment, one can determine if the mean score for a BMP is statistically above some level (say 0.5) at the 95% confidence level. The results of this analysis show that the mean scores for the following BMPs were not above 0.5 at the 95% confidence level in Lincoln: stabilization (residential); sediment tracking (all construction types); linear controls (commercial/industrial); and inlet protection (commercial/industrial and residential). The mean scores for the following BMPs were not above 0.5 at the 95% confidence level in Omaha: sediment tracking (all construction types); inlet protection (general and residential). These statistical measures could be used to assist evaluation and comparison of the BMP types (e.g., identifying differences in implementation quality between BMP types or changes in BMP implementation quality through time).

Figures 8 and 9 show the distributions of the overall average scores for each construction site visited. These figures show fairly uniform distributions of scores from 0.0 to 1.0, showing that a range of implementation and maintenance quality is present in both communities. The uniform distributions in Figures 8 and 9 show that the numbers of construction sites with very good, very poor, and mediocre implementation and maintenance of BMPs are roughly equal. However, the scoring system used in this study is likely rather generous. That is, typically sites were given the top score of 1.0 if only two-thirds of the site’s BMPs were installed and maintained properly, and sites were given a score of 0.5 if the BMPs were present but not properly maintained. Under this scoring system, a site could have essentially ineffectual BMPs and still receive a score of 0.5. Therefore, the distributions showing half of the sites with overall scores below 0.5 likely indicates that more than half of the sites have largely ineffectual BMPs.

Considerable research has been done on the removal efficiencies of various BMPs for controlling certain pollutants (USEPA 2004). For example, silt fence has been reported to remove from 75% to 85% of sediments (USEPA 2004). However, these results are based on properly installed and maintained BMPs. The actual removal effectiveness of a BMP is a function of how well a BMP works if installed properly and how well it is installed and maintained in the field. Therefore, the overall effectiveness of a BMP program should consider both the types of BMPs that are to be used and the quality of implementation and maintenance of those BMPs.

Based on the results of this study, proper installation and maintenance occurs on fewer than 50% of the sites studied. Therefore, it is likely that construction-site BMP systems perform significantly below the optimal levels. This result should be considered when planning and evaluating BMP programs and when estimating the potential benefits (i.e., pollutant removal efficiency) of proposed BMP programs.

Improvements to Construction-Site BMP Inspection Protocol
The metrics developed for this study used a three-tiered grading system, generally: 1.0 for sites with BMPs present and properly maintained over two-thirds of the site, 0.5 for sites with BMPs present but not properly installed and maintained over two-thirds of the site, and 0.0 for sites lacking appropriate BMPs. The simple three-tier metric was selected to reduce ambiguity between grading levels. Analysis of the results showed that while the simple three-tier system was simple and reproducible, a scoring system with more grading steps would likely make the results more realistic. For example, sites with BMPs present but completely ineffectual received scores of 0.5, implying that they were “half-good,” while in reality they provided little to no pollutant removal capability.

Recommendations for Further Study
It is recommended that additional study be conducted to revise the inspection scoring protocol to better describe the actual performance of the BMPs. In conjunction with that study, methods to determine the acceptable threshold for BMP quality should be evaluated.

Conclusions
Construction-site assessments were conducted at 87 construction sites in Lincoln and Omaha, NE. The results of these assessments showed:

• Quality of construction-site controls varied greatly and produced a uniform distribution of scores, from very well-installed and -maintained sites (score of 1) to those where no controls were witnessed (score of 0). Therefore, at least half of the construction sites have significant shortcomings regarding BMP implementation and maintenance and, therefore, performance.
• Because construction-site BMPs are only installed and maintained properly about 50% of the time, actual BMP effectiveness may be only 50% of reported values for specific BMP types (e.g., silt fences are reported to remove approximately 80% of sediments, but if they are installed improperly 50% of the time, the actual effectiveness is significantly less).
• Installation and maintenance of BMPs on construction sites appears to be limiting the effectiveness of construction-site controls.
• The scoring method has proven to be a useful tool for identifying BMP types that may require additional monitoring or contractor training and for identifying changes through time of BMP installation quality.