Critical Infrastructure Resiliency In the National Capital Region

“Emergency Generation Capability & Reliability In the National Capital Region: A Pilot Survey” (CIPAUAS6)
Submitted To:
Office of Commonwealth Preparedness Commonwealth of Virginia 1111 East Broad Street Richmond, VA 23219
Prepared By:
Advanced Research Institute Virginia Polytechnic Institute and State University Arlington, Virginia 22203
November 17, 2009
Contents
Executive Summary 1
I. Introduction 7 A. Background B. Measuring Readiness C. Project Inception D. Project Objectives E. Project Approach & Methodology F. Project Survey G. Additional Information Sources
II. Survey Summary 12 A. Survey Objectives B. Survey Approach & Methodology C. Overall Survey Totals D. Cell Phone Service Organization Participation
III. Facilities & Equipment 16 A. Introduction B. Arlington County, Virginia C. Montgomery County, Maryland D. District of Columbia
IV. Local-Regional Strategies: Obtaining Additional Units 27 A. Introduction B. Approach for Defining Outside Resource Needs C. Timeline for Obtaining Additional Emergency Generators D. NCR Intra-Jurisdiction Loans E. NCR Inter-Jurisdiction Loans F. Preferences for Inter-Jurisdiction Loans G. Discussions with Regional Equipment Suppliers H. Discussions with Area Construction Firms and Building Associations
ii
V. Critical Issues 33 A. Introduction B. Jurisdiction Personnel C. Contracted Support Services D. Practices Within & Across Jurisdictions 1. Operation, Maintenance, & Testing 2. Facility, Equipment, & Fuel 3. Refueling Contracts – Emergency Provisions 4. Age and Condition of Units 5. Administrative Support E. Fuel Suppliers & Distributors F. Equipment Maintenance & Testing Organizations G. Regional Hospital & Medical Facilities H. Designated Municipal & County Shelters I. ID System for Fuel Distributor & Generator Maintenance Personnel
VI. Findings & Conclusions 43 A. Introduction B. Better & Best Practices C. Facility & Equipment Conditions D. Coordinated Maintenance & Testing Contracts E. FEMA/USACE/State-Supplied Generators F. U. S. Army Corp. of Engineers’ Prime Power Project G. Regional Fuel Supply H. Additional Survey Information of Value
VII. Recommendations 47 A. Introduction B. Complete the NCR Jurisdiction Database C. Coordinate Jurisdiction-Wide Emergency Generator Activities D. Future Equipment/Facility Design, Procurement, & Installation E. Equipment Operation, Maintenance, & Testing F. Fuel Contracts – Emergency Provisions G. Interactions with FEMA/U. S. Army Corps of Engineers H. Jurisdiction’s Most Critical Facilities I. Regional Strategies to Obtain Additional Units during Emergencies J. Emergency Generator Units at Medical Facilities K. District & County Emergency Shelters
VIII. Appendices 51 A. Prior Emergency Generator-Related Projects B. Project Information Package C. Emergency Generator Survey Questionnaire D. Fuel Suppliers & Distributors – Questions
iii
E. Equipment Maintenance & Testing Organizations – Questions
F. Jurisdiction & Regional Strategy Questions
iv
G. Project Workshop – Agenda
Figures
I-1 National Capital Region & Participating Jurisdictions I-2 Arlington County, Virginia – Organization Chart I-3 Montgomery County, Maryland – Organization Chart I-4 District of Columbia – Organization Chart I-5 Private Cell Phone Service Installations I-6 Fuel Suppliers & Distributors I-7 Equipment Vendors & Technical Support Organizations II-1 Arlington Emergency Generator Size Distribution II-2 Montgomery County Emergency Generator Size Distribution II-3 District of Columbia Emergency Generator Size Distribution III-1 Arlington Emergency Generator Age Distribution III-2 Montgomery County Fire/Rescue Size Distribution III-3 Montgomery County Fire/Rescue Unit Vintage III-4 Montgomery County Schools Unit Size Distribution III-5 Montgomery County Schools Unit Vintage V-1 Arlington County Fuel Tank Capacity vs. Maximum Running Time
v
Tables
II-1 Departments Providing Equipment/Facility Data
III-1 Arlington Survey Responses
III-2 Capacity Ratings for Arlington County Units
III-3 Emergency Functions/Loads Served – Arlington County (Total)
III-4 Emergency Functions/Loads Served – Arlington Schools
III-5 Engine and Generator Manufacturers – Arlington County
III-6 Unit Capacity Ratings – Montgomery County
III-7 Emergency Functions/Loads Served – Montgomery County Fire/Rescue Facilities
III-8 Engine and Generator Manufacturers – Montgomery County Fire Departments/Rescue Squads
III-9 Generator Manufacturers – Montgomery County High/Middle Schools
III-10 Capacity Ratings for District of Columbia Units
III-11 Generator Manufacturers – District of Columbia MPD
vi
III-12 Generator Manufacturers – District of Columbia OPM
Acronyms & Abbreviations
ARI Advanced Research Institute – Virginia Polytechnic Institute and State University AT&T American Telephone & Telegraph Co. DCFD D.C. Fire Department DHS U. S. Department of Homeland Security DPW D.C. Department of Public Works EOC Emergency Operations Center EPA U. S. Environmental Protection Agency FCC U. S. Federal Communications Commission FEMA U. S. Federal Emergency Management Agency FRAC First Responder Authentication Credentials GSA U. S. General Services Administration MPD D.C. Metropolitan Police Department MWCOG Metropolitan Washington Council of Governments NCR National Capital Region OCTO D.C. Office of the Chief Technology Officer OPM D.C. Office of Property Management UASI Urban Area Security Initiative (A DHS program) USACE U. S. Army Corp of Engineers VDEM Virginia Department of Emergency Management Virginia Tech Virginia Tech Project Team
vii
Acknowledgements
ƒ To the many employees of Arlington County, Virginia; the District of Columbia; and Montgomery County, Maryland who gave their time and shared their knowledge and experience with the project team ƒ Lt. Gen. Robert Newman of the Virginia National Guard, who, in his previous position as Deputy Director of the Virginia Governor’s Office of Commonwealth Preparedness, saw the value and need for this effort for the NCR and supported its implementation ƒ Mr. George Nichols, staff coordinator for the Metropolitan Washington Council of Government’s ESF #12 Task Force, who helped shepherd the project’s proposal through the UASI funding cycles for FY05 and FY06 and has been very supportive of the effort. He also serves on the Regional Critical Infrastructure Committee which provided valuable advice during the project ƒ Members of the NCR Regional Critical Infrastructure Committee, chaired by Mr. Mike McAllister, and previously by Ms. Constance McGeorge, of the Virginia Governor’s Office of Commonwealth Preparedness, who provided advice, council, and direction during the project ƒ The personnel at the five private firms that agreed to be interviewed for this project and freely shared their knowledge and experience: 1) American Tower Co., 2) Fannon Petroleum Co., 3) Continental Petroleum Co, 4) Curtis Engine and Equipment, Inc., and Johnson & Towers Inc.
viii
Notice
This project was conducted by the following personnel who are located at Virginia Tech’s Advanced Research Institute in Arlington, Va.:
ƒ Dr. Frederick Krimgold, Principle Investigator ƒ John E. Bigger, Adjunct Research Professor ƒ Dr. Michael Willingham, Adjunct Professor
This project final report, Emergency Generation Capability & Reliability in the National Capital Region: A Pilot Survey, provides a summary of the survey information collected, interviews conducted, and data analyzed during the project. However, for security reasons, it is presented in a form that does not support correlation of specific facilities and equipment with their characteristics, capabilities, or limitations.
This report should be cited as: Bigger, J. and Willingham, M. (2009). Emergency Generation Capability & Reliability in the National Capital Region: A Pilot Survey. Arlington, VA: Advanced Research Institute.
ix
Executive Summary
Background Hurricane Isabel, which struck the National Capital Region (NCR) in September 2003, dramatically highlighted the problems related to poor operation of emergency generation facilities and equipment. More recently, the four hurricanes that struck Florida in a period of 45 days in August and September of 2004 and the three hurricanes that struck the Gulf Coast in 2005 again demonstrated the major impacts of loss of the electric utility service and the inadequate operation of emergency generators on the health, safety, and recovery of an area. Review of numerous “after-action,” “lessons learned” and “performance assessment” reports by government agencies, private firms, and consultants almost always identifies the inadequate performance of emergency generators as a significant factor in reduced capability of organizations during an emergency and subsequent recovery period.
Project Description The overall project goal is to help public jurisdictions and private firms increase the reliability and security of emergency generation facilities in the NCR. The project is funded by the U. S. Department of Homeland Security’s Office of National Capital Region Coordination – Urban Area Security Initiative (UASI) program. The approach used was to initially identify emergency generation equipment already installed in selected organizations in three NCR jurisdictions: Arlington County, Virginia; Montgomery County, Maryland; and the District of Columbia. The project team documented the characteristics of a number of facilities/units, as well as present practices for operation, maintenance, and testing by means of a Survey Questionnaire, meetings, and interviews with persons responsible for equipment-related activities. Primary emphasis was on emergency generators larger than 5 kW. The Virginia Tech team also inquired about contracts for refueling and testing and maintenance services. A second round of meetings was conducted to identify experience and document procedures, from a regional perspective, to obtain additional emergency generators in the event of an emergency or natural disaster. To increase the regional perspective on resilience of the jurisdictions, the Virginia Tech team then interviewed two equipment vendors and two fuel distribution companies that serve the selected jurisdictions. This effort identified other practices that public jurisdictions and organizations in the region use related to emergency generator operation, maintenance, and testing.
Significant Findings In all three participating jurisdictions, the departments surveyed have contracts with private firms in the region for regular operation, maintenance, and testing of their emergency generators. With the transfer of these activities to outside contractors, it is noteworthy that Virginia Tech team members found the jurisdiction personnel interviewed had good first-hand knowledge about and experience with the department’s emergency generating units.
1
2

ƒ The Virginia Tech team identified a number of “better or best” practices that are currently being followed in the three participating jurisdictions ƒ In all three participating jurisdictions, the emergency managers do not have a complete list of the jurisdiction’s emergency generators, whether permanently installed, mobile, or portable. These managers indicated that having such complete/up-to-date lists would be valuable during emergency situations, especially in the event of electric service outages which are not infrequent in all jurisdictions. ƒ Generators/quick connects at jurisdiction critical facilities, but jurisdiction personnel did not know how far down their critical facilities list this was true. ƒ Generators at hospitals, major medical facilities and almost all shelters, both emergency and weather, do not have installed emergency generator capacity to support required facility emergency systems AND HVAC systems.1 Thus, if there is an extended and widespread power outage, in summer or winter, these facilities might become unusable and patients and personnel would have to be evacuated. ƒ No common ID system for refueling and repair personnel exists in the NCR jurisdictions
Operation, Maintenance, and Testing Issues Emergency generator operation, maintenance, and testing practices and procedures vary not only across the three participating jurisdictions – this was expected – but also among the departments within each jurisdiction – this is not an uncommon finding. ƒ Operation, maintenance, and testing activities have been contracted out to local private firms. The competition for these contracts is very price-competitive and leaves little room for suggestions or recommendations by the proposing organizations. ƒ Each jurisdiction’s departments may have operation, maintenance, and testing contracts and refueling contracts with different firms; the contract “boilerplate” may be the same or similar, but the technical requirements and provisions vary. This can lead to paying different prices/rates to different firms for the same or similar services. ƒ Annual, full-load testing of emergency generators is considered critical in maintaining equipment reliability and security. Within the three participating jurisdictions, there is a range of commitment to this exercise: annual 4-hr tests, biannual 2-hr tests, and some units are not full-load tested at all. ƒ Each jurisdiction has a number of emergency generators that are 25 years of age or older; two jurisdictions have units in the 40-50 year age range. However, no jurisdiction makes any change in maintenance procedures, whether regularly scheduled or annual, for these older units.
Regional Strategies to Obtain Additional Units during Emergencies
1 Two Middle/High Schools in Arlington have full generation capabilities.  Thomas  Jefferson Middle School and Community Center, along with Washington‐Lee High  School and the adjacent Arlington Education Center, are fully supported with  emergency generators.  Both schools can be used as shelters.
3
The departments in the three jurisdictions have considerable experience with borrowing and lending emergency generators within their respective jurisdictions. However, in response to questions related to borrowing emergency generators from nearby jurisdictions, none of the department personnel interviewed in the three participating jurisdictions remembered having borrowed or lent units with any nearby jurisdictions during an emergency. Managers in all three jurisdictions indicated that mutual assistance agreements existed with the surrounding jurisdictions where additional units could be obtained. All jurisdictions are members of statewide mutual aid organizations and can obtain needed resources through their state Departments of Emergency Management. In addition, the states and the District are members of Emergency Management Assistance Compact (EMAC), which is a nation-wide organization administered by the National Emergency Management Association. Questions were asked about any experience with borrowing units from FEMA/USACE during an emergency, but again, there was no experience with obtaining units from this source.
One issue raised was that the local emergency managers and their staffs did not have reliable estimates for response times when requesting additional emergency generating units through either the NCR regional mutual aid agreements or the EMAC aid agreements. And they did not, with one exception, have information about the types, sizes, numbers of units and response times that FEMA Region III had to support local jurisdiction requests.
None of the departments in the three participating jurisdictions had approached local or regional emergency generator equipment suppliers to discuss or develop pre-negotiated agreements to obtain additional units on a quick-response basis. In discussions with local equipment dealers, this quick-response service is available and a few organizations in the NCR have these types of agreements in place.
Recommendations As a result of the equipment survey and the meetings and interviews, the Virginia Tech team members have a number of recommendations that will increase the emergency generation capability and reliability in the NCR. These recommendations are divided into categories and are briefly listed below; some of the categories include more than one recommendation. A more complete description of situations and findings leading to these recommendations are contained in Sections I – V of the report and expanded descriptions of the recommendations can be found in Section VI. For a few of the recommendations, the Virginia Tech team has included suggestions for addressing the issue(s).
The Virginia Tech team recognizes the financial situation of not only the three jurisdictions that participated in the study but also all the other NCR jurisdictions. These situations have resulted in a shortage of funds and personnel to implement these recommendations; the recommendations made are those where it is felt the jurisdiction’s security, reliability, and resilience will be increased. The majority of these recommendations relate to changes in emergency generator-related processes, procedures, or documentation by the participating jurisdictions. However, a few of the recommendations will require the investment of capital or additional operating funds.
Complete & Update NCR Jurisdiction Emergency Generator Database 1. Recommendation: Each NCR jurisdiction conduct an all-department emergency generator survey, including the school system, and develops a jurisdiction-wide summary of
4
emergency generators for use by the jurisdiction’s public works department, emergency manager, and the jurisdiction’s EOC staff. It is also suggested that each NCR jurisdiction updates the database annually. It is suggested that this would be an excellent project for a summer intern, such as a 2nd year student from a nearby community college enrolled in an Engineering Technology or a Pre-Engineering program. Coordinate Jurisdiction-Wide Emergency Generator Activities 2. Recommendation: Each NCR jurisdiction create a city, county, or District emergency generator committee, including the local school system, to coordinate activities on a jurisdiction-wide basis. It is suggested that the committee be patterned after the Williamsburg, Virginia’s “Best Practice” committee described in Section V.D.5.
Jurisdiction Equipment/Facility Design, Procurement, & Installation 3. Recommendation: Reducing no. of equipment suppliers (each jurisdiction has procured equipment from more than ten different suppliers)– reduce spare parts 4. Recommendation: When designing new, permanently installed units/facilities, there should be increased recognition of operation, maintenance, testing, and refueling personnel requirements.
Equipment Operation, Maintenance, and Testing 5. Recommendation: Each jurisdiction modifies emergency generator installations and test procedures to allow testing of the Load Transfer Switches when regularly testing is conducted on permanently installed generator units. 6. Recommendation: Each jurisdiction inspects its emergency generator units and, where fuel tank gauges or fuel level sight glasses are either not installed or operating, that these gauges are installed or inoperable gauges be repaired. 7. Recommendation: Each jurisdiction conduct an annual lubricating oil analysis for each emergency generator unit that is 25 years of age or older. This is standard practice for modern fleets in order to diagnose and anticipate problems. 8. Recommendation: Each jurisdiction ensures that an annual four-hour, full load test is conducted on each emergency generator units.
Fuel Contracts – Emergency Provisions 9. Recommendation – Each jurisdiction review its fuel supply contracts with respect to the feasibility, desirability, cost, and resulting increase in reliability and security of including one or more of the following emergency provisions when contracts are renewed: 1. “Top Off” Provisions – when an emergency situation can be predicted 2. “Time to Respond” – response time after notification 3. Priority of Service – within the fuel supplier’s own priority list 4. On-Site Emergency Generator – fuel supplier has on-site generation capability installed

Note: Where the phrase “…highest (10, 20…TBD) priority critical facilities” is used in this report, the intent is for the jurisdiction to make the decision of the number of facilities to examine.
Interactions with FEMA/U. S. Army Corps of Engineers
10. Recommendation – All NCR jurisdiction emergency managers participate in a joint meeting with FEMA Region III and USACE personnel regarding federal response to local requests for additional emergency generators in the event of an emergency. Information covered in such a meeting should include: 1) types of units available from FEMA, 2) federal agency process and response times, 3) specific size range of units available, and 4) type of advanced preparation at receiving facilities to connect the units in the event of an emergency. One option for such a meeting is that it be coordinated through the MWCOG R-ESF #5 (Emergency Planning) Committee. 11. Recommendation: Each jurisdiction coordinate with and submit the information needed for a USACE Site Assessment Input form for each of their top (10, 25…TBD) critical facilities to FEMA Region III Logistics Office. This effort will significantly reduce response time for the FEMA/USACE/contractor to supply needed emergency generators in the event of a major, wide-spread disaster with an accompanying electric service outage. (See Section VI-E and Appendix H)
Jurisdictions’ Most Critical Facilities 12. Recommendation: Each jurisdiction evaluate its highest (10, 20…TBD) priority critical facilities and for those without a permanent emergency generator installed, the jurisdiction determine what building electric service entrance modifications must be made to accept portable or mobile generators. Any needed service entrance modifications and quick connect installation be designed and added to the jurisdiction’s mitigation plan and capital improvement budget and schedule. For those facilities with permanently installed emergency generators, the jurisdiction evaluates the loads connected to the emergency electric bus to determine if the emergency generator has adequate capacity to serve all the connected loads. This recommendation could be developed into a single project for all NCR jurisdictions and submitted to the DHS UASI Program’s next procurement cycle for funding. 13. Recommendation: Each jurisdiction evaluate its highest (10, 20…TBD) priority critical facilities and evaluate the costs, benefits, and increased security obtained by increasing on-site fuel storage to support 72 hours of operation without refueling.
Regional Strategies to Obtain Additional Units during Emergencies 14. Recommendation – Each jurisdiction develops a time-sensitive plan for obtaining additional emergency generators in the event of a large-impact emergency event where electric service may be interrupted for a considerable time (See Section IV-C and Table IV-1) and consider the option of pre-negotiating agreements with local/regional equipment suppliers and/or rental firms for quick-access to units. One option is to consider this recommendation on an NCR-wide basis. Negotiations could be conducted on behalf of all the NCR jurisdictions which could result in a cost 5
6
savings when compared to a number of jurisdictions going out individually seeking this service. If this recommendation is considered for implementation, then it would be beneficial for individual jurisdictions to have completed an emergency generator survey (Recommendation No. 1, above) so each jurisdiction is knowledgeable about its own emergency generator capabilities and limitations.
Emergency Generator Units at Medical Facilities 15. Recommendation: Each jurisdiction reviews the on-site emergency generation capability of the hospitals and major medical facilities within its borders; documents internal systems and critical services connected to the emergency circuits; and documents what additional emergency generation capabilities might be required in the event of a long-term electric power outage to maintain patients in the facility. Such a review could be conducted with the facility’s annual fire department inspection. 16. Recommendation: Each jurisdiction reviews the incoming electric service at each of the hospitals and major medical facilities in its jurisdiction to determine emergency service capabilities and limitations (e.g., single electric service feeder, dual service on the same distribution pole line, etc.). It is suggested that these evaluations be conducted by a team with members from the utility (Pepco, Dominion Virginia Power, etc.), the respective jurisdiction’s emergency management and public works department, and the hospital’s facility and security department. Again, such a review could be coordinated with the facility’s annual fire department inspection 17. Recommendation: Conduct an investigation with local equipment dealers and regional equipment rental firms to determine the cost and availability of larger emergency generators on a quick-delivery basis. This would provide emergency back up units for use at hospitals and other medical facilities in the event of a large emergency situation in the NCR that includes a widespread electric service outage. This would significantly reduce the possibility of having to evacuate these large, patient-filled facilities.
District & County Emergency Shelters 18. Recommendation: Each jurisdiction reviews their primary emergency and weather shelters to determine capability of existing emergency generator to serve loads other than emergency circuits, such as the HVAC system, and documents additional emergency generation capabilities required in the event of a long-term electric power outage to maintain use of the facility. Such a review could be conducted with the facility’s annual fire department inspection.
I. Introduction
A. Background Government agencies, private firms, and private citizens have purchased emergency generators to provide power to critical facilities and activities in the event that normal electric service is interrupted, whether by natural events, technical failures, or human intervention. However, many emergency generators historically have not operated properly during emergencies. When called upon, a large number don’t start, and many of those that do start do not operate for as much as one hour. The reasons can be traced to a variety of factors: inadequate design, improper installation, or poor installation quality; lack of training and/or experience of facilities personnel; inadequate maintenance and regular testing of the units; insufficient on-site fuel storage for extended operation; and lack of adequate refueling provisions. Hurricane Isabel, which struck the National Capital Region (NCR) in September 2003, dramatically highlighted the problems related to poor operation of emergency generation facilities and equipment. More recently, the four hurricanes that struck Florida in a period of 45 days in August and September of 2004 and the three hurricanes that struck the Gulf Coast in 2005 again demonstrated the impacts of loss of the electric utility service and the inadequate operation of emergency generators have on the health, safety, and recovery of an area. Review of numerous “after-action,” “lessons learned” and “performance assessment” reports produced by government agencies, private firms, and consultants almost always identifies the inadequate performance of emergency generators as a significant factor in the reduced capability of organizations to serve citizens during an emergency and the subsequent recovery period. The National Capital Region is a formal region that includes a total of 17 government jurisdictions: areas of the State of Maryland the Commonwealth of Virginia and the District of Columbia along with 13 cities and counties in the two states, and the federal government. Figure I-1 shows all the jurisdictions in the NCR along with pointers to the three jurisdictions that participated in the project: Arlington County, Virginia; the District of Columbia, and Montgomery County, Maryland. Figure 1-1: National Capital Region (NCR)
Montgomery Co.
Arlington Co.
District of Columbia
7
8

B. Measuring Readiness One major question being raised in the homeland security community is: How does a jurisdiction determine its readiness to respond to an emergency situation, whether that emergency is a natural disaster, technical failure, accident, or intentional damage. Government agencies, academic teams, independent organizations, and consultants have spent considerable time and resources in trying to develop a set of metrics to measure the state of readiness of an organization, system, emergency plan, or the jurisdiction’s facilities; thus far, the results have not been very usable for decision-makers. Until “readiness” can be measured and confidence is developed in that measure, one approach is to regularly exercise or test the system and/or equipment and see how it responds. The term “Response Reliability” has been recently coined to capture this approach2. As discussed above, the emergency generators procured by jurisdictions and other public agencies, at considerable capital expense, have not always produced the reliable service they were installed to provide. And in many instances, various department and emergency management personnel in a jurisdiction do not have an accurate understanding of what the capabilities and limitations of the emergency generator “resource” is in their jurisdiction. Thus, this project is meant to help address that situation.
C. Project Inception Since mid-2000, a small group of researchers at Virginia Tech’s Advanced Research Institute has been involved in a series of projects that examined the capabilities, reliability, security, and resilience of four critical infrastructures – communications, energy, transportation, and water. The approach used in these projects involved the direct interaction between the researchers and personnel at infrastructure organizations, both public and private, and at jurisdictions served by these utilities and companies. The participants in the “guided” meetings and interviews ranged from maintenance personnel to top management. These research projects covered geographic areas in the National Capital Region (District of Columbia, Maryland, and Northern Virginia) after Hurricane Isabel struck in 2003; central Florida after three hurricanes struck the area during the 2004 hurricane season (Charlie, Francis, and Jeanne); and in other areas of southern and southeastern Virginia. One topic that came up repeatedly in the meetings and interviews in all the projects, with both the infrastructure organization and jurisdiction personnel, was the poor performance and insufficient numbers of emergency generators when significant weather-related events occurred, such as heavy storms, hurricanes, floods, etc. At the urging of Lt. Gen. Robert Newman, then Deputy Director of the Virginia Governor’s Office of Commonwealth Preparedness, the research group submitted a proposal to the NCR’s UASI FY2006 funding cycle. The proposal was to conduct an emergency generator survey in all 17 jurisdictions in the NCR. The proposal was submitted through the Metropolitan Washington Council of Government’s (MWCOG) R-ESF #12 Committee (Energy); the Committee rated the proposal No. 1 for that cycle and forwarded it (with other proposals) for review and funding. However, the UASI Senior Policy Group (SPG) did not support the recommendation and the project was not funded. An updated proposal was resubmitted for the UASI FY2007 funding cycle. The MWCOG RESF #12 Committee again rated it in the top category and forwarded it for review and funding.
2 Jackson, Brian A. The Problem of Measuring Emergency Preparedness: The Need for Assessing “Response Reliability” as Part of Homeland Security Planning. Rand Report OP-234-RC, 2008
9
The SPG did not support the recommendation and the proposal was not funded. However, in late2007, the Virginia Tech group was notified that unspent UASI funds were available and that the project could be funded, but at a much-reduced funding level. The smaller project was initiated in April 2008. During the final project planning phase, project team members met with the NCR Regional Critical Infrastructure Committee to review all aspects of the project. Committee members recommended that the proposed project include not only three proposed NCR public jurisdictions but also a private sector group or industry to allow comparison between private and public sector approaches to emergency generator policies and procedures. After consideration, the Virginia Tech team recommended that an evaluation of cell phone service organizations that provide service in Arlington County be included in the project survey. This was accepted and this private industry sector was added to the proposed project survey organizations.
D. Project Objectives The overall goal of this effort is to help increase the reliability, availability, and capacity of the emergency generator “resource” in the National Capital Region to increase the resilience of the region. To achieve that goal, this project has the following specific goals:
1. Identify and document characteristics of public emergency generators (> 5 kW) in three departments at each selected NCR jurisdiction and private communications (cell phone) firms in Arlington County 2. Document capabilities and limitations demonstrated by a sample of units/facilities in three jurisdictions and one private industry sector in the NCR 3. Develop a strategic approach to help NCR jurisdictions and private firms to manage emergency generating resources 4. Develop an initial assessment of information on selected publicly and privately owned emergency generators located in NCR 5. Document procedures used to ensure refueling services to support longer term unit/facility operation during emergencies
E. Project Approach & Methodology The information, experience, and results obtained from previous infrastructure- and emergency generator-related projects and research conducted by Virginia Tech team members provided a foundation on which to conduct this project. A brief description of seven of these recent infrastructure- and emergency generator-related projects is contained in Appendix A. A literature search also was conducted using the Internet and other technical databases (e.g., U. S. federal government scientific and technical resources). The overall project was divided into two major parts: a) conducting the survey of equipment presently owned by the selected jurisdictions and b) conducting a series of interviews at the jurisdictions to document regional strategies in place for borrowing or lending emergency generators, quick access to units during an emergency, and any pre-negotiated agreements with equipment suppliers or manufacturers. The initial contact at each jurisdiction was made through each jurisdiction’s Director of Emergency Management. A meeting was held where the project was described, the jurisdiction’s support needed for the project outlined, and the benefits of participation in the effort described. A
10
project information package, containing a two-page Project Description, the Project Data Security document, and the Survey Questionnaire, was distributed at this initial meeting. Copies of the Information Package documents are contained in Appendix B. If the jurisdiction agreed to participate in the project, a meeting with key personnel involved with emergency generators within the jurisdiction was scheduled. In this meeting the project was described in more detail, the specific departments within the jurisdiction were selected for participation, and the Project Information package and Survey Questionnaire were distributed. Similar meetings were conducted with the three departments in each jurisdiction: Fire or Fire & Rescue, Public Schools, and Public Works. Primary emphasis in the project was on emergency generators larger than 5 kW. Similar departments in each jurisdiction were selected so comparisons of policies and procedures could be made among the jurisdictions. In past projects, the Virginia Tech team has seen not only the need to contact and interview critical “upstream” suppliers of emergency generator products and services, which serve the jurisdictions, but also that product and service suppliers can provide a broader scope to the support and supply situation in the region in which the jurisdictions exist. During the early meetings and interviews with the jurisdiction personnel, specific support contractors were identified and company contacts obtained. Subsequently, telephone interviews were conducted with two companies that provide refueling services and two companies that provide equipment maintenance and testing services to the participating jurisdictions. Copies of the questions used in the interviews are included in Appendix D – Questions for Fuel Suppliers & Distributors and in Appendix E – Questions for Equipment Maintenance & Testing Organizations. After the survey questionnaire information and additional data had been obtained and initial evaluation completed, the preliminary results of the project were initially shared with personnel from the participating jurisdictions and limited outside invitees in a Project Workshop held on July 7, 2009. A copy of the Workshop Agenda is contained in Appendix G.
F. Project Survey The Survey Questionnaire was developed by the Virginia Tech team and is based on the team’s experience in previous projects that related to critical infrastructure and emergency generators. In these earlier projects, Virginia Tech team members met with personnel from a large number of jurisdictions after hurricanes had struck specific areas – the NCR, Central Florida, and southern Virginia. Also, copies of surveys from organizations identified in the Internet search were obtained and reviewed. The final Survey contains questions in the five areas listed below. Questions included in each of these areas were shown to be important in one or more of the previous Virginia Tech projects. This produced a long (2 ½ pages) document and it was recognized that and this would impose quite a burden on the jurisdiction personnel if the requested information was not readily available. The project personnel accepted the risk that this might impact the level of response. The Virginia Tech team also documented practices for operation, maintenance, and testing during the various interviews and meetings with the jurisdiction personnel. A copy of the complete Survey Questionnaire is contained in Appendix C.
11
Survey Questionnaire – Areas of Inquiry 1. Unit/Facility Ownership and Responsibility 2. Basic Facility/Equipment Characteristics 3. Operation, Maintenance, and Testing History and Practices 4. Emergency Loads Served 5. Fuel and Refueling Information
G. Additional Information Sources The Internet Literature Search led the Virginia Tech team members to a number of additional sources of information on the design, installation, operation, testing, and general experience with emergency generators. These included standards documents, industry magazines, technical papers, books, and general articles from both U. S. and international sources.
II. Survey Summary
A. Survey Objectives The overall objective of this Emergency Generator Survey is to provide detailed information on emergency generating equipment already installed in the selected NCR jurisdictions. The collected data and subsequent analyses are expected to provide the basis for recommendations to increase the reliability and performance of all units for any future emergency situations.
B. Survey Approach & Methodology The first step in developing a program to improve the reliability and performance of publicly owned emergency generators is to document already-installed units and their characteristics, which would allow informed decision-making in developing an effort to increase the reliability of all units. The Survey questionnaire was developed to obtain information not only about each unit’s physical and performance characteristics but also about key areas identified in “lessons learned” and after-action reports as significant weaknesses during emergencies and natural disasters. These latter areas included: 1) unit operation, maintenance, and testing; 2) refueling procedures; and 3) specific responsibilities for all aspects of emergency generator activities. The questionnaire sought specific information in five (5) areas (listed below). A copy of the complete questionnaire is included in Appendix B.
ƒ Basic Equipment/Facility Characteristics ƒ Fuel & Refueling Information ƒ Operation, Maintenance, and Testing Practices ƒ Critical Loads Served ƒ Unit/Facility Ownership and Responsibility
C. Overall Survey Totals Completed survey forms were returned from all requested facilities in Arlington County and partially from Montgomery County. No forms were returned from the District of Columbia. In the latter two jurisdictions, information was extracted from secondary sources, such as individual department equipment lists (paper and electronic). These department lists, however, did not provide information and data in all the areas that the Survey Questionnaire requested. Summarized versions of equipment characteristics and other data and information from each jurisdiction are presented in Section III. A summarized version of information obtained from the three jurisdictions is presented in this section.
12
Total Capacity of Units Surveyed by Jurisdiction • Arlington County, Virginia: 19,120 kW • Montgomery County, Maryland 8,088 kW • District of Columbia 17,106 kW
Table II-1: No. of Units Surveyed by Jurisdiction & Departments
Juris./Dept.
Emergency Services/ F&R
Public Works
Public Schools
Other TOTAL
Arlington County
27 14 33 10 84
District of Columbia
29 43 — 48 120
Montgomery County
40 22 63 — 125
Figure II-1: Arlington Emergency Generator Size Distribution
13
Figure II-2: Montgomery County Emergency Generator Size Distribution
Figure II-3: District of Columbia Emergency Generator Size Distribution
14
15
D. Cell Phone Service Organization Participation In response to the NCR Regional Critical Infrastructure Committee’s request for participation in the Survey from private industry, a list of cell phone service and equipment owners in Arlington County was developed. In developing the list of potential contacts for cell phone service providers and equipment owners, two principal sources were utilized: The U. S. Federal Communications Commission (FCC) and Arlington County itself. The FCC maintains a publicly available database of all cell phone tower installations and radio antennas by geographic area. In the FCC database, certain antenna and tower characteristics are included (e.g., tower height) along with contacts at the company. Arlington County also maintains a list of the tower owners because building permits, etc. are required; this list is not generally available to the public. Initial contact was made with three organizations: American Tower Co., Verizon, Inc., and American Telephone & Telegraph Co. (AT&T). The American Tower Co. was contacted, agreed to participate, and a telephone interview was conducted. American Tower has a total of over 20,000 facilities located in all 50 states; almost 8,000 are installed on buildings or other structures with the rest being stand-alone towers. However, the company presently has very few installations with emergency generators installed on the site and these are spread around the country. The Survey Questionnaire was used during the interview, but no specific installation characteristic data was collected. The results of the interview are integrated in the various issues and findings in this report. When contacted, Verizon and AT&T declined to participate in the NCR Survey because of pending FCC action. After Hurricane Katrina, the FCC conducted a review of telecommunication failures. One result of the review is a proposed change in regulations related to backup power at cell tower sites and other communication facilities. The potential change could have a major impact on the industry as there are over 210,000 cellular sites and over 20,000 other facilities. FCC hearings on the pending regulations are being scheduled later in 2009 and/or in 2010. Thus, the reluctance of the cell service companies to participate in the NCR project on that exact same topic.
III. Facilities & Equipment
A. Introduction This section provides a summary of statistics and information based on data and documentation, including returned Survey Questionnaires and equipment data lists, from the departments in the three participating jurisdictions. When data lists were provided, whether on paper or digitally, the information on the lists did not provide all the information on the facility/unit that was requested in the Survey. An attempt has been made to present findings in a consistent manner, although in some instances lack of specific data precluded this from being fully realized.
B. Arlington County, Virginia Following the initial meetings with Arlington County emergency personnel, survey packages were mailed/e-mailed to selected Arlington County departments. A breakdown of the organizations that received the survey packages is given below, along with the number of responses from each organization.
Table III-1 Arlington Survey Responses3 County Organization Responses Water Pollution Control Bureau 14 Facilities Management Bureau 27 Public Schools 33 Parks & Recreation 2 Adminstrative Offices 8 Total Survey Responses 84
There were 84 responses, four with two units, and one with no unit information. Responses covering a total of 87 emergency generating facilities/equipment units were received from Arlington County. This included four sites where two units were installed at each site. The Survey asked for nameplate electric generating capacity of each unit was requested. The total capacity for the 87 emergency generator units reported on in Arlington is 19,757 kW.
Table III-2 shows statistical measures of unit capacity rating for the 87 units for all the participating Arlington organizations.
Table III-2: Capacity Ratings for Arlington County Units (87 Units)
Mean Capacity 221 kW
16
3 Arlington County and Public Schools are separate legal and political entities with combined and coordinated emergency response programs.
Median Capacity 105 kW Mode Capacity 40 kW
The Survey Questionnaire asked about the type of fuel used in each unit. Of the 87 Arlington units, 75 reported this information: 73 are Diesel-fueled and 2 are fueled by natural gas.
The Survey Questionnaire asked for the date of unit installation or purchase and this information was provided for 59 of the 87 units. The age/vintage distribution for the 59 units is shown in Figure III-1 and indicates that 53 units are 25 years or less and only 6 of the 59 units are 25 years or older. The average age/vintage of units was 5.3 years for the Arlington Public Schools, 9.9 years for the fire department, and 15.9 years for the Public Works.
Figure III-I: Arlington Emergency Generator Age Distribution
17
A Survey question asked about the general equipment condition. Information on 62 of the 87 Arlington units was provided: New = 2, Excellent = 11, Good = 47 and two described as “Okay.”
The Survey asked about Total Engine Operating Hours and Annual Engine Operating Hours. For 36 units that reported this information, the Average Total Operating Hours was slightly in excess of 600 hours with Average Annual Operating Hours of 49.
The survey asked about unit run time based on available fuel onsite. The unit run-time on a full tank of fuel for 46 of the 87 units that reported this data was 43.2 hours, with a median run time of 24.1 hours.
18
A Survey question asked about the Scheduled Maintenance of the units. Information on 38 units was reported and Scheduled Maintenance was reportedly conducted every quarter on 26 units including oil change and analysis, with a more extensive Annual Maintenance check on five units.
The Survey asked about unit failures to start/run during scheduled tests over the last two years. This information was provided for Arlington’s Water Pollution Control Bureau’s 14 units. A total of 10 of the 14 units had reported failure(s) during the past two years for scheduled startand-run-with-no-load tests. No failures for annual full load or other scheduled duration tests were reported
The Survey asked about the type of load transfer switch the permanently installed units had. Information for 63 installations was provided: 61 units had automatic transfer switches and 2 had manual transfer switches.
The Survey asked about what type of emergency functions/emergency loads were served by the emergency generator unit. The questionnaire listed nine (9) specific categories for the responders to check. Information was provided on 85 of the 88 installations county-wide and is shown in Table III-3. The responses for thirty-three (33) Arlington public schools are shown in Table III-4.
Table III-3: Emergency Functions/Loads Served (Arlington County; 85 Responses) Load Served No. Emergency Exit Lighting 69 Critical Process Supply/Control 33 Interior Emergency Lighting: 70 Security Systems 55 Internal Communications: PA Systems 55 Facility Air Conditioning 14 External Communications: Phones/Chargers 32 Door Locks/Gate Openers 19 Life-Support Supply/Control 15
19
Table III-4 Emergency Functions/Loads Served Arlington School System: 33 Schools Load Served No. Emergency Exit Lighting 32 Critical Process Supply/Control 2 Interior Emergency Lighting: 32 Security Systems 32 Interior Communications: PA Systems 32 Facility Air Conditioning 1 Ext. Communications: Phones/Chargers 1 Door Locks/Gate Openers 0
The Survey asked about the equipment manufacturers for the emergency generators and associated engines. Information provided for 83 generators and 87 engines installations countywide is shown in Table III-5.
Table III-5: Engine and Generator Manufacturers – Arlington County All Departments
Generator Mfgr. No. of Gen’s Engine Mfgr. No. of Eng’s Marathon 33 Caterpillar 4 Katolight 20 Cummins 16 Kohler 12 Detroit Diesel 2 Onan 9 Doosan 1 Caterpillar 3 Ford 1 General 2 John Deere 46 Olympian 2 Katolight 2 Detroit 1 Kohler 1 Ingersoll Rand 1 Mitsubishi 1 Onan 2 Perkins 8 TOTAL 83 TOTAL 87
The Survey asked about fuel suppliers for the emergency generators and associated engines. The two principal suppliers were Continental Petroleum (Fairfax, Virginia) and Fannon Petroleum Services (Alexandria, Virginia). A Fannon representative stated that Tri Gas & Oil (Federalsburg, Maryland) was available to provide emergency fuel for Fannon customers. Washington Gas was identified as supplying natural gas to one facility.
C. Montgomery County, Maryland
A total of 84 emergency generation units have been identified in Montgomery County from supplied equipment lists, with a combined generating capacity of 8,088.4 kW. The departments responding to the survey form distributed by ARI included the Montgomery County Fire Departments and Rescue facilities. A total of 38 fire station reports and 2 rescue squads
responded to the questionnaire. As discussed below, information from the school system was obtained from a variety of equipment lists; these lists did not provide the total data requested in the ARI Survey questionnaire.4
Fire Department and Rescue Units
Table III-6 shows statistical measures of unit capacity rating of the 33 units for the Montgomery County Fire Stations and Rescue Units.
Table III-6: Unit kW Ratings for Montgomery County Fire/Stations/Rescue Units (33 Units) Average Capacity 108.1 kW Median Capacity 60 kW Mode Capacity 30 kW
The size distribution for the Montgomery County Fire/Rescue units is shown in Figure III-2.
Figure III-2: Montgomery County Fire/Rescue Size Distribution
The Survey Questionnaire asked about the type of fuel used in each unit. Of the 40 Fire/Rescue units reporting, 35 reported this information: 24 are Diesel-fueled, 9 are fueled by natural gas, one by propane and one by gasoline.

20 4 Montgomery County and Montgomery County Public Schools are separate legal and political entities.
A Survey question asked about the general equipment condition. Information on 30 of the 40 Fire Department/Rescue units was provided: New = 5, Excellent = 2, Very Good = 4; Good = 14; Fair – 2; and Poor = 3.
The survey asked about unit run time based on a full fuel tank. Unit run-time for the 24 units that reported this data averaged 83.3 hours, with a median run time of 72 hours. This means the Fire Chief’s are nearly in place.
The Survey Questionnaire asked for the total operating hours of the units. For the 22 units reporting data, the average total operating hours was 542.6 hours and the median total operating hours was 375.9 hours.
A Survey question asked about the Scheduled Maintenance of the units. Information on 33 units was reported, with Scheduled Maintenance reportedly conducted every quarter on 1 unit, monthly on 1 unit, semi-annually on 5 units, and annually on 26 units.
Startup-Run tests were reported for 35 units, with 1 unit reporting twice weekly tests, 32 units reporting weekly tests, and 2 units reporting monthly tests.
The Survey Questionnaire asked for the date of unit installation or purchase and this information was provided for 33 of the 40 units. The age/vintage distribution for the 33 reporting units is shown in Figure III-3 and indicates that 8 of the 33 units are 25 years or older. The average age/vintage of units for the Fire Departments and Rescue Facilities was 15.6 years.
Figure III-3: Montgomery County Fire/Rescue Unit Vintage
21
22
The Survey asked about unit failures to start/run during scheduled tests over the last two years. This information was provided for 10 Fire Department units. A total of 7 of the 10 units had reported failure(s) during the past two years for scheduled start-and-run-with-no-load tests. No failures for annual full load or other scheduled duration tests were reported.
The Survey asked about the types of emergency functions/emergency loads were served by each emergency generator unit. The questionnaire listed nine (9) specific categories for the responders to check. Information was provided for 33 of the 40 Fire/Rescue installations countywide and is shown in Table III-7. There is no information for Montgomery County public schools.
Table III-7: Emergency Functions/Loads Served Montgomery County Fire/Rescue Facilities: 40 Responses Specific Emergency Function No. of Replies Interior Emergency Lighting: 33 Security Systems 8 Int. Communications: PA Systems 33 Facility Air Conditioning 6 Ext. Communications: Phones/Chargers 32 Door Locks/Gate Openers 2 Life-Support Supply/Control 24 Emergency Exit Lighting 32 Critical Process Supply/Control 13
The Survey asked about the equipment manufacturers for the emergency generators and associated engines. Information provided for the 40 fire/rescue installations county-wide is shown in Table III-8
Table III-8: Engine and Generator Manufacturers Montgomery County Fire Departments/Rescue Squads Generator Mfgr No. of Gen’s Engine Mfgr No. of Eng’s Detroit/Spectrum 1 Denso 1 Elliot Magnetek 1 Detroit Diesel 1 Lima 1 Ford 6 MTU Energy 2 Hercules 1 SMDO 2 John Deere 10 Unidentified 2 Perkins 2 Generac 2000 5 Katolight 5 Kohler 5 Onan 12 Unidentified 19 TOTAL GENERATORS 36 TOTAL ENGINES 21
High Schools and Middle Schools
Information was obtained from Montgomery County equipment lists for 26 high schools and 37 middle schools. The data has been incorporated to the degree possible consistent with the categories included in the ARI Survey Questionnaire form.
Information regarding unit size shown in Figure III-4 was based on information for 51 high schools and middle schools. The total capacity of the 51 units is 452 kW. The average unit size is 88.7 kW, the median size is 56.2 kW, and the mode is 37.6 kW.
Figure III-4: Montgomery County Schools Unit Size Distribution (51 Units)
Generator manufacturers for the 63 emergency generators in Montgomery County high schools and middle schools are shown in Table III-9
Table III-9: Generator/Engine Manufacturers Montgomery County High/Middle Schools Manufacturer Number of Units Cat 1 Cummins Power Generator 1 Detroit Diesel 3 Dmt Corporation 1 GENERAC-GENERATOR 19 Kohler Power Systems 12 Magnamax 1 Olympian 1 Onan 24 23
TOTAL 63
Information regarding the date of unit installation or purchase was provided for the 63 high schools and middle schools. The age/vintage distribution for 62 reporting units is shown in Figure III-5 and indicates that 51 units have been in service 20 years or less and only 11 of the 62 units are 20 years or older. The average age/vintage of units for the high schools was 15.2 years.
Figure III-5: Montgomery County Schools Unit Vintage
D. District of Columbia No information was provided on the ARI Survey Questionnaire from the participating departments in the District of Columbia. Data was provided by means of equipment lists and a copy of a 2005 Critical Facilities Survey report that included emergency generators in the Metropolitan Police Department (MPD), the Fire Department (DCFD), the Office of Property Management (OPM), the Department of Public Works (DPW), and the Office of the Chief Technology Officer (OCTO).5 The equipment lists and the Facilities Survey report did not include information and data on unit vintage, service and maintenance statistics, unit testing, or primary emergency loads and functions. Although 120 generator units were identified in the District of Columbia, only 89 reported unit size. Figure II-3 provides a graphic summary of unit size distribution. Table III10 shows statistical measures of unit capacity rating for the 89 units identified in the District of Columbia.
24 5 District of Columbia Generator Testing Program – Generator Data Catalog, December 2005.
25
Table III-10: Capacity Ratings for District of Columbia Units (89 Units)
Mean Capacity 211 kW Median Capacity 100 kW Mode Capacity 15 kW
Metropolitan Police Department (MPD) Information for generators identified under the authority of MPD listed 22 units with a total capacity of 1465.9 kW. The average unit size of the14 units for which information was provided was 104.7 kW. The median unit size was 60.5 kW. For the 20 units reporting fuel type, 16 were identified as diesel and 4 were natural gas. All Diesel-fueled units were listed as capable of operating from 28-32 hours under full load, with onsite fuel capacity from 75 gallons to 275 gallons. Generator/Engine manufacturers for 15 MPD emergency generators with available information are shown in Table III-11:
Table III-11: Generator/Engine Manufacturers District of Columbia MPD (15 Units)
Manufacturer Number of Units Empire 1 General 1 Katolight 1 Kohler 1 Onan 4 Power Zone 1 Trade Wind 6 TOTAL 15
`
Fire Department (DCFD) Information from the District of Columbia Fire Department was obtained for 27 units with a total capacity of 640 kW. All reported units were manufactured by Generac. The average unit size was 23.7 kW, and the median was 15 kW. A total of 25 units were identified as fueled by diesel, and 2 units were fueled by natural gas. All diesel-fueled units had on-site capacity of 137 gallons.
Department of Public Works (DPW)
Information for DPW identified 5 units. Total unit capacity for three reporting units was 295 kW, with an average unit size of 98.3 kW. The median unit size was 70 kW. Three units were diesel-based and two units were fueled by natural gas. Manufacturers included Kohler (1), Katolight (2), Alban (1), and General Electric (1).
26
Office of Property Management (OPM)
Equipment lists for OPM identified 43 units, with a total capacity of 14,702 kW and an average unit size of 341.9 kW. The median unit size was 250 kW. Some 40 units were diesel-fueled and 3 units were fueled by natural gas. For 39 units reporting data, the average hours under full load with full fuel tanks is 22.9 hours, with a median value of 20 hours. Three natural gas units were reported as providing continuous operation.
Generator manufacturers for 43 OPM emergency generators with available information are shown in Table III-12:
Table III-12: Generator Manufacturers District of Columbia OPM Manufacturer Number of Units All Power 1 Caterpillar 5 Cummins 6 Detroit Diesel 1 Generac 1 General 2 John Deere 1 Katolight 3 Kohler 4 Lima 2 Marathon 2 Onan 12 Tradewinds 1 Waukesha 1 Western Branch 1 TOTAL 43
Office of the Chief Technology Officer (OCTO) Information for OCTO identified 2 units. Total unit capacity for two reporting units was 1650 kW, with both diesel-based units rated at 825 kW. One unit with on-site fuel capacity of 1000 gallons was capable of operating for 12 hours under full load. The other unit with on-site fuel capacity of 5000 gallons was capable of operating for 72 hours under full load. … Arlington County and Public Schools are separate legal and political entities
IV. Local-Regional Strategies: Obtaining Additional Units
A. Introduction In addition to the Survey questions about existing emergency generator units and facilities and questions related to specific equipment characteristics, operation, maintenance, and testing, Virginia Tech team members surveyed jurisdiction personnel about any existing local, regional, state, or national-level strategies in place or used in obtaining additional emergency generators during or after a local area disaster. In previous projects, Virginia Tech team members have interviewed jurisdiction and infrastructure personnel in the NCR (after Hurricane Isabel struck in 2003), in central Florida (after the 2004 Hurricane season), two areas in the Commonwealth of Virginia, along with the USACE evaluation of the situation in New Orleans before, during, and after Hurricane Katrina (2005). In these emergency events and disasters, many individual jurisdiction personnel indicated they didn’t have enough generators to meet their needs during the emergency or that some of the units they did have failed. In the meetings and interviews associated with these projects, the Virginia Tech team identified various approaches used to obtain additional emergency generator units. Nine of these are listed in Table IV-1 One illustration of the response to not having enough emergency generators on hand during the 2004 Florida hurricane season: the Florida Department of Transportation purchased 400 new generators and stored them at sites around the State in preparation for the 2005 hurricane season.
Table IV-1 Options Used by Non-NCR Jurisdictions to Obtain Additional Emergency Generators Procedure Approach Description 1 Borrow Informal From surrounding jurisdictions 2 Formal Request Formal Through mutual assistance agreements – with local/regional jurisdictions 3 Formal Request Formal – Governor declaration of disaster Jurisdiction Æ State-level mutual assistance agreements 4 Formal Request Formal – Presidential declaration of disaster Jurisdiction Æ State Æ Federal Æ FEMA 5 Contract Pre-negotiated With local/regional enginegenerator sales & service firms 6 Contract Pre-negotiated Regional/national engine-generator rental companies 7 Contract Pre-negotiated Engine-generator manufacturers or integrators
27
28
Procedure Approach Description 8 Borrow Informal Local/regional construction companies 9 Purchase Formal Purchase Order Local/regional engine-generator dealers and manufacturers or integrators
B. Approach for Defining Outside Resource Needs When developing an approach or strategy for defining outside resource needs, a number of aspects are usually considered by jurisdiction emergency management personnel and executives; these include, but are not be limited to: a) magnitude of emergency or natural disaster impact, b) response time for needed support, c) expected duration of emergency situation, d) administrative complexity of obtaining added resources, and e) funding sources for needed resources. 1. Magnitude of Impact: ƒ Small impact area w/ only a few critical facilities impacted ƒ Moderate impact area w/ a number of critical emergency functions & facilities impacted ƒ Large impact area w/ many critical emergency functions and facilities impacted 2. Response Time for Support ƒ Resources needed w/in very limited hours for critical functions & facilities (e.g., hospitals, waste water processing & pumping, etc.) ƒ Resources needed w/in 10s of hours for less critical functions & facilities (e.g., highway signal lighting, railroad signal system and grade-level crossing gates, etc.) 3. Duration of Emergency Situation ƒ Anticipated to be only a few hours ƒ Anticipated to be one or more days ƒ Anticipated to be an extended period – many days to over a week 4. Administrative Complexity ƒ Borrowing from surrounding jurisdictions w/in same state ƒ Requesting interstate resources ƒ Requesting federal resources
5. Funding Sources for Resources ƒ Borrow resources locally and let accounting people sort it out later ƒ Borrow from state-level sources where terms & conditions already agreed to ƒ Borrow from federal-level sources where terms & conditions are defined
29
C. Timeline for Obtaining Additional Emergency Generators During the discussions with jurisdiction personnel and the equipment maintenance and testing contractors (both firms are dealers for a number of equipment brands), questions related to response times for delivering a generator to a site during an emergency were raised. Because of the lack of documented experience by the jurisdictions with obtaining units through various mutual aid agreements or from equipment firms, only “rough” estimates were made. A summary of these is shown in Table IV-2. These response time estimates could be significantly changed depending on whether the jurisdictions had warnings and could prepare by pre-positioning equipment (e.g., spring floods or hurricanes) or the emergency situation happened suddenly (e.g., large tornados, flash floods, or earthquakes).
Table IV-2 Roughly Estimated Timeline to Obtain Additional Emergency Generators for NCR Situations Response Time Source Description 0 – 2 Hr. ƒ Intra-Department Borrowing within a single jurisdiction < 4 Hr. ƒ Inter-Jurisdiction Borrowing from other NCR jurisdictions via Mutual Aid Agreement ƒ Local Equipment Supplier Firms w/ Pre-Negotiated Agreements 4 – 24+ Hr. ƒ State-Level Mutual Aid Agreement (EMAC)* ƒ Regional Equipment Rental Firms w/ PreNegotiated Agreements < 24** – 72 Hr. ƒ Jurisdiction Æ State Æ Federal Æ FEMA Æ USACE*** * Governor’s declaration required ** Assumes pre-positioning of units *** Presidential declaration required
D. NCR Intra-Jurisdiction Loans 1. Borrowing/Lending Process The departments in the three jurisdictions have ongoing experience with the borrowing/lending of emergency generators within their respective jurisdictions. Different procedures are used in each jurisdiction. For example, in one jurisdiction a request for a generator (in either an emergency or non-emergency situation) goes to the Emergency Manager who, in turn, contacts the designated person in the Public Works Department responsible for providing units to the requesting organization or facility. The reasoning behind this approach is that the Emergency Manager will then know where the emergency generator resources are deployed.
30
In a second example, all requests for an emergency generator in this jurisdiction (in either an emergency or non-emergency situation) go directly to the person in the Public Works Department responsible for providing the unit to the requesting organization or facility. However, in the interviews where the jurisdiction emergency managers were present, it was learned that NONE of the emergency managers have a complete list of the emergency generators owned by the jurisdiction, whether permanently installed, mobile, or portable.
2. Borrowing/Lending Procedures Again, once a request for an emergency generator has been made, different procedures are followed in each jurisdiction in what is supplied with the unit and what support is provided. For example, in one jurisdiction the provided unit is brought to the location requested and that completes the Public Works responsibility. It is up to the site facilities manager to hook the unit up, operate the unit, and call for fuel when needed. In a second procedure example, the jurisdiction’s Public Works Department brings all connection equipment needed, works with the site facilities manager to hook the unit up, get the unit running, and then turns it over to the site facility manager.
E. NCR Inter-Jurisdiction Loans In response to questions related to borrowing or lending emergency generators to nearby jurisdictions, none of the department personnel interviewed in the three participating jurisdictions remembered having borrowed units from any nearby jurisdictions during an emergency. Since the Virginia Tech team did not meet with all department personnel, one could conclude only that the experience of borrowing/lending is extremely low in these jurisdictions. Managers in all three jurisdictions indicated that mutual assistance agreements existed with the surrounding jurisdictions where additional units could be obtained. All jurisdictions are members of state-wide mutual aid organizations and can obtain needed resourced through the state Departments of Emergency Management. In addition, the states and the District are members of Emergency Management Assistance Compact (EMAC), which is administered by the National Emergency Management Association. The compact is an agreement among all 50 states, the District of Columbia, Puerto Rico, and the Virgin Islands to provide assistance across state lines when a disaster occurs. One issue raised about requesting units through EMAC was the time it would probably take to get the unit to the building or facility where it was needed. The jurisdiction personnel were asked about any experience with borrowing units from State National Guard units or from FEMA/USACE. Again, the response was negative. One issue raised in these discussions was that the local emergency managers and their staffs did not have information about what types, sizes, or numbers of units that FEMA Region III had to support local jurisdiction requests. Again, the issue of response time to local requests that went up through the state, to the federal government, to FEMA, and then the units are distributed to the jurisdictions.
31
F. Preferences for Inter-Jurisdiction Loans During the Project interviews and meetings, jurisdiction personnel were asked for any suggestions or recommendations that should be included in the development of an NCR-wide regional strategy for borrowing/loaning emergency generators during an emergency. The following is a list of the suggestions most frequently made by the jurisdiction personnel (Note: the order of appearance does not imply any priority). ƒ Have every jurisdiction emergency manager have a complete list of their emergency generators so with one phone call, a manager needing additional units could find out if the other jurisdiction had units that met specific requirements. ƒ Have the U. S. Army Corp of Engineers survey the NCR region and develop a list of all emergency generator units and their characteristics that would be available for loan. ƒ Have a common set of Maintenance & Testing Procedures for NCR Jurisdictions that the units would have undergone prior to being loaned to help ensure operability and reliability. ƒ Have all emergency generator units available for loans undergo an Annual Full-Load Test to help ensure operability and reliability.
G. Discussions with Regional Equipment Suppliers A question was asked of the jurisdiction personnel: Had they initiated any discussions with regional emergency generator suppliers or equipment rental firms regarding the quick availability of additional generator units before or shortly after an emergency? The general response to the question was no. A few indicated they knew of such agreements with other governments agencies, such as the federal General Services Administration (GSA). The interviewees felt that additional units they might need could be obtained from surrounding jurisdictions or, if that wasn’t successful, the needed units could be obtained through the various city/county/state mutual assistance agreements that were in place. However, as mentioned above, the procedures for obtaining units under these mutual assistance agreements hadn’t been exercised or included in any area or regional emergency exercises, to date, nor had any actual borrowing under these agreements taken place. Thus, estimates of response times were only rough estimates.
A similar question was asked of the participating equipment and service suppliers: Had they been approached by or had they marketed a “quick-response service” to local jurisdictions? The response of both firms was that they did provide such a service to a few organizations in the NCR, but this service hadn’t been marketed to all the NCR jurisdictions. One supplier indicated they had over a dozen mobile/transportable engine-generator units stored at their headquarters, ranging in size from 100 kW to 2 MW. These units are available for rent by customers; the units are also used by the supplier when performing maintenance or repair on critical units (e.g., hospital units). The other supplier indicated they already have a negotiated agreement in place with a regional rental firm to have quick delivery of units in an emergency. The delivery of these units can be within 10s of hours. This supplier also indicated they already provide this short time availability and delivery service to some NCR organizations.
Many engine manufacturers and their regional equipment integrators and retailers have equipment supply programs where short-time availability and delivery of packaged enginegenerator units can be contracted for. However, the jurisdiction personnel indicated they had not discussed this service with the local firms.
32
H. Discussions with Area Construction Firms and Building Associations Questions were asked of the jurisdiction personnel about any discussions they may have had with local construction companies about the feasibility of borrowing and the availability of emergency generator units during emergencies. No discussions have been held by jurisdiction personnel with local and area construction firms or their association regarding the possibility of borrowing units during and after an emergency. In a previous project, Virginia Tech team members suggested this source for additional units. When the jurisdiction followed up, the response was positive. The local construction companies usually have smaller (< 150 kW) portable and mobile units that use gasoline or diesel fuels. These units are frequently used and maintained in good condition; when used they usually operate under load.
V. Critical Issues

A. Introduction During the course of this project, a wide range of issues related to emergency generators were identified; these were obtained through three main channels: a) the data provided on the returned Survey Questionnaires and subsequent analyses, b) those raised during the meetings, interviews, and the project workshop by personnel from the participating jurisdictions and companies, and c) those identified in previous infrastructure- and emergency generator-related projects and research efforts conducted by Virginia Tech team members that were relevant to this project. Where issues come from c), this is clearly identified that they are of non-NRC origin and are used for illustration only. The following sections provide brief discussions of the issues that were identified in a wide range of areas during the project interviews and meetings.
B. Jurisdiction Personnel Although the majority of the work related to operation, maintenance, testing, and servicing the emergency generators is contracted out, the jurisdiction personnel involved in this area that the Virginia Tech team members met with had a high level of direct knowledge and hands-on experience with the equipment, systems, and procedures. However, the Virginia Tech team members noted that a number of these knowledgeable people have “a bit of gray hair” and may be getting close to retirement. No questions were asked on the Survey Questionnaire or during the interviews specifically about succession planning or training, but it is hoped that the jurisdictions are training new staff members so the corporate experience and memory in this area is not lost. An opportunity to increase training, knowledge and experience of jurisdiction personnel was raised by one of the participating maintenance and testing companies. They indicated that when multi-hour, full-load testing of a unit is conducted; the contractor’s technician is on site for the test’s duration (2 to 4 hours) and is willing to conduct training on the specific unit for a jurisdiction’s new hires and persons seeking to increase their knowledge about the unit.
C. Contracted Support Services The regular maintenance and testing of most emergency generators in the three jurisdictions studied are conducted under contract with equipment maintenance and testing firms in the region. These support contracts are competitively bid and run from one to three years in length. The Requests for Proposals (RFP) issued by the jurisdictions for maintenance and testing services are usually detailed and the awards are primarily based on cost. When asked if the public agencies were open to suggestions by bidding organizations, the response was “usually not.” Departments within a single jurisdiction may have emergency generator service contracts with different firms. The public school system in each jurisdiction deals with their emergency generators independently of other county and District departments; they usually have independent contracts for fuel supply and for maintenance and testing services.
33
34
D. Practices Within & Across Jurisdictions 1. Operation, Maintenance, & Testing Emergency Generator Fuel Capacity – The Survey Questionnaire asked two questions related to emergency generator fuel tanks: a) the tank capacity (in gallons) and b) the length of time the engine would run on a full tank of fuel (in hours). The answer to the first question was most always known or the information was readily available to staff. The answer to the second question, however, was not often known. For a unit that has been operating for a period of time during an emergency, someone must visually check and, if needed, call for fuel. Having information about the length of time the generator will run with a full tank of fuel will help in scheduling refueling and will reduce considerably the risk of running out of fuel. Emergency Generator Scheduled Testing – An issue raised by both jurisdiction personnel and the maintenance and testing contractors dealt with the regularly scheduled operation of the units, whether it is done on a weekly, monthly, or other basis. For these regular tests, the units are usually started, run for about 15 minutes in a no-load condition, and then shut down. The Diesel engine industry and some of its largest use sectors – power generation and marine applications – have come to the conclusion that this procedure is detrimental to the engines: it results in buildup of carbon and other materials on cylinder surfaces. And operation in a no-load condition does not truly exercise the engine and running the unit for only 15 minutes does not allow the engine and its lubricating oil to come up to recommended operating conditions (e.g., temperatures). Just running the engine-generator by itself is not a test of the complete emergency backup system. The installation’s load transfer switch also needs to be exercised for the “complete system” to be tested. 2. Facility, Equipment, & Fuel How Much Fuel to Store – In most all the meetings and interviews, the question came up about “…how much fuel should be stored at each emergency generator facility?” Local personnel have looked for information in numerous DHS, FEMA, and state publications, but found only general guidance. The general guidance given in federal and state emergency publications directed toward the public indicates that citizens and businesses should have the capability to survive in an emergency for 72 hours without outside assistance. The Montgomery County Fire Chief has instituted a policy that will require each fire station in the County to have 72 hours of fuel for the station’s emergency generator stored on site. This policy will take some time to implement as the stations do not presently have this capability. The extent to which this can represent a problem also is shown by a scatter plot of individual generator units in Arlington County, as shown by Figure V-1. The graph clearly indicates that all units with fuel storage capacity less than 2,000 gallons have less than the DHS/FEMA general guidance on-site storage capacity.
Fig V-1: Arlington County Fuel Tank Capacity vs. Maximum Running Time
35
0
10
20
30
40
50
60
70
80
90
0 500 1000 1500 2000 2500
Maximum Run Time (Hours)
Generator Tank Size (Gallons) [Excluding Tanks ≥ 2000 gallon capacity] Arlington County Emergency Generators Tank Size vs. Maximum Running Time
72 Hour Minimum Recommended Time
During the Project Workshop in July 2009, Mr. Robert Hoban, who is responsible for distributing emergency generators for FEMA Region III after a disaster, stated that it could take up to 72 hours to get an emergency declaration signed off, emergency response activities underway, and stored emergency generators distributed to where they are needed. Thus if outside assistance is needed (e.g., more generators) agreements (formal or informal) with other local agencies or private sources need to be developed and exercised. Fuel Tank Fuel Levels –Equipment maintenance and testing service personnel brought up the fact that a number of the emergency generator fuel tanks in the NCR do not have a fuel gauge or sight glass that allows a quick determination of remaining fuel in the tank. On numerous occasions when contract personnel arrived at a site for scheduled maintenance or testing, it was discovered that the unit’s fuel level is low. Stored Fuel Quantity & Quality – As indicated in Section III, the average emergency generator fuel tank capacity for units in the three jurisdictions studied is 879 gallons (Arlington), 396 gallons (Montgomery County), and 1,472 gallons (DC) gallons. Plans are in place in each of the jurisdictions to increase on-site fuel storage capacity at a number of emergency generator installation sites. As more fuel is stored at these sites, the issue of fuel quality and degradation needs to be addressed. A rough “rule of thumb” shared with Virginia Tech team members in a previous jurisdiction infrastructure project was: If the unit does not go through two tanks of fuel in a year (for regularly scheduled operation and testing), then additional fuel quality measures need to be considered.
36
Fuel Additives – The fuel distributors interviewed were asked about jurisdictions in the NCR requesting the addition of fuel additives for Diesel and gasoline tanks (e.g., biocides); they indicated that very few of their customers request any of these. The fuel distributors indicated they can add these and other products to the Diesel or gasoline fuels as part of their service, if requested by their customers. Personnel from the equipment maintenance and testing organizations raised a related issue: If the emergency generator unit ever runs out of fuel, then any organic growth, accumulated water, and/or solids that are in the tank will wind up in the engine and cause problems. Most engine installations have a 10 micron filter in the fuel line, but this will not keep everything in the bottom of a fuel tank out of the engine if the tank is run dry. One suggestion was that for tanks where the fuel is not cycled often, these tanks should be cleaned or that a fuel “polishing” application be conducted every two years. Jurisdiction-Owned Tankers – Not all jurisdictions and/or departments own their own fuel tankers that could provide fuel to individual generating units during an emergency. Some departments are dependent upon their fuel supply contractor to refuel individual units. In a situation where the fuel supply truck is delayed (e.g., roadways blocked in a storm or hurricane), critical emergency generating units might run out of fuel. One jurisdiction interviewed in a different Virginia Tech project had built a structure (it includes two fuel tanks and a gasoline-powered pump) which can be put on a flat-bed truck when needed and taken off after the emergency is over. That jurisdiction’s fuel supply contractor normally services the emergency generator fuel tanks. Fuel Testing – In general, fuel testing is not done either by customers or the distributors interviewed in the NCR project for a variety of reasons. First, it is expensive and second, the turn-around time to obtain the results is usually one to two weeks; however, both firms offer fuel testing services when requested by customers. When a delivery is made to the distributors, they receive a “certificate of content” from their supplier. When a fuel contamination question does arise at a jurisdiction’s unit, the fuel distributors or jurisdiction personnel can take a quick sample to determine if there is water in the fuel tank that might be causing problems. Ultra-Low Sulfur Diesel Fuel – In 2008, the Diesel fuel provided by the nation’s refineries was changed to Ultra-Low Sulfur (Sulfur content < 0.5 %) to comply with new EPA regulations. The fuel distributors were asked about any problems during the change-over or if any are lingering today. Both distributors indicated a few problems were experienced early on, but there are no problems today. Bio-Diesel Fuel – Some NCR jurisdictions are beginning to use biodiesel fuel in a few of their engines. The fuel distributors interviewed are beginning to supply this alternate fuel; however, they are finding the biodiesel fuel is not as “clean” as regular Diesel or gasoline fuels coming from the regional oil refineries. In Arlington County, biodiesel was used in the past in a number of units; however, these units developed fuel-related operating problems. The biodiesel fuel was removed, the tanks cleaned, and regular Diesel fuel is now being used again.
3. Refueling Contracts – Emergency Provisions The Project team asked specific questions, both on the Survey Questionnaire and in the Initial and Regional Strategy meetings, about emergency provisions in
37
jurisdiction/department fuel supply contracts. The provisions specifically asked about are listed below along with a summary of the answers obtained from the jurisdiction and personnel from the fuel distribution firms. a. Response Times: The fuel contracts for the three jurisdictions did not have any required response times specified. The fuel distributors indicated that a number of organizations in the region, including public agencies, have a response time of four (4) hours in their contracts. However, this implicitly means “best efforts” to respond because of possible road blockages, unsafe conditions, etc. during an emergency. b. County/District Service Priority: No fuel contracts contained a provision that specified a priority for any of the jurisdictions or departments. The fuel distributors indicated they “know their customers and their needs” and prioritize as the situation warrants. c. “Top-Off” Provisions: No fuel contracts contained a provision that would require a supplier to “top-off” large storage facilities or individual emergency generator units in the event of a predicted emergency (e.g., hurricane) unless called by the customer. The fuel distributors indicated they can and have provided this service, but it can be very expensive. d. Emergency Generators: None of the fuel supply contracts contained a requirement for the supply contractor to have emergency generator(s) installed at its storage/pumping facilities to enable the company to continue operation and distribution in the event of an electric service outage. One distributor indicated that the company would not bid on an RFP that contained such a provision.
4. Age and Condition of Units The Survey Questionnaire asked specific questions about the age of the unit/facility (year of purchase/installation) and Engine Operating Hours (Total and Estimated Annual). As shown in Section III (Figs III-1, III-3 and III-5), both Arlington and Montgomery Counties have a number of units that are in the 25 – 50 year age range. However, because of the usual low operation hours, both total and per year, for this type of emergency power application, these older units can have considerable operating life remaining.
Older Unit Maintenance – The maintenance and testing contractor personnel were asked about customers requesting changes in maintenance as a unit ages. In general, maintenance procedures are not changed as a unit gets older. Sometimes, an oil analysis will be conducted to help determine if any abnormal conditions exist. Arlington County has an annual oil analysis done for each unit, regardless of age, in order to obtain early warnings of any developing problems. Replace vs. Rebuild – One of the maintenance and testing contractors suggested that when a jurisdiction is considering replacing an older unit, that the cost of a major rebuild of the unit also be considered. The cost of the rebuild is much lower than that of purchasing a new unit. However, the contractor personnel also indicated that after 25+ years of operation, the installation’s electric wiring and switches, controls, and sensors would probably also need replacing so the difference in cost between rebuild vs. new purchase is not as large as it may first appear.
38
5. Administrative Support Jurisdiction Budget and Contract Changes – In this period of major budget reductions for the NCR’s jurisdictions, there have also been reductions in the maintenance and testing funds available for emergency generators. The maintenance and testing personnel contractors indicate that when this happens, they generally get a phone call from their jurisdiction contact informing them that “…the budget for the contract has been cut and no further activities are to be done or anything charged to the contract…” They indicated that they would rather be informed earlier in the budget-reduction process and to work with the jurisdiction personnel to ensure that its most critical units are serviced and that activities on the other units reduced or eliminated to reduce the jurisdiction’s expenditures. This would maintain the jurisdiction’s confidence in the capability of its most critical emergency generators. Leased Facilities – During a number of the interviews, it was learned that all the jurisdictions lease facilities to house various functions, personnel groups, or activities. When questioned, interviewees indicated that some of the jurisdiction’s critical facilities or groups were located in these leased buildings. However, they indicated that emergency power supplies for these critical groups was not part of the lease negotiations; since the jurisdiction did not own the facility, any emergency power would be what the building owner had already installed. Non-NCR Best Practice: Emergency Generator Coordinating Committee – During a previous project, the Virginia Tech team identified an Emergency Generator Coordinating Committee Best Practice while interviewing city emergency personnel at Newport News, Virginia. This approach was judged to be a “Best Practice” after having interviewed a large number of emergency service and public works personnel in the NCR, the Central Florida region, and two other areas within Virginia about how emergency generator activities were coordinated within their jurisdiction. A bulleted description of the Newport News committee is given below:
Fig V-2 Emergency Generator Coordinating Committee – Best Practice
Best Practice: Jurisdiction Emergency Generator Coordinating Committee
City of Newport News, Virginia ¾ Chair: Public Works Dept. Engineer – committee meets bimonthly ¾ All departments w/ emergency generator units represented, including schools ¾ Individual departments maintain responsibility & authority over units ¾ Committee reviews maintenance & testing agreements ¾ Committee reviews fuel service contracts & provisions ¾ Helps set “next-unit” priorities when funds become available ¾ Helps with technical support for departments (e.g., procurement)g ¾ Brings “visibility” to emergency generator value & requirements ¾ A Significant Benefit: 12+ Pairs of Eyes Looking for Funding!!!
E. Fuel Suppliers & Distributors Supplier & Distributor Capabilities – Questions were asked by the Virginia Tech team of the jurisdiction personnel regarding their knowledge about their fuel suppliers: size of company, number of delivery trucks/tankers, where they obtain their fuel, etc. In most all instances, the jurisdiction personnel had little knowledge of the distribution companies. This question was prompted by similar questions and responses in a previous Virginia Tech project, which involved interviewing jurisdictions and infrastructure personnel in Central Florida in early 2005 after three hurricanes struck the area in 45 days in the late summer of 2004. If a large electric outage occurred that resulted in a significant number of engines running at the same time in the NCR region, would the fuel distribution firm be capable of providing timely response to the jurisdiction’s needs AND provide service to all its regular customers? The general response was that the jurisdiction personnel didn’t know the answer to the question. A similar question was put to the fuel distribution firms. The two firms participating in this project are not large firms. One fuel distribution company is fairly small and indicated that during the Hurricane Isabel (2003) response and recovery period, their drivers were taking naps and sleeping on the office floors. Since the Department of Labor limits on the number of hours that commercial drivers can spend behind the wheel in each 24-hour period are lifted during a Presidential-declared emergency, the drivers push themselves and work longer hours; their reward is the overtime pay. The firm’s personnel indicated that they were able to supply their regular customer’s needs but, if a non-regular customer called asking for fuel delivery, they would be put at the end of the waiting list and some of these did not get served at all.
39
40
The other distribution firm is larger. Their personnel also worked very long hours during Hurricane Isabel and were also able to keep up with their regular customers’ needs and the demands of the emergency generators operating at the time.
Secondary Fuel Suppliers – A question was asked of both fuel distribution firms regarding their contracting with an organization to serve as a back-up fuel supplier. Only one organization had provided this service and they had done this only once. This same firm’s personnel indicated that, to their knowledge, this was not a common practice for public or private organizations in the NCR.
Site Limitations for Refueling – The fuel distribution personnel raised a safety issue that was present at a few of the permanently installed facilities in the NCR: below-grade installations. When units are installed at an elevation significantly below grade, there can be problems with over-pressure at the fuel tank in the beginning of the refueling process. At some sites in the NRC, two people are required to safely refuel the tanks because of this situation.
F. Equipment Maintenance and Testing Organizations Questions similar to those asked of and about the fuel distributors (Section E above) were asked of and about the two participating equipment maintenance and testing firms. Maintenance and Testing Firm Capabilities – Again, the jurisdiction personnel generally did not know the size of the maintenance and testing organizations (e.g., number of technicians); they generally saw the same contractor personnel at the equipment checks and tests. The companies’ capability to serve NCR jurisdictions in the event of a large and/or extended electric service outage was unknown To obtain similar information from the maintenance and testing firms, they were asked about their experience during Hurricane Isabel and any other situation when a large number of emergency generators were running. Again, their technicians and support personnel put in very long hours during the response and recovery periods. G. Regional Hospital & Medical Facilities During the meetings between jurisdiction personnel and Tech team members, the subject of emergency generators at hospitals and other medical-related facilities was always brought up. This was usually in the context of discussing emergency generators at critical facilities in the jurisdictions. Hospital Emergency Generator Capacity: – The NCR region has about 30 hospitals and it was learned that NONE have emergency generators with capacity to support more than their required critical facilities: operating rooms, critical care units, and some additional critical service systems. It was stated that only four (4) local hospitals have larger-than-minimum sized emergency generating units. The required hospital emergency generating units DO NOT have the capacity to ALSO run facility HVAC equipment. And dialysis facilities and local clinics – most all are privately owned – are not required to have emergency generators! During the discussions, it was stated that none of the three jurisdictions/departments could provide enough large, mobile/moveable generators to support the HVAC systems if there were a major emergency that included an electric service outage over a large area and extended period of time.
41
Electric Utility Service to Hospitals: – The interviews revealed that many of the DC regional hospitals are served by only ONE electric distribution feeder and in many locations the hospital shares that single feeder with other utility customers. If this is indeed the situation, this is an elevated risk situation, a “single point of failure” situation, for some of the NCR’s most critical facilities.
H. Designated Municipal & County Shelters Emergency Generators in Designated Shelters – In the three jurisdictions studied, designated emergency shelters are primarily high schools and a few recreational centers. There are emergency generators installed in most of the school facilities, but these are sized to support emergency systems that facilitate safe evacuation of the facility in an emergency. The units have an electric generator capacity sufficient to support certain critical systems: telephones, public address systems, emergency lighting, etc. The units, however, are not sized to support any significant emergency shelter requirements, such as the facility’s HVAC system – in total or in part. In the summer, without the HVAC system the facility would warm up within a very few hours and become extremely uncomfortable for evacuees and possibly unusable as a shelter. In the winter, without heat the situation would also become extremely uncomfortable for evacuees and unusable as a shelter.
Designated Shelter Quick-Connect Equipment – In response to questions, jurisdiction personnel did not know if those designated shelters without permanently installed emergency generators had “quick connect” equipment and properly modified electric service panels and circuits in place to allow timely connection of mobile/portable emergency generators. Most doubted this was the case.
Weather Shelters – In the District, some recreation centers and other public buildings are used for “cooling shelters” when summer temperatures exceed 95oF. However, none of these shelters have emergency generating units of sufficient size to continue running the building’s HVAC system in the event of an extended electric service outage. Without the HVAC system, the facility would warm up and within a very few hours become extremely uncomfortable and possibly unusable as a cooling shelter.
I. ID System for Fuel Distributor & Generator Maintenance Personnel In past disasters, problems have been documented where fuel trucks were unable to gain access to some of the hardest hit areas to refuel emergency generators at vital facilities. This was brought to the fore in New York during the September 11, 2001 event and its aftermath. In that situation, NO commercial fuel trucks were allowed into a wide area around “Ground Zero” because no previous arrangements had been authorized or even discussed among the organizations involved. After-action reports indicated one result was that a number of communications facilities that the rescuers were using failed because the emergency generators ran out of fuel. The Virginia Tech team documented a similar situation when Hurricane Charlie struck central Florida in August 2004. In some of the hardest-hit areas, neither the drivers nor the fuel trucks were immediately known to the police/traffic persons or jurisdiction EOC staffs. This situation resulted in delays in getting fuel to critical facilities in these areas.
42
In the NCR, there is no common, region-wide ID system nor does one exist in any jurisdiction to ensure positive identification of drivers and fuel trucks in a timely manner during an emergency. Fuel distributor personnel indicated the process used today if fuel trucks are stopped by police/traffic personnel, the truck driver will call 911; be transferred to the local EOC – police desk; explain the situation; the person on the police desk speaks with/radios the on-site officer and gives approval; and the fuel truck proceeds. In the past, this has worked reasonably well in this area without causing too much delay.
In late December 2008, FEMA issued preliminary NIMS Guidelines for credentialing state and local first responders: fire, police, medical, mutual assistance etc. and private sector workers who provide fuel and restore power, water, etc. Presently, Arlington County and the City of Alexandria are participating in pilot efforts of the DHS First Responder Authentication Credential (FRAC) program, which verifies the identities and attributes of emergency responders, allowing them to gain access in to and out of secured areas and across multi-jurisdictions. Arlington County has issued ID cards to County employees.
Some federal organizations take additional steps to ensure the fuel trucks get through with the least delay. One government agency (security/intelligence-related) provides letters to the fuel distribution company and their drivers to get them through. Another (intelligencerelated) provides a rider, with proper credentials, for each refueling truck to ensure the fuel gets through to the organization’s facility. And another government agency (intelligencerelated) has the fuel distributor park a loaded truck on the organization’s property, prior to or during an emergency, to ensure fuel is there when needed.
This same situation is present for emergency generator service and repair firms in the NCR. Again, maintenance personnel and their trucks may pass through a number of jurisdictions when going from their headquarters to the site where repairs are needed. And they may be stopped and a procedure similar to the one described above has to be gone through, with resulting delays at each stop.
VI. Findings & Conclusions
A. Introduction Project Survey Approach: The approach used in this pilot project to obtain survey data on emergency generators turned out to be very time consuming. The first contact with a jurisdiction was with its emergency manager. It was expected that the responsibility for support of the survey would be passed on through at least two other people; this was understood at the beginning of the project. However, in some instances the survey actually passed through a total of six people before the right person was identified. And at one jurisdiction, the information package was sent to the legal department, where it is still being reviewed. In a smaller pilot survey effort conducted in a city with a population of 50,000, the first contact was with an assistant city manager, then the department heads, and then to the specific department persons. However, as this survey was part of a much larger project and the project team members had met with the assistant city manager and the department heads a number of times prior to initiating the emergency generator survey effort, this transfer process went smoothly and occurred in a timely manner. Thus, if conducting a survey covering all NCR jurisdictions, the approach used in this project to get to the persons with direct responsibility for and information about the emergency generating units needs to be reviewed. Major jurisdiction activities occurring at the time the project data request was distributed – in the middle of the planning/budget process and during the planning for the inauguration of a new president – had a significant impact on the availability of personnel to support this project.
B. Better & Best Practices During the course of the meetings and interviews conducted as part of this project, a number of policies, practices, and procedures were identified as “better and best practices.” The term better and best practices used here is based on situations the Virginia Tech team members have found in other jurisdictions where the local personnel have identified significant benefits over time associated with the specific activities Montgomery County Fire & Rescue – It is the policy of the F&R department that whenever a F&R facility is scheduled for refurbishment or modification, facilities personnel conduct an evaluation of the new/revised design to determine if the existing emergency generator and associated equipment is adequate to serve the changed or increased loads. If not judged adequate, then new emergency generator equipment is procured or exchanged and all facility construction and equipment installation is done at one time. The Montgomery County Fire Chief has initiated a policy whereby each fire station in the county will have 72 hours of emergency generator fuel stored on site. As this is not the present situation, when major changes or upgrades are made to the facilities, the increased fuel storage capacity will be installed.
43
44
Montgomery County Public Schools – The School System has recently enacted a policy that for each type of school – Elementary, Middle, and High – the county has a specified size of emergency generator and specific loads connected to the emergency circuit. The complete implementation of this policy will take time to implement as the existing schools have different sized generator units and different loads served at each school. District of Columbia – Prior to the beginning of this project, the District of Columbia formed a 5-person task force, with a public works person as chair. The first task undertaken by the task force was to identify and survey emergency generators owned by the D. C. government. Arlington County Water Pollution Control Bureau – The Arlington County Water Pollution Control Bureau had the best data base on their emergency generators. They were able to provide the material requested in the Survey Questionnaire in a very short period of time, including photographs of buildings and equipment. All the information on their units is kept on an electronic database. In addition, as part of each emergency generator’s annual maintenance check, the contractor’s technician takes a sample of the engine oil and it is analyzed for a number of parameters.
C. Facility & Equipment Conditions Present Equipment Conditions – One question on the Survey asked about the present condition of specific units. The categories used were: new, excellent, OK, and poor, Information was provided only for Arlington County (all departments) and Montgomery County Fire/Rescue, and is discussed in Section III. Test & Starting Failures – The Survey Questionnaire also asked about any unit failures when testing over the last two years. This information also was provided only by Arlington County (all departments) and Montgomery County Fire/Rescue, and is summarized in Section II. Age-Related Maintenance Procedures – All jurisdictions have a number of emergency generator units that are 25+ years old, and two units are 45 – 50 years old! Yet no jurisdiction makes any changes in the maintenance procedures for these older units. Regular compression testing and oil analyses are two of the actions (e.g., at the annual maintenance service) that can be taken to maintain the confidence of operating and management personnel in the capability of these older units.
D. Coordinated Maintenance & Testing Contracts A jurisdiction’s maintenance and testing contracts for the different departments are usually handled and negotiated separately. This is always true for the jurisdiction’s school system. And each contract may be awarded to different firms. Therefore, although the “contract boilerplate” may be the same (they all go through the same purchasing department), the technical terms and requirements may vary among the contracts within the same jurisdiction. This can sometimes lead to paying a different price/rate for similar services from different companies.
E. FEMA/USACE/State-Supplied Generators No one interviewed in the participating jurisdictions remembers having borrowed emergency generators through a FEMA/USACE procedure, a State emergency management department request, or through the various regional or state mutual aid agreements during or after an emergency. And none of the emergency managers or staff had a reasonable estimate about how long it would take to obtain a unit through such requests; it was roughly estimated by one manager that it would take a few 10s of hours to get a unit to a site in his jurisdiction.
45
When asked about sizes and types of emergency generators that were available through these types of requests, the response was that the size needed was always part of the request. Only one emergency manager had knowledge of the specific types, size ranges, etc. of units that would be available through a FEMA/USACE request. During the July 2009 Emergency Generator Workshop, Mr. Robert Hoban of the FEMA Region III Logistics Office, and the person responsible for distributing emergency generators for FEMA Region III after disasters, indicated that getting an emergency generator to a requesting jurisdiction’s site is normally a three step process: Step 1 The jurisdiction’s request is processed through the state emergency management agency, sent to the federal government where it is forwarded to the appropriate FEMA region for action. Step 2 If FEMA logistics personnel do not have information about the facility to which the emergency generator is to be connected, then a person from the USACE or its contractor, is sent to the site to document the physical details and requirements of the facility; a USACE Site Assessment Input form is filled out and sent to the equipment distribution site. Step 3 The properly sized emergency generator, with needed connection equipment, is taken to the requesting jurisdiction and installed at the site.
It should be noted that if the FEMA Region Logistics office has a filled-out Site Assessment Input form on file, then Step 2 (above) is not needed and the response time to get a unit to a specific site is considerably shortened.
F. U.S. Army Corp. of Engineers’ Prime Power Project A number of years ago (~2004), the USACE’s Prime Power group at Fort Belvoir, Virginia participated in a “Prime Power Project” with NCR jurisdictions. The objectives of that effort included:
ƒ Assist NCR jurisdictions in prioritizing their most critical facilities ƒ Inspect the highest priority critical facilities in each jurisdiction and determine if any installed emergency generator had sufficient capacity to serve the connected emergency loads ƒ If no generator was installed at the facility, determine the modifications necessary to connect FEMA-supplied generators to the facility’s critical loads during an emergency. The effort was curtailed while the first three jurisdictions were being surveyed (DC, Fairfax County, and Montgomery County). The Army’s Prime Power personnel involved in the project were sent to Iraq and Afghanistan for duty. Questions about this USACE Project were asked at all the jurisdiction meetings. But, with one exception, no one remembered the effort or had any documentation from the project: Mr. Kerry Payne of the DC Emergency Management and Homeland Security Agency remembered the effort but had no project documentation. In discussions with emergency managers and others in each jurisdiction, it was considered this type of effort was still needed in the NCR. When asked, the interviewees believed their own jurisdiction’s most critical facilities had either emergency generators or quick connects installed,
46
but they didn’t know how far down their Critical Facility List this was true; the interviewees indicated they couldn’t confirm this situation without doing some investigation.
G. Regional Fuel Supply During interviews with the two fuel distributors, general issues related to the region’s fuel supply and sources were discussed. It was roughly estimated that the NCR could operate up to a week if its regular supply sources were interrupted by a large disaster. Depending on the product, the NCR is served by pipelines from the Gulf and also served through the Ports of Baltimore and Tidewater-Hampton Roads. Also, a number of fuel distribution firms are owned by larger regional companies and in the event of an emergency, fuel would be brought in by tanker trucks to serve customers in the NCR.
H. Additional Survey Information of Value ƒ The existing Survey Questionnaire asks questions related to loads served by the emergency generator. An additional valuable data point would be the total emergency electrical load, in kW or Amps, served by the emergency generator. ƒ To be of increased value to EOC staff, emergency management staff, and public works personnel responsible for getting emergency generators to sites in the event an in-place emergency generator fails during an emergency, it is suggested that an additional handdrawn sheet be added to the Survey Questionnaire. The sheet would show the following information: a) the site where the replacement unit would be placed in relation to the existing building, b) the location of the electric service panel within the building where the emergency electrical connection will be made, and c) the estimated length of the temporary electric power cable run from the generator location to the electric service panel. .
VII. Recommendations
A. Introduction As a result of the equipment survey and the meetings and interviews, the Virginia Tech team members have a number of recommendations that will increase the emergency generation capability and reliability in the NCR. These recommendations are divided into a number of categories and are listed below; some of the categories include more than one recommendation. For a few of the recommendations, the Virginia Tech team has included some suggestions for addressing them.
The Virginia Tech team recognizes the financial situation of not only the three jurisdictions that participated in the study but also the other NCR jurisdictions. This situation has resulted in a shortage of funds and personnel to implement any of these recommendations; the recommendations made are those where it is felt the jurisdiction’s security, reliability, and resilience will be increased. The majority of these recommendations relate to changes in emergency generator-related processes, procedures, or documentation by the participating jurisdictions. A few of the recommendations will require the investment of capital or additional operating funds.
B. Complete the NCR Emergency Generator Database 1. Recommendation: Each NCR jurisdiction conducts an all-department emergency generator survey, including the school systems, and develops a jurisdiction-wide summary of emergency generators for use by the jurisdiction’s emergency manager and the EOC staff. It is also suggested that each NCR jurisdiction updates the database annually. It is suggested that for each jurisdiction this would be an excellent project for a summer intern, such as a 2nd year student from a nearby community college enrolled in an Engineering Technology or a Pre-Engineering program. C. Coordinate Jurisdiction-Wide Emergency Generator Activities 2. Recommendation: – Each NCR jurisdiction create a city, county, or District emergency generator committee, including the local school system, to coordinate activities on a jurisdiction-wide basis, patterned after the Newport News, Virginia “Best Practice” organization (Section V.D.5).
D. Jurisdiction Equipment/Facility Design, Procurement, & Installation 3. Recommendation: Reducing no. of equipment suppliers – reduce spare parts 4. Recommendation: When designing new, permanently installed units/facilities, there should be increased recognition of operation, maintenance, testing, and refueling personnel requirements.
47
48
E. Equipment Operation, Maintenance, & Testing 5. Recommendation: Each jurisdiction modifies emergency generator installations and test procedures to allow testing of the Load Transfer Switches when regularly testing is conducted on permanently installed generator units. 6. Recommendation: Each jurisdiction inspects its emergency generator units and, where fuel tank gauges or fuel level sight glasses are not installed or operating, that new gauges are installed or inoperable gauges be repaired. 7. Recommendation: Each jurisdiction initiate an annual lubricating oil analysis for their emergency generator units that are 25 years of age or older. 8. Recommendation: Each jurisdiction ensures that an annual four-hour, full load test is conducted on each emergency generator units.
F. Fuel Contracts – Emergency Provisions 9. Recommendation – Each jurisdiction review its fuel supply contracts with respect to the feasibility, desirability, cost, and resulting increase in reliability and security of including one or more of the following emergency provisions: ƒ “Top Off” Provisions – when an emergency situation can be predicted ƒ “Time to Respond” – response time after notification ƒ Priority of Service – within the fuel supplier’s own priority list ƒ On-Site Emergency Generator – fuel supplier has on-site generation capability installed
G. Interactions with FEMA/U. S. Army Corps of Engineers 10. Recommendation – All NCR jurisdiction emergency managers participate in a joint meeting with FEMA Region III and USACE personnel regarding federal response to local requests for additional emergency generators in the event of an emergency. Information covered in such a meeting should include: 1) types of units available from FEMA, 2) federal agency response times, 3) specific size range of units available, and 4) type of advanced preparation at receiving facilities in the event of an emergency. One option for coordinating such a meeting could be through the MWCOG R-ESF #5 Task Force (Emergency Planning). 11. Recommendation: Each jurisdiction submit the information needed for a USACE Site Assessment Input form for each of their top (10, 25…TBD) critical facilities to FEMA Region III Logistics Office to reduce response time by USACE/contractor to supply needed emergency generators in the event of a Presidential Declared Disaster. (See Section V-E and Appendix H)
H. Jurisdictions’ Most Critical Facilities 12. Recommendation: Each jurisdiction evaluate its highest (10, 20 …TBD) priority critical facilities and for those without a permanent emergency generator installed, the jurisdiction determine what building electric service entrance modifications must be made to accept portable or mobile generators and that any needed service entrance modifications and quick connect installation be designed and added to the jurisdiction’s mitigation plan and capital improvement budget and schedule. For those facilities with permanently installed
49
emergency generators, evaluate the loads connected to the emergency electric bus to determine if the emergency generator has adequate capacity to serve the emergency loads. This recommendation could be developed into a single project for all NCR jurisdictions and submitted to the annual DHS UASI program’s next procurement cycle for funding. 13. Recommendation: Each jurisdiction evaluate its highest (10, 20 …TBD) priority critical facilities and evaluate the costs, benefits, and increased security obtained by increasing on-site fuel storage to support 72 hours of operation without refueling.
I. Regional Strategies to Obtain Additional Units during Emergencies 14. Recommendation – Each jurisdiction develops a time-sensitive plan for obtaining additional emergency generators in the event of a large-impact emergency event where electric service may be interrupted for a considerable time (See Section IV-C and Table IV-1) and consider the additional option of pre-negotiating agreements with local/regional equipment suppliers and/or rental firms for quick-access to units. One option is to consider this recommendation on an NCR-wide basis. Negotiations could be conducted on behalf of all the NCR jurisdictions which could result in a costsavings when compared to a number of jurisdictions going out individually seeking this service. If this recommendation is considered for implementation, then it would be beneficial for individual jurisdictions to have conducted an emergency generator survey (Recommendation No. 1, above) so each jurisdiction is knowledgeable about its own emergency generator capabilities and limitations.
J. Emergency Generator Units at Medical Facilities 15. Recommendation: Each jurisdiction reviews the on-site emergency generation capability of the hospitals and major medical facilities within its borders; documents internal systems and critical services connected to the emergency circuits; and documents what additional emergency generation capabilities might be required in the event of a long-term electric power outage to maintain patients in the facility. Such a review could be conducted with the facility’s annual fire department inspection. 16. Recommendation: Each jurisdiction reviews the incoming electric service at each of the hospitals and major medical facilities in its jurisdiction to determine emergency operating capabilities and limitations (e.g., single electric service feeder, dual service on same pole line, etc.). It is suggested that these evaluations be conducted by a team with members from the utility serving the facility (Pepco, Dominion Virginia Power, etc.), the respective jurisdiction’s emergency management and public works department, and the hospital’s facility and security departments. Again, such a review could be coordinated with the facility’s annual fire department inspection 17. Recommendation: Conduct an evaluation with local equipment dealers and regional equipment rental firms to determine the cost and availability of larger emergency generators on a quick-delivery basis. This would provide emergency back up units for use at hospitals and other medical facilities in the event of a large emergency situation in the
50
NCR that includes a widespread electric service outage. This would significantly reduce the possibility of having to evacuate these large, patient-filled facilities.
K. District & County Emergency Shelters 18. Recommendation: Each jurisdiction reviews their primary emergency and weather shelters to determine capability of existing emergency generator to serve loads other than emergency circuits, such as the HVAC system, and documents additional emergency generation capabilities required in the event of a long-term electric power outage to maintain use of the facility. Such a review could be conducted with the facility’s annual fire department inspection.
VIII. Appendices
A. Prior Virginia Tech Emergency Generator-Related Projects B. Emergency Generator Project Information Package C. Emergency Generator Survey Questionnaire D. Fuel Supplier & Distributor Questions E. Equipment Maintenance & Testing Organization Questions F. Jurisdiction Regional Strategy Questions G. Project Workshop – Agenda & Attendees H. U. S. Army Corp of Engineers – Site Assessment Input Forms
51
52
Appendix A
Prior Virginia Tech Emergency Generator-Related Projects
Appendix B
53
“Critical Infrastructure Resiliency: Increasing Emergency Generation Reliability and Capability in the NCR” (CIPAUAS6)
Project Description
The Problem Government agencies, private firms, and private citizens have purchased emergency generators to provide power to critical facilities and activities in the event that normal electric service is interrupted, whether by natural events, technical failures, or human intervention. However, most emergency generators historically have not operated properly during emergencies. When called upon, a large number don’t start, and many of those that do start do not operate for as much as one hour. The reasons can be traced to a variety of factors: inadequate design, improper installation, or poor installation quality; lack of training and/or experience of facilities personnel; inadequate maintenance and regular testing of the units; insufficient on-site fuel storage for extended operation; and lack of adequate refueling provisions. Background Hurricane Isabel, which struck the National Capital Region (NCR) in September 2003, dramatically highlighted the problems related to poor operation of emergency generation facilities and equipment. More recently, the four hurricanes that struck Florida in a period of 45 days in August and September of 2004 and the three hurricanes that struck the Gulf Coast in 2005 again demonstrated the major impacts of loss of the electric utility service and the inadequate operation of emergency generators on the health, safety, and recovery of an area. Review of numerous “after-action,” “lessons learned” and “performance assessment” reports by government agencies, private firms, and consultants almost always identifies the inadequate performance of emergency generators as a significant factor in reduced capability of organizations during an emergency and subsequent recovery period. The Project The overall project goal is to help public jurisdictions and private firms increase the reliability and security of emergency generation facilities in the NCR. The project is funded by the U. S. Department of Homeland Security’s Office of National Capital Region Coordination – Urban Area Security Initiative (UASI) program. This project is supported and guided by the NCR Regional Infrastructure Working Group, chaired by Mr. Mike McAllister of the Virginia Governor’s Office of Commonwealth Preparedness. The approach to be used will initially identify emergency generation equipment already installed in selected organizations, both public and private in three NCR jurisdictions. The project team will document the capabilities of a number of facilities/units, as well as present practices for operation, maintenance, and testing. Primary emphasis will be on emergency generators larger than 5 kW. The project team will examine contracts for refueling services and also interview a number of equipment vendors and fuel distribution companies in the region. The results of the project will be disseminated at a workshop where the information obtained and “better practices” found will be presented to a wide audience of project participants, NCR facilities managers, and Metropolitan Washington Council of Governments’ (MWCOG) R-ESF #5 and R-ESF #12 committee members.
Appendix B
54
A brief description of the five project tasks is provided below: Task 1: Emergency Generator Survey – The project team will conduct an emergency generator equipment/facility survey in three public jurisdictions and one private sector in the NCR: Arlington County, Va.; Montgomery County Md.; the District of Columbia; and private cell phone service suppliers serving Arlington County. Task 2: In-Depth Facility Examination – For selected facilities in each public jurisdiction and the private sector, the team will conduct an in-depth, on-site examination of the facilities, including monitoring of normally scheduled equipment/facility testing. Task 3: Issues, Conclusions, & Recommendations – The project team will analyze the results of the survey, the on-site demonstrations, and personnel interviews to: (1) identify issues presently impacting the facility/equipment performance and (2) make recommendations that will increase overall performance, reliability, and security. Task 4: Project Results Workshop – The project team will develop and conduct a “results & findings” workshop for participating organizations, other NCR facilities personnel, and MWCOG R-ESF #5 and R-ESF #12 committee members Task 5: Documentation & Reporting – A project final report and workshop notebook will be produced; the validated survey methodology will be included in the project final report. A facility/equipment characteristic database will be produced for each participating jurisdiction and organization of its own facilities/equipment.
Project Benefits & Deliverables ¾ Documentation of recommended practices and procedures to increase emergency generator performance, reliability, and security in the NCR ¾ Database of characteristics for larger, permanently installed and portable emergency generators at selected departments in three NCR jurisdictions and cell phone service providers in Arlington County (For participating organizations only). ¾ Workshop for NCR jurisdiction and private sector personnel on “better practices” in emergency generator operation, maintenance, and testing to enhance performance, capability, and reliability of this equipment ¾ Validated survey methodology that will enable other jurisdictions and organizations to conduct their own emergency generator survey and assessment Contacts ¾ John E. Bigger Advanced Research Institute 4300 Wilson Blvd. Suite 750 Arlington, VA 22203 Ph: (703) 387-6032 E-Mail: jobigger@vt.edu
Courtesy of Generac Power Systems, Inc. ©2006
¾ Dr. Michael Willingham Advanced Research Institute 4300 Wilson Blvd. Suite 750 Arlington, VA 22203 Ph: (703) 387-6036 E-Mail: mwilling@vt.edu
Appendix B
55

Critical Infrastructure Resiliency: Increasing Emergency Generation Reliability and Capability in the NCR (CIPAUAS6)
Project Data Security
Introduction This project is designed to help public jurisdictions and private firms in the National Capital Region (NCR) increase the reliability and security of emergency generation facilities. The project is funded by the U.S. Department of Homeland Security (DHS), and is supported and guided by the NCR Regional Infrastructure Committee, chaired by Mr. Michael McAllister of the Virginia Governor’s Office of Commonwealth Preparedness. Data Sensitivity Information to be requested from participants to successfully conduct the project is considered sensitive from both security and proprietary perspectives. Specific safeguards and legal procedures for information handling, protection, and ultimate disposition are being implemented to insure the data is protected. First, the data and information collected have certain protection from the Commonwealth of Virginia: they are exempt from disclosure under the Freedom of Information Act (FOIA)6. Second, the only persons collecting, handling, and analyzing the data are the three team members, who are Commonwealth of Virginia employees. Third, for another DHS project, the team has been certified and procedures put in place under the joint Federal/Commonwealth “Protected Critical Infrastructure Information” (PCII) Program. The project team will handle all data and information in accordance with the PCII procedures, although this project is not subject to that program. Finally, information satisfying the requirements of the Critical Infrastructure Information Act of 2002 is protected from both FOIA and State and local disclosure laws, and also may not be used in civil litigation. Data Collection, Handling, and Storage ƒ Collected by Project Team members only; expected in both electronic & paper formats ƒ Secure Computer – Isolated installation, password protection, installed in locked room ƒ Physical Storage – Printed documents kept in locked room and locked cabinets ƒ Room keys and passwords issued only to Project Team members ƒ Project data analyses conducted only on Project’s Secure Computer by Project Team members
6ref: Code of Virginia § 2.2-3705.2 (3), (4), (6)
Appendix B
56
Data Dissemination ƒ ESF #5,12 Briefings ¾ No jurisdiction or communications company raw data included in ESF briefings presentations ¾ Data used in ESF briefings will be summarized and masked to prevent attendees from tracing workshop information back to a specific installation/equipment ƒ Project Final Workshop ¾ No jurisdiction or communications company raw data included in workshop materials & presentations ¾ Data used in workshop will be summarized and masked to prevent attendees from tracing workshop information back to a specific installation/equipment ƒ Project Final Report ¾ No jurisdiction or communications company raw data included in final report ¾ Data used in final report will be summarized and masked to prevent any reader from tracking report information back to a specific installation/equipment ¾ Project Final Report – one electronic and one paper copy submitted to Project Technical Manager, Mr. Mike McAllister. ƒ Project Completion: ¾ One electronic copy of each jurisdiction’s facility/equipment data provided only to that jurisdiction’s Emergency Manager or designee ¾ One electronic copy of each communications company’s facility/equipment data provided only to that company’s designee ¾ Project raw data deleted from Project’s Secure Computer
* * * * *
Appendix C
57
National Capital Region Form Emergency Generator Survey
(Please use one questionnaire for each engine-generator unit)
Facility Owner
Organization: Dept./Div./System: Location: Address: Zip Code: Building Name: GIS: (If avail.) Lat. Long. Contact: Name: Telephone: Facility Characteristics
Unit No. Mfg. No. Jurisdiction No.* (Engine only) * If Applicable Manufacturer Engine: Generator: Model No. Engine: Generator: Nameplate Cap. kW Hp: Type Permanent Installation: Portable Unit: Fuel Primary: Secondary: Fuel Storage Gal. Unit Run Time Hrs. (On full fuel tank) Age Yr. Mfg’d: Years: Load Transfer Switch: Manual Auto. General Equipment Condition –
Appendix C
58
Loads Served
Please check all that apply. Emergency Exit Lighting: Critical Process Supply/Control: Interior Emergency Lighting: Security Systems: Int. Communications: PA Systems: Facility Air Conditioning: Ext. Communications: Phones/Chargers: Door Locks/Gate Openers: Life-Support Supply/Control:
Fuel Supplier
Supplier Name: Address: Telephone: Contact: Fuel Contract Refuel Provisions: Top-Off Provisions: w/in Hrs. Facility Priority at Supplier:
Operating History
Documentation – As-Built & One-Line Drawings: – Users Manual: – Maintenance Manual: – Testing Manual; – Site Log:
O&M History – Total Operating Hrs: Ave. Ann. Hrs. – Major Rebuilds/Overhauls/Repairs; – Scheduled Maintenance:
Appendix C
59
Testing History – Scheduled Testing (Start & Run w/ no or partial load): Successes: Last 2 Yrs Failures: Last 2 Yrs – Full-Load Testing Successes: Last 2 Yrs Failures: Last 2 Yrs – Scheduled Duration Testing (>1 Hr) Successes: Last 2 Yrs Failures: Last 2 Yrs
Contract Operation, Maintenance, & Test Support – Company Name: – Address: – Telephone: – Contact:
Google Earth Overhead Photo (Supplied by Project Staff)
Equipment Photo (Supplied by jurisdiction – if available)
Please mail, fax, or e-mail filled-out form(s) to either one of the following: John E. Bigger Dr. Michael Willingham Adjunct Professor Adjunct Professor Advanced Research Institute Advanced Research Institute 4300 Wilson Blvd, Suite 750 4300 Wilson Blvd, Suite 750 Arlington, VA 22203 Arlington, VA 22203 Ph: 703 387-6032 Ph: 703 387-6036 Fax: 703 528-5543 Fax: 703 528-5543 E-Mail: jobigger@vt.edu E-Mail: mwilling@vt.edu * * * * *
Appendix D
60
Critical Infrastructure Resiliency: Increasing Emergency Generation Reliability and Capability in the NCR (CIPAUAS6)
Questions: Fuel Distribution Companies
Fuel Distribution to Jurisdictions ƒ Delivery to major fuel facilities ƒ Delivery to individual installations
Fuel Testing ƒ Do any jurisdictions do their own fuel testing ƒ Is fuel testing offered/required ƒ Sample/test as delivered ƒ Offer Fuel Stabilizers &/or Microbiocides ƒ Offer fuel processing/polishing
Emergency Contract Provisions ƒ Does company offer emergency provisions in their “standard” contract ƒ Emergency provisions required by jurisdictions ƒ Response times after being called ƒ Any automatic “Top-Off’ provisions when an emergency is pending (e.g., hurricane ƒ Customer priorities (e.g., public agencies, emergency facilities) ƒ How does your company set priorities among customer calls ƒ Do you offer priority delivery service for a fee
Company Fuel Distribution Facility ƒ How are tankers filled if your facility looses electric power ƒ Do you have emergency generators on site
Appendix D
61
ƒ – At what capacity can you operate with only emergency generators ƒ Do any jurisdictions have contract provisions that require you to have on-site emergency generators in order to win contract ƒ Primary source of gasoline or diesel fuel
– Regional port (e.g., Baltimore or Norfolk) – Regional refinery – Petroleum pipeline – Tanker truck delivery ƒ How easy is it to get additional fuel during emergencies when many emergency generators are running – Primary supply – Any alternate supply
General Experience during Hurricane Isabel ƒ Any problems with fuel supplies to company
ƒ Any problems with delivery to customers ƒ Do company personnel or delivery drivers contact jurisdiction Emergency Operating
Centers or local police when delivery is impeded (e.g., roadways blocked)
Customer Fuel Use
ƒ Any customer problems with Ultra-Low Sulfur diesel fuel
Appendix E
62
Critical Infrastructure Resiliency: Increasing Emergency Generation Reliability and Capability in the NCR (CIPAUAS6)
Questions: Emergency Generator Supply, Maintenance, & Testing Companies
Unit/Facility Testing
Testing Frequency Customer Specified On-Site or Remotely Controlled Tests: Weekly and Quarterly are usual Test Frequency vs. Unit Size Test Frequency vs. Type of Facility Test Frequency vs. Age of Unit Regularly Scheduled Testing Unit Run Time w/o load, Unit Run Time w/ Load Available Test Load Transfer Equipment Pull Parallel Switch and Time Unit Start & Load Pickup Annual Testing Unit Run Time Run w/ Load Available Run w/ Full Load – Bring in resistive load Test Load-Transfer Equipment Pull Parallel Switch and Time Unit Start & Load Pickup
Appendix E
Maintenance Procedures Weekly/Monthly/Quarterly Checks Remotely Monitored Units Local Timer Installed Start & Run Units Annual/Scheduled Maintenance Maintenance vs. Age Procedures Measure Output Voltage & Frequency Measure Power Output Analysis of Used Oil Fuel Analysis Sound Test/Analysis Vibration Test/Analysis Infra-Red Check of Electric Terminals & Connections Contract Maintenance Response Time Customer Priority On-Site Generator(s) for Business Continuity Unit Suppliers & Integrators Unit Supply during Emergencies
Does Company Offer O&M Training? At Time of Purchase
1
Regularly Scheduled Throughout Year
Appendix F
Critical Infrastructure Resiliency: Increasing Emergency Generation Reliability and Capability in the NCR (CIPAUAS6)
Questions: Jurisdiction & Regional Emergency Generator Strategy
Historic Experience w/ Power Outages ƒ What is general history of power outages: frequency, magnitude, length of outages, etc.? o Outages in general o Outages where emergency generators were required o How did emergency generators function during these outages? o Has County/District ever asked for any service reliability data in general/at critical facilities from PEPCO/Dominion? ƒ Did units just support County/District requirements or can they serve others – private, non-profit, etc.? ƒ Were enough generators available? What is size range of existing fleet? ƒ Have there been incidents where generator availability made a significant difference in the emergency situation? ƒ Critical facilities, such as hospitals (and nursing homes in Maryland) are required to have emergency generators and test periodically. o Do County/District personnel monitor these tests? o Have any of these units failed and County/District units been needed for backup?
Present Approach for Borrowing/Lending Units ƒ Does the County/District have sufficient number of units on hand? Are there incidents where additional units were needed by County/District? ƒ Is the number/capacity of units (permanent, mobile, portable) known for County/District departments and jurisdiction total? ƒ Any borrowed/bought during Hurricane Isabel? ƒ Search locally – jurisdictions: cities, counties, District?
1
Appendix F
ƒ Are there any pre-conditions on borrowing/lending units? o What equipment/documents accompany units? o Who repairs unit when there is a failure? ƒ Does the County/District have uniform testing and maintenance procedures? ƒ Who approves borrowing/lending units: department to department and jurisdiction-to-jurisdiction? ƒ Are there any priorities established for borrowing units?
Intra-Jurisdiction Borrowing/Lending ƒ Can mobile &/or portable units (Size > 5 kW) be easily borrowed from department to department? ƒ Are there historic records of units used/needed during past events or emergencies? ƒ Are there any examples of past borrowings? ƒ What is supplied with the unit? o Operator/installer o Materials for interconnection – quick connects, power cables, service bus connections. o Operating manuals o Responsibility for refueling
Inter-Jurisdictional Borrowing ƒ Mobile &/or portable units (Size > 5 kW) ƒ Historic records of units borrowed during past events or emergencies ƒ Examples of past borrowings ƒ Are there any formal agreements w/ other jurisdictions? ƒ What is supplied with the unit? o Operator/installer o Equipment for interconnection – quick connects, power cables, service bus o Operating manual o Responsibility for refueling, maintenance, and repair ƒ Any discussions regarding establishing jurisdiction/NCR inventory of emergency generator units and formal borrowing agreements
2
ƒ Any discussions w/ other jurisdictions or COG ESF-#5, #12 Committees, etc. to join together and pre-negotiate contracts with local dealers/suppliers for additional units during emergencies?
Appendix F
ƒ What would you see as necessary requirements in any generator borrowing agreements? ƒ Would you need a minimum level of unit maintenance and testing?
Federal- & State-Jurisdiction Borrowing ƒ Has County/District borrowed units from State organizations? o Any experience borrowing from Department of Emergency Management or D.C./Maryland/Virginia National Guard ƒ Has County/District borrowed units from FEMA – Region III? ƒ Has County/District borrowed units from Corp. of Engineers/U. S. Army – Ft. Belvoir? o What has been the County’s/District’s experience in the past with these borrowed units?
Industry-Jurisdiction
Support for a Local Private or Non-Profit Facility ƒ Can/have you supply units to private companies or non-profit organizations? ƒ Can/have you support vulnerable population facilities, (e.g., nursing homes)?
Local Generator Package Suppliers ƒ Has County/District had discussions w/ local firms regarding general inventory & capability to supply? ƒ Have there been any informal agreements or pre-negotiated contracts with local dealers/suppliers for units during emergencies? ƒ Rent, lease, or purchase?
Generator Package Manufacturers ƒ Have there been any discussions regarding manufacturers’ general inventory & capability to supply? ƒ Have there been any agreements or pre-negotiated contracts with manufacturers for quick delivery of units during emergencies? ƒ Are there any pre-negotiated rent, lease, or purchase agreements in place? ƒ Discussions w/ other jurisdictions, COG ESF-#5, #12 Committees, etc. to join together and pre-negotiate contracts with regional dealers/suppliers for units during emergencies?
3
ƒ In Florida, a few generator manufacturers/packagers pre-position units in area during the hurricane season when impacts on land are predicted.
Appendix F
Generator Package Users/Dealers ƒ Have there been any interactions with local power utilities, construction companies, or builders regarding borrowing units during emergencies? ƒ Have there been any interactions with County/regional/state and construction or building associations regarding support during an emergency situation? ƒ Have there been any discussions with retail organizations regarding support during an emergency situation: Home Depot, Lowe’s, etc.
County/District Mitigation Program ƒ Does the County’s/District’s (TBD) most critical facilities all have installed emergency generators? o Do the ones that don’t have generators installed all have installed “quick connects”? ƒ What was the outcome of the U. S. Army Corp. of Engineers’ “Prime Power Project” that worked with NCR jurisdictions about four years ago to review emergency generation needs for a jurisdiction’s most critical facilities? ƒ Has the electric service panel at each critical facility been modified for emergency service support? ƒ Has jurisdiction submitted proposal(s) to the DHS Mitigation Program for funding to modify electric service panels and install “quick connect hardware” at its most critical facilities that do not have an emergency generator installed? ƒ Should there be a regional list/map of critical facilities and their emergency generator needs in case of longer term power outages? o This would allow for the estimation of potential demand, allocation of currently available resources, and needs for formal requests to state or federal sources.
County/District Refueling Activities ƒ Does jurisdiction have its own tanker(s) for refueling generators when outages are long duration – hours to days? ƒ Does County/District depend on jurisdiction employees or private distributors for refueling? o Are there provisions in the supply/service contracts for a) response times, b) priorities among other customers, c) “top-off” when natural disasters are predicted?
4
ƒ Does the County’s/District’s refueling contract require the distributor to have emergency generation installed on-site to maintain service capability during an electric service outage/emergency?
Appendix F
5
ƒ Does the County/District Emergency Operations Center staff have information on fuel delivery vehicles & procedures during emergencies or disasters? o Such as the companies involved tanker and truck descriptions, drivers, destinations, and routes o Have ID documents been issued to allow private fuel-delivery company drivers/vehicles entry to “restricted” areas in County/District to make deliveries?
* * * * *
Appendix G
Critical Infrastructure Resiliency: Increasing Emergency Generation Reliability and Capability in the NCR (CIPAUAS6)
Project Workshop: Agenda
July 7, 2009 Advanced Research Institute Virginia Polytechnic Institute & State University Arlington, Virginia
Workshop Objectives ƒ Present preliminary survey findings & project recommendations ƒ Obtain feedback from project participants Workshop Attendees ƒ Arlington and Montgomery County and District of Columbia participants ƒ Virginia Governor’s Office of Commonwealth Preparedness ƒ Regional fuel distributors and maintenance & testing service companies ƒ Project Team Members
09:30 – 09:40 Opening Remarks Dr. Fred Krimgold Mike McAllister (Invited) Virginia Governor’s Office of Commonwealth Preparedness
Workshop Participant Introductions
09:40 – 10:00 Project Overview & Objectives Dr. Fred Krimgold
10:00 – 10:25 Summary of Survey Findings & Comparable Findings w/ Q&A Dr. Michael Willingham
10:25 – 10:50 Unit Maintenance & Test Findings w/ Q&A John Bigger
10:50 – 11:15 Fuel Delivery & Contracting Practices w/ Q&A
1
John Bigger
Appendix G
2
11:15 – 11:40 Equipment Maintenance, & Test Practices w/ Q&A John Bigger
11:40 – 12:00 Regional Strategies & Project Recommendations w/ Q&A John Bigger
12:00 – 12:30 Workshop Discussion & Preliminary Recommendations FK, MW, & JB
12:30 Workshop Adjournment
* * * * *
Appendix H
1
EMERGENCY GENERATOR CRITICAL FACILITY SITE SURVEY CRITICAL FACILITY PRIORITY: LIFE SAVING LIFE SUSTAINING INFRASTRUCTURE FACILITY USE / PURPOSE: Emergency Response Crew lodging facilities – Agency Dispatch location – Agency Radio Base Station NAME OF FACILITY: Virginia Department of Forestry – Salem Regional Office COUNTY : Roanoke SITE NUMBER: 3 MUNICIPALITY: Salem PRIMARY POC: John Miller PHONE: 434-977-6555 ADDRESS: 210 Riverland Drive
CITY: Salem STATE: VA FL ZIP: 24153 LATITUDE: 37 16’ 30” LONGITUDE: 80 02’ 30” FACILITY POC: Ed Zimmer PHONE: 804-834-2300 NAME OF POWER COMPANY: Dominion Va Power FACILITY PEAK LOAD: unknown UTILITY CONNECTION ABOVE OR BELOW GROUND: Above Ground TOTAL AMP DRAW: 200 AMP TRANSFORMER TYPE: 25KV MAX VOLTAGE: 250 SERVICE DROP TYPE: Overhead Underground Transformer MOUNT TYPE: Pad Pole
GENERATOR PLACEMENT SITE OBSTRUCTIONS: (Gate, fence, plants etc.)
none EMERGENCY GENERATOR CONNECTION POINT:
Side of office
GENERATOR KW RATING REQUIRED AT 75% LOAD:
60KW SIDE OF FACILITY:
right
Appendix H
DISTANCE FROM TRAILERED GENERATOR TO CONNECTION POINT:
50’ METHOD OF CONNECTION TO FACILITY:
Double Throw Transfer Switch
LENGETH OF CABLE REQUIRED TO COMPLETE CONNECTION:
50’ DOES FACILITY HAVE AN ATS OR MTS?
ATS MTS NONE
ABILITY TO OFFLOAD GENERATOR: TYPE EQUIPMENT:
YES NO Fork Lift
STAFF AVAILABLE TO MAINTAIN GENERATOR:
Yes
ON-SITE REFUELING CAPABILITY:
No DATE OF SURVEY
11/28/07 BY Wayne Huskey
COMMENTS:
2
Appendix H
SITE
PLAN

3
VDOF Mechanics Shop
VDOF Office
VDOF Shed
River Road
Apperson Drive
Ban
Restauran t
Generator
Appendix H
4

1.
Mission Number:
1a.
Priority: 1 LIFE SAVING 2 LIFE SUSTAINING
2. Facility Name and Building use:
3 OTHER 3. Address: 4. City: 5. State: 6. County: 6a. Latitude: 6b. Longitude: 7. Site_POC:
8. Phone: 9. Alt_Phone: 10. Fax: 11. Date and Time Tasking passed to 249th:
12. Team member names:
13. KW: (assessed Gen. size)
14. Phase: (assessed Gen. Phases)
15. Voltage: assessed Gen. Voltage)
16. Configuration: Y ______ D ______ 17. Load Cable Size: MCM ______ #awg______ 18. Qty/UI: (ground wire length)
19. Ground Cable Size: MCM ______ #awg______ 20. Qty/UI: (ground wire length)
21. Remarks: (include ground rod and additional electrical supplies needed for installation)
22. Date and time assessment is physically completed:
23 Date and time tasking received by mission manager:
PLACE THIS ORDER OR A SIMILAR ORDER WITH US TODAY AND GET AN AMAZING DISCOUNT