Corps of oil, Part 1
Part 2, Part 3
The problem
A few years back, I was tracking oil spills occurring at the Corps' three lakefront gate-and-pump conglomerations (17th Street, Orleans Avenue, and London Avenue). I stopped counting at about 800 gallons with my April 18, 2007 post, "11 spills and counting."
Since then, through review of documents obtained through federal and state records requests, I've determined that the oil spill problems at the gates run far deeper than those 11 spills. Thus far, I've documented a total of 40 incidents over the past four years, 29 of which appear to have not been reported to the federal authorities as required by federal law. There appears to have been similar laxity in state-level reporting as well. The total quantity of spills is difficult to determine. From reports that were made, it is definitely 2000-3000 gallons of oil, but I believe adding in the unreported spills brings the total into the range of 5000-6000 gallons, most of it hydraulic fluid coming out of the pumps.
I have repeatedly asked the Corps New Orleans District for copies of whatever spill reports they might have, even sharing my findings with them along the way. But in response to my inquiries and a formal FOIA request, they only turned over two reports, both of which were already available on the state of Louisiana's website.
In light of recent news of what is going on in the Gulf of Mexico, I feel it is particularly timely that this information get out. On a more local basis, this constant spilling of oil is another indication of the consequences of poor decisions made by Corps Engineering and Operations personnel in dealing with the flood protection works at the gates over the past four years. In addition, it is unclear if all the oil was even cleaned up. In recent years the Corps has contracted to clean up oil spills, but early on, it is impossible to determine if that was happening on a regular basis. Their apparent failure to report many of the spills certainly does not inspire confidence that they were cleaning them up.
I've divided the lengthy spill history into three segements:
a) Installation, testing and repairs - May, 2006 through June, 2007
b) Acceptance testing - July, 2007 through September, 2007
c) Post-acceptance testing and corrosion repairs - October, 2007 through present day
In the first two periods, we know of oil spills through Corps testing records. Some spills occurred as water was being pumped from the canals during tests. In fact, some records actually note hydraulic oil being poured into engine reservoirs as pumps were running! Other spills were discovered when those reservoirs were found to be partially full or nearly empty at the start of testing on a given day. Occasionally sheens were observed as well. Reporting to the authorities was scattershot, with contractors calling in the spills to the Coast Guard or the state as much as the responsible party for the sites - the Corps. Some very large spills, in which the reservoirs on drive skids were emptied of their 300 gallons of hydraulic oil, do not appear to have ever been called in.
Spill response also appears to have been similarly hit-or-miss in that early period. Unlike now - when the danger of spills due to the poor design, redesign and maintenance of the pumps is well known, and solid as well as absorbent oil boom is permanently deployed (details on current booms here) - in 2006 and 2007 the response scheme was ad hoc. Only absorbent boom was being deployed, and then only when pumps were turned on. At least two separate oil spill response companies are mentioned in the testing records, and it appears they were not under contract to the responsible party - the Corps. This "catch as catch can" response (along with many other chaotic activities happening at the gates in 2006 and 2007, including lack of reporting) make me wonder if the Corps was meeting the federal requirement for possession of a Spill Prevention Control and Countermeasure plan at each site. They have them currently, but those versions are dated September, 2008:
17th Street SPCC plan
London Avenue SPCC plan
Orleans Avenue SPCC plan
The situation did not improve when the pumps were turned over to the Operations Division. In some ways it got worse. In nearly three years of stewardship over the pumps, the Operations Division had - until this month - not reported a single spill to the federal authorities. By my count there were 18 such spills at the sites, some of them releases of hundreds of gallons. One spill dropped between 1000 and 2000 gallons of diesel. More troubling perhaps is that for these later spills, there is a money trail to follow. The Corps formalized spill response when they transferred the pumps and gates to Operations, signing successive contracts with a Baton Rouge firm, Quaternary Resource Investigations, or QRI. According to contract documents, the Corps has paid QRI over $93,000 in task orders to clean up 9 of the spills in the last three years, yet none of those QRI spills appears to have been reported to the federal authorities, and only two seem to have been reported to the state (though a few of the QRI spills may have been below the state reportable quantity of 42 gallons).
However, oil spill response is not the focus of this series; oil spill reporting is. And in sum, the Corps appears to have been routinely failing to report oil spills for four years at the lakefront gates, despite being required by federal and state law to do so.
The background on reporting spills
I don't know how to explain this failure to report. Spill reporting on the federal level is not difficult. If even a drop of oil is spilled in U.S. navigable waters or on land which eventually drains into those waters (which includes all three gate sites, since they are on the outfall canals, which are extensions of Lake Pontchartrain, and the land at the sites drains to the New Orleans and Jefferson Parish drainage systems, which also lead to Lake Pontchartrain), the facility's owner must report it to the Coast Guard's National Response Center, or NRC, as soon as they learn of it. The NRC is - by federal law - the central repository and contact point for any oil spill into U.S. navigable waters. One can call a toll free number or input a report online. The NRC database is publicly available and updated daily, giving everyone in the world near-real-time access to all the spill reports the NRC has received since 1990.
The amount of oil to be reported is laid out in the federal regulations:
For all intents and purposes, that's any amount of oil, especially when talking about the relatively thin hydraulic fluid and fuels kept at each site.
The federal penalties for failure to report oil spills are serious. They include (I believe) imprisonment up to five years and possible civil penalties of $25,000 to $50,000 per day the spill went unreported.
At the state level, reporting is also easy. Louisiana provides a toll free phone number. The state of Louisiana also requires reporting of oil to the Louisiana Department of Environmental Quality (LDEQ), though they have a 1 barrel (42 gallon) threshold. I'm having trouble figuring out the state level penalties for not reporting. The LDEQ reporting website is a little less user friendly than the NRC's, though not impossible to master. I've managed to squeeze a few reports out of it on spills that I hadn't known about.
The documents
When I made that last post on oil spills in 2007, I had tallied 11 spills. Even at that point, with just NRC records and newspaper accounts, I had found that the Corps was violating Federal law by not reporting spills (specifically a spill on Septemebr 15, 2006). What has allowed me to discover that the problem is much larger than that is additional documents. They are in three basic batches.
First, credit where credit is due. A lot of work has been done from outside the Corps researching exactly what the heck was happening with the hydraulic pumps in 2006 and 2007. Much of the digging was done by KPCC-FM reporter Molly Peterson, who filed FOIA request after FOIA request with the Corps New Orleans District in an effort to pry documents out of them. To a large degree, she was remarkably successful, though only after years of effort. She complied her findings into an award-winning four part report on KPCC last August titled "Pumps Under Pressure."
As far as oil spills are concerned, the first particularly valuable set of documents - which were obtained by Ms. Peterson - is a lengthy (though probably incomplete) set of testing records for the pumps during the period from May, 2006 through May, 2007. Among other things, they certainly put the lie to the Corps' reassurances during that period. The pumps were undergoing near-constant "testing" and repair during that time, and it certainly appears they could not have provided much protection to the city. Often, they couldn't even make it a few minutes under sunny skies. Heavy vibrations, jumping hoses, oil leaks, equipment failures, overheating, and belches of black smoke were common. If there's an overall theme that emerges from these testing records, it is "chaos."
The May, 2006 to May, 2007 testing records also provide a second, contemporaneous source for oil spill information. They are a gold mine of information. Where I can, I have included information from them in my descriptions of oil spills below, and in many cases - where there has not been a report properly filed with the authorities - they are the primary source for those descriptions.
A second set of testing records (available here or here), roughly covering the period from June through September of 2007 and also obtained by Ms. Peterson through FOIA, describes the "acceptance" testing performed by the Corps to prove the pumps would work. Those records are not as detailed as the earlier set, but they nonetheless include clear descriptions of more oil spills. Owning to the extreme lack of release reporting to the authorities, they too are primary records in describing spills.
Finally, as I mentioned earlier, since the end of acceptance testing the Corps has had a succession of two contracts in place for oil spill cleanup with Quaternary Resources Investigations of Baton Rouge. The first one covered the period from mid-2007 to late-2008, while the second one - a potentially three year deal - has run from late-2008 to the present. Task orders issued on those contracts provide documentation of spills, and the dollars spent to clean them up. So do contract actions on the pump repair contracts I've written about previously.
Which brings me to the point of this post. I wanted to show the public as much information as possible on oil spills at the gates, so I've provided links to all these documents. Most of the spills recorded in these documents do not show up on the NRC or LDEQ websites.
I've already mentioned the lone pair of spill reports I got from the Corps through my FOIA inquiry and subsequent requests, as well as the public access provided by the federal NRC and state LDEQ websites. But I could not simply rely on downloads - or lack thereof - from those websites to say whether spills had been properly reported.
So I filed public records requests with the state to ensure I got all the LDEQ spill reports. Very helpful LDEQ personnel uncovered some reports that had not been uploaded to their website, while also verifying they had definitely not received reports for other spills. Finally, I contacted personnel at the Coast Guard who assured me in writing that everything on the NRC website is everything they have. With those communications in hand, I feel confident saying that anything that doesn't appear on those two sites wasn't properly reported by the responsible party, the Corps, barring any sudden "discoveries" by the Corps (which would be an implicit acknowledgement of failing to fulfill my FOIA request).
There is a caveat to this. Some stuff not reported to the state could conceivably have been below the 1 barrel threshold. However, smart personnel should have been reporting everything to everyone out of an abundance of caution. I've noted below-42-gallon spills where applicable.
How the pumps work
Before getting in to the big list, I just wanted to provide a brief refresher on how the pumps work, in order to help readers understand where these spills were occurring.
The hydraulic oil is stored in a 300 gallon reservoir on the drive skid for each pump. Here's the best picture I've found thus far of a drive skid (via SCPR Flickr).
Zooming into to the skid:
Here's the flow of hydraulic oil:
The hydraulic oil comes out of the reservoir at low pressure, and flows into the Denison hydraulic pump. Actually there are two Denisons per skid. They are driven by the Caterpillar engine through the gearbox.
The Denison raises the oil to around 3000 pounds per square inch (psi). The Denison oulets (2 per Denison) flow to the HP lines through a manifold. The outlets of the manifold (pictured below in the photo of what I believe is the reservoir level gauge) are connected by hose to long lengths of pipe which go out to the water pumps in the canal. At the manifold, there's also a bypass device and a relief valve too. They are supposed to keep the pressure of the oil from getting too high by diverting some of the oil off at specified pressure.
The inlets to the Denisons were changed in late spring and early summer of 2007, after nearly a year of unexplained delay. The reservoirs were raised and the inlets lowered in direct response to Corps Pump Team Leader Maria Garzino's finding and the Corps' agreement that the Denison pumps were sucking in air and getting damaged with the original piping configuration. In that old configuration the Denison inlet lines had to be primed by an operator using a portable air compressor before the pumps could be started (the testing records note frequent priming at pump startup throughout 2006 and 2007). That un-automated procedure would likely have been impossible to do during higher category hurricanes, because the gate structures are evacuated during those storms.
Here's the revised configuration from MWI's drawing of the reservoir modification (page 34 of the MWI contract documents):
Similar arrangements were earlier installed on the "Phase 2" drives at 17th Street in 2006:
and on the drives for the 17th Street bridge pumps in early 2007 (via Flickr):
Given the timing of those installations, the year-long delay in implementing the flooded suctions on the original order of 34 pumps is odd.
In any case, let's return to following the flow of hydraulic oil. The drive skids are housed in sheds at each site. The only variation on this is at 17th Street, where some pumps installed after the original 12 at that site have their drive skids right next to them. Specifically, the "Phase 2" 60" pumps and the "Phase 3" 42" bridge pumps:
However, for the rest of the hydraulic pumps, the Orleans Avenue site is typical:
Once the oil is raised to pressure by the Denisons, it travels along hundreds of feet of pipe out to the pumps in the canal. The pipes are mounted in pipe racks, also noted in the image above.
According to Black & Veatch's analysis of the current gate structures, these pipe lengths violate MWI's own guidelines.
When the oil gets to the pump, it travels through a short section of hose to the pipe extensions the Corps had installed in late 2006 and early 2007. The history on those piping extensions is here. Here's the end of the main pipe from the sheds and beginning of those hoses at London Avenue in June, 2006 (via SCPR Flickr):
[At the risk of burying a lede here... As I've described in previous posts, the hose connections on the pumps in the canals were corroding within months of installation. They were leaking oil. In fact, the corrosion may have started within just weeks of installation, as this photo of hose connections only exposed to salty air in June of 2006 shows. Here's a detail of one of those rusty hose connections taken at the same time as the picture above (via SCPR Flickr):
So, it would appear the corrosion problem with the hose connections started pretty much instantly in June, 2006, and continued to fester until the piping extensions were installed in early 2007. Of course, the overall corrosion problem has been going on from the very start, and continues to this day.]
And here's the piping extensions that were installed on the pumps in early 2007 (Before those piping extensions were installed, the piping connections at the pumps were short sections of threaded pipe):
Inside the pump, there are two high pressure inlet lines which convey the pressurized oil to the Rineer motor:
That motor spins the shaft, to which is attached the impeller:
The interior of the pump was - and is - a prime place for oil leaks with numerous seals and gland plates. In 2006, there were threaded pipe connections, another excellent place for oil to leak out under pressure. Most of the materials of construction of those pump guts were originally carbon steel, and thus not suitable for the brackish water from Lake Pontchartrain. It seems likely some of the leaks in 2006 and early 2007 were due to corrosion of the pump guts.
The spinning impeller sends water flowing upward through the pump and into 5 foot diameter piping, which feeds into 9 foot diameter piping leading to the lake on the other side of the gates. Both pipe sizes are noted on this picture of the Orleans Avenue site:
Back inside the pump, with the oil having imparted its energy to the rotating Rineer motor, it flows out the low pressure outlet lines:
And travels to the oil coolers on the outside of the pumps:
Theoretically, the residual heat of the oil is transferred to the water of the canal. However, rapid marine growth in the brackish Lake Pontchartrain water fouls the coolers, reducing their efficiency. The marine growth can be seen in Conhagen's 2009 report of their repairs to 17th Street pumps E5 and E7:
After the coolers, the oil travels through more piping along the same route as the high pressure pipes back to reservoir. And the cycle repeats continuously.
How some of the spills were recorded
Some of the incidents I've catalogued are based on notations in the 2006-07 testing reports that speak about, in part, level gauge or sight glass readings on the drive skid reservoirs. The notes often talk about a percentage of oil visible in the sight glass, and that percentage translating into how full the reservoirs are. It happens time and time again, in reports generated by many different individuals.
As best as I can determine, the inspectors observing the testing were probably referring to this instrument on the side of the reservoir:
The photo is taken from the Corps' April 26, 2006 shop inspection report.
A particular level in that sight glass might or might not translate into a particular level in the reservoir. The conduit coming out of the bottom of the instrument is a little confusing, since I'm not aware of level instrumentation sending a signal to the control panels on the skids, though there is mention of a "low hydraulic level light indicator" on the skid control panel on page 9 of the July, 2006 testing records.
When I learned to do document analysis, I was taught that it is easy to get to wrapped up in interpretation and to lose sight of the plain meaning of the words. So since the actual words of the test reports say that a certain percentage full on a reservoir level gauge/sight glass translated into the same percentage full for the reservoir itself, and since those statements are often backed up with contemporaneous notations about MWI refilling the reservoirs or with actual spill reports, I'm going to trust that the records are - on their face - correct, even if I can't verify the exact function of the instrumentation those Corps inspectors were viewing.
The spills
If you've gotten this far, I'm not going to drag you through every spill report. The remainder of this post will cover just the install and testing period from May, 2006 through June, 2007. Even so, that's still 19 spills. Spill reports from later periods will be included in future posts.
I'm going to speculate that many of the spills during this period were due to corrosion and poor design at the waterline on the pumps; specifically, the corrosion of the zinc-coated hose connections (see photo above of similar connections just weeks after installation) and the poor selection of threaded pipe fittings to which they were connected. I wrote about how the Corps dealt with those hose connections during the period between October, 2006 and January, 2007 here. I didn't note how foolish it was to put threaded connections in a brackish environment, so let me do so now: it's very foolish. That's why the replacement piping extensions installed in 2006-07, and the stainless steel replacements for those replacements installed in 2009-10 were all welded.
However, the gusher of spills during - and more importantly - before that October, 2006-January, 2007 period says the Corps was way too late in dealing with those problems, and clearly saw putting oil in the water as little more than an annoyance alongside what they viewed as more important issues. Indeed, during the period of highest chatter/inaction on the hose connection issue (October 1 through mid-November, 2006), I count 10 hydraulic oil spills which may have dumped over 1200 gallons into the canals.
So, let's get to it...
July 9, 2006 at London Avenue
Quantity released: Unknown. (I guess 50 gallons from description)
NRC report: No
LDEQ report: No
Mention in testing records: Yes. On page 11 of the July, 2006 records is a description of testing on pump E2:
This was typical for testing on this day and the following one. Many of the pump drives had low hydraulic oil levels, possibly indicating widespread leaks.
August 8, 2006 at 17th Street
Quantity released: 50 gallons (from NRC report)
NRC report: Yes
"Caller states that Army Corp of Engineers ran hydraulic pumps in 17th street canal, when a coupling on a pump came loose and discharged hydraulic oil into the canal in amount of 50 gallons."
LDEQ report: Yes
"Coupling on a pump came loose and discharged hydraulic oil into canal"
Mention in testing records: No. Testing records for August, 2006 include testing at London Avenue that day, not 17th Street, but testing records could be incomplete.
August 26, 2006 at Orleans Avenue
Quantity released: Unknown. (I guess 270 gallons based on description)
NRC report: No
LDEQ report: No
Mention in testing records: Yes. Testing records for August, 2006 indicate pump W1 was shut down immediately after startup on this day. The hydraulic oil reservoir on the drive skid for W1 was only at 10% of its maximum 300 gallon capacity. The records also mention MWI personnel refilling the reservoir. The lost oil must have gone somewhere, and just as with the July 9, 2006 spill(s), the canal seems the most likely spot.
September 8, 2006 at London Ave
Quantity released: Unknown. (I guess 600 gallons based on description)
NRC report: No
LDEQ report: No
Mention in testing records: Yes. On pages 31 and 32 of the records for September, 2006, tests of pumps W4 and W5 at London Avenue had instances of low oil. In the record for the W4 test are two notes: "low/no oil in sight glass" and "low oil, none in sight glass." The test record for the W5 test says that on warm up of the engine, "low on oil - site [sic] glass at half way." It also notes, "MWI to get more oil today."
September 14, 2006 at Orleans Ave
Quantity released: 55-225 gallons (from NRC, LDEQ, and testing reports)
NRC report: Yes
"Quantity released: 200 gallons
Description of incident: During a test of pumps stationed on the Orleans Avenue drainage canal in New Orleans, hydraulic oil was discovered to have leaked from a pump on the east side of the canal."
LDEQ report: Yes
"US Army Corps of Engineers performed a test of temporary drainage pumps, located on the Orleans Drainage Canal @ Lakeshore Drive> During the test, a hydraulic line broke causing 55 gal of hydraulic oil to spill into the Orleans Canal. The USCG was notified and Oil Mop was contracted to clean the spill."
This estimate, based on the test reports, seems wildly low.
Mention in testing records: Yes. On pages 12-14 of the records for September, 2006, a test of pumps E4 and E5 on this day is described. The entire test run took about three hours, but E4 was shut down less than an hour into the test due to the oil leak. The notes indicate the oil continued leaking even after E4's engine was shut off.
The description hints that oil added by MWI workers to E4's engine during (or after) the test may have gone straight in the canal:
This implies that three quarters of the reservoir's contents - or 225 gallons of hydraulic oil - and maybe more, went into the canal, perhaps after the pump was shut off.
The report also tantalizingly mentions, "Photos and video available upon request."
Finally, this spill, unlike all others, set off some kind of alarm in the Corps engineering folks. Jim St. Germain, one of the guys titularly in charge at the time, directed MWI to clean up this spill at their expense (though it is unclear who Oil Mop was working for at this time). It even got memorialized as modification P00016 to the MWI contract (found on pages 89 through 91 of the MWI contract). No other spill (like perhaps the one the following day, or the one a week previous, or any of the cluster in October and November, 2006) garnered such attention from Corps leadership. I don't know why.
September 15, 2006 at London Avenue
Quantity released: Unknown. (I guess 50 gallons based on description)
NRC report: No
LDEQ report: No
Mention in testing records: Yes. On page 7 of the September 2006 testing records (the only page that appears to be run through the scanner cockeyed), we find mention of this spill:
The rest of the notation is cut off.
That page also includes the notation "WWLTV 4," meaning this testing was likely caught on video by local news station WWL-TV.
Page 9 includes a fuller description of the spill:
Translating that... The Corps was trying to solve the vibration problem that came about from use of undersized hydraulic motors. The tests on September 15th at London Avenue were one of many attempting to diagnose the problem. Some new hydraulic motors had been apparently been installed. However, there were other problems still plaguing the system. A burst hose and blown gasket are symptomatic of overheating or undersizing, problem picked up months earlier during the Florida testing. The mention of "air in system" could refer to the problem that Maria Garzino diagnosed during her time in Florida - the Denison pumps on the engine skids sucking in air.
The mention of backflow in the 9 foot and 5 foot pipes refers to backflow of water through the 9 foot main pipes that exit to the lake and the 5 foot pipes that come off the top of each pump leading to those 9 foot pipes. Backflow indicates that the siphon breakers installed on each of those 5 foot pipes were not functioning properly when pumps cut off, allowing water to be sucked back into the city, a detail left out by now-retired Colonel Bedey in his comments to the Times-Picayune, who may have been on site that day:
Whatever "environmentally friendly" oil is, the Corps is no longer using it today. Their hydraulic oil of choice is Mobil DTE Excel 68, a hydraulic oil whose product data studiously avoids any reference to environmental amity.
There've been a lot of spills without a report to the NRC, but I believe this is the only one that was also mentioned in the paper.
September 18, 2006 at Orleans Ave
Quantity released: 25 gallons (from NRC report)
NRC report: Yes
"During a site inpection of the temporary pump station on the Orleans Avenue canal, I noticed a sheen on the water, originating from the east side of the canal, underneath the platform of hydraulic pumps. I alerted a contractor onsite, and he disconnected the source of the oil from the suspected leak, and the slick stopped growing. The contractor consulted suspects that no more than 25 gallons of hydraulic oil (Mobil 68) entered the canal from the leak. Containment and absorbent booms had been previously placed on site, and have contained the new leak."
LDEQ report: No (though may not be required, since 25 gallons is below state reportable quantity of 42 gallons.)
Mention in testing records: No. The testing records for September, 2006 do not include September 18, 2006, which makes sense because the description in the NRC report does not include a test but merely someone noticing a sheen.
October 4, 2006 at London Avenue
Quantity released: Unknown. (I guess 30 gallons based on description)
NRC report: No
LDEQ report: No
Mention in testing records: Yes. Pages 27 and 28 of the October, 2006 testing records note that the oil reservoir for pump 4W was only 90% full, and there were oil leaks present at the manifold. It's unknown whether they were talking about the main piping manifold for the entire west side (less likely), or simply piping on the drive skid (more likely). Either way, this may not have been a spill over water, but over land. It should still have been reported, since the drainage systems for the sites lead to New Orleans' and Jefferson Parish's drainage systems, which both lead to Lake Pontchartrain.
October 11, 2006 at 17th Street
Quantity released: Unknown. (I guess 150-270 gallons based on description)
NRC report: No
LDEQ report: No
Mention in testing records: Yes. Page 20 of the October, 2006 testing records shows that pump W1 (or perhaps W3, it's not clear), connected to drive skid number 8860 (mounted on the west side of 17th Street), was shut down during a test of pumps W1, W2, and W3 due to a hydraulic oil leak: "oil leak, reservoir site glass is half full."
The Corps continued running the other two pumps for another 40 minutes after discovering the leak.
A note on page 21 muddies the issue further: "Engine 3 leaked oil as result of user error. Site glasses were filled with oil on all three engine tank at the start of the test. Engine 3 had approximately 10% oil seen in the site glass after the oil spill."
A 90% drop in oil level would mean a spill of 270 gallons.
October 13, 2006 at 17th Street
Quantity released: Unknown. (I guess 300 gallons based on description)
NRC report: No
LDEQ report: No
Mention in testing records: Yes. Page 1 of the October, 2006 testing records shows oil being added to the drive for pump W2 during a test of pumps W2 and W3. Even at the end of the test, the sight gauge on W2's reservoir still only read 1/3rd full.
October 15, 2006 at 17th Street
Quantity released: Unknown. (I guess 135 gallons based on description)
NRC report: No
LDEQ report: No
Mention in testing records: Yes. Page 15 of the October, 2006 testing records mentions that at the conclusion of testing on pumps W1, W2, and W3 at 17th Street, the sight glass for the W2 reservoir showed about 75% full and the sight glass for the W3 reservoir was at about 80%. That oil likely ended up in the canal.
October 22, 2006 at 17th Street
Quantity released: 250-300 gallons (from NRC and LDEQ reports)
NRC report: Yes
"The caller is reporting a release of materials into the 17th Street Canal from a hydraulic pump due to equipment failure."
LDEQ report: Yes
"Army Corp of Engineer's located at 17th Street Locks in New Orleans experienced a non hazardous hydraulic oil leak on a new pump installation on 10/22/06 at 1200 hours. The leak was secured by MWI (Mobility[sic] Water Industries), booms were deployed, Oil Mop was dispatched for clean-up, and repairs were made to the pipe the next morning. I arrived on 10/23/06 at 1230 hours and was told approximately 250 gallons of oil was recovered. The recovery team was still placing oil pads to remove a slight oil sheen remaining. The release is not considered to be preventable due to unforseen equipment failure and no human error is suspected at this time. No further action will be conducted concerning this release."
It seems unlikely MWI or the Corps mentioned to LDEQ that there had been three other releases within the previous two weeks on the same set of three drives and three pumps. It's impossible to trace the problems on each of these drive units and pumps during October, because the Corps and MWI were switching the pumps around in their slots on the platform in some attempt to figure out what was causing the vibrations.
Mention in testing reports: Yes. The description of October, 2006's additional testing on pumps W1, W2, and W3 notes that the drives for W1 and W3 already needed oil at the beginning of the test that day, meaning there was a leak even before they got started. After refilling the reservoirs and tracking the oil levels carefully during the duration of the approximately hourlong test, everything was shut down when it was found that the drive for pump W1 had lost all its oil and there was a sheen in the canal. That was the leak that got reported to LDEQ and the NRC, not the additional loss of oil before the test.
October 31, 2006 at 17th Street
Quantity released: 50 gallons (based on NRC report)
NRC report: Yes
"The caller reports the RP [responsible party] was doing construction on a pump station of an internal drainage canal when a fitting dislodged on a hydraulic oil hose which resulted in the discharge of material."
LDEQ report: No.
Mention in testing reports: No. There are no records of testing on October 31 in the October, 2006 testing records.
November 2, 2006 at 17th Street
Quantity released: 20 gallons (from NRC report)
NRC report: Yes
"The caller states that while removing a pump for inspection and repair, oil spilled into the 17th Street Canal when the pump was lifted up out of the water."
LDEQ report: No (not required because quantity released was below state reportable quantity)
Mention in testing reports: No. According to the November, 2006 testing records, there was testing on November 4, but not on November 2.
November 4, 2006 at 17th Street
Quantity released: Unknown. (I guess 200 gallons based on description)
NRC report: No
LDEQ report: No
Mention in testing reports: Yes. Pages 3 and 4 of the November, 2006 testing records detail the testing done on November 4. They show that the sight glass on the reservoir for pump W2 was at 50% and the reservoir for pump W3 was at 90% at the start of testing that day. Oil was added to the W2 reservoir before the test. W2 shut down about half way through the approximately two hour test due to overheating of the Denison hydraulic pumps on the engine skid. At the end of the test, they tried to restart the W2 engine, but could not because they needed to add oil again. That oil had gone somewhere, i.e. the canal.
November 7, 2006 at 17th Street
Quantity released: 50 gallons (from NRC and LDEQ reports)
NRC report: Yes
"Caller stated a fitting came off of a pump as it was being pulled out of the 17th Street Canal. This resulted in the release of material into the canal."
LDEQ report: Yes
"This incident occurred at the London Ave [sic - obvisouly an error] pump station construction/installation site on the 17th Street Canal. The incident occurred on 11/7/06 when a pump was being moved and a fitting was broken off. Approximately 50 gallons of hydraulic fluid was released into the 17th St canal. MWI, Moving Water Industries, the company supplying and installing the pumps for the U.S. Army Corps of Engineers was the responsible party creating the spill during work activities. John Wisinger with the Corps reported the spill. I spoke to Mr. Wisinger on 11/14/06 to discuss the events that had occurred. Mr. Wisinger stated that ES&H was the clean-up contractor responding to the incident on the 7th. The spill was cleaned up. He said that the Coast Guard responded to the spill on the 8th and agreed the spill was cleaned up. The site continually has boom installed up and downstream of the installation location as a precautionary measure. No further action will be taken concerning this incident."
Mention in testing reports: No, but that's not a surprise since the incident occurred during movement of a pump rather than testing.
It is unknown who ES&H was contracted to.
November 13, 2006 at London Avenue
Quantity spilled: 10 gallons (from NRC report)
NRC report: Yes
"The caller states that there is leak of hydraulic oil at the temporary pump station due to equipment failure. The material is discharging into a drainage canal."
LDEQ report: No (not required because quantity released was below state reportable quantity)
Mention in testing reports: No. The November, 2006 testing records do not include records for testing on November 13.
January 25, 2007 at 17th Street
Quantity released: 20 gallons (from NRC report)
NRC report: Yes
"Quantity released: 20 gallons
Description of incident: The caller is reporting a release of materials into the 17th Street canal from a water pump due to equipment failure on the pipe."
LDEQ report: No (not required because quantity released was below state reportable quantity)
Mention in testing reports: Unknown. January testing records not sent.
March 24, 2007 at 17th Street
Quantity released: 50 gallons (from NRC, LDEQ and testing reports)
NRC report: Yes
"The caller stated that while testing a pump, a hydraulic line failed and caused a release of hydraulic oil into the canal. The release is secured and booms have been placed.
Sheen size length: 40 feet.
Sheen size width: 30 feet."
LDEQ report: Yes
"On March 24, 2007, Mr. John Wisinger with the Army Corp of Engineers verbally notified and submitted notification in acordance with LAC 33:I.3925. At the time of the investgation, I met with Mr. Wisinger on March 28, 2007 at the 17th Street Canal Pumping Station in New Orleans. Mr. Wisinger informed me that a stainless steel fitting on a hydraulic hose separated, spilling approximately 50 gallons of hydraulic oil to the drainage canal during an equipment test run on the new pumps. At the time of this investigation only a small sheen was detected within the boomed area. All of the visible oil was recovered. The release is not considered to be preventable due to unforseen equipment failure and no human error is suspected at this time."
Mention in testing reports: Yes. On page 34 of the March, 2007 test reports, notes for a test of pump W2 include the following:
The records for testing that day also mention two other hydraulic oil releases. Pump W5 was shut down because of a "minor" oil leak at the pump.
Also, vibration consultant Carl Eyman of Measurements, LLC in New Orleans was on site that day to check the whether pulsations had been reduced in the pumps with the rebuilt Rineer motors (they had). I'll let the report describe what happened:
Yikes!
This spill is notable because it is the last one at the lakefront gates reported to the National Response Center until a 15 gallon spill on June 2, 2010, over three years later. There were many spills in between.
The next part of this series covers the acceptance test period during the summer of 2007.
Karen Gadbois and Molly Peterson contributed to this report.
The problem
A few years back, I was tracking oil spills occurring at the Corps' three lakefront gate-and-pump conglomerations (17th Street, Orleans Avenue, and London Avenue). I stopped counting at about 800 gallons with my April 18, 2007 post, "11 spills and counting."
Since then, through review of documents obtained through federal and state records requests, I've determined that the oil spill problems at the gates run far deeper than those 11 spills. Thus far, I've documented a total of 40 incidents over the past four years, 29 of which appear to have not been reported to the federal authorities as required by federal law. There appears to have been similar laxity in state-level reporting as well. The total quantity of spills is difficult to determine. From reports that were made, it is definitely 2000-3000 gallons of oil, but I believe adding in the unreported spills brings the total into the range of 5000-6000 gallons, most of it hydraulic fluid coming out of the pumps.
I have repeatedly asked the Corps New Orleans District for copies of whatever spill reports they might have, even sharing my findings with them along the way. But in response to my inquiries and a formal FOIA request, they only turned over two reports, both of which were already available on the state of Louisiana's website.
In light of recent news of what is going on in the Gulf of Mexico, I feel it is particularly timely that this information get out. On a more local basis, this constant spilling of oil is another indication of the consequences of poor decisions made by Corps Engineering and Operations personnel in dealing with the flood protection works at the gates over the past four years. In addition, it is unclear if all the oil was even cleaned up. In recent years the Corps has contracted to clean up oil spills, but early on, it is impossible to determine if that was happening on a regular basis. Their apparent failure to report many of the spills certainly does not inspire confidence that they were cleaning them up.
I've divided the lengthy spill history into three segements:
a) Installation, testing and repairs - May, 2006 through June, 2007
b) Acceptance testing - July, 2007 through September, 2007
c) Post-acceptance testing and corrosion repairs - October, 2007 through present day
In the first two periods, we know of oil spills through Corps testing records. Some spills occurred as water was being pumped from the canals during tests. In fact, some records actually note hydraulic oil being poured into engine reservoirs as pumps were running! Other spills were discovered when those reservoirs were found to be partially full or nearly empty at the start of testing on a given day. Occasionally sheens were observed as well. Reporting to the authorities was scattershot, with contractors calling in the spills to the Coast Guard or the state as much as the responsible party for the sites - the Corps. Some very large spills, in which the reservoirs on drive skids were emptied of their 300 gallons of hydraulic oil, do not appear to have ever been called in.
Spill response also appears to have been similarly hit-or-miss in that early period. Unlike now - when the danger of spills due to the poor design, redesign and maintenance of the pumps is well known, and solid as well as absorbent oil boom is permanently deployed (details on current booms here) - in 2006 and 2007 the response scheme was ad hoc. Only absorbent boom was being deployed, and then only when pumps were turned on. At least two separate oil spill response companies are mentioned in the testing records, and it appears they were not under contract to the responsible party - the Corps. This "catch as catch can" response (along with many other chaotic activities happening at the gates in 2006 and 2007, including lack of reporting) make me wonder if the Corps was meeting the federal requirement for possession of a Spill Prevention Control and Countermeasure plan at each site. They have them currently, but those versions are dated September, 2008:
17th Street SPCC plan
London Avenue SPCC plan
Orleans Avenue SPCC plan
The situation did not improve when the pumps were turned over to the Operations Division. In some ways it got worse. In nearly three years of stewardship over the pumps, the Operations Division had - until this month - not reported a single spill to the federal authorities. By my count there were 18 such spills at the sites, some of them releases of hundreds of gallons. One spill dropped between 1000 and 2000 gallons of diesel. More troubling perhaps is that for these later spills, there is a money trail to follow. The Corps formalized spill response when they transferred the pumps and gates to Operations, signing successive contracts with a Baton Rouge firm, Quaternary Resource Investigations, or QRI. According to contract documents, the Corps has paid QRI over $93,000 in task orders to clean up 9 of the spills in the last three years, yet none of those QRI spills appears to have been reported to the federal authorities, and only two seem to have been reported to the state (though a few of the QRI spills may have been below the state reportable quantity of 42 gallons).
However, oil spill response is not the focus of this series; oil spill reporting is. And in sum, the Corps appears to have been routinely failing to report oil spills for four years at the lakefront gates, despite being required by federal and state law to do so.
The background on reporting spills
I don't know how to explain this failure to report. Spill reporting on the federal level is not difficult. If even a drop of oil is spilled in U.S. navigable waters or on land which eventually drains into those waters (which includes all three gate sites, since they are on the outfall canals, which are extensions of Lake Pontchartrain, and the land at the sites drains to the New Orleans and Jefferson Parish drainage systems, which also lead to Lake Pontchartrain), the facility's owner must report it to the Coast Guard's National Response Center, or NRC, as soon as they learn of it. The NRC is - by federal law - the central repository and contact point for any oil spill into U.S. navigable waters. One can call a toll free number or input a report online. The NRC database is publicly available and updated daily, giving everyone in the world near-real-time access to all the spill reports the NRC has received since 1990.
The amount of oil to be reported is laid out in the federal regulations:
For purposes of section 311(b)(4) of the Act, discharges of oil in such quantities that the Administrator has determined may be harmful to the public health or welfare or the environment of the United States include discharges of oil that:
(a) Violate applicable water quality standards; or
(b) Cause a film or sheen upon or discoloration of the surface of the water or adjoining shorelines or cause a sludge or emulsion to be deposited beneath the surface of the water or upon adjoining shorelines.
- from 40 CFR 40, Part 110.3
For all intents and purposes, that's any amount of oil, especially when talking about the relatively thin hydraulic fluid and fuels kept at each site.
The federal penalties for failure to report oil spills are serious. They include (I believe) imprisonment up to five years and possible civil penalties of $25,000 to $50,000 per day the spill went unreported.
At the state level, reporting is also easy. Louisiana provides a toll free phone number. The state of Louisiana also requires reporting of oil to the Louisiana Department of Environmental Quality (LDEQ), though they have a 1 barrel (42 gallon) threshold. I'm having trouble figuring out the state level penalties for not reporting. The LDEQ reporting website is a little less user friendly than the NRC's, though not impossible to master. I've managed to squeeze a few reports out of it on spills that I hadn't known about.
The documents
When I made that last post on oil spills in 2007, I had tallied 11 spills. Even at that point, with just NRC records and newspaper accounts, I had found that the Corps was violating Federal law by not reporting spills (specifically a spill on Septemebr 15, 2006). What has allowed me to discover that the problem is much larger than that is additional documents. They are in three basic batches.
First, credit where credit is due. A lot of work has been done from outside the Corps researching exactly what the heck was happening with the hydraulic pumps in 2006 and 2007. Much of the digging was done by KPCC-FM reporter Molly Peterson, who filed FOIA request after FOIA request with the Corps New Orleans District in an effort to pry documents out of them. To a large degree, she was remarkably successful, though only after years of effort. She complied her findings into an award-winning four part report on KPCC last August titled "Pumps Under Pressure."
As far as oil spills are concerned, the first particularly valuable set of documents - which were obtained by Ms. Peterson - is a lengthy (though probably incomplete) set of testing records for the pumps during the period from May, 2006 through May, 2007. Among other things, they certainly put the lie to the Corps' reassurances during that period. The pumps were undergoing near-constant "testing" and repair during that time, and it certainly appears they could not have provided much protection to the city. Often, they couldn't even make it a few minutes under sunny skies. Heavy vibrations, jumping hoses, oil leaks, equipment failures, overheating, and belches of black smoke were common. If there's an overall theme that emerges from these testing records, it is "chaos."
The May, 2006 to May, 2007 testing records also provide a second, contemporaneous source for oil spill information. They are a gold mine of information. Where I can, I have included information from them in my descriptions of oil spills below, and in many cases - where there has not been a report properly filed with the authorities - they are the primary source for those descriptions.
A second set of testing records (available here or here), roughly covering the period from June through September of 2007 and also obtained by Ms. Peterson through FOIA, describes the "acceptance" testing performed by the Corps to prove the pumps would work. Those records are not as detailed as the earlier set, but they nonetheless include clear descriptions of more oil spills. Owning to the extreme lack of release reporting to the authorities, they too are primary records in describing spills.
Finally, as I mentioned earlier, since the end of acceptance testing the Corps has had a succession of two contracts in place for oil spill cleanup with Quaternary Resources Investigations of Baton Rouge. The first one covered the period from mid-2007 to late-2008, while the second one - a potentially three year deal - has run from late-2008 to the present. Task orders issued on those contracts provide documentation of spills, and the dollars spent to clean them up. So do contract actions on the pump repair contracts I've written about previously.
Which brings me to the point of this post. I wanted to show the public as much information as possible on oil spills at the gates, so I've provided links to all these documents. Most of the spills recorded in these documents do not show up on the NRC or LDEQ websites.
I've already mentioned the lone pair of spill reports I got from the Corps through my FOIA inquiry and subsequent requests, as well as the public access provided by the federal NRC and state LDEQ websites. But I could not simply rely on downloads - or lack thereof - from those websites to say whether spills had been properly reported.
So I filed public records requests with the state to ensure I got all the LDEQ spill reports. Very helpful LDEQ personnel uncovered some reports that had not been uploaded to their website, while also verifying they had definitely not received reports for other spills. Finally, I contacted personnel at the Coast Guard who assured me in writing that everything on the NRC website is everything they have. With those communications in hand, I feel confident saying that anything that doesn't appear on those two sites wasn't properly reported by the responsible party, the Corps, barring any sudden "discoveries" by the Corps (which would be an implicit acknowledgement of failing to fulfill my FOIA request).
There is a caveat to this. Some stuff not reported to the state could conceivably have been below the 1 barrel threshold. However, smart personnel should have been reporting everything to everyone out of an abundance of caution. I've noted below-42-gallon spills where applicable.
How the pumps work
Before getting in to the big list, I just wanted to provide a brief refresher on how the pumps work, in order to help readers understand where these spills were occurring.
The hydraulic oil is stored in a 300 gallon reservoir on the drive skid for each pump. Here's the best picture I've found thus far of a drive skid (via SCPR Flickr).
Zooming into to the skid:
Here's the flow of hydraulic oil:
The hydraulic oil comes out of the reservoir at low pressure, and flows into the Denison hydraulic pump. Actually there are two Denisons per skid. They are driven by the Caterpillar engine through the gearbox.
The Denison raises the oil to around 3000 pounds per square inch (psi). The Denison oulets (2 per Denison) flow to the HP lines through a manifold. The outlets of the manifold (pictured below in the photo of what I believe is the reservoir level gauge) are connected by hose to long lengths of pipe which go out to the water pumps in the canal. At the manifold, there's also a bypass device and a relief valve too. They are supposed to keep the pressure of the oil from getting too high by diverting some of the oil off at specified pressure.
The inlets to the Denisons were changed in late spring and early summer of 2007, after nearly a year of unexplained delay. The reservoirs were raised and the inlets lowered in direct response to Corps Pump Team Leader Maria Garzino's finding and the Corps' agreement that the Denison pumps were sucking in air and getting damaged with the original piping configuration. In that old configuration the Denison inlet lines had to be primed by an operator using a portable air compressor before the pumps could be started (the testing records note frequent priming at pump startup throughout 2006 and 2007). That un-automated procedure would likely have been impossible to do during higher category hurricanes, because the gate structures are evacuated during those storms.
Here's the revised configuration from MWI's drawing of the reservoir modification (page 34 of the MWI contract documents):
Similar arrangements were earlier installed on the "Phase 2" drives at 17th Street in 2006:
and on the drives for the 17th Street bridge pumps in early 2007 (via Flickr):
Given the timing of those installations, the year-long delay in implementing the flooded suctions on the original order of 34 pumps is odd.
In any case, let's return to following the flow of hydraulic oil. The drive skids are housed in sheds at each site. The only variation on this is at 17th Street, where some pumps installed after the original 12 at that site have their drive skids right next to them. Specifically, the "Phase 2" 60" pumps and the "Phase 3" 42" bridge pumps:
However, for the rest of the hydraulic pumps, the Orleans Avenue site is typical:
Once the oil is raised to pressure by the Denisons, it travels along hundreds of feet of pipe out to the pumps in the canal. The pipes are mounted in pipe racks, also noted in the image above.
According to Black & Veatch's analysis of the current gate structures, these pipe lengths violate MWI's own guidelines.
When the oil gets to the pump, it travels through a short section of hose to the pipe extensions the Corps had installed in late 2006 and early 2007. The history on those piping extensions is here. Here's the end of the main pipe from the sheds and beginning of those hoses at London Avenue in June, 2006 (via SCPR Flickr):
[At the risk of burying a lede here... As I've described in previous posts, the hose connections on the pumps in the canals were corroding within months of installation. They were leaking oil. In fact, the corrosion may have started within just weeks of installation, as this photo of hose connections only exposed to salty air in June of 2006 shows. Here's a detail of one of those rusty hose connections taken at the same time as the picture above (via SCPR Flickr):
So, it would appear the corrosion problem with the hose connections started pretty much instantly in June, 2006, and continued to fester until the piping extensions were installed in early 2007. Of course, the overall corrosion problem has been going on from the very start, and continues to this day.]
And here's the piping extensions that were installed on the pumps in early 2007 (Before those piping extensions were installed, the piping connections at the pumps were short sections of threaded pipe):
Inside the pump, there are two high pressure inlet lines which convey the pressurized oil to the Rineer motor:
That motor spins the shaft, to which is attached the impeller:
The interior of the pump was - and is - a prime place for oil leaks with numerous seals and gland plates. In 2006, there were threaded pipe connections, another excellent place for oil to leak out under pressure. Most of the materials of construction of those pump guts were originally carbon steel, and thus not suitable for the brackish water from Lake Pontchartrain. It seems likely some of the leaks in 2006 and early 2007 were due to corrosion of the pump guts.
The spinning impeller sends water flowing upward through the pump and into 5 foot diameter piping, which feeds into 9 foot diameter piping leading to the lake on the other side of the gates. Both pipe sizes are noted on this picture of the Orleans Avenue site:
Back inside the pump, with the oil having imparted its energy to the rotating Rineer motor, it flows out the low pressure outlet lines:
And travels to the oil coolers on the outside of the pumps:
Theoretically, the residual heat of the oil is transferred to the water of the canal. However, rapid marine growth in the brackish Lake Pontchartrain water fouls the coolers, reducing their efficiency. The marine growth can be seen in Conhagen's 2009 report of their repairs to 17th Street pumps E5 and E7:
After the coolers, the oil travels through more piping along the same route as the high pressure pipes back to reservoir. And the cycle repeats continuously.
How some of the spills were recorded
Some of the incidents I've catalogued are based on notations in the 2006-07 testing reports that speak about, in part, level gauge or sight glass readings on the drive skid reservoirs. The notes often talk about a percentage of oil visible in the sight glass, and that percentage translating into how full the reservoirs are. It happens time and time again, in reports generated by many different individuals.
As best as I can determine, the inspectors observing the testing were probably referring to this instrument on the side of the reservoir:
The photo is taken from the Corps' April 26, 2006 shop inspection report.
A particular level in that sight glass might or might not translate into a particular level in the reservoir. The conduit coming out of the bottom of the instrument is a little confusing, since I'm not aware of level instrumentation sending a signal to the control panels on the skids, though there is mention of a "low hydraulic level light indicator" on the skid control panel on page 9 of the July, 2006 testing records.
When I learned to do document analysis, I was taught that it is easy to get to wrapped up in interpretation and to lose sight of the plain meaning of the words. So since the actual words of the test reports say that a certain percentage full on a reservoir level gauge/sight glass translated into the same percentage full for the reservoir itself, and since those statements are often backed up with contemporaneous notations about MWI refilling the reservoirs or with actual spill reports, I'm going to trust that the records are - on their face - correct, even if I can't verify the exact function of the instrumentation those Corps inspectors were viewing.
The spills
If you've gotten this far, I'm not going to drag you through every spill report. The remainder of this post will cover just the install and testing period from May, 2006 through June, 2007. Even so, that's still 19 spills. Spill reports from later periods will be included in future posts.
I'm going to speculate that many of the spills during this period were due to corrosion and poor design at the waterline on the pumps; specifically, the corrosion of the zinc-coated hose connections (see photo above of similar connections just weeks after installation) and the poor selection of threaded pipe fittings to which they were connected. I wrote about how the Corps dealt with those hose connections during the period between October, 2006 and January, 2007 here. I didn't note how foolish it was to put threaded connections in a brackish environment, so let me do so now: it's very foolish. That's why the replacement piping extensions installed in 2006-07, and the stainless steel replacements for those replacements installed in 2009-10 were all welded.
However, the gusher of spills during - and more importantly - before that October, 2006-January, 2007 period says the Corps was way too late in dealing with those problems, and clearly saw putting oil in the water as little more than an annoyance alongside what they viewed as more important issues. Indeed, during the period of highest chatter/inaction on the hose connection issue (October 1 through mid-November, 2006), I count 10 hydraulic oil spills which may have dumped over 1200 gallons into the canals.
So, let's get to it...
July 9, 2006 at London Avenue
Quantity released: Unknown. (I guess 50 gallons from description)
NRC report: No
LDEQ report: No
Mention in testing records: Yes. On page 11 of the July, 2006 records is a description of testing on pump E2:
"0822 - Startup - Leak at control side HP [high pressure] line - Stopped test.
Oil noted at pump in canal
0945 - Filled tank, restarted."
This was typical for testing on this day and the following one. Many of the pump drives had low hydraulic oil levels, possibly indicating widespread leaks.
August 8, 2006 at 17th Street
Quantity released: 50 gallons (from NRC report)
NRC report: Yes
"Caller states that Army Corp of Engineers ran hydraulic pumps in 17th street canal, when a coupling on a pump came loose and discharged hydraulic oil into the canal in amount of 50 gallons."
LDEQ report: Yes
"Coupling on a pump came loose and discharged hydraulic oil into canal"
Mention in testing records: No. Testing records for August, 2006 include testing at London Avenue that day, not 17th Street, but testing records could be incomplete.
August 26, 2006 at Orleans Avenue
Quantity released: Unknown. (I guess 270 gallons based on description)
NRC report: No
LDEQ report: No
Mention in testing records: Yes. Testing records for August, 2006 indicate pump W1 was shut down immediately after startup on this day. The hydraulic oil reservoir on the drive skid for W1 was only at 10% of its maximum 300 gallon capacity. The records also mention MWI personnel refilling the reservoir. The lost oil must have gone somewhere, and just as with the July 9, 2006 spill(s), the canal seems the most likely spot.
September 8, 2006 at London Ave
Quantity released: Unknown. (I guess 600 gallons based on description)
NRC report: No
LDEQ report: No
Mention in testing records: Yes. On pages 31 and 32 of the records for September, 2006, tests of pumps W4 and W5 at London Avenue had instances of low oil. In the record for the W4 test are two notes: "low/no oil in sight glass" and "low oil, none in sight glass." The test record for the W5 test says that on warm up of the engine, "low on oil - site [sic] glass at half way." It also notes, "MWI to get more oil today."
September 14, 2006 at Orleans Ave
Quantity released: 55-225 gallons (from NRC, LDEQ, and testing reports)
NRC report: Yes
"Quantity released: 200 gallons
Description of incident: During a test of pumps stationed on the Orleans Avenue drainage canal in New Orleans, hydraulic oil was discovered to have leaked from a pump on the east side of the canal."
LDEQ report: Yes
"US Army Corps of Engineers performed a test of temporary drainage pumps, located on the Orleans Drainage Canal @ Lakeshore Drive> During the test, a hydraulic line broke causing 55 gal of hydraulic oil to spill into the Orleans Canal. The USCG was notified and Oil Mop was contracted to clean the spill."
This estimate, based on the test reports, seems wildly low.
Mention in testing records: Yes. On pages 12-14 of the records for September, 2006, a test of pumps E4 and E5 on this day is described. The entire test run took about three hours, but E4 was shut down less than an hour into the test due to the oil leak. The notes indicate the oil continued leaking even after E4's engine was shut off.
The description hints that oil added by MWI workers to E4's engine during (or after) the test may have gone straight in the canal:
"Although this unit [the drive for E4] was off, there was additional loss of oil in the system. The site[sic] glass had less than 1/4 oil when Drive unit 8840 [the drive unit for the other pump, E5, which continued running] was shut down [at the conclusion of the test]. Prior [to] shutdown of Drive unit 8840, MWI at one point did fill the oil level on unit 8849 [the drive for pump E4, the one that was leaking] such that the site glass was full again. I'm not sure how much additional oil was added."
This implies that three quarters of the reservoir's contents - or 225 gallons of hydraulic oil - and maybe more, went into the canal, perhaps after the pump was shut off.
The report also tantalizingly mentions, "Photos and video available upon request."
Finally, this spill, unlike all others, set off some kind of alarm in the Corps engineering folks. Jim St. Germain, one of the guys titularly in charge at the time, directed MWI to clean up this spill at their expense (though it is unclear who Oil Mop was working for at this time). It even got memorialized as modification P00016 to the MWI contract (found on pages 89 through 91 of the MWI contract). No other spill (like perhaps the one the following day, or the one a week previous, or any of the cluster in October and November, 2006) garnered such attention from Corps leadership. I don't know why.
September 15, 2006 at London Avenue
Quantity released: Unknown. (I guess 50 gallons based on description)
NRC report: No
LDEQ report: No
Mention in testing records: Yes. On page 7 of the September 2006 testing records (the only page that appears to be run through the scanner cockeyed), we find mention of this spill:
"at 2:32 a line burst"
The rest of the notation is cut off.
That page also includes the notation "WWLTV 4," meaning this testing was likely caught on video by local news station WWL-TV.
Page 9 includes a fuller description of the spill:
"hydraulic oil/ hose burst - blown gasket on a flange.
loud noise at engine unit - air in system
had backflow w/in 9 ft/5 ft pipe"
Translating that... The Corps was trying to solve the vibration problem that came about from use of undersized hydraulic motors. The tests on September 15th at London Avenue were one of many attempting to diagnose the problem. Some new hydraulic motors had been apparently been installed. However, there were other problems still plaguing the system. A burst hose and blown gasket are symptomatic of overheating or undersizing, problem picked up months earlier during the Florida testing. The mention of "air in system" could refer to the problem that Maria Garzino diagnosed during her time in Florida - the Denison pumps on the engine skids sucking in air.
The mention of backflow in the 9 foot and 5 foot pipes refers to backflow of water through the 9 foot main pipes that exit to the lake and the 5 foot pipes that come off the top of each pump leading to those 9 foot pipes. Backflow indicates that the siphon breakers installed on each of those 5 foot pipes were not functioning properly when pumps cut off, allowing water to be sucked back into the city, a detail left out by now-retired Colonel Bedey in his comments to the Times-Picayune, who may have been on site that day:
"A pipe joint began leaking during the tests on Friday [at London Avenue], spilling a few gallons of hydraulic fluid into the canal, but officials said repairs would be finished within 24 hours. On Thursday, a leak in the Orleans Canal also spilled hydraulic fluid into that canal.
'We're using environmentally friendly hydraulic oil, and you don't want it to happen. But that's why we have booms set up every time to absorb (spills), just in case,' [Corps of Engineers Hurricane Protection Office Chief Colonel Jeff] Bedey said."
Whatever "environmentally friendly" oil is, the Corps is no longer using it today. Their hydraulic oil of choice is Mobil DTE Excel 68, a hydraulic oil whose product data studiously avoids any reference to environmental amity.
There've been a lot of spills without a report to the NRC, but I believe this is the only one that was also mentioned in the paper.
September 18, 2006 at Orleans Ave
Quantity released: 25 gallons (from NRC report)
NRC report: Yes
"During a site inpection of the temporary pump station on the Orleans Avenue canal, I noticed a sheen on the water, originating from the east side of the canal, underneath the platform of hydraulic pumps. I alerted a contractor onsite, and he disconnected the source of the oil from the suspected leak, and the slick stopped growing. The contractor consulted suspects that no more than 25 gallons of hydraulic oil (Mobil 68) entered the canal from the leak. Containment and absorbent booms had been previously placed on site, and have contained the new leak."
LDEQ report: No (though may not be required, since 25 gallons is below state reportable quantity of 42 gallons.)
Mention in testing records: No. The testing records for September, 2006 do not include September 18, 2006, which makes sense because the description in the NRC report does not include a test but merely someone noticing a sheen.
October 4, 2006 at London Avenue
Quantity released: Unknown. (I guess 30 gallons based on description)
NRC report: No
LDEQ report: No
Mention in testing records: Yes. Pages 27 and 28 of the October, 2006 testing records note that the oil reservoir for pump 4W was only 90% full, and there were oil leaks present at the manifold. It's unknown whether they were talking about the main piping manifold for the entire west side (less likely), or simply piping on the drive skid (more likely). Either way, this may not have been a spill over water, but over land. It should still have been reported, since the drainage systems for the sites lead to New Orleans' and Jefferson Parish's drainage systems, which both lead to Lake Pontchartrain.
October 11, 2006 at 17th Street
Quantity released: Unknown. (I guess 150-270 gallons based on description)
NRC report: No
LDEQ report: No
Mention in testing records: Yes. Page 20 of the October, 2006 testing records shows that pump W1 (or perhaps W3, it's not clear), connected to drive skid number 8860 (mounted on the west side of 17th Street), was shut down during a test of pumps W1, W2, and W3 due to a hydraulic oil leak: "oil leak, reservoir site glass is half full."
The Corps continued running the other two pumps for another 40 minutes after discovering the leak.
A note on page 21 muddies the issue further: "Engine 3 leaked oil as result of user error. Site glasses were filled with oil on all three engine tank at the start of the test. Engine 3 had approximately 10% oil seen in the site glass after the oil spill."
A 90% drop in oil level would mean a spill of 270 gallons.
October 13, 2006 at 17th Street
Quantity released: Unknown. (I guess 300 gallons based on description)
NRC report: No
LDEQ report: No
Mention in testing records: Yes. Page 1 of the October, 2006 testing records shows oil being added to the drive for pump W2 during a test of pumps W2 and W3. Even at the end of the test, the sight gauge on W2's reservoir still only read 1/3rd full.
October 15, 2006 at 17th Street
Quantity released: Unknown. (I guess 135 gallons based on description)
NRC report: No
LDEQ report: No
Mention in testing records: Yes. Page 15 of the October, 2006 testing records mentions that at the conclusion of testing on pumps W1, W2, and W3 at 17th Street, the sight glass for the W2 reservoir showed about 75% full and the sight glass for the W3 reservoir was at about 80%. That oil likely ended up in the canal.
October 22, 2006 at 17th Street
Quantity released: 250-300 gallons (from NRC and LDEQ reports)
NRC report: Yes
"The caller is reporting a release of materials into the 17th Street Canal from a hydraulic pump due to equipment failure."
LDEQ report: Yes
"Army Corp of Engineer's located at 17th Street Locks in New Orleans experienced a non hazardous hydraulic oil leak on a new pump installation on 10/22/06 at 1200 hours. The leak was secured by MWI (Mobility[sic] Water Industries), booms were deployed, Oil Mop was dispatched for clean-up, and repairs were made to the pipe the next morning. I arrived on 10/23/06 at 1230 hours and was told approximately 250 gallons of oil was recovered. The recovery team was still placing oil pads to remove a slight oil sheen remaining. The release is not considered to be preventable due to unforseen equipment failure and no human error is suspected at this time. No further action will be conducted concerning this release."
It seems unlikely MWI or the Corps mentioned to LDEQ that there had been three other releases within the previous two weeks on the same set of three drives and three pumps. It's impossible to trace the problems on each of these drive units and pumps during October, because the Corps and MWI were switching the pumps around in their slots on the platform in some attempt to figure out what was causing the vibrations.
Mention in testing reports: Yes. The description of October, 2006's additional testing on pumps W1, W2, and W3 notes that the drives for W1 and W3 already needed oil at the beginning of the test that day, meaning there was a leak even before they got started. After refilling the reservoirs and tracking the oil levels carefully during the duration of the approximately hourlong test, everything was shut down when it was found that the drive for pump W1 had lost all its oil and there was a sheen in the canal. That was the leak that got reported to LDEQ and the NRC, not the additional loss of oil before the test.
October 31, 2006 at 17th Street
Quantity released: 50 gallons (based on NRC report)
NRC report: Yes
"The caller reports the RP [responsible party] was doing construction on a pump station of an internal drainage canal when a fitting dislodged on a hydraulic oil hose which resulted in the discharge of material."
LDEQ report: No.
Mention in testing reports: No. There are no records of testing on October 31 in the October, 2006 testing records.
November 2, 2006 at 17th Street
Quantity released: 20 gallons (from NRC report)
NRC report: Yes
"The caller states that while removing a pump for inspection and repair, oil spilled into the 17th Street Canal when the pump was lifted up out of the water."
LDEQ report: No (not required because quantity released was below state reportable quantity)
Mention in testing reports: No. According to the November, 2006 testing records, there was testing on November 4, but not on November 2.
November 4, 2006 at 17th Street
Quantity released: Unknown. (I guess 200 gallons based on description)
NRC report: No
LDEQ report: No
Mention in testing reports: Yes. Pages 3 and 4 of the November, 2006 testing records detail the testing done on November 4. They show that the sight glass on the reservoir for pump W2 was at 50% and the reservoir for pump W3 was at 90% at the start of testing that day. Oil was added to the W2 reservoir before the test. W2 shut down about half way through the approximately two hour test due to overheating of the Denison hydraulic pumps on the engine skid. At the end of the test, they tried to restart the W2 engine, but could not because they needed to add oil again. That oil had gone somewhere, i.e. the canal.
November 7, 2006 at 17th Street
Quantity released: 50 gallons (from NRC and LDEQ reports)
NRC report: Yes
"Caller stated a fitting came off of a pump as it was being pulled out of the 17th Street Canal. This resulted in the release of material into the canal."
LDEQ report: Yes
"This incident occurred at the London Ave [sic - obvisouly an error] pump station construction/installation site on the 17th Street Canal. The incident occurred on 11/7/06 when a pump was being moved and a fitting was broken off. Approximately 50 gallons of hydraulic fluid was released into the 17th St canal. MWI, Moving Water Industries, the company supplying and installing the pumps for the U.S. Army Corps of Engineers was the responsible party creating the spill during work activities. John Wisinger with the Corps reported the spill. I spoke to Mr. Wisinger on 11/14/06 to discuss the events that had occurred. Mr. Wisinger stated that ES&H was the clean-up contractor responding to the incident on the 7th. The spill was cleaned up. He said that the Coast Guard responded to the spill on the 8th and agreed the spill was cleaned up. The site continually has boom installed up and downstream of the installation location as a precautionary measure. No further action will be taken concerning this incident."
Mention in testing reports: No, but that's not a surprise since the incident occurred during movement of a pump rather than testing.
It is unknown who ES&H was contracted to.
November 13, 2006 at London Avenue
Quantity spilled: 10 gallons (from NRC report)
NRC report: Yes
"The caller states that there is leak of hydraulic oil at the temporary pump station due to equipment failure. The material is discharging into a drainage canal."
LDEQ report: No (not required because quantity released was below state reportable quantity)
Mention in testing reports: No. The November, 2006 testing records do not include records for testing on November 13.
January 25, 2007 at 17th Street
Quantity released: 20 gallons (from NRC report)
NRC report: Yes
"Quantity released: 20 gallons
Description of incident: The caller is reporting a release of materials into the 17th Street canal from a water pump due to equipment failure on the pipe."
LDEQ report: No (not required because quantity released was below state reportable quantity)
Mention in testing reports: Unknown. January testing records not sent.
March 24, 2007 at 17th Street
Quantity released: 50 gallons (from NRC, LDEQ and testing reports)
NRC report: Yes
"The caller stated that while testing a pump, a hydraulic line failed and caused a release of hydraulic oil into the canal. The release is secured and booms have been placed.
Sheen size length: 40 feet.
Sheen size width: 30 feet."
LDEQ report: Yes
"On March 24, 2007, Mr. John Wisinger with the Army Corp of Engineers verbally notified and submitted notification in acordance with LAC 33:I.3925. At the time of the investgation, I met with Mr. Wisinger on March 28, 2007 at the 17th Street Canal Pumping Station in New Orleans. Mr. Wisinger informed me that a stainless steel fitting on a hydraulic hose separated, spilling approximately 50 gallons of hydraulic oil to the drainage canal during an equipment test run on the new pumps. At the time of this investigation only a small sheen was detected within the boomed area. All of the visible oil was recovered. The release is not considered to be preventable due to unforseen equipment failure and no human error is suspected at this time."
Mention in testing reports: Yes. On page 34 of the March, 2007 test reports, notes for a test of pump W2 include the following:
"Emer. shut down
High pressure line blew on pump platform
About 50 gallons lost"
The records for testing that day also mention two other hydraulic oil releases. Pump W5 was shut down because of a "minor" oil leak at the pump.
Also, vibration consultant Carl Eyman of Measurements, LLC in New Orleans was on site that day to check the whether pulsations had been reduced in the pumps with the rebuilt Rineer motors (they had). I'll let the report describe what happened:
"A second leak occurred when Carl Eyman attempted to insert his pressure transducers into the couple on pump [W]5. The fluid in the hose was under pressure and hydraulic oil was sprayed on several observers. To prevent a recurrence, the drive unit will be shut down while Mr. Eyman inserts his transducers."
Yikes!
This spill is notable because it is the last one at the lakefront gates reported to the National Response Center until a 15 gallon spill on June 2, 2010, over three years later. There were many spills in between.
The next part of this series covers the acceptance test period during the summer of 2007.
Karen Gadbois and Molly Peterson contributed to this report.
Labels: Rusty pumps
posted by mcbrid35 at
4:19 PM
0 Comments:
Post a Comment
<< Home