Fix the pumps

Thursday, May 11, 2017

Corps refuses to pay for corrosion repairs to St. Bernard Parish pumps they knew would corrode

[Editor's note: Following the initial publication of this post on May 11, 2017, WWL-TV's Paul Murphy did a follow-up story about these pumps on May 12, 2017, linked below. Screenshots and quotes from that report have been incorporated into the post below. Also, since the WWL-TV story included video from station 2, I have added pictures of station 2 from October, 2005 and February, 2009.]

Over the last year, all thirteen pumps at three St. Bernard Parish drainage pump stations have had to be pulled out and completely refitted because of oil leaks brought on by corrosion, costing cash-strapped local authorities over $600,000. The leaks caused the pumps to fail, making them non-functional if they had been needed during rainstorms. These three stations were rebuilt by the U.S. Army Corps of Engineers between 2007 and 2009. The stations were destroyed by the failure of the Corps' floodworks after Hurricane Katrina's passage in 2005. The stations were turned over to local authorities in 2009, but Corps-funded work continued on them for at least a year longer, and the Corps contract stayed open for another two years.

The facts of the story in St. Bernard have direct parallels to one in New Orleans, where in recent years (this link sends you to all the entries with the tag "Rusty pumps") hydraulically-driven pumps - the same design as those in St. Bernard - at all three Corps-built and Corps-operated outfall canal sites have been pulled out and refitted due to corrosion, sometimes in multiple instances, over 50 times. The Corps has spent at least $8.5 million on those repairs since 2009, with work continuing as recently as 2016, when three pumps were pulled out. Such federal attention has been severely lacking in St. Bernard, and last year the inattention actively turned into an abandonment of responsibility by the Corps.

In 2016, the Corps refused to pay for the repairs, saying they do not have the authority to do so. This despite: 1) Already determining they had the authority to pay over $10 million for other post-turnover repairs at the same St. Bernard pump stations; 2) Their knowing these corrosion failures would occur, thanks to copious evidence of identical failures accumulated over a decade on their own pumps in New Orleans; 3) Trumpeting this knowledge publicly throughout 2009 as part of a public lobbying effort; 4) Implementing fixes to other known problems on the St. Bernard pumps, while ignoring established fixes for corrosion; and 5) Paying millions of dollars in post-turnover corrosion repairs on numerous other projects throughout the greater New Orleans area.

A brief orientation

St. Bernard Parish, downriver from New Orleans, was completely devastated by the federal levee failures following Katrina. Every building but a handful was flooded, and many people died. A 23,000 barrel oil spill from the Murphy Oil refinery in Meraux during the flooding - 3,000 barrels of which escaped containment and entered flooded neighborhoods - added an extra layer of misery to St. Bernard's residents. The Parish has never regained its' pre-storm population.

Among the structures damaged or destroyed in 2005 were all eight of St. Bernard's perimeter drainage pumping stations. Within the miles-long ring of federal hurricane protection levees and floodwalls, St. Bernard pumps its rainwater into the nearby bodies of water and swamps through these eight stations, which are run by the Lake Borgne Basin Levee District (LBBLD). Outside of the levees, seven small pumping stations remove tidally driven floodwaters in neighborhoods like Reggio, Delacroix, and Yscloskey. Those stations, run by St Bernard Parish directly, were also destroyed and were replaced using FEMA funds. Under special authority granted following the storm, the Corps took responsibility for repair, rehabilitation, and reconstruction of the eight larger pumping stations within the levees:

Three of those stations - stations 2, 3, and 5 - housed a mix of hydraulically driven and direct driven  pumps when the levees broke. Here's pictures of station 2 showing the extent of the damage, excerpted from the Corps' post-Katrina St. Bernard pumping station damage survey report (Vol. 1 and Vol. 2, written by New Orleans engineering firm Waldemar S. Nelson):

In rebuilding these stations, which were total losses, the Corps' New Orleans District decided to start from scratch. Among the many changes they made from the original designs were the elimination of all direct drive pumps, replacing them with hydraulically-driven pumps across all three stations

Hydraulically-driven pumps are the same design as those initially selected and installed by the Corps at the three New Orleans outfall canals. The Corps continues to operate and maintain the New Orleans outfall canal sites. The decisions for both the New Orleans and St. Bernard pumps happened at the same time, in the first weeks of 2006, just months after the levee failures. The only difference in St. Bernard was the pumps' size; the pumps in New Orleans are 60" and 42" in diameter while those in St. Bernard are 54" and 42". Seven 54" pumps were installed at two stations: four at station 2, three at station 3. The six pumps at station 5 are all 42". A primer with illustrations describing how the pumps work can be found here, and all the posts related to corrosion of the New Orleans hydraulic pumps can be found here. They were updated earlier this year to reflect the current state of knowledge, based on internal Corps documents obtained via FOIA.

While the decision to use exclusively hydraulically-driven pumps at the three stations was made in early 2006, it still took nearly another year and a half to award a contract. In May, 2007, well known federal contractor CDM was awarded an $18 million design-build contract for rebuilding of stations 2, 3, and 5. The contract was one of five awarded in 2006 and 2007 to fix the damaged pump stations in St Bernard. The other four contracts covered roofing, provision of generators, repair of taintor gates, and miscellaneous electrical, mechanical, and structural repairs at stations 1, 4, 6, 7, and 8. Those four contracts' eventual combined value - $2.1 million - was dwarfed by the CDM design-build contract, which grew to just over $20 million.

A design-build contract means that one firm is responsible for everything on the project, from the engineering and design all the way through the commissioning and startup. The intent was to have construction substantially complete by the middle of August, 2008, but that did not happen until early in 2009. Here's pictures of station 2 on February 4, 2009, a few days before a ribbon cutting ceremony (pictures from Flickr):

Punch list items were worked on throughout 2009 and 2010, and spending on the contract stopped in July, 2011. In November, 2009, while the contract remained open, the Corps and CDM won the National Design-Build Award in the Rehabilitation/Renovation/Restoration category from the Design-Build Institute of America, an engineering industry lobbying group. DBIA said it was in recognition of the use of design-build "to quickly and cost effectively rehabilitate the three pump stations." The award was officially presented to the Corps and CDM in May, 2010:

What was gained in award-winning speed of construction was lost in durability. The St. Bernard pumps quickly started suffering corrosion failures. How do we know this? Because the LBBLD reported it to the Corps:
"Since shortly after completion of the project, hydraulic leaks at Pump Stations 2, 3, and 5 have developed and become more frequent and severe. Investigation by LBBLD Personnel, Conhagen and Associated Pump and Supply (AP and S) have identified issues related to the use of high carbon steel fittings that have corroded over time, leading to hydraulic fluid leaks and the failure of hydraulic motor seals."

The history of St Bernard's leaky pumps

Documents from the period of the rebuilding and first few years of operation of St Bernard pump stations 2, 3, and 5 (2008-2012) speak to the hydraulic fluid leaks cited by LBBLD. Indeed, just one month following the February 9, 2009 ribbon-cutting ceremony for the stations, leaks on the pumps were being plugged. Minutes of a March 25, 2009 CDM-led construction progress meeting (the February ceremony was just that, a ceremony. Work continued on the stations throughout 2009 and 2010.) describe a visit by Associated Pump and Supply, the local service representative for the pumps' manufacturer, MWI of Deerfield Beach, FL (emphasis mine):
"Associated [Pump] came out to fix leak at PS [pump station] 3 and found that on Pump 2 the high pressure line is being pushed up against the pump support. Anthony [Bertucci of prime subcontractor and project manager MR Pittman Group, LLC of St. Rose, LA] was made aware of the issue when it was discovered and will address it. Also the No 4 pump at PS 2 has a leak at the flange directly above the suction bell. Associated pump will be back out to address that ASAP."

Were these the only problems, it would not be worth mentioning. But they are part of a pattern, because leaks are mentioned repeatedly in the period after substantial completion of the stations:

- All three punch lists for the stations (undated but likely prepared and constantly updated in late 2008 and the first half of 2009) mention hydraulic leaks at hose connections and flanges.

- An email chain among the job's contractors from March, 2010, over a year after the ribbon cutting, showed work continuing (apparently at Federal expense) to address pump hydraulic leaks. Two of the pumps at pump station 5 were found to have leaks. The situation was summarized by CDM's Kristin Celantano, writing CDM's Stephen Thanner on March 19, 2010:
"PS 5
Pulled the number 3 pump to locate the hydraulic leak. Found that the leak is in the same spot as the leak from the number 5 pump. The "T" or terminal block was damaged at the threaded connection. Also, after taking the impeller off another leak was discovered coming from the motor. MWI rep Keith said he will bring back all necessary seals and parts next week to repair the pump and reinstall."

This describes the exact same failures as were occurring on the New Orleans pumps at the same time: leaks at pipe joints and leaks out of the Rineer hydraulic motors at the hearts of these pumps. The Corps has gone through dozens of Rineer motors during their years of repairs on the New Orleans pumps.

Notably, a number of Corps personnel are copied on - or authored - these emails and some mention they were personally visiting the St. Bernard sites observing the leaks. Some of these people - including resident engineer Randy Persica and senior project manager Dan Bollinger, pictured above  - were also deeply involved in the New Orleans pumps.

These 2010 leaks were substantial enough to get reported up to the LBBLD's parent organization, the  east bank levee authority. In their April, 2010 monthly status report to the Southeast Louisiana Flood Protection Authority - East (SLFPAE), the LBBLD reported,
"Rebuild project for Pump Stations 2, 3, and 5: The majority of the Contractor's punch list items are finished. Pump #5 and #3 at Station #5 have been removed for inspection by the manufacturer MWI. The designer has found a flaw in the design of the flange assembly that connects the hydraulic lines to the oil cooler. MWI is reviewing this matter and will inspect the other four pumps with the same flange assembly and replace them with redesigned parts at no cost to LBBLD."

This additional detail seems to also describe the leaks from March, 2009 mentioned above. Also, based on what we have learned over the years from the copious failures at the New Orleans pumps (as well as a 2014 inspection of one of the pumps from pump station 5 described below), it is likely the fixes applied by Associated and MWI for these leaks were ultimately temporary in nature and did not address the root cause of corrosion. It is also notable that no mention was made of no-cost repairs to the Rineer motors, a point just as vulnerable as the carbon steel pipes.

At literally the very same time these St Bernard pump station leaks were being band-aided in March, 2010, the Corps was performing massive refits on the pumps in New Orleans. Just three days before the email quoted above, two refitted pumps had been reinstalled at the 17th Street site with brand new stainless steel piping and oil coolers. In fact, the reinstallation was captured by the local media when Fox 8 aired video of a refitted pump being dropped back into place on March 16, 2010 as part of a story about the permanent pumps project agreement signing ceremony:

Then a few days later, four pumps were pulled out of the London Avenue site for complete overhauls. I wrote about this - not once, but twice - at the time. That is, while the Corps was spending hundreds of thousands of dollars on their New Orleans pumps to address corrosion (though ultimately it would be only semi-successful), they were allowing the St. Bernard pumps to languish, having the contractors apply tiny fixes while leaving all the carbon steel piping to rust away.

- In 2012, leaks were continuing in St. Bernard. This was a year after the CDM contract was closed. A March 23, 2012 quarterly inspection report by LBBLD of pump station 2 noted:
"Some hydraulic fluid has leaked from the systems. In investigative phase now with Associated Pump."

- By 2014, the leaks had gotten a lot worse. In March of that year, leaks were found in four of the seven pumps across stations 2 and 3. A report compiled by Associated Pump showed failures in two pumps at each station. Both stations had two flavors of leaks: a) failed oil coolers; and b) failed mechanical seals. These are failure modes identical to those in the Corps' New Orleans pumps. At station 2, the leaks were so severe that both of the failed pumps had lost all of their oil - hundreds of gallons - into the water. This leak severity also tracks failures at the New Orleans pumps, where thousands of gallons of hydraulic oil have been spilled into the outfall canals (see three part series on leaks here, here, and here).

Which brings us back to the December, 2015 quote near the top of this entry. The quote is taken from a December, 2015 "distress report" filed with the Corps by the LBBLD. The report consolidated findings from LBBLD-funded leak examinations by the local pump repair shop (Conhagen in Kenner, LA) and a by a local engineering firm (Burk-Kleinpeter, Inc. in New Orleans). It also included the Associated Pump report of the March, 2014 leaks.

In September, 2014, Conhagen removed one of the six pumps from station 5 and took it back to their shop for inspection. The pictures from Conhagen's report look maddeningly identical to the reports generated years earlier when they also performed repairs on the New Orleans outfall canal pumps. Just as in 2010 (and literally dozens of other times), the piping and hose connections were badly corroded:

Also, the pump suction bell had suffered severe erosion damage, with large chunks missing, just like the New Orleans pumps:

Additional pictures of the St. Bernard pump in 2014 appear to show corrosion at the locations described in the 2009 and 2010 St. Bernard failures - piping connections just above the suction bell ...

... and at the connections to the oil coolers:

The oil coolers themselves were also rusting:

Following Conhagen's work, BKI was asked to analyze the observations and the pumps' repair histories to date. They concluded in late 2015 corrosion was the likely culprit, and even though some pumps had yet to fail, the similarity of the conditions those pumps experienced made it inevitable they too would eventually succumb.

The repair recommendation was unsurprising: replacement of as much carbon steel as possible with stainless steel, and refurbishment of the remaining carbon steel components. Again, this is exactly what Conhagen themselves said in 2009 and again in 2010 about the New Orleans pumps at the start of that now-eight year old repair effort.

LBBLD accepted BKI's and Conhagen's recommendation and proceeded to have all six pumps at station 5 pulled out and repaired during the spring of 2016. The District paid $183,452.50 for this work, and the pumps were back in service prior to the beginning of the 2016 hurricane season.

Since LBBLD has been in a severe cash crunch for many years now (they had 46 employees before the storm and are now down to 21, with major maintenance of the perimeter levee system going undone), the pump repairs were limited to what was thought to be absolutely essential. For instance, while all the oil coolers and the carbon steel piping were replaced with stainless steel, the Rineer motors at the hearts of the pumps - with their mostly carbon steel exterior construction and proven record of failure in New Orleans and St. Bernard - were not refurbished by Conhagen. This will leave these pumps vulnerable to future failures. On their standard suite of repairs for leaky pumps, the Corps always replaces Rineers with refitted models. No spare Rineers were provided by the Corps with the St. Bernard pumps.

With the 2016 completion of Corps-funded seepage repairs at stations 2 and 3 (described below), the LBBLD turned their attention to the seven pumps at those stations this past winter. Damage to these pumps appears to have been more severe, with all the Rineers from station 3 getting refurbished. Station 2's Rineers however are not getting fully refurbed, due to cost. The rest of the suite of repairs is identical to those done on the station 5's pumps, as well as the pumps in New Orleans: replacement of carbon steel piping and hose components with those made of stainless steel, along with weld repairs to the eroded/corroded inlet bells. Those seven pumps are currently out of the water at stations 2 and 3, and are expected to be reinstalled in early June. Here's pictures of station 2 with all four pumps removed (pictures courtesy WWL-TV, from their May 12, 2017 report on this issue):

Again, only the essential repairs were part of the scope. So at their February 16, 2017 board meeting, the Southeast Flood Protection Authority - East (or SLFPAE, the LBBLD's parent organization) authorized the District to issue a purchase order to Conhagen for $460,453 for the repairs to the pumps at stations 2 and 3 (the resolution actually shorted the District $30,000 due to a typographical error, but the District issued the PO in the correct amount anyway). The February 1, 2017 Conhagen quote for those repairs is here.

Additional repair costs pushed that cost up another $32,345, to $492,798, which the SLFPAE was due to discuss at their May 18, 2017 board meeting.

[Ed note, 6/18/17: the board adopted a resolution authorizing the extra expenses at their May 18, 2017 meeting, pursuant to a recommendation by their Operations Committee made earlier the same day. The minutes for both meetings indicate the extra expenses covered more expensive crane work and additional repairs on Rineer hydraulic motors. I will endeavor to obtain more detailed information.]

The Levee District - despite the Corps' foreknowledge of the failure mode and their inaction in preventing it - is completely on the hook for these expenses. Six months after the LBBLD submitted their distress report, the Corps responded with a one page letter. They agreed the repairs were warranted but,
"United States Army Corps of Engineers (USACE) is unable to provide reimbursement for these repairs. The Cooperation Agreement and Project Information Report only allowed USACE to 'Repair and rehabilitate the pump stations to pre-storm conditions.'"

One day following the initial publication of this article, WWL-TV's Paul Murphy followed up with a report about the issues discussed here. He got additional comment from the Corps:
"Rene Poche, a spokesman for Corps says they are aware of the pump failures and signed off on a remediation plan.

'In this particular case, we received a request for technical assistance from the local levee district,' Poche said. 'They asked us to review their proposal to remediate, repair and replace these parts, we reviewed it an approved it.'

The Southeast Louisiana Flood Protection Authority – East is now paying $600,000 to $800,000 price tag to fix the pumps.

Friday, the Army Corps said it would consider a request for reimbursement.

'Let's get them up and running and doing what they're supposed to do,' Poche said. 'We haven't received a request for reimbursement. We'll address that if the issue comes up.'"

Given the context and knowledge about how these pumps and the New Orleans pumps have done over the years, the 2016 written - and apparently unsolicited - refusal of reimbursement is infuriating. But to then go on camera and pretend that refusal didn't exist - well that's just beyond the pale. Based on everything happening around the St. Bernard pump stations, the New Orleans outfall canals, and throughout the rest of the Corps' post-Katrina construction efforts, the absolute minimum the Corps must do is reimburse LBBLD for the pump repairs and testing at stations 2, 3, and 5. What should really be done is what will be done in New Orleans: the pumps should be ripped out and permanently replaced.

There's a lot of arguments against the Corps stingy position. Let's go through them.

The Corps has already paid for post-completion repairs at the very same stations

One does not have to look very far to find a counter-example to the Corps' claim. In fact, two of the three rebuilt pump stations with rusty pumps had other problems after they were turned over to the locals. After initially claiming they did not have the authority or funding to make repairs, the Corps reversed themselves and fixed the problems at full federal expense.

In 2008, even before the stations were finished, the sheet pile walls next to the pump discharge pipes at pump stations 2 and 3 were found to have a significant amount of water seeping through them. Here's a picture of the seepage at pump station 3 from the LBBLD December, 2009 status report:

As with almost everything, the locals had to fight tooth and nail with the Corps to get them to admit the problem was related to Katrina. The Corps said they had no authority to repair the leakage no matter how severe it was, seizing on the information that the sheet piling had minor leakage before the storm, but ignoring the fact the seepage became worse following the storm. Notably, in their initial December, 2008 rejection letter, the Corps agreed the leaks could lead to the walls' failure.

The LBBLD responded in October, 2009, arguing that the seepage at two different stations grew worse simultaneously, and that worsening took place immediately following Katrina. They also noted that the Corps raised the levee through which the discharge pipes pass (called the 40 Arpent or "back" levee) immediately following Katrina, even though the levee was not part of the federal levee system. They also questioned the Corps position that they (the Corps) had no authorization to repair the seepage issue.

This letter was followed by a meeting two months later between the LBBLD and the Corps. Reading the meeting minutes is gut wrenching, as the locals try to convince the Corps the worsening seepage  is the Corps' responsibility, and the Corps personnel try to use every excuse not to fix it. The locals point out the back levee, which was about 8 feet tall at the time of the storm, saw over 18 feet of surge that poured through the Corps' failed hurricane "protection" levees following Katrina. The minutes put the lie to any idea of "cooperation" between the Corps and the local partners. It is ugly. To drive the knife in fully, the Corps sent a follow up letter in January, 2010 again denying they were responsible. That letter and all the other documents to which I refer above are here.
The Corps' refusal to take responsibility kicked off a multiyear odyssey for the levee district to fix the problem. The next suitable funding source was FEMA, which agreed to fund repairs, implicitly agreeing the problem was caused by the inundation brought on by the failure of the Corps' levees following Katrina's passage. In summer, 2010, the leakage got so bad at station 3 that the east bank board declared an emergency, allowing for expedited repairs. A temporary coffer dam was installed to stem the leaking out the discharge pipes for pumps 2 and 3. FEMA paid for this as well.

After two more years of wrangling, during which a design for the repairs was worked on, the Corps' reversed itself in 2012. In a March 29, 2012 letter to Senator Vitter, then-Acting Corps Commanding General "Bo" Temple wrote:
"St. Bernard Pump Station: Your request is that the Corps reverse its decision on whether or not it may address the current seepage problem and affect a solution in order to prevent abandonment of the $20 million pump rehabilitation project.

"The Corps has reexamined its decision and has determined that it has authority and funding to implement a solution to the seepage problem and will immediately begin to develop plans and specifications to complete repairs to discharge pumped water without further leakage and risk to the pump station."

That is, they admitted their mistake in providing pump stations which didn't work - the seepage was bad enough to consider "abandonment" of stations 2 and 3 - and said they would provide ones that did. They determined they actually did have the authorization to fix problems of their own creation at the St. Bernard pump stations. While another two years passed before a contract was finally issued, and construction took another two years - including a year of delays - to complete, the seepage problems were solved. The Corps spent $10.3 million just in construction funds correcting this mistake, or more than 17 times what the LBBLD has asked for to fix the corrosion error. One must squint to the point of blindness not to see the parallels between paying for the seepage repairs and paying for the pump repairs, especially considering they are at the same place!

The Corps knew

One cannot consider the corrosion problems of the St. Bernard pumps without looking at the events happening simultaneously at the New Orleans pumps. Fortunately, I've been monitoring work on those pumps since 2006, so we can understand how deep the abandonment of the St. Bernard pumps was.

Corrosion was a problem with the New Orleans hydraulic pumps right from the start, with rust appearing on hose fittings just a month after the pumps were installed: (via SCPR Flickr):

Here's a closeup of one of those rusty hose connections (via SCPR Flickr):

The efforts to deal with corrosion on the New Orleans temporary pumps were continuous from installation of those pumps in May, 2006 all the way through today. They have cost over $8.5 million in direct repair expenses, and millions more in oil spill responses and various other needs. The corrosion was severe in many cases, as shown in pictures from the Corps' repair contractor:

The efforts ramped up in 2009, just at the time the St. Bernard pumps - which are identical in all but size - were being feted as a wonderful permanent solution. The New Orleans pump corrosion problem was so well known and so (supposedly?) worrisome to the Corps that it became a talking point for them in their 2009 public lobbying effort to wrangle an agreement from the state to begin construction - using the Corps-preferred, publicly detested "Option 1" design scheme - on the three permanent pump stations in New Orleans. That effort was part of a titanic battle raging from Washington to Baton Rouge to New Orleans between the Corps, Congress, and the local and state authorities over what shape the multimillion dollar permanent pump project would take.

To apply pressure to the locals and the state, the Corps claimed that corrosion would limit the life of the New Orleans temporary pumps or the entire Corps-built temporary pump stations (they deliberately kept this vague) to 5 to 7 years, with the 7 year clock expiring in 2013. So the permanent pumps had to get started as soon as possible. Delay of the permanent pumps, the Corps claimed, could place New Orleans at higher risk of flooding. Implicit was a threat by the Corps not to fix the corrosion problem in the intervening years - a hollow threat since by the time the lobbying effort really got going in the last half of 2009, they had already repaired a number of pumps and were planning to fix many more (Some of the pictures above come from the reports of those 2009 repairs). Nonetheless, the claim of imminent temporary pump self-destruction was used in the Corps' own print materials, at presentations with civic and government leaders, and in interviews with the media.

I wrote quite a bit about that PR effort at the time it was unfolding, and my perspective was skewed by the Corps' reluctance to release any backup for their assertions. Only when I started getting pump repair reports a year later did the corrosion threat - and the Corps' countermeasures - become clear. Of course, by then the Corps had gotten their permanent pump stations agreement signed, so they had no further need to speak about corrosion. Indeed, the topic disappeared from the public conversation, even when the calendar flipped way past the supposed 7 year end of life of the New Orleans pumps.

Taken at face value, the Corps was saying that the New Orleans pumps - without remediation for corrosion - would be useless within seven years of their 2006 installation. Using the same clock, this would mean the St. Bernard pumps, installed in 2009 and unremediated for corrosion since then, would be useless in 2016. That is the exact time they actually started getting pulled out and refitted at LBBLD's expense. In retrospect, one has to wonder how the Corps balanced the cognitive dissonance; on the one hand, they claimed the New Orleans pumps would be dead by 2013 while they were somewhat secretly extending their lives; on the other hand, they lauded their own work in St. Bernard with the exact same pumps while not performing any corrosion refits. It's twisted. But if we put aside the possible motivations, it's useful to examine the numerous mentions of corrosion at the Corps' New Orleans pump stations throughout 2009, because they all apply to the St. Bernard pumps as well.

In June, 2006, the Corps created a new bureaucracy to deal with all the post-Katrina construction they were managing. It was called "Task Force Hope." Part of the Corps' public relations strategy - which after 2007 was heavily carried out by a firm named Outreach Process Partners - was issuance of online newsletters with updates on Task Force Hope's activities. The newsletters were distributed widely in order to get the Corps' message out. As such, they represent an excellent contemporaneous record of the Corps' undistilled PR messages.

In 2009, there were 19 Task Force Hope newsletters issued, of which 5 spoke to the New Orleans pump problem. In April, the talking point linking the temporary structures' limited lifespan to a need for speed on the permanent pumps appeared for the first time:
"Public Safety is the Corps’ No.1 priority. The temporary pump stations must be replaced by the end of their 5- to 7-year project life."

This was just two months after the ribbon cutting at the St. Bernard pump stations.

On May 19, 2009, they said:
"The interim structures provide 100-year level risk reduction, but these structures have a limited project life. The new, permanent pump structures are planned for completion in 2013, coinciding with the obsolescence of the temporary structures."

Then on July 31, 2009, they said:
"Any delay in one part [of the hurricane protection system] puts the community at extended risk from hurricane surge flooding...The clock is ticking. The temporary pumps and closure structures at the three Outfall Canals have a limited service life."

In between those last two statements, they pulled out at least five pumps from the 17th Street site and replaced the pumps' carbon steel pipe with stainless steel. Two more 17th Street pumps would come out later that summer for identical repairs. The pumps were indeed horribly corroded, as can be seen in a report on two of them. The Corps appear to have attempted to conceal the work by keeping the media away from the 17th Street site during their annual pre-hurricane season drill/media opportunity on May 27, 2009.

Revealing the temporary pumps not only could be repaired - possibly extending their lives beyond the 5-7 year lifespan - but were actually being repaired would have undermined the Corps' argument for a quick agreement on the permanent pump stations. So the entire scope of the repair work - rebuilding all 54 of the hydraulic pumps across all three sites, an effort which would take four years - was not mentioned anywhere publicly until I sniffed it out while it happened between 2010 and 2012. The only mention in 2009 was a solitary, brief news story by Maya Rodriguez on WWL-TV on August 5, 2009, which gave hints to what was going on, but kept the narrative decidedly low key. The story, titled "Inspectors find problems with 17th Street canal pumps" is long gone from the internet, but I quoted some of it in an unpublished blog post at the time:
"They are the pumps designed to drain flood water in the event of a tropical storm or hurricane. However, some of the pumps at the 17th Street Canal have been pulled out of the site after inspectors discovered problems with six of the hydraulic pumps there."

Actually, there were at least seven pumps pulled at 17th Street, not six.
"'It's taken the three years, that they've been in service, for this problem to got bad enough to require this attention,' said Ray Newman, Army Corps of Engineers 17th Street Canal captain.

A routine inspection uncovered major corrosion on six of the pumps. Corps officials said a high salinity content in the water may be to blame.

'The 17th Street Canal site is closer to the lake than the other sites, so it's salinity in the water, and combined we actually have a set of six pumps of all of our pumps, only six of them are installed deeper than the others,' Newman said."

Newman's quotes about 17th Street being worse were wrong, as severe damage would also turn up at the other two sites. Also, it is extremely likely a London Avenue pump (E1) was pulled out for corrosion repairs months before Newman was interviewed, as I would uncover in 2011, further undermining the "17th Street-only" facts of the story as presented by the Corps.

Separate quotes from the WWL-TV story appeared on the New Orleans News Ladder:
"'I'm not really surprised. Those pumps really aren't the best pumps that our city has,' said H.J. Bosworth, a civil engineer and the director of research at the watchdog group 'These hydraulic pumps, if they last 20 years, the folks that make them are happy. So, they're not the most durable.'

'Sure, they'll do the job,' he said. 'But I'm glad that someone sounded the alarm and decided to go ahead refurbish these things, re-fit them and do something about the corrosion they found.'"

One must understand this was a single story in a sea of New Orleans news, but I do feel compelled to point to its existence. In the meantime, the full court press by the Corps continued.

On August 14, 2009, they said:
"To provide the permanent replacements of these temporary facilities, the Army and the State must sign a [Project Partnership Agreement] by late August 2009. Any delay in signing the PPA could delay this project past its scheduled construction completion milestone of December 2013, putting the public at extended risk."

Then, with that August deadline passed and no agreement, on October 13, 2009 they had the following to say:
"The Corps continues to remind the public that every delay in the start of construction of the permanent structures delays the completion of robust perimeter protection, and puts the public increasingly at risk when we exceed the service life of the temporary structures."
"'If we start today to replace the temporary structures,' said Dan Bradley, Branch Chief for Permanent Pumps, 'it will take until spring of 2014 to complete the construction work. That will be pushing the temporary pumps' service life to the limit. That’s risky business to the surrounding community.'"

The repairs which had taken place just months earlier were left unmentioned, in favor of a PR narrative. Of course, the newsletters weren't the only source of the Corps' 2009 message on imminent failure of the New Orleans pumps. Corps personnel also appeared in person at various public meetings to hammer home the point, especially in the last half of the year. The civilian head of Task Force Hope, Karen Durham-Aguilera, made appearances at the SLFPA-E in May and before the New Orleans City Council in November. Dan Bradley, quoted above, also showed up at the SLFPA-E September and October board meetings. They sang from the same hymnal, saying the same things written in the newsletters. You can read what I wrote at the time about the October and November appearances.

While all this was happening, the St. Bernard pump station contract was still active. In fact, there was a walk-through of punch list items on October 26, 2009, right between Bradley's and Durham-Aguilera's appearances. However, it appears there was no public linkage whatsoever between the Corps' public stance on the known corrosion of the New Orleans pumps and the private reality that the St. Bernard pumps were just as vulnerable. Going back through the records, I also can't find any clear, unambiguous public acknowledgement of the Corps' then-active efforts to stem corrosion of all the New Orleans pumps. The August 5, 2009 WWL-TV story comes the closest, but it was a tiny, quickly-forgotten blip. The St. Bernard pumps were just left out of the conversation, an unfortunate casualty of the Corps' PR and lobbying strategy.

In sum, the Corps publicly, repeatedly warned about corrosion on the New Orleans pumps for many months, worked to address the problem for years, but did nothing about the St. Bernard pumps which suffered the same difficulties.

The Corps transferred some knowledge from New Orleans to St. Bernard, but not about corrosion

What about in private? Out of the public eye, was there linkage between the New Orleans pumps and the St. Bernard pumps? In at least one key case, there was. It shows the opportunity for transferring the corrosion experience from New Orleans to St. Bernard existed, but was never taken advantage of.

A key component on the New Orleans pumps' drive skids for the hydraulic pumps was redesigned in 2006 and 2007 to prevent failures. Changes were implemented on all the drive skids in New Orleans during that time. The component was an inlet to the core hydraulic pumps on each skid, pumps manufactured by a company called Denison.

Here's a drive skid from the early 2006 period, before the design change (via Flickr):

The photo above was taken by the Corps' chief pump inspector Maria Garzino during her time at MWI's Florida facility in early 2006, while the pump systems were being assembled. That is why the drive skid does not appear in MWI's school-bus yellow.

Zooming into to the skid, we see the Caterpillar engine driving the Denison pumps, the fuel tank providing fuel for the engine, and the hydraulic reservoir holding the hydraulic fluid which ultimately powers the water pump:

Here's the flow of hydraulic oil:

The hydraulic oil comes out of the reservoir at low pressure, and flows into the Denison hydraulic pumps (there are two Denisons per skid) through the inlet line shown in green above. They are driven by the Caterpillar engine through the gearbox.

The Denisons raise the oil to around 3000 pounds per square inch (psi). The pressurized oil flows from the Denison outlets (2 outlets per Denison) to the high pressure (HP) lines through a manifold, shown in red above. The outlets of the manifold are connected by hoses to long lengths of pipe which go out to the water pumps in the canal. At the manifold, there's also a bypass device and a relief valve too, shown in yellow above. They are supposed to keep the pressure of the oil from getting too high by diverting some of the oil off at specified pressure. Further information on how the pressurized oil turns the water pump impellers can be found here.

In 2006, Ms. Garzino found the Denison pumps were sucking in air through the original piping configuration, which had the oil coming out the top of the reservoir through a dip tube inside the reservoir. Air caused damage to the Denison pumps, causing the system to fail. This happened repeatedly on the New Orleans pumps in 2006 and 2007.

During that time, the Corps implemented a temporary fix. They would have operators prime the Denison inlet lines using a vacuum pump and a portable air compressor before the drainage pumps were started (The May, 2006 to May, 2007 pump testing records note priming at pump startup throughout the period). That manual procedure would likely have been impossible to do during actual higher category hurricanes, because the gate structures were to be evacuated during those storms. Who would run the compressor to reprime those lines if something went wrong?

A permanent fix was issued as a contract modification in 2006, but did not get implemented until 2007 under the combined effects of media coverage of the pumps and the publication of the Corps' own internal investigation. That investigation termed the temporary fix "a façade in addressing the real issue" and questioned the year-long delay in implementing the permanent fix. The newer configuration had the reservoir raised and the inlet to the Denison pumps pulling oil from nearer to the bottom of the reservoir, ensuring the inlet was always flooded. Here's a detail from MWI's drawing of the reservoir modification (page 34 of the MWI contract documents):

This was termed a "flooded suction." It was an improvement over the old design, but to my eye it could still get air in it if the oil level got too low in the reservoir due to oil leaks (due to, say, corrosion of the pumps out in the canals). For purposes of this discussion, I'll refer to it as a "semi-flooded suction." The changes to the new semi-flooded configuration began in May or June, 2007, as I wrote about at the time. Here's a 2008 picture of one of the drive skids - the spare at Orleans Avenue - following the design change:

In this photo, the reservoir is raised and the inlets to the Denison pumps are lowered.

Part of the delay in implementing this fix to the original purchase of 34 pumps was due to an objection by MWI. They claimed it was unnecessary. However, the objection couldn't have been particularly strong, because the six "Phase 2" pumps that went in at 17th Street in the fall of 2006 included the reservoir modification:

In 2007, MWI and the Corps went even further with the redesign, turning the inlets to the Denisons into true flooded suctions by having the inlet lines pull hydraulic oil off the very bottom of the raised reservoirs. They implemented this improved design on the drive skids for the 17th Street bridge pumps, which were procured and installed between January, 2007 and June, 2007 (via Flickr):

Following that, the true flooded suction inlet apparently became a standard feature of MWI's drive skids, because they showed up on the St. Bernard pumps. Here's a picture I showed earlier of the St. Bernard station 2 drive skids taken February 4, 2009, five days before the St. Bernard pump station ribbon cutting. They have the raised reservoirs (via Flickr)...

...and the true flooded suction inlet on the Denisons, with hydraulic oil coming off the bottom of the reservoirs (from the February, 2009 Task Force Hope newsletter):

And just so there is no doubt this was a change of design from before the storm, we can examine pictures of the original pre-Katrina St. Bernard skids. As mentioned above, New Orleans engineering firm Waldemar S. Nelson - under contract to the Corps - in October and November, 2005 performed a Damage Survey Report of all the St. Bernard pump stations immediately following the devastation wrought by the federal levee failures. That report is in two volumes - the text, and the pictures.

Here's the relevant shots:

They show before the storm the St. Bernard hydraulic pump skids had their Denison pump inlets oriented to the top. That is, the Corps made a change - a substantial change - from the "pre-storm conditions" of the St. Bernard pumps. This change was intended to ensure the continued operation of the pumps through their intended lifetime.

In addition, this shows there was a transfer of knowledge about the Denison pump inlet orientation from the New Orleans pumps to the St. Bernard pumps long before the St. Bernard pumps were turned over to the locals. That is, behind the scenes, solutions to known problems at the New Orleans pumps were being implemented in St. Bernard. This makes the absolute failure to implement the corrosion refit repairs (replace carbon steel with stainless steel) in St. Bernard even more egregious, since the planned refits were occurring in New Orleans at the same time the St. Bernard contract remained open (2009-2011), and for a year afterward. The Corps knew what needed to happen - was actually doing it on the New Orleans pumps - and didn't do it on the St. Bernard pumps.

Corps paid for other corrosion repairs

Besides the St. Bernard pumps, the Corps has had many other corrosion-related problems which had to be addressed after they turned projects over to the locals. From 2013 through this year, there have been a series of findings of deficiencies in the painting of nearly every major marine closure structure the Corps constructed in the post-Katrina period. Failure of paint in a marine structures can eventually cause structural failure due to corrosion. In all these paint cases, the Corps paid to fix the problem. Never were the locals asked to pay a dime for the remedies.

Bayou Bienvenue vertical lift gate
The bottom ten feet of the Bayou Bienvenue vertical lift gate - part of the Inner Harbor Navigational Canal (IHNC) Surge Barrier - was still being repainted when the entire structure was turned over to the locals in December, 2013. It appears this work was rolled into the original construction contract, since the work was done prior to the turnover. I cannot find the actual contract action for the painting on FPDS-NG, but it was most assuredly pricey.

Here's a couple of pictures of the vertical lift gate on March 24, 2011, the day of its original installation:

Remarkably, we can read the actual paint specification inscribed on the side of the gate:

Here's the gate a little over a year later in 2012, showing damage to the bottom section of the gate:

And then in 2015, with the bottom part painted:

IHNC Surge Barrier bulkheads
Other very large parts of the IHNC Surge Barrier had to be repainted well after being turned over to the locals, specifically the stackable bulkheads for the sector gate. The bulkheads are removable steel structures intended to be used during maintenance of the sector gate. They would be placed on either side of the gate, allowing the space between to be pumped out and opened up for maintenance of the gates.

Here's a Corps picture of the bulkheads in their storage location behind the IHNC surge barrier sector gate:

And here's a closeup shot,, taken from the Corps' July, 2013 report of a contracted inspection of the bulkheads' painting system:

According to the report (compiled six months before the Corps turned the barrier over to the locals) the speckles on the faces of the bulkheads in the photo are barnacles, resulting from extended immersion. It seems four of the eight bulkheads, including those shown above, were in use for a while during the sector gate's construction. According to the report, the paint system on those immersed bulkheads had "failed on all counts." The coating system chosen for the bulkheads was apparently not suitable for long term submergence, an odd choice given their intended use. The ones that hadn't been in the water didn't do too well either, suffering from improper application and improper repair methods. That is, the contractor did not follow the specifications and there was inadequate quality control and assurance by the Corps.

In June, 2014, six months after the turnover, the Corps issued a contract to Quality First Construction, LLC for sandblasting and repainting the bulkheads. Quality First's contract's eventual value would be slightly over $3 million. The actual blasting and painting would be performed by Crescent Coatings of Gibson, LA. Pictures of the repainting reside on Quality First's website. Those pictures include people so the enormous scale of these bulkheads (and the cost of their repainting) can be appreciated.

Seabrook vertical lift gates
The huge vertical lift gates at the Seabrook closure structure on the New Orleans lakefront also had faulty painting and corrosion. In fact, the same day the Corps turned the Seabrook complex over to the locals, December 3, 2013, the state had a paint expert out to look at the gate. That expert's report pointed out a number of flaws, the most serious of which was possible delamination between coats of paint. The report said the delamination could represent "a significant risk to the long-term performance of the coating system."

In August, 2014, the Corps had its inspectors look at the vertical lift gates and they too noticed flaws, including the intercoat delamination. A third visit took place in November, 2014, documenting further damage.

Here's a couple of Corps pictures of the Seabrook complex, including one showing just how big the vertical lift gates are:

The report of the state paint inspector contains dozens of examples of painting problems. I am not a paint expert by any means, so I cannot judge how bad these are. But I found a few particularly interesting.

This shows the delamination referred to in the report as a "significant risk:"

This shows a closeup of an area which had been touched up:

The paint had been applied directly over barnacles and is blistering, clear violations of specifications for adequate surface preparation during repairs. The next photo shows what was found under the blistered area:

The base metal underneath is rusting. The report recommended all touched up areas - and there were many of them - be completely repainted, with a certified paint inspector present.

All the reports eventually led to complete repainting of the vertical lift gates under a nearly $1.2 million Corps-funded contract to Pontchartrain Partners, LLC, awarded in December, 2015. The lift gates were removed and taken offsite to be repainted, and were reinstalled just before the start of hurricane season in June, 2016.

Caernarvon sector gate
Finally, the large sector gate on the Caernarvon canal - also in St Bernard Parish - had its paint start peeling off "in sheets" months after its August, 2013 handover to the locals. Here's a Corps picture of the Caernarvon gate:

Here's the paint failures shown in a March, 2014 report by the state's inspectors:

In October, 2014, the Corps' own paint experts determined the problem - delamination of the top coat of coal tar epoxy - was caused by inadequate surface preparation by the Corps' construction contractor. Nearly half the coating thickness was gone in these areas. A contract for nearly $1.5 million was awarded to Quality First Construction (the same firm as got the IHNC bulkhead contract) in December, 2015. After numerous delays, the Corps started sandblasting and repainting the entire thing in January of 2017. That work is due to finish before the start of the 2017 hurricane season.

A final note on all these paint failures... In many cases, the paint experts subcontracted by the Corps or directly employed by the Corps themselves brought into question the specification, the application, and the contractor oversight on these jobs. One must wonder if these structures' paint systems were so poorly engineered and applied, requiring millions of dollars of rework within a scant period of time of their original completion, what does that say about the rest of the structures?

Corps paid for other types of repairs

The listing above of paint problems are hardly the only post-Katrina design or construction errors later corrected at the Corps' expense. Here's a small sample of some more:

- In 2008, painting the rusty carbon steel hydraulic pipes at the outfall canal closure structures for $1.9 million. Yet more rust.

- In 2013, having to pile more dirt on 3250 feet of a super-duper levee (LPV-109.02a) in New Orleans East that settled 2.5 feet below its design height within months of its supposed completion.

- In 2014, having to also put up to an extra foot of dirt on top of 1000 feet of another New Orleans East levee (LPV-111)...
No, sorry, it was 4000 feet...
No, sorry, it was 6000 feet...
No, sorry again, it was 9000 feet...
No, sorry once again, it was 10,400 feet at a cost of at least $1 million. The Corps is not great about letting the locals know things in a timely fashion.
Also, the entire length of the levee was 28,000 feet, so almost half of the $350 million LPV-111 levee sunk below its design height before it was even turned over to the locals.

- In 2014, having to re-remediate the 17th Street and London Avenue outfall canals after having done so once before in 2011. That cost over $13.6 million.

That doesn't even get into the biggest one: the possibility that the entire eastern flank of the hurricane protection system, known as the Chalmette Loop, is not up to snuff due to corrosion and bending of subsurface sheet piles. There's currently an independent review of that possibility taking place. The Corps could be on the hook for extensive future repairs.

The point is the Corps went back and fixed flawed projects after they were turned over to the locals all the time.

Rebutting the Corps' excuse for not paying

We've gone through arguments the St. Bernard Parish pump station 2, 3, and 5 pump corrosion repairs - performed after the Corps turned over the pump stations to the locals - should be reimbursed by the Corps:

- The Corps already paid for over $10 million of repairs at the very same St. Bernard pump stations after they were turned over to the locals and the original construction contract was completed. They decided that work was authorized after previously saying it was not.
- The Corps knew the identically designed New Orleans pumps were corrosion-prone, and even used that fact as a key talking point in their high-profile 2009 public effort to force the state and locals into an agreement for the permanent pumping stations. At the same time, they made no mention of the St. Bernard pumps, which were just as corrosion-prone. This has been borne out by the St. Bernard pumps failing on nearly the exact schedule the Corps claimed for the unrepaired New Orleans pumps: five to seven years after installation.
- As a result of that knowledge, the Corps engaged in an eight-year long, $8.5 million effort (with mixed success) to address corrosion at the New Orleans pumps, while never doing anything to address the completely predictable corrosion on the St. Bernard pumps.
- The Corps was privately willing to transfer some technical knowledge about the New Orleans pumps to the St. Bernard pumps, but refused to do so for corrosion prevention.
- The Corps paid tens of millions of dollars for tons of other repairs on other projects, most of them after they were handed over. Many of them were repairs due to corrosion, just like the St. Bernard  pump repairs.

All of these make sense, but they do not directly address the Corps' own language in their terse rejection of the LBBLD's reimbursement request:
"The Cooperation Agreement and Project Information Report only allowed USACE to 'Repair and rehabilitate the pump stations to pre-storm conditions.'"

From what I've found, the "Cooperation Agreement" refers to an agreement to allow the Corps to raise the top of the 40 Arpent levee by a couple of feet in 2005. In fact, by definition, raising a levee is not restoring something to "pre-storm conditions." So we will move past the mention of the Cooperation Agreement, since it does not seem to bear directly on the pumps.

The Corps' quote about "repair and rehabilitate to pre-storm conditions" is taken from the Project Information Report (or "PIR") for the St. Bernard pump stations, issued January 22, 2006. The PIR is the governing document for the repair and rehabilitation of all eight St. Bernard stations. It was prepared in the months following Katrina, incorporating the findings from the Damage Survey Reports mentioned above and proposing solutions. It also includes the legal justifications for the Corps' decisions and actions. PIR's were prepared for each Katrina-affected parishes' (Jefferson, Orleans, St. Bernard, and Plaquemines) pump station repairs, as well as for the outfall canals and other New Orleans work.

I have the original version of the St. Bernard PIR. I do not know if it was updated. However, the work laid out in the PIR is what eventually happened, so I feel comfortable referencing it.

The PIR did not simply describe simple repairs to the pump stations, restoring them to their pre-storm conditions. In fact, it laid out four different options, ranging from "no action" (standard for these types of government reports) to what was actually done. The two options of interest are numbers 3 and 4.

Option 3 was simple repair and rehabilitation for seven of the eight stations. Everything would just be replaced in kind. At station 2, due to the extensive damage to the building and the cost involved in replacing it in kind, the pumping equipment would be replaced with weather-proofed equipment and small control room would be constructed.

The Corps' rejection of St. Bernard's request for reimbursement, in its brevity, implies this is what was done for the pump stations. In fact, it was the far more complex option 4 which was actually chosen. Option 4 did indeed call for simple repair of five of the stations to their pre-storm conditions. Those were the stations that mostly had elevated equipment already.

However, the three stations of interest to us - numbers 2, 3, and 5 - were assigned much more extensive work which went far beyond replacement in kind. Stations 2 and 3 each had one direct drive pump at the time. They were to be replaced with hydraulic pumps. The remainder of the pumps at each station (three pumps at station 2, two at station 3 and all six at station 5) were hydraulically-driven, and that is what the Corps decided to stay with. Finally, at all three stations, the drive skids for all pumps would be placed on elevated platforms 8 to 10 feet above the existing concrete slabs to keep them above future flood water. So the quote in the Corps' reimbursement request is taken out of context, and does not directly describe the work done on stations 2, 3, and 5. In fact, within option 4, the phrase "repair and rehabilitate" is not even used for stations 2, 3, and 5.

So the work on stations 2, 3, and 5 was quite far from just "repair and rehabilitation to pre-storm conditions." In fact, the stations were effectively new structures, with new equipment and - in the case of stations 2 and 3 - a radically different design. That radical difference was also the case for the Denisons, which were installed literally upside down relative to their "pre-storm conditions." Again, here's the pump engines before the stations were rebuilt:

And here's the pump engines after:

They did not return the pump engines to their "pre-storm conditions" because if they had, they would not have worked! If the Corps could make a change like that, why couldn't they make the pumps more resistant to corrosion, especially when they absolutely knew they would rust to the point of failure quite quickly?

In reality, the Corps has wide latitude to do what it wants. Much of the work post-Katrina - including all the work on all the pump stations across all four parishes - was conducted under a "one time deviation" to the Corps' own rules, an exception granted to the Corps by the Corps. That exception was used to justify their reversal for the seepage repairs at stations 2 and 3, over six years after the storm. Nothing changed in the text of the deviation between when it was issued in October, 2005 and when the Corps decided it actually did apply in 2012. The only thing that changed was the Corps' minds.

If they could change their mind then, why not now? The reasons they should, as I've demonstrated above, are numerous, compelling and now out in the public domain. Let's hope they do the right thing and reimburse the Lake Borgne Basin Levee District.

Many but not all of the prior entries in this series are linked throughout the report above. For convenience, here's the complete set:

Imminent, originally posted May 13, 2010, covers the halting first repairs in 2009

How did the pumps get from..., originally posted May 27, 2010, covers the original 2006-07 decisions on corrosion

This year's scramble, originally posted June 3, 2010, covers the spring 2010 repairs

Worse than previously known, originally posted June 11, 2010, also covers the spring 2010 repairs

Corps of oil, Part 1, originally posted June 20, 2010, covers oil spills from 2006 and early 2007.

Corps of oil, Part 2, originally posted June 21, 2010, covers oil spills in later 2007

Corps of oil, Part 3, originally posted June 22, 2010, covers oil spills from 2008 through 2010

No urgency, originally posted August 18, 2010, reports on repairs during summer, 2010

Quick update (with pretty new trees!), originally posted August 30, 2010, gives a summary of corrosion repair events to date at the fifth anniversary of Katrina, as well as revealing the Corps' wasteful spending on tree planting around the closure sites

The latest on lakefront pump repairs, originally posted March 23, 2011, looks at the repairs from summer 2010 through early 2011

The 2011 pump rebuild scramble, originally posted May 9, 2011, covers the spring, 2011 repairs

Brief update on London Avenue pumps, originally posted June 24, 2011, also covers the spring, 2011 repairs

Finally, originally posted July 13, 2011, provides an update on receipt of FOIA-requested documents, all of which were eventually transmitted by the Corps

No surprise, originally posted July 15, 2011, looks at the summer, 2011 re-repairs of pumps also pulled out in 2009

No surprise, Part 2, originally posted July 20, 2011, covers possible contamination of the pumps' hydraulic fluid with water

2011-12 rusty pump update, originally posted July 9, 2012, chronicles repairs to ten 60" pumps between August, 2011 and March, 2012. This post was updated in February, 2017.

Birth of a no bid contract, originally posted July 28, 2012, uses internal Corps emails from November and December, 2009 to tell the story of how Healtheon's second contract was created in the 2009-10 hurricane off season

Rusty pumps - the summing up, originally posted August 26, 2012, just days before Hurricane Isaac provided the most serious test of the New Orleans pumps, provides a wrap up of the whole corrosion saga and details pumps pulled out in 2012. This post was updated in January 2017.

Rusty pumps keep rusting, Corps yawns, originally published May 31, 2015, covers the details of pumps pulled from 2013 through 2016. This post was updated in January and February, 2017.


  • Great post. Thanks.

    Good coatings require a lot of different factors for success. A marine grade coating system can last at least 5 years if the surface is properly prepared and the coating is properly applied.

    The single biggest factor on the success or failure of a coating system is the surface prep. Poor surface prep will sabotage the best paint system.

    The painted-over barnacles are an example of a either pencil-whipping the coating reports or just not giving a shit you're doing a terrible job. No serious paint shop can look at that and say with a straight face that they did a quality job.

    This is probably the painter's cousin:
    (A personal favorite of mine).

    Common themes of failures with the Corps:
    * Galvanic corrosion? What's that?
    * Civil Engineers trying to do Mechanical work (HPU's are big time MechE scope and I'll be they had a civil supervising that).

    Also, I hate hydraulics, as a general rule. Hydraulics, particularly hydraulic systems with lots of connections, aren't that reliable. Direct drive electric is always more reliable, dollars to donuts.


    By Anonymous Clay, at May 14, 2017 11:55 AM  

  • One more practical comment: a piece that I always include in contracts for important painting is a NACE certified paint inspector signing off on the paint reports.

    The paint booth is usually down the totem pole of a fabricator. Some paint inspectors fail downward into the job. A non-NACE paint inspector that pencil whips a report and walks down the road and gets a new job. A NACE guy can lose his certification and have it follow him if it's apparent he pencil whipped a report.

    Always insist on the DFT reports (with humidity data, etc.) be signed off by a NACE certified paint inspector before the job is turned over to the client.

    Failure to do that on a job where the coating system is an integral part of the work is ignorance or negligence.

    By Blogger Clay, at May 14, 2017 12:05 PM  

  • That photo with the paint spec... That's a very good paint spec. Maybe even a little overboard.

    SSPC SP-5 is a white metal blast, as good as you get (if they actually do it). You must then come back very quickly to take advantage of that blast prep with your primer. In South Louisiana, you'll have scaling start in minutes.

    To save a couple nickels, SSPC SP10 (near white metal) yields nearly equal performance to SSPC SP-5 at a fraction of the cost.

    It also looks like there's quite a few number of coats. The painter might have had trouble keeping his total films underneath the net cap. They have a good zinc rich primer as a base.

    The devil is in the details....

    By Blogger Clay, at May 15, 2017 9:14 AM  

  • Much of the components contain very hot hydraulic fluid. The coating selected should obviously be selected for these high temperature and their cyclical nature. After failure of the coating, the high heat further accelerates the corrosion rate.

    By Anonymous Anonymous, at May 30, 2017 12:33 PM  

  • We were told yesterday by a Parish official that the pumps have been repaired. Well, not actually repaired, but "up and running." Can you confirm whether they have in fact been repaired or is this just political speak???

    "Kerri Callais
    OK St. Bernard residents just want to clarify our status since I'm being bombarded with an article about our pumps. The information in this article is out dated. ALL OF OUR PUMPS ARE UP AND RUNNING! ...."

    By Anonymous Anonymous, at June 21, 2017 6:55 AM  

  • Kerri Callais, Please prove it is up to date with photos, bc from the pictures takes on 6/21/17 is proves 2 out of 3 stations are not working. Pump Station 3 is not even connected and Pump Station 2 is not running. We just want the truth even if it's our own fault for not voting funds for maintenance since we not explained what we were voting for.

    By Anonymous Anonymous, at June 22, 2017 11:06 AM  


    By Anonymous Anonymous, at June 22, 2017 11:11 AM  

  • Matt McBride sighting!!!

    By Blogger Clay, at August 11, 2017 9:04 PM  

Post a Comment

<< Home

Go to older posts Go to newer posts