The 2011 pump rebuild scramble
This entry was updated July 14, 2011 to bring it in line with revisions made elsewhere on this blog. These revisions came about due to a large pump repair document release in June and July, 2011 in response to FOIA requests. As such, the tense is an odd mix of tone, mixing voices from both the original date of composition on May 9, 2011 and the later revision date of July 14, 2011.
Last spring, we looked at a flurry of hydraulic pump repair activity at the New Orleans lakefront gates. This spring, there seems to be another such push.
Third Healtheon contract, task order #1
At the end of the last update, we noted that two pumps at the London Avenue site - E1 and E5 - had been removed and taken to Conhagen's shop for rebuilding around March 22, 2011, based on an oil spill report from that day. Here's pictures of the empty E1 and E5 slots in early April:
As of late April, E1 and E5 were back in at London Avenue:
This work was performed under $598,101.68 task order #1 to Healtheon contract W912P8-11-D-0015, the third pump repair contract. As described in the previous post, that task order also called for the repair of 17th Street pumps W4 and W7:
Modification 1 to task order #1, issued for $12,433.26 on June 8, 2010 - after the work had been completed - accounted for the bearing and seal work that has become standard on all these repairs:
It also allowed for a smaller crane at the London Avenue site.
The Conhagen repair reports for these four pumps (London Avenue E1 and E5 report, 17th Street W4 and W7 report) include the now-usual photos of very, very, very rusty pumps. And they also help solve a mystery.
The pictures of London Avenue pump E1 include this shot:
Notice the multiple zinc anodes, and how they are fastened to the pump's suction bell. The original pump order included only a single pair of anodes for each pump, and they were fastened in a different manner. Here's pictures of the pumps in 2006, with their original anodes:
And here's one of the 2009 Conhagen-repaired pumps going back in the water:
All this indicates this pump had previously been removed and partially repaired. This helps explain the May 22, 2009 spill report I noted over a year ago in Corps of oil, Part 3:
May 22, 2009 at London Avenue
Quantity released: 40 gallons (from QRI task order)
NRC report: No
LDEQ report: No
Description: From $5399.16 task order #4 to the second QRI contract: "Provide labor, material, and use of eqipment [sic] to clean up about 40 gallons of hydraulic fluid that spilled from pump 1E at London Avenue Canal."
At that time, I noted that E1 was not among the four pumps that had been pulled out at the London Avenue site before June, 2010. Indeed, I don't know of any contractual evidence of the repair of London pump E1, but it obviously happened.
It was probably repaired by the Corps themselves, before they started sending pumps to Conhagen through the first Healtheon contract. And since this happened so early on in the repairs (May, 2009), it also likely had the very limited scope of work involving just replacement of the carbon steel pipes inside the pump bore with stainless steel, and simple cleaning of the external piping and coolers followed by a new coat of paint.
Indeed, other pictures of E1 from the 2011 repair report back this up. The interior pipes are not rusted at all, indicating they're stainless:
While the external piping and coolers look like they've taken some serious corrosion damage since the first round of repairs in June, 2009:
Five other 60" pumps - all at 17th Street - also received this misguidedly small set of repairs before the Corps realized they needed to do more.
This makes London Avenue pump E1 the first 60" pump to be pulled out twice, doubly compounding the errors in corrosion judgement made in 2006 and 2007. This is a massive waste of money. It is extremely likely this pattern will be repeated on the 17th Street pumps repaired in 2009 as well.
Anyhow, the other three pumps rebuilt under this task order had all the normal rust damage to their Rineer motors and attached piping:
And to their oil coolers and external piping:
Conhagen completed all the repairs (it's unclear whether they got new or refitted Rineer motors) in April, 2011. London Avenue pumps E1 and E5 went back in the water April 27, 2011. 17th Street pumps W4 and W7 were reinstalled three days later, on April 30, 2011.
Third Healtheon contract, task order #2
Work continued this spring at London Avenue, with pumps W5 and W6 coming out:
We can tell they're out because we can see the bottom flanges of the elbows unattached to any pumps below them. We also don't see the distinctive piping extensions added to the pumps in 2007, as we can on the adjacent pump W4.
This work likely took place on April 26th, since there was a spill report from London Avenue that day:
The Corps claimed 3 gallons for this spill. Again, like the last spill report in March, I find this implausibly tiny, and a likely underreporting. However, at least it was reported, unlike dozens of others over the last five years.
There is another possibility for the source of this spill. It could have resulted from the reinstallation of London Avenue pumps E1 and E5. The acceptance testing for those pumps occurred on April 27th, the day after this spill, but they could have been getting put back in on the 26th. Without more detail, it's unfortunately impossible to know.
Meanwhile, there was also work at 17th Street, with E4 coming out on May 2nd:
The corrosion on the outer pump housing and the oil coolers is obvious:
The corrosion damage to the innards of the pump, including the Rineer motor, is likely similarly severe.
E3 was also due to come out that day, as evidenced by the missing flange bolts on its elbow:
E3 and E4 were untouched before this latest activity.
All four of these pumps came out under $523,862.74 task order #2 to the third Healtheon/Conhagen contract, issued April 25, 2011. The text of that task order reads,
However, as the Corps' self-imposed June 1st deadline for the completion of the 100 year protection system around the New Orleans area approached, apparently someone wanted results faster. Modification 1 to task order #2, issued after the fact on June 8, 2011 did two things:
1) It moved the deadline for the work on these four pumps up to May 31, 2011.
2) It added the now standard suite of repairs to bearings and seals:
It didn't do these things cheap. It cost $139,913.16, bringing the total value of this task order to $663,775.90.
My FOIA request also produced the Conhagen repair reports for both pairs of pumps (repair report for 17th Street pumps E3 and E4 and repair report for London Avenue pumps W5 and W6)
As usual, these reports are full of troubling photos. Here's a before and after of the seal flange on London Avenue pump W6:
Here's the seal flange on 17th Street pump E3:
I'm assuming this photo was taken to show the oil that had gotten past the seal and had spread. When they removed some of the outer housing, presumably this is what they found. This is bad, because big oil leaks like this mean lubrication for the bearings is gone, potentially causing pump failure.
Anyhow, Conhagen accelerated the repairs (hopefully completing them properly in the process). The London Avenue pumps went back in the water May 21, 2011. Here they are in place on June 1, 2011:
The 17th Street pumps were back in May 24, 2011, beating the Corps' artificial deadline.
[I realize this information and the charts below include information that occurred after the date of this post, but that's because I'm writing it after I got information that was missing at the time I originally composed it. It's timey-wimey.]
Stepping back a bit, this means the Corps has pulled ten pumps in five pairs for planned repairs (the planned repairs always occur in pairs) since the 2010 hurricane season:
London Avenue pumps E2 and E6
London Avenue pumps E1 and E5 (second round of repairs for E1)
London Avenue pumps W5 and W6
17th Street pumps W4 and W7
17th Street pumps E3 and E4
With the receipt of all the documentation on these pumps, we can update the pump status for the London Avenue site:
We can also update the 17th Street site. It is important to realize that the five 60" pumps reworked in 2009 with an incomplete set of repairs are extremely liable to oil spills.
The Orleans Avenue site has had nearly zero attention paid to it, with only a single pump pulled out in the last two years, and even that was only in response to an oil spill, not as a planned repair:
So, even after all the pumps we believe have gone for repairs, there will still be twelve 60" pumps still untouched - including 9 of the 10 at the Orleans Avenue site:
17th Street: W2, W3
London Avenue: W2
Orleans Avenue: E1, E2, E3, E4, E5, W1, W2, W3, W5
and another five with incomplete repairs:
17th Street: E5, E7, W8, W9, W10
which are likely to break again very soon.
And it's taken two years to get to this point. Since the Corps only pulls pumps out during hurricane season if they break, it's either unlikely (in the Corps' view) or likely (in view of all the photos of broken pumps I've published over the last year) we'll see more activity before the end of the 2011 hurricane season.
Matt Davis and Steve Beatty of The Lens contributed to this report.
Last spring, we looked at a flurry of hydraulic pump repair activity at the New Orleans lakefront gates. This spring, there seems to be another such push.
Third Healtheon contract, task order #1
At the end of the last update, we noted that two pumps at the London Avenue site - E1 and E5 - had been removed and taken to Conhagen's shop for rebuilding around March 22, 2011, based on an oil spill report from that day. Here's pictures of the empty E1 and E5 slots in early April:
As of late April, E1 and E5 were back in at London Avenue:
This work was performed under $598,101.68 task order #1 to Healtheon contract W912P8-11-D-0015, the third pump repair contract. As described in the previous post, that task order also called for the repair of 17th Street pumps W4 and W7:
CLIN 1001A - Remove, recondition and reinstall pump #4W and #7W at 17th Street Canal @ $262,376.76.
CLIN1001BC - Remove, recondition and reinstall pump #1E and #5E at London Avenue Canal @ $334,724.92. Contractor may opt to use 240-ton crane at this site in lieu of 500-tonn crane as stated in contract."
Modification 1 to task order #1, issued for $12,433.26 on June 8, 2010 - after the work had been completed - accounted for the bearing and seal work that has become standard on all these repairs:
"Add the following additional work for all four (4) pumps:
Repair Mechanical Seals at subcontractor shop (4 Each)
Remove Pump Shaft Bearings
Replace Radial bearings Impeller End (4 Each)
Replace Radial Bearings Motor End (4 Each)
Replace Thrust Bearings (4 Sets)
Replace bearing Spacers (4 Each)
Inspect Shafts and Housing
Mount New Bearings
Clean All Parts for Assembly
Reinstall Pump Shafts"
It also allowed for a smaller crane at the London Avenue site.
The Conhagen repair reports for these four pumps (London Avenue E1 and E5 report, 17th Street W4 and W7 report) include the now-usual photos of very, very, very rusty pumps. And they also help solve a mystery.
The pictures of London Avenue pump E1 include this shot:
Notice the multiple zinc anodes, and how they are fastened to the pump's suction bell. The original pump order included only a single pair of anodes for each pump, and they were fastened in a different manner. Here's pictures of the pumps in 2006, with their original anodes:
And here's one of the 2009 Conhagen-repaired pumps going back in the water:
All this indicates this pump had previously been removed and partially repaired. This helps explain the May 22, 2009 spill report I noted over a year ago in Corps of oil, Part 3:
May 22, 2009 at London Avenue
Quantity released: 40 gallons (from QRI task order)
NRC report: No
LDEQ report: No
Description: From $5399.16 task order #4 to the second QRI contract: "Provide labor, material, and use of eqipment [sic] to clean up about 40 gallons of hydraulic fluid that spilled from pump 1E at London Avenue Canal."
At that time, I noted that E1 was not among the four pumps that had been pulled out at the London Avenue site before June, 2010. Indeed, I don't know of any contractual evidence of the repair of London pump E1, but it obviously happened.
It was probably repaired by the Corps themselves, before they started sending pumps to Conhagen through the first Healtheon contract. And since this happened so early on in the repairs (May, 2009), it also likely had the very limited scope of work involving just replacement of the carbon steel pipes inside the pump bore with stainless steel, and simple cleaning of the external piping and coolers followed by a new coat of paint.
Indeed, other pictures of E1 from the 2011 repair report back this up. The interior pipes are not rusted at all, indicating they're stainless:
While the external piping and coolers look like they've taken some serious corrosion damage since the first round of repairs in June, 2009:
Five other 60" pumps - all at 17th Street - also received this misguidedly small set of repairs before the Corps realized they needed to do more.
This makes London Avenue pump E1 the first 60" pump to be pulled out twice, doubly compounding the errors in corrosion judgement made in 2006 and 2007. This is a massive waste of money. It is extremely likely this pattern will be repeated on the 17th Street pumps repaired in 2009 as well.
Anyhow, the other three pumps rebuilt under this task order had all the normal rust damage to their Rineer motors and attached piping:
And to their oil coolers and external piping:
Conhagen completed all the repairs (it's unclear whether they got new or refitted Rineer motors) in April, 2011. London Avenue pumps E1 and E5 went back in the water April 27, 2011. 17th Street pumps W4 and W7 were reinstalled three days later, on April 30, 2011.
Third Healtheon contract, task order #2
Work continued this spring at London Avenue, with pumps W5 and W6 coming out:
We can tell they're out because we can see the bottom flanges of the elbows unattached to any pumps below them. We also don't see the distinctive piping extensions added to the pumps in 2007, as we can on the adjacent pump W4.
This work likely took place on April 26th, since there was a spill report from London Avenue that day:
"CALLER IS REPORTING A DISCHARGE OF HYDRAULIC OIL FROM A HYDRAULIC PUMPS OVER THE WATER AT THE PUMPING STATION DUE TO MAINTENANCE ACTIVITY (MAINTAINING THE PUMPS)."
The Corps claimed 3 gallons for this spill. Again, like the last spill report in March, I find this implausibly tiny, and a likely underreporting. However, at least it was reported, unlike dozens of others over the last five years.
There is another possibility for the source of this spill. It could have resulted from the reinstallation of London Avenue pumps E1 and E5. The acceptance testing for those pumps occurred on April 27th, the day after this spill, but they could have been getting put back in on the 26th. Without more detail, it's unfortunately impossible to know.
Meanwhile, there was also work at 17th Street, with E4 coming out on May 2nd:
The corrosion on the outer pump housing and the oil coolers is obvious:
The corrosion damage to the innards of the pump, including the Rineer motor, is likely similarly severe.
E3 was also due to come out that day, as evidenced by the missing flange bolts on its elbow:
E3 and E4 were untouched before this latest activity.
All four of these pumps came out under $523,862.74 task order #2 to the third Healtheon/Conhagen contract, issued April 25, 2011. The text of that task order reads,
"CLIIN 1001A: Removal, reconditioning and reinstallation of pump #3E and #4E at 17th St. Canal Interim Closure Structure.
CLIN 1001BD: Removal, reconditioning and reinstallation of pump #5W and #6W at London Ave. Canal Interim Closure Structure.
All work shall be completed by 17 June 2011."
However, as the Corps' self-imposed June 1st deadline for the completion of the 100 year protection system around the New Orleans area approached, apparently someone wanted results faster. Modification 1 to task order #2, issued after the fact on June 8, 2011 did two things:
1) It moved the deadline for the work on these four pumps up to May 31, 2011.
2) It added the now standard suite of repairs to bearings and seals:
"The Scope of Work is also modified to add the following additional work for all four (4) pumps:
Repair Mechanical Seals at subcontractor shop (4 Each)
Remove Pump Shaft Bearings
Replace Radial bearings Impeller End (4 Each)
Replace Radial Bearings Motor End (4 Each)
Replace Thrust Bearings (4 Sets)
Replace bearing Spacers (4 Each)
Inspect Shafts and Housing
Mount New Bearings
Clean All Parts for Assembly
Reinstall Pump Shafts"
It didn't do these things cheap. It cost $139,913.16, bringing the total value of this task order to $663,775.90.
My FOIA request also produced the Conhagen repair reports for both pairs of pumps (repair report for 17th Street pumps E3 and E4 and repair report for London Avenue pumps W5 and W6)
As usual, these reports are full of troubling photos. Here's a before and after of the seal flange on London Avenue pump W6:
Here's the seal flange on 17th Street pump E3:
I'm assuming this photo was taken to show the oil that had gotten past the seal and had spread. When they removed some of the outer housing, presumably this is what they found. This is bad, because big oil leaks like this mean lubrication for the bearings is gone, potentially causing pump failure.
Anyhow, Conhagen accelerated the repairs (hopefully completing them properly in the process). The London Avenue pumps went back in the water May 21, 2011. Here they are in place on June 1, 2011:
The 17th Street pumps were back in May 24, 2011, beating the Corps' artificial deadline.
[I realize this information and the charts below include information that occurred after the date of this post, but that's because I'm writing it after I got information that was missing at the time I originally composed it. It's timey-wimey.]
Stepping back a bit, this means the Corps has pulled ten pumps in five pairs for planned repairs (the planned repairs always occur in pairs) since the 2010 hurricane season:
London Avenue pumps E2 and E6
London Avenue pumps E1 and E5 (second round of repairs for E1)
London Avenue pumps W5 and W6
17th Street pumps W4 and W7
17th Street pumps E3 and E4
With the receipt of all the documentation on these pumps, we can update the pump status for the London Avenue site:
We can also update the 17th Street site. It is important to realize that the five 60" pumps reworked in 2009 with an incomplete set of repairs are extremely liable to oil spills.
The Orleans Avenue site has had nearly zero attention paid to it, with only a single pump pulled out in the last two years, and even that was only in response to an oil spill, not as a planned repair:
So, even after all the pumps we believe have gone for repairs, there will still be twelve 60" pumps still untouched - including 9 of the 10 at the Orleans Avenue site:
17th Street: W2, W3
London Avenue: W2
Orleans Avenue: E1, E2, E3, E4, E5, W1, W2, W3, W5
and another five with incomplete repairs:
17th Street: E5, E7, W8, W9, W10
which are likely to break again very soon.
And it's taken two years to get to this point. Since the Corps only pulls pumps out during hurricane season if they break, it's either unlikely (in the Corps' view) or likely (in view of all the photos of broken pumps I've published over the last year) we'll see more activity before the end of the 2011 hurricane season.
Matt Davis and Steve Beatty of The Lens contributed to this report.
Labels: Rusty pumps
posted by mcbrid35 at
10:00 AM
0 Comments:
Post a Comment
<< Home