Would you buy a used car from these folks?
[UPDATE, June 4, 2010: Many of the conclusions in this post have been superceded by my May 27, 2010 post, "How did the pumps get from...". I have seeded updates through the post to provide perspective.]
There's been a lot of talk since the story about the defective MWI pumps broke on March 12. And while the media and the Corps have covered various aspects of the story around the pumps, no one has publicly looked at the system itself to figure out exactly what problems may still need to be addressed. Fortunately, nearly everything about the system is fully on display for anyone to walk up to and examine.
Over the next few posts, I'm going to be concentrating on specific deficiencies in the design of the pumping system. Some relate to fabrication, some relate to engineering design, and some relate to installation. This is nitty-gritty stuff that engineers love (or should love) to dig into. Keep in mind that engineering is not a static field - it is subject to opinion on how to do things the best way. These are my opinions, backed up with my experience as a mechanical engineer and consultations with others familiar with similar installations. Some of this may seem a bit didactic, but I'll always show that there's a point.
I'm going to start with the pump units themselves. As a refresher, you may wish to read my earlier post about the fundamentals of how the pumps and their drive units are set up.
Out at the Orleans canal floodgates, all ten pumps have been pulled in anticipation of refits by MWI. The five west pumps were pulled Wednesday, March 22. The five east pumps have been on the deck since last October. They're all up on the deck on the east (Marconi Drive) side, and anyone can go up and check them out. So I did so last Sunday. So did at least five other people while I was there, including two joggers with their dogs.
Frankly, the workmanship is ... not great. Let me lead off by showing you something that I had to laugh at to keep from crying:
Yes, that's duct tape wrapped around a hose meant to hold at least 1000 psi of hydraulic oil (I think - I'm not sure of the pressure on the outlet of the coolers). To top it all off, that particular fitting sits below the waterline when the pump is in the canal. Are they kidding us with this?
[UPDATE: This hose fitting appears to be one of the ones on the east pumps that were sandblasted and painted during October, 2006 as a first cut at repairing corrosion damage on the hose fittings. The duct tape may have been acting as masking for the painting and sandblasting, or it may have been part of the protection. One can see what also appears to be duct tape on other painted fittings in this shot from October, 2006:
There's more details on this hose fitting painting scheme and how it fits into the overall corrosion chronology in my May 27, 2010 post, "How did the pumps get from...". I don't think this particular method of corrosion repair ever ended up back in the water, though see the UPDATE further down that speaks to confusion over this.]
Allow me to further explain what you're looking at in that picture. The black piece is the fitting between the hydraulic hose coming out of the discharge of the Rineer hydraulic motor and the inlet of one of the two hydraulic coolers. The hose comes down from the top of the frame, and the duct tape is around the very section where the hose joins the fitting. I couldn't believe what I was seeing when I snapped this picture.
Let's step back a bit. Here's a picture of three of the five west pumps (the ones that were just pulled out of the canal last week):
I've noted the location of the typical waterline, which is based on the growth of those little barnicle things all over the pumps. As you can see, the fittings joining the hoses to the hydraulic cooler inlets on these pumps have all rusted. Here's a close-up of another cooler fitting from a west pump:
What is moresurprising [UPDATE: This should read "perplexing," and it is explained in the next UPDATE paragraph as being in the "splash zone"] is that the fittings on the discharges of the coolers - which for most of the last year have been above the waterline, and thus only exposed to air and humidity - have also rusted. How could this happen? These pumps have only been in the canal since last June.
It's pretty simple. It appears that the manufacturer, MWI, skimped on the purchase of those fittings, buying carbon steel instead of stainless steel despite knowing they would be exposed to water all the time. Curiously, the Corps' specifications are oddly silent on the material of the hose fittings:
Considering the criticality of the installation (protection of life and property), use of stainless steel fittings in this application to prevent corrosion should have been a no-brainer.
[UPDATE: This very issue was the subject of considerable debate between the Corps and MWI at the time. The fittings are zinc-plated steel, and obviously could not hold up to brackish water, whether immersed or in the splash zone (the area above the water line, but subject to spray and waves, which are just as corrosive). However, MWI stated that since the plated fittings were part of their standard pump package, they were under no obligation to supply anything better. Of course, since the Corps copied and pasted MWI's specs for the standard package, it shouldn't have been a surprise that the pumps wouldn't work very well in conditions for which the standard package is not designed. Also, MWI claimed they didn't know the water was salty, so why would they provide anything better than what they thought was needed for fresh water?
Again, it's best to read over my May 27, 2010 post, "How did the pumps get from...". Also, the letter MWI sent to the Corps at the same time the Orleans Avenue east pumps were initially on the deck (during the period when the Corps and MWI were trying to figure out what to do about corrosion) is quite illuminating. It can be found starting on page 201 of the complete MWI contract file]
You may be asking, why is the fitting in the duct tape photo not rusted? I believe that fitting is on one of the pumps which has been out of the canal since October (the ones from the east side of the project), so it was only in the water for a little over four months. The pumps shown in the other photos have been in the water for approximately ten months. This is troubling, since the expected service life of these pumps is about four years - the Corps has promised permanent pump stations at the lakefront by the end of 2010.
[UPDATE: We now know the fitting in that picture was painted after being removed from the water in September. It was rusty before that. Also, thanks to tons of delays by the Corps, the permanent pumps are now not scheduled to be on line until the 2015 hurricane season.]
There is another possibility why the fittings on the east pumps don't show rust: they leak so much hydraulic oil that rust cannot form. There is some evidence for this. Below is a picture of one of the east pumps (which has been on the deck since October). I've highlighted the fitting on the discharge of the cooler.
Here's the highlighted area blown up:
I've noted the places where oil has apparently leaked.
[UPDATE: This is curious, because this photo is of one of the pumps from the west side of the site. It was pulled up in March, 2007 ahead of the piping extension repairs mentioned below and detailed in my May 27, 2010 post "How did the pumps get from...". Yet the hose fitting I've highlighted appears to have been sandblasted and painted like was done to the fittings on the east side pumps back in October, 2006. It's difficult to figure out, since the contract documentation gives the impression the sandblasting and painting technique was only done on the five east pumps. I'm not sure what to make of this.
However, the idea that the photo also shows evidence of leaks on the oil cooler is eminently possible, especially after reading about the repairs to the pumps in 2009 and 2010. The 2009 Conhagen report of repairs to 17th Street pumps E5 and E7 mentions weld repairs to the coolers. In the 2010 repairs, the coolers were apparently so corroded that they were completely replaced with stainless steel units.]
This is not good manufacturing. Material selection is a pretty easy part of the engineering on this job, and it appears the ball was dropped. It certainly brings into question the quality of the work elsewhere on the job.
This part of the story may have a happy ending. As part of the refits the Corps and MWI are undertaking - apparently at the shop of Associated Pump & Supply in Houma, LA - all of the below-the-waterline connections which previously were between hose and pipe have been replaced with hard pipe, whioch hopefully won't leak. Now all the pumps have pieces of pipe sticking up out their sides, terminating with connection points which will be above the deck on the support platforms. Frankly, this should have been done in the first place by MWI, and if they knew there were going to be connections which remained underwater constantly (which they did, since the pumps are designed to be immersed in the canal), those fittings should have been corrosion-proof. That means using stainless steel, which is the industry standard. Keep in mind that none of this has ever been acknowledged as a problem by the Corps.
[UPDATE: It was never acknowledged, but it was paid for, to the tune of over $500,000 for the piping modifications. And those modifications turned out to be nearly an entire waste of money, because while the elevated hose fittings may have been a little further from the brackish water, the entire rest of the pumps were completely corroding to bits just two years later.
See my May 13, 2010 post, "Imminent," my May 27, 2010 post, "How did the pumps get from..." and my June 3, 2010 post "This year's scramble" for the results of the total lack of consideration on dealing with corrosion in the late 2006/early 2007 timeframe. They screwed this one up big time, and then used permanent pump station money for the "fix," which in some cases was no fix at all.]
In future entries, I'll be looking at other problems with the pumping system. There are many, and none of them have been acknowledged by the Corps.
There's been a lot of talk since the story about the defective MWI pumps broke on March 12. And while the media and the Corps have covered various aspects of the story around the pumps, no one has publicly looked at the system itself to figure out exactly what problems may still need to be addressed. Fortunately, nearly everything about the system is fully on display for anyone to walk up to and examine.
Over the next few posts, I'm going to be concentrating on specific deficiencies in the design of the pumping system. Some relate to fabrication, some relate to engineering design, and some relate to installation. This is nitty-gritty stuff that engineers love (or should love) to dig into. Keep in mind that engineering is not a static field - it is subject to opinion on how to do things the best way. These are my opinions, backed up with my experience as a mechanical engineer and consultations with others familiar with similar installations. Some of this may seem a bit didactic, but I'll always show that there's a point.
I'm going to start with the pump units themselves. As a refresher, you may wish to read my earlier post about the fundamentals of how the pumps and their drive units are set up.
Out at the Orleans canal floodgates, all ten pumps have been pulled in anticipation of refits by MWI. The five west pumps were pulled Wednesday, March 22. The five east pumps have been on the deck since last October. They're all up on the deck on the east (Marconi Drive) side, and anyone can go up and check them out. So I did so last Sunday. So did at least five other people while I was there, including two joggers with their dogs.
Frankly, the workmanship is ... not great. Let me lead off by showing you something that I had to laugh at to keep from crying:
Yes, that's duct tape wrapped around a hose meant to hold at least 1000 psi of hydraulic oil (I think - I'm not sure of the pressure on the outlet of the coolers). To top it all off, that particular fitting sits below the waterline when the pump is in the canal. Are they kidding us with this?
[UPDATE: This hose fitting appears to be one of the ones on the east pumps that were sandblasted and painted during October, 2006 as a first cut at repairing corrosion damage on the hose fittings. The duct tape may have been acting as masking for the painting and sandblasting, or it may have been part of the protection. One can see what also appears to be duct tape on other painted fittings in this shot from October, 2006:
There's more details on this hose fitting painting scheme and how it fits into the overall corrosion chronology in my May 27, 2010 post, "How did the pumps get from...". I don't think this particular method of corrosion repair ever ended up back in the water, though see the UPDATE further down that speaks to confusion over this.]
Allow me to further explain what you're looking at in that picture. The black piece is the fitting between the hydraulic hose coming out of the discharge of the Rineer hydraulic motor and the inlet of one of the two hydraulic coolers. The hose comes down from the top of the frame, and the duct tape is around the very section where the hose joins the fitting. I couldn't believe what I was seeing when I snapped this picture.
Let's step back a bit. Here's a picture of three of the five west pumps (the ones that were just pulled out of the canal last week):
I've noted the location of the typical waterline, which is based on the growth of those little barnicle things all over the pumps. As you can see, the fittings joining the hoses to the hydraulic cooler inlets on these pumps have all rusted. Here's a close-up of another cooler fitting from a west pump:
What is more
It's pretty simple. It appears that the manufacturer, MWI, skimped on the purchase of those fittings, buying carbon steel instead of stainless steel despite knowing they would be exposed to water all the time. Curiously, the Corps' specifications are oddly silent on the material of the hose fittings:
"2.4 Hydraulic Piping and Hose: ... All reinforced supply hose shall be double wire braid reinforcement and shall have minimum safe working pressure of 3,000 psi. All pipe fittings shall be socket weld type (with socket weld to thread fittings at conversion point of pipe to reinforced hose)..."
Considering the criticality of the installation (protection of life and property), use of stainless steel fittings in this application to prevent corrosion should have been a no-brainer.
[UPDATE: This very issue was the subject of considerable debate between the Corps and MWI at the time. The fittings are zinc-plated steel, and obviously could not hold up to brackish water, whether immersed or in the splash zone (the area above the water line, but subject to spray and waves, which are just as corrosive). However, MWI stated that since the plated fittings were part of their standard pump package, they were under no obligation to supply anything better. Of course, since the Corps copied and pasted MWI's specs for the standard package, it shouldn't have been a surprise that the pumps wouldn't work very well in conditions for which the standard package is not designed. Also, MWI claimed they didn't know the water was salty, so why would they provide anything better than what they thought was needed for fresh water?
Again, it's best to read over my May 27, 2010 post, "How did the pumps get from...". Also, the letter MWI sent to the Corps at the same time the Orleans Avenue east pumps were initially on the deck (during the period when the Corps and MWI were trying to figure out what to do about corrosion) is quite illuminating. It can be found starting on page 201 of the complete MWI contract file]
You may be asking, why is the fitting in the duct tape photo not rusted? I believe that fitting is on one of the pumps which has been out of the canal since October (the ones from the east side of the project), so it was only in the water for a little over four months. The pumps shown in the other photos have been in the water for approximately ten months. This is troubling, since the expected service life of these pumps is about four years - the Corps has promised permanent pump stations at the lakefront by the end of 2010.
[UPDATE: We now know the fitting in that picture was painted after being removed from the water in September. It was rusty before that. Also, thanks to tons of delays by the Corps, the permanent pumps are now not scheduled to be on line until the 2015 hurricane season.]
There is another possibility why the fittings on the east pumps don't show rust: they leak so much hydraulic oil that rust cannot form. There is some evidence for this. Below is a picture of one of the east pumps (which has been on the deck since October). I've highlighted the fitting on the discharge of the cooler.
Here's the highlighted area blown up:
I've noted the places where oil has apparently leaked.
[UPDATE: This is curious, because this photo is of one of the pumps from the west side of the site. It was pulled up in March, 2007 ahead of the piping extension repairs mentioned below and detailed in my May 27, 2010 post "How did the pumps get from...". Yet the hose fitting I've highlighted appears to have been sandblasted and painted like was done to the fittings on the east side pumps back in October, 2006. It's difficult to figure out, since the contract documentation gives the impression the sandblasting and painting technique was only done on the five east pumps. I'm not sure what to make of this.
However, the idea that the photo also shows evidence of leaks on the oil cooler is eminently possible, especially after reading about the repairs to the pumps in 2009 and 2010. The 2009 Conhagen report of repairs to 17th Street pumps E5 and E7 mentions weld repairs to the coolers. In the 2010 repairs, the coolers were apparently so corroded that they were completely replaced with stainless steel units.]
This is not good manufacturing. Material selection is a pretty easy part of the engineering on this job, and it appears the ball was dropped. It certainly brings into question the quality of the work elsewhere on the job.
This part of the story may have a happy ending. As part of the refits the Corps and MWI are undertaking - apparently at the shop of Associated Pump & Supply in Houma, LA - all of the below-the-waterline connections which previously were between hose and pipe have been replaced with hard pipe, whioch hopefully won't leak. Now all the pumps have pieces of pipe sticking up out their sides, terminating with connection points which will be above the deck on the support platforms. Frankly, this should have been done in the first place by MWI, and if they knew there were going to be connections which remained underwater constantly (which they did, since the pumps are designed to be immersed in the canal), those fittings should have been corrosion-proof. That means using stainless steel, which is the industry standard. Keep in mind that none of this has ever been acknowledged as a problem by the Corps.
[UPDATE: It was never acknowledged, but it was paid for, to the tune of over $500,000 for the piping modifications. And those modifications turned out to be nearly an entire waste of money, because while the elevated hose fittings may have been a little further from the brackish water, the entire rest of the pumps were completely corroding to bits just two years later.
See my May 13, 2010 post, "Imminent," my May 27, 2010 post, "How did the pumps get from..." and my June 3, 2010 post "This year's scramble" for the results of the total lack of consideration on dealing with corrosion in the late 2006/early 2007 timeframe. They screwed this one up big time, and then used permanent pump station money for the "fix," which in some cases was no fix at all.]
In future entries, I'll be looking at other problems with the pumping system. There are many, and none of them have been acknowledged by the Corps.
posted by mcbrid35 at
9:17 AM
0 Comments:
Post a Comment
<< Home