More on the pipes ... and rust

Updated 6/11/08. See below.

Updated 5/27/08. See below.

Updated 10/3/08. See below.

As promised, I wanted to show pictures of the hydraulic pipes, so people can get a better feel for what I'm talking about.

First, let me point out where they are:

This is the east bank of the Orleans Avenue canal floodgate site. I'll be returning to this picture in a later post to talk about some details on the elbow, but for now, it serves to help place the hydraulic lines in context.

Let's look at a closeup:


First off, I've noted the confirmation that the high pressure lines are schedule 80 with a white rectangle.

You can see other pipes from 17th Street also marked as "Sched 80" on this picture from my September 30, 2006 post:


For reference, the pipes marked "Sch. 40" are the schedule 40 low pressure lines coming back to the drive units from the pumps. The schedule 80 pipes are the high pressure lines going out to the pumps from the drive units.

What is obvious, but somehow has escaped everyone's notice until now, is that the pipes are unpainted, and have been since they were installed almost a year ago. Naturally, they're rusting.

Here's two more closeups of the Orleans Avenue pipes:




You can see dings and marks all over the pipes. I'm not sure what they are. They could be areas that were struck by the welding torch or the welding rod during welding. Or they could be marks where tools or other pipes hit during assembly or shipment. They could be points of local thickness loss; I'm not sure.

I can bet the marks are even worse at the 17th Street site. Remember my post last October about lack of fall protection? That has pictures of the installation contractors walking all over these pipes up in the rack. Such behavior would never be tolerated in private industry.

I've been looking at this situation since the lines were installed. I've always thought the pipes were eventually get painted, despite the difficulty in doing it after they were installed (normally, on any job, pipes are sandblasted, primed and painted in a shop before they are shipped to the job site. That wasn't done here.). Despite all the dignataries, Senators, Representatives, Corps officials, press, and contractors that have passed through these three sites over the last year, it seems inconceivable that no one has asked, "Aren't the pipes going to be painted?" However, here we are almost a year later, and I think we're stuck with these unpainted pipes for the coming storm season, as well as the three following that. I think it would be nearly impossible to sandblast and paint the pipes on the interior of the pipe racks without disassembling the ones on the outer perimeter. Plus, doing it over water introduces a whole new set of complications. So obviously the pipes, which already have walls that are too thin for 3000 psi service, are going to continue to rust, further thinning the walls.

Just so you know, here's what the bid specifications say about the pipe:

"Supply pipe shall be ASTM A106, Schedule 80 seamless black steel pipe, and return pipes shall be ASTM A106, Schedule 40 seamless black steel pipe. All hydraulic pipe shall be pickled, oiled and plugged (P.O.P.)."

There's no mention of painting, which is weird. Nearly every pipe specification I've ever seen discusses painting. I've helped write both paint and pipe specs, and have read numerous ones during those jobs. There's tons of standards and other publications on this stuff from SSPC.

Also, here's the relevant portion about painting from chapter 12, "Corrosion Protection," in the Corps' Engineering Manual that deals with piping, EM-1110-1-4008, "Engineering and Design - Liquid Process Piping:"

12-1 Corrosion Protection
[...]
b. Above Grade Installations
The external surfaces of metallic piping installed above grade will also exhibit electrochemical corrosion. The corrosion rate in air is controlled by the development of surface-insoluble films. This development is, in turn, affected by the presence of moisture, particulates, sulfur compounds, nitrogen-based compounds, and salt. This corrosion is typically uniform, although pitting and crevice corrosion are also common. Besides selecting a material of construction that is appropriate for the ambient environment, the primary method of corrosion control in above grade piping system is the application of protective coatings.

I don't know why such a simple thing as painting the pipes to keep them from rusting has been neglected. It doesn't make sense. However, physics and electrochemistry will not stop because of whatever excuse the Corps gives. The pipes will continue to rust in a moisture-rich environment.

This isn't the only rust on the system. Look back at the shot with the duct tape in my post two posts ago :

You'll notice the bolts are also rusting. There seems to be rust on the flanges as well.

Those bolts should have been stainless steel. The coal tar-like coating on the pumps should have been waterproof. But once again, skimping by MWI left the bolts as carbon steel, and I don't think the coating's holding up either.

This is so basic, that I think it's escaped everyone's notice until now. After all, who would leave these pipes to simply corrode? It sounds crazy. But I'm forced to ask, are we supposed to have confidence in a system that isn't even protected from the moisture in the air? At this point, the only way to restore that confidence is to ultrasonically or radiometrically test every foot of the 17,000 feet or so of high pressure line, and then release the results to the public immediately. Short of that, there is no conclusive way the Corps can say that the pipes - as they exist today, not when they were delivered - are the thickness they were specified as. Of course, even if they are that thickness, they're still too thin, but that's another kettle of fish.

Update 5/27/08

Over two years after the rusty pipes were installed, the Corps has finally decided to address the corrosion problem. The more fundamental problem of the pipes being undersized will go unaddressed.

On May 23, 2008 a presolicitation was issued to hydroblast and paint all the hydraulic and fuel lines at all three outfall canals:

New Coating System for Hydraulic Pipes-Cleaning and coating hydraulic and fuel lines Interim Closure Structures-Orleans Avenue, London Avenue, and 17th Street

Of course, they're not exactly moving at light speed. The presolicitation indicates that potential bidders will have until half way through the peak of hurricane season (July 31, 2008) to get their bids in, and the contract would probably be issued weeks after that, meaning that the pipes will continue to rust for many more months.

One other thing: this means the Corps is literally covering up the problems with the pipes.

Update 6/12/08

Further documents for the pipe painting job have been issued. They are a treasure trove of information.

Both the specifications and drawings have been placed on line:

Specs: https://www.fbo.gov/utils/view?id=7e334641286b8cb4daba96f2fdcaf561
Drawings: https://www.fbo.gov/utils/view?id=f654bc8f34386efd84e6fe67abbea8f3

The scope of the job is gigantic. Just about every pipe on all three floodgate sites must be hydroblasted and painted. It is shameful that this much work has been delayed for this long, while the pipes have been rusting for years.

However, while the information associated with the solicitation is interesting in its own right, what attracted my attention is that many of the drawings are actually collections of digital pictures of the three sites.

According to the date stamps on some of the pictures, they were probably taken in December of 2007.

I'd like to zero in on one picture in particular: picture 2 on sheet 32 (drawing M-30):

The two areas highlighted in red are what I'm interested in.

The top area shows...

duct tape.

The bottom area shows...

Rope suspending hoses which convey hydraulic fluid to and from the pumps. Not steel, but rope. I hope those knots are tied tight.

My guess on the reason those ropes are there? Two winters ago, the Corps pulled the pumps to install extended piping on them which moved the previously submerged hose connections out of the water (to prevent further rusting of the fittings as captured in the pictures above). They probably decided to reuse the same hoses.

Before, those hoses ran from the deck above down to fittings which were underwater. With the extended piping, there was a shorter distance between the pipe on the deck and the new connections on the pumps. But they still had the old hoses, which were now too long and heavy, and they needed to make sure the hoses were not pulling down on the new fittings. So they roped them up. Hardly the best solution, and one has to wonder how well the ropes will hold up in extended storm conditions with the pumps running for eight or twelve hours.

Other possible solutions?
1) Fabricate shorter hoses
2) Run the piping above the deck and hard pipe the entire system except for standard flexible connections. That is, eliminate the hoses. That would make it more difficult to maintain the pumps, but would make their operation a little more reliable.

Another picture in the drawing package caught my eye. It is picture 33 on sheet 69 (drawing M-67):



The hose on the left is a hydraulic fluid hose leaving the drive unit, meaning that the pressure of the fluid is at its maximum at that point.



The labeling on the hose says the following:
3000 PSI (21.0 MPa) MAX

That's interesting, because there are red signs on all the drive units:




which say the following:

DO NOT EXCEED 3200 PSI

or 200 psi more than the rating of the hose.

When one has a piece of equipment with a rating of "X" psi connected to another piece of equipment that can generate "X" + 200 psi, one should not be exceeding "X" psi. Or else, one might damage the hose and take a pump out of service.

Would it have cost that much more to provide hydraulic hose with a rated pressure greater than the rating of the pump feeding it?

Update 10/3/08

The Corps took its sweet time awarding this one, finally announcing it on October 2nd, 2008:
Contract award for new coating systems (sandblasting and painting) at New Orleans outfall canals

With the actual document to be found here.

I assume the Corps realized it would be foolish to have contractors out in the middle of the canals during the height of hurricane season. Of course, considering the pipes have been rusty since they went in over two years ago, it's not like they didn't have time to do this work last winter and spring. Or perhaps even before the pipes went in...

Anyway, the work's (contract W912P8-09-C-0001) been awarded to Creek Services out of Gretna, just across the river. Creek Services is a favorite contractor for the Corps in New Orleans, doing tens of millions of dollars in business since Katrina. This contract, at a value of $1,972,867, is peanuts for them.

Interestingly, the Corps posted the bid summary as well:
Bid summary for pipe coating systems

With the actual document here.

The bids were opened July 7th, 2008.

It appears there were only two bidders. Creek Services and Truckla Services of Vicksburg. Truckla was actually a little cheaper than Creek.

The contract can be found here.

[Update 6/1/11]

Over the years, the Corps protested publicly that these pipes were adequate, even though the most basic of calculations show they are not.

They even tried that with their own consultants. In 2006, the Corps had their outfall canal consultants Black & Veatch investigate ways to reduce the costs of the permanent pump stations. One scheme was to reuse parts of the Interim Closure Structures. The resulting report, dated December 12, 2006 and titled the "Alternative Considerations Report," detailed what B&V thought of various components of the ICS's. As such it is a contemporaneous account of design deficiencies. Here's what B&V wrote about the hydraulic piping:

"The field staff indicated that the hydraulic piping wall thickness is sized for the rated pressure of 3500 psi and has little to no extra wall thickness for corrosion. The piping is not coated and could soon begin to reduce to the safety factor and eventually lead to failure. The adequacy of the wall thickness for the discharge piping was not determined but it is not coated and could pose problems as corrosion reduces the wall thickness."

So the "field staff" either lied or didn't know about the inadequate sizing. However, they did know the pipe effectively had no corrosion allowance. That's yet another problem the Corps knew about at the time, but waited years to address, and even then literally only covered up the problem rather than addressing it fully.