Fix the pumps

Debris Part 11

2012-01-26T11:35:00.006-06:00

Part 1
Part 2
Part 3
Part 4
Part 5
Part 6
Part 7
Part 8
Part 9
Part 10

After the flurry of debris reports in east bank projects during the spring of 2010, the reports - oddly - seemed to stop for about a year. But they started appearing again in the spring of 2011 as the Corps rushed to beat their June 1st deadline.

In Orleans Parish, the first spring 2011 mention of debris was on LPV-106 in the April 8, 2011 report:

"Still clumpy rocks in section between [Corps project LPV-]107 & [Corps project LPV-]108 where seeding has taken place."

Here's the locations of those three projects:

LPV-106 came up again about a month later, in the May 4, 2011 report:

"Still concrete and debris in turf area."

These referred to areas of finished levee. The contrator (L&A) had apparently placed soil with excessive debris in it and had yet to remove the debris from areas where grass was attempting to be grown.

We know the reports from April and from May were referring to separate areas of LPV-106 because of the May 5, 2011 report, which had more detail:

"Contractor has started a crew from [project LPV-]107 west towards Read blvd clearing debris from levee section to be grassed next. They will also clear out left debris in reach 5 and reseed in areas that grass has not established. Tuesday I will inspect levee clean up as contractor plans to seed next week."

The section of LPV-106 from LPV-107 west toward Read Blvd. is part of the much larger western section of the project, as opposed to the report in April of the smaller eastern patch between LPV-107 and LPV-108.

The same report noted debris in the soil on the flood side of the levee at adjacent project LPV-107 (contractor: Shavers-Whittle):

"debris removal and grading on flood side of levee."

About a week later, the May 11, 2011 report mentioned the contractor and the Corps had declared victory over the debris at LPV-106:

"contractor has cleared access rubble/debris from 107 tie in till Jahncke pump station [aka Sewerage & Water Board drainage pump station 14]. Contractor has not been able to confirm when seeding will take place."

Here's the location of Read Blvd. and the Jahncke Pump Station:

It was clear the debris problems were not conquered in May, because a month later they were still getting mentioned in LPV-106 updates. In the June 16, 2011 report we learned:

"Contractor is continuing to address debris in levee."

Frustratingly, none of these reports include photos. However, it is clear debris continued to be an issue even past the vaunted June 1, 2011 deadline when the Corps declared the system 100 year storm ready.

A second batch of spring 2011 debris reports comes from the Corps projects in St Bernard Parish. They're located here:

On February 24, 2011, the SLFPA-E inspector wrote about project LPV-148.02 (contractor: Cajun Construction):

"Observed fill material at top of levee slope appear to be non compliant with more than one percent contaminants in work area 1A and 1B."

The following day's report had a bit more detail:

"Discussed issues found on previous day's observation with Butter Compton. 1. Quality of backfill used for dressing levee does not conform to specification. Also quite a bit of dirt was left on stabalization slab. Butter talked with contractor and a coordination meeting is scheduled for Tuesday March 01, 2011."

We never find out what happened. The inspection reports are silent. However, debris continued to be a concern elsewhere.

LPV-146 (contractor: St Bernard Levee Partners), April 1, 2011:

"Contractor backfilled area on protective side with concrete rubble at the 90 degree turn in the wall at MRGO and Verrett [sic] wall."

See the map above for the sharp bend at the southeast corner of LPV-146.

Fortunately, we got pictures of this incident:

The reports don't indicate whether the concrete rubble was later removed.

LPV-145 (contractor: Chalmette Levee Constructors), April 4, 2011:

"There was discussion about debris and gravel in the slopes of the flood side. It appeared that the haul road material was pushed into the slopes of the flood side."

LPV-147 (contractor: Integrated Pro Services), April 5, 2011, after a pre-final inspection:

"There is unsuitable material and debris in the earthwork of the protective side slopes.

LPV-149 (contractor: Conti Group), June 16, 2011:

"There are concerns for backfilling with debris and trash."

That's seven projects with 2011 reports of debris in the dirt. That's on top of the six projects with debris concerns in the spring of 2010.

If you then combine that with the 9 west bank projects with debris problems...

You get 22 projects where debris has been a problem either in the existing levee, in the new levee, or in both.

22 projects.

Is it now obvious there is a systemwide problem with the integrity of the levees built by the Corps? It should be.

Debris Part 10

2012-01-16T20:40:00.009-06:00

Part 1
Part 2
Part 3
Part 4
Part 5
Part 6
Part 7
Part 8
Part 9
Part 11

Much pressure was brought to bear upon the Corps over the past few months to address the debris problems in the west bank levees, and there have been halting measures to do so. A "tiger team" of other Corps people from outside New Orleans was brought in, they dug some holes in the levee at a single one of the projects (in the wrong spots), and then dug some more holes (in the right spots) while they literally tried to bury the evidence, which included a shopping cart and hubcaps. They also promised Senator Vitter they would add the west bank levees to their third party review process - a process meant to review designs before they're built, not after they're all done and shown to be defective.

But the west bank levees were not designed or constructed in a vacuum apart from the rest of the region. One must ask if the same problems have arisen with the rest of the system. The answer is "Yes."

We know of two large debris fields in the Lake Pontchartrain lakefront levees in Kenner, because the Times-Picayune reported on them in November, 2009 and January, 2010. In those cases, the existing levee was found to be filled with junk from a site on North Claiborne Avenue in New Orleans. The spoiled soil had been placed there by the Corps in 2000-01. We never heard about any further east bank levee problems after that, and the Corps certainly attempted to give the impression they were isolated - albeit 8000 foot long - cases.

A few months back I noted the receipt of the reports from the inspectors assigned to Corps projects within the geographic area assigned to the east bank levee authority (the SLFPA-East). I have finally finished plowing through them, and will be presenting what I've dug out over the next few posts. First up is evidence of many more instances of debris in the east bank levees. All the excerpts below are taken from these reports, downloadable at the earlier post.

Let's start at project LPV-104.02 (LPV stands for Lake Pontchartrain and Vicinity, the official Corps name for all the levees on the east bank of the Mississippi). Here's the location of LPV-104.02:

LPV-104.02 is a catchall project involving a few small projects bundled together along the lakefront between the London Avenue canal and the Industrial Canal. One of those little projects was the installation of a vehicle ramp over the levee behind the main campus at UNO (University of New Orleans).

The lead contractor on LPV-104.02 is Quality Enterprises. They were placing the first "lift" of the ramp on the UNO side of the levee in May, 2010 (a "lift" is a layer of dirt, usually six to twelve inches thick). On May 13, 2010, the SLFPA-E inspector wrote the following about his inspection on the prior day:

"1st 6" lift was applied on protected side of the levee. Material from stock pile on LPV-104.01 [adjacent project - ed.] was used for this lift, and placed over the area that was cleared and grubbed 10" to 12" as stated by specifications. The material despite how much was cleared, was filled with small shells. These shells mixed with the material used for the first lift which caused it to have more than 5% debris in material. Material used was also considered to be too wet to pass moisture test, but will be cut up to dry out before next lift would be applied. Billy Rossignol [Corps resident engineer for the project - ed.] was asked to come and make a decision regarding the material being filled with the shells from the site. The decision was made that nothing can be done about this lift and to leave as is."

Isn't that nice? The bottom six inches of the earthwork are filled with shells (at 5 times the specified limit, no less), as shown in this picture from the same report:

But don't worry, they did something:

"Corrective measure will be done to prevent this from happening in the remaining lifts. Corrective action include not stockpiling material on the site, and scraping the surrounding area to move shelled material away from toe of ramp. (40ft offset to be exact)"

Note their "corrective" measures did not correct the actual problem by removing the debris. Their response to that was simply to leave it in place, which it is now.

Isolated incident of debris in the dirt? Not so much. Over at LPV-108 (contractor: Boh Brothers), a lakefront levee project at the exreme end of New Orleans East as shown on this map:

the SLFPA-E inspector wrote in his April 16, 2010 report:

"Corps inspector mentioned being unhappy with the size and amount of rock that is in the soil being used. Will follow up with this."

The follow-up came in the April 27th report:

"The dirt as mentioned in previous report is has not been filtered properly. It is filled with debris and material over the specified allowable size and over the specificed percentage of the material. The Contractor will be getting a formal letter requesting a correction plan from Billy Rossignol."

The inspector included this photo:

along with a caption informing us the material had been placed in the levee, debris and all:

"An example of the size of rocks that have been pulled from the surface of the material. This is a very small amount compared to what is on the surface of the entire project. Also if this is what is on the surface, what is located through the entire soil subsurface[?]"

That's a damn good question. Based on what we've seen at the west bank projects, the answer is probably that the LPV-108 levee had debris inside it.

A couple of weeks later, the May 13th report read,

"Corrective action has been done regarding soil conditions. Boh Bros was asked to remove a 5' wide by 6" deep section of material along the entire wall. Perform compaction and moisture tests on the area. Replace with stock pile material from pit, compact, and test to specifications."

Anyone who thinks the debris only went 6" down, raise their hands.

Just the Corps folks and their contractors? Okay, moving on...

Throughout the spring of 2010, debris was found at three other lakefront projects. At project LPV-103.1A1 (contractor: Aquaterra-CAYO), a levee project located in Orleans Parish between the Orleans Avenue and London Avenue canals...

...SLFPA-E inspectors reported the following:

"Starting to clean out dirt on the west side. The embankment material is full of large rocks and debris which is unacceptable for final use. Process will continue till the material is to specification." (April 20, 2010 report)

"Working on filtering out unwanted material in the back fill that is onsite." (April 30, 2010)

"Continue to remove debris in material on west side of Bayou St. John." (June 17, 2010)

Unfortunately, this is all the information we have from the reports. However, the events at two other projects were described more fully.

At LPV-19.2 (contractor: Phylway), a Jefferson Parish lakefront levee enlargement placement project located on either side of the Causeway:

this got reported on May 5, 2010:

"Found unsuitable material on F/S of levee at stations 409+00 and 419+50. It appears to be an old ramp that was covered up. Once contractor has provided enough material on site to fill the area they will start to do exploratory excavation to see exactly how much material will be removed."

Those locations are just 1000 feet apart, yet the Corps appears to have not dealt with anything in between them, viewing them as isolated instances. Pictures from the report give some more perspective, albeit only on the easternmost debris site at station 419+50, right next to the Causeway:

Te black stuff in the lower photo appears to be about a foot thick section of asphalt.

It is unclear if Phylway covered up the debris-laden area during prior work placing a berm at the base of the levee, or if they were excavating it in anticipation of normal work. Whatever the choronology, it is interesting to see their name on yet another project involving debris.

However, unlike the west bank projects Phylway worked on, in this east bank case, the Corps decided to excise the debris from the existing levee. Subsequent inspection reports tell us Phylway started moving in replacement material very quickly (5/6/10: "Began stockpiling material for areas where they are to remove unsuitable material."), and the removal of the bad dirt started around a week later (5/14/10: "Dug out unsuitable material area near Sta. 419+50. Unsuitable area around Sta. 409+00 still to be dug out."). Photos from the May 14, 2010 report show the excavation:

The dig out of the debris-laden material continued for about a week. The May 20, 2010 report said Phylway had moved to the second area: "Unsuitable material was removed from the F/S of the levee at Sta. 409+00. The area was approx. 150' wide." Photos also show that removal effort:

The Corps had a walkthrough on May 24, 2010 to determine whether Phylway had gotten everything. A third area was actually found with debris, according to the May 25, 2010 report: "Area at Sta. 407+00 near the Severn ramp where unsuitable material was found was dug out some more to remove a little excess that was found during inspection on Monday 5/24/10. Area was reinserted and approved for contractor to begin placing material and bring levee section back up to grade." Here's a picture:

But they declared victory on the other two areas and started backfilling them at the same time. Here's a photo of that backfilling in the May 27, 2010 report:

About a week later, the backfilling was nearly done, according to the June 2, 2010 report: "Work at this reach is almost done with the two sites that had material removed almost rebuilder [sic]."

Other than a single mention buried on page 21 of a SLFPA-E set of board meeting minutes from May, 2010 (minutes which were not published until two months later), none of this activity at LPV-19.2 was ever revealed to the press or the public by anyone. Given how close these events were happening to the beginning of the 2010 hurricane season, that's understandable from a public relations viewpoint. From a "complete disclosure," "partnership," or "protection of citizens" viewpoint though, I can't think of a justifiation.

But there were even more debris-related problems out at the lakefront besides all these. In April, 2010, debris was found in at LPV-20.1 (contractor: L&S/CKS Joint Venture), the project to enlarge the Jefferson Parish lakefront levee from Bonnabel over to the 17th Street canal. Here's the location of that project:

The first mention was in the April 13th report. It was just one sentence: "Special report filed unsuitable material in Levee." The "special report" was not included with the regular report.

The following day the SLFPA-E inspector reported "follow up" at the site, but gave no details. Reports over the following two days revealed the location (a ramp in front of the Coast Guard station) and the fact that more debris had been found.

On April 19th, it was revealed the Corps held a walkthrough to decide whether enough debris had been excised to allow the ramp to be rebuilt. This report (by a different inspector) also included much more detail:

"The USACE held a final walk through to approve the removal of unsuitable material in the Ramp and Levee. After a second check of levee the levee was deemed to be clean. The material that was removed will be stockpiled at Williams Blvd. Boat Launch. The plan is to use the material in foreshore protection. The USACE is still waiting on redesign of ramp."

This report also included photos, which show how extensive the dig out was. The photos are shot looking east toward the 17th Street canal, progressing from the protected (land) side of the levee to the flood (lake) side:

It's that last photo - showing the LPV-20.1 debris removal effort was not confined strictly to a protected side ramp, but also involved the flood side facing the lake - which is the most disturbing by virtue of the caption: "Flood side of levee any remaining large chumks [sic] of concrete will be removed."

"Large chunks of concrete?" Yikes.

Like the debris problems at LPV-19.2, this also got a single mention in the SLFPA-E board minutes, in this case on page 16 of the April, 2010 minutes. There was nothing else.

That's five different projects across two parishes with debris problems in the spring of 2010, almost none of which was revealed to the public. In the next part we'll discuss even more projects where debris was found as recently as last spring, after the Corps declared the system 100-year ready.

Seriously?

2011-12-16T12:07:00.002-06:00

Times-Picayune, October 26, 2011:

"Mark J. Titus, 48, pleaded guilty to conspiracy to commit mail fraud for bilking $1.2 million from a construction firm in a scheme allegedly aided by his brother-in-law Dominick Fazzio, River Birch's chief financial officer.

Fazzio, 58, of Slidell, used two construction companies he owns to send seven fraudulent invoices from June 2008 through March to Garner Services, a New Orleans construction management firm, according to a five-page "factual basis" for Titus' guilty plea."

FedBizOpps.gov, December 9, 2011:

Y--West Bank and Vicinity, New Orleans, LA HSDRRS, Lake Cataouatche Levee Enlargement, Phase 2, First Lift, Jefferson Parish, LA ED 11-203Solicitation Number: WBV15A-12-S-0001
Agency: Department of the Army
Office: U.S. Army Corps of Engineers
Location: USACE District, New Orleans

"This is a combined synopsis/solicitation for commercial items prepared in accordance with the format in Subpart 12.6, as supplemented with additional information included in this notice. This announcement constitutes the only solicitation; proposals are being requested and a written solicitation will not be issued.

THIS IS BEING POSTED FOR SUBCONTRACTING OPPORUNITIES ONLY. PROPOSALS WILL ONLY BE ACCEPTED FROM MATOC HOLDERS AS FOLLOWS:
[...]
W912P8-10-D-0009
Garner Services, LLC
348 S Diamond St
New Orleans, LA 70130-3808

POC: Mark Titus
Email: mtitus@garnersvc.com
Office: 504-566-1192
Fax: 504-566-1173"
[...]
SYNOPSIS:

The work consists of clearing and grubbing; removal of steel sheet pile; Contractor furnished borrow pit preparation and excavation; placement of earthen material, the earthen material shall be graded and compacted; placement of stone armor; fertilizing, seeding and mulching all disturbed areas and all other incidental work.

What exactly does it take for the Corps to cut ties with companies? This is a solicitation for new business, not an existing contract, but they are insisting on allowing Garner to bid on it. At the very least, don't they read the papers?

Plus or minus two projects

2011-10-27T06:45:00.000-05:00

After more than 500 days, the Corps finally released the documents I had been asking for in the permanent pumps saga. It is obvious why they held them back.

In March, 2009 the Corps delivered a report to Congress on the releative costs of Option 1 (keep the existing outfall canal walls, levees, and pump stations, while adding a pump station at the end of each canal) and Option 2 (line the canals with concrete, get rid of the walls and levees, demolish the existing pump stations, and have new lakefront pump stations to handle all drainage). There's also an Option 2a, which adds a diversion of drainage from the Hoey's Basin section of Jefferson and Orleans Parishes. That stormwater would be routed to the Mississippi River.

The report presented these as the costs:

Over $3.4 billion dollars for Option 2. That's certainly an eye-popping number and makes Option 2 look inconceivably expensive compared to the $797 million for Option 1. The Corps, unlike everyone else - including the people protected by this project - wanted Option 1.

There's one problem: it appears those cost numbers are not the truth, according to these new documents.

In a December, 2008 internal cost estimate prepared by Corps consultants Black & Veatch as part of the effort to write the Report to Congress, Option 2's engineering and construction costs were estimated at about $1.8 billion, not the $3 billion+ number.

Let's make sure we're comparing apples to apples here. The internal estimate is strictly for the engineering and construction of the facilities. It leaves out stuff like land acquisition, relocations, and construction management. The line items from the Report to Congress for just the Option 2 pump stations, canal revisions, and engineering were these:

The sum of the highlighted items is $3.219 billion.

There's one further clarification: the internal estimate (a document titled, "Basis of the Cost Estimate Developed by Black & Veatch to Support the 2008 Report to Congress" - the mention of 2008 indicates the original deadline of the report, which was busted by the Corps) calls out a couple of different flavors for Option 2, one with breakwaters in front of the 17th Street canal and Orleans Avenue canal stations, one without. However, that's a minor distinction, because both are about $1.8 billion:

The estimate for the same items as reported to Congress is $1.6 billion more - 77% greater - than what the Corps' internal numbers showed.

Put another way, that's a difference of about double the Congressional appropriation of $804 million for the entire project.

But wait there's more.

The Report to Congress called out the total price for Option 1 as $797 million, conveniently just below the appropriated amount. I wrote quite a bit in 2009 about how the Corps likely slashed their Option 1 estimate to make it fit the appropriation, moving the costs to Option 2. The slashing is confirmed explicitly by the new documents, in a part when Black & Veatch is describing the process of updating their cost estimates from an earlier round performed in 2006:

"Material Costs Update. The 2006 estimate was then updated to current (October 2008) costs. There were significant upward cost revisions in many areas, the greatest being in the fabrication costs for the pumps and drivers. When these costs were added, the Option 1 cost estimate appeared to be very close to, perhaps over, the appropriated amount. This cost estimate was termed the “Robust Adaptable Option 1”. HPO leadership asked B&V to find ways to reduce costs, and an aggressive review of all costs was undertaken.

Base Adaptable Pump Station. Careful examination of the design concepts incorporated in the “Robust” design resulted in significant cost savings, and the development of a new design called the “Base Adaptable Option 1”. This concept and the estimate developed from it, became the basis of the “Base to Go”, the cost of Option 1 for the Report to Congress."

So the "Base Adaptable Pump Station" is the flavor of Option 1 that became the basis for the estimate in the Report to Congress. So what was the internal estimate for the Option 1 Base Adapatable flavor?

$480 million.

Just to make sure we're doing apple to apples, here's what the Report to Congress called out for just the pump station, small revisions to the canals, and engineering:

That total - released in March, 2009 - is $752 million. That's an inflation of 56% from the internal estimate published in December, 2008.

I'm sure there's lots of people out in the field who know loads more about Corps estimating than I do, and would very much like to poke holes in what I'm saying here. However, I believe such criticisms will likely amount to nibbling around the edges of what appears to be a massive overestimation of the costs of the project likely designed to perform two Corps objectives:

1) Make Option 2 look so costly as to be impossible to construct.
2) Make Option 1 look to fit perfectly within the Congressional appropriation.

The public suspected objective 1 was going on. After all, Orleans and Jefferson Parishes commissioned a study showing the actual costs of Option 2 were over a billion dollars less than what the Corps was reporting. However, objective number two, while speculation on my part, appears to be new news. It comes out of the very surprising result that the Corps was radically inflating the Option 1 estimate as well. To what end were they doing this? Perhaps to avoid having the appropriation slashed by Congress. If Congress knew the Corps believed Option 1 could be built for half what had been appropriated, there likely would have been pressure to cut the funds.

What this appears to come down to is that the Corps may have, let's say, "stretched the truth" in reporting the costs of the permanent pumping project to Congress and the public. A $1.6 billion difference in internal and external cost estimates for Option 2 is very tough to explain away, as is the 56% increase in Option 1 costs. But I look forward to the Corps attempting to do so.

5701 Pratt

2011-10-07T11:35:00.007-05:00

This is very odd.

A demolition permit for a privately owned house along the London Avenue canal was given to the contractor on the London Avenue canal remediation contract, Integrated Pro Services, on October 4, 2011.

Permit 11BLD-06904 authorizes IPS to demolish 5701 Pratt.

This property falls within an area the Corps said they would use as a staging area during the remediation repairs; it is not behind any of the areas actually remediated. Supposedly all the substantial work on the London Avenue canal is done. So what is going on here?

[UPDATE: More than likely, this is just for the removal of the contractor's construction trailer at this site (see the last page of the Corps' monthly remediation updates - such as this one from the end of April, 2011 - for confirmation of the location), and the New Orleans Safety & Permits department incorrectly typed in the permit as a private home. So it's probably not odd at all.]

[UPPERDATE: I have been told this demolition is actualy for flood protection purposes, and that the house was to be purchased by the Orleans Levee District, though the assessor's record doesn't reflect such a sale. According to a comment by the owner on an unrelated nola.com article last year, the purpose of the sale was to allow better access to the London Avenue canal levee.]

500 days

2011-09-06T19:21:00.000-05:00

Since I filed a FOIA request with the Corps of Engineers' New Orleans District for the following documents:

"1) The Comparative Cost Analysis generated by Black & Veatch under task order CZ05 of contract W912BV-07-D-1002 (original and modification attached). This report is described as a deliverable on page 7 of the original task order and is described on on page 4:

"Cost Differential Analysis. Review and analyze the cost differential and methodology between the 2006 Conceptual Design Report and the 2008 90-day study and report to congress."

2) The Basis of Cost Estimates generated by Black & Veatch under task order CZ05 of contract W912BV-07-D-1002 (original and modification attached). This report is described as a deliverable on page 7 of the original task order."

These are the reports the Corps has never released which show in detail why they chose the cheaper, technically inferior Option 1 (canal depths stay where they are, pumping stations at each end of each canal which must be coordinated every time lakefront gates are closed, New Orleans taxpayers must pay extra $10 million in annual property taxes upon completion of construction to pay for maintenance of three extra pump stations) over the favored-by-everyone-except-the-Corps Option 2 (canals depths dug to ground level and lined with concrete, single pump station at each canal mouth, no property tax increase).

But 500 days isn't the oldest FOIA request I have into the Corps. That goes to this one:

"1) Any and all emails, memoranda, and other documents related to the operation, maintenance, design, or any other matter relating to the pumps, gates, and all other associated equipment at the interim closure structures (17th Street, Orleans Avenue, London Avenue) sent or received by the following individuals between 12:01 AM November 5, 2009 and 11:59 PM November 14, 2009, as well as between 12:01 AM, December 6, 2009 and 11:59 PM December 16, 2009:

a) Colonel Alvin B Lee
b) Karen Durham-Aguilera
c) Christopher Accardo
d) Carl Robinson
e) Michael Stack Jr
f) Nancy Powell
g) Raymond Newman
h) Donald Constantine
i) Dan Bradley

Responsive documents should include all attachments to the emails, as well as the emails themselves.

2) After action reports for Corps New Orleans District activities undertaken in preparation for and during Hurricane Ida and the December 11-13 rainstorms."

The Corps doesn't want the curtain pulled back on exactly how they operate the gates (in the case of the Hurricane Ida event) or how they don't (in the case of the December event, when water repeatedly rose to dangerous levels in the outfall canals and the gates were never closed, at the same time it appears there was a systemwide 6-day-long failure of their SCADA system.

That FOIA request has been pending for 538 days. That places it and the other one in rarefied company. According to the latest DoD FOIA Annual Report , from fiscal year 2010, only 1.1% of all simplified or complex requests to the Army are answered in 400+ days. The average is supposed to be fewer than 50 days.

I'm not interested in setting a record. I wish they would just turn this stuff over.

Drip, drip, drip

2011-09-03T01:47:00.001-05:00

When we last looked in on the rusty lakefront pumps, two pumps had been pulled out after the start of the 2011 hurricane season due to oil leaks. Both were at 17th Street (W8 and W9). I know W8 went back in the water, and I'm assuming W9 is also back in.

But that still leaves 15 of the 40 large size (60") hydraulic pumps rusting away in the brackish Lake Pontchartrain waters. 9 of those 15 are at the Orleans Avenue site:

There's one at the London Avenue site:

And, with the assumed return of pump W9, five at 17th Street:

With 9 of the 15 rusty pumps at the Orleans Avenue site, the chances were much greater of an Orleans Ave. pump being the next one to spill oil. That's exactly what happened on August 10, 2011, according to this spill report from the National Response Center (the Corps seems to be improving their spill reporting):

Location: "ORLEANS AVE INTERIM CANAL STRUCTURE PUMPING STATION"

Description: "CALLER STATED THERE IS A LEAK FROM A PUMP AT THE PUMPING STATION. CALLER STATED THERE MAY BE A CORROSIVE ELEMENT THAT IS CAUSING THE SPILL. THE SPILL IS WITHIN THE STRUCTURE OF THE FACILITY. CALLER STATED THERE IS NO SPILL TO LAKE PONTCHARTRAIN."

Released material: HYDRAULIC OIL"

Let's parse that. First, the only corrosive element causing the spill was salt water.

Secondly, there was most definitely a spill to Lake Pontchartrain. The waters within the canals aren't somehow separated from the lake with some invisible barrier - it's all the same thing: navigable waters of the United States. Under the law, if there's an oil spill of any amount into those waters, it must be reported to the National Response Center.

Anyhow, the Corps claimed the spill was 20 gallons and that their spill response contractor - QRI - had been summoned. Activities like this always portend a removal of a pump.

Indeed, a pump was pulled out from the Orleans Avenue site: E3. Here's pictures of E3 out:

Steve Beatty tells me the pump remains out while New Orleans gets rain from Tropical Storm Lee on September 2. If it came out sometime around August 10th, it should be back in around September 10th.

However, that still will leave 80% of the Orleans Avenue pumps dangerously close to failure should they be needed during this hurricane season. The Corps could have issued multiple repair contracts for these pump repairs and had them finished a long time ago. Instead they stuck with a solitary small business vendor (Healtheon, with Conhagen doing all the actual repair work) for the last two years. Combine that with the sections of the Orleans Avenue canal levees possessing Safe Water Elevations of zero feet which were left unremediated this spring, and the Orleans Avenue canal is not looking so well prepared.

Steve Beatty of The Lens contributed to this report.

Debris Part 9

2011-08-30T19:45:00.006-05:00

Part 1
Part 2
Part 3
Part 4
Part 5
Part 6
Part 7
Part 8
Part 10
Part 11

Good news on the project at the center of the debris problems on the west bank: WBV-14c.2. That project has had serious problems with concrete, wood, steel, and other flotsam in its dirt supply.

As I've posted in the past, the debris laden dirt was coming out of a borrow pit owned by the River Birch organization, a landfill firm currently at the center of a large federal investigation of Jefferson Parish politics.

This month, the Corps finally (after nearly a year) told the WBV-14c.2 contractor (Phylway) to stop using the cruddy River Birch pit and to switch to the much cleaner Willow Bend pit near Donaldsonville. It was announced as part of an inspection of the site on August 9, 2011:

"USACE recently instructed contractor (via email) to change borrow pit and start hauling in material from the Willow Bend Pit for the remainder of the borrow material for the site. A modification to the contract will be issued in the incoming days.

Approximately 30,000 cubic yards are left to be hauled in."

That same day, the inspectors were told about a rather large chunk of concrete that had been dug out of the existing levee. Remember that this project involves using the existing levee to build up a new, taller levee behind it:

When one looks a picture of the concrete, it doesn't look so bad:

Then when one sees the size of it compared to a truck, it's clear how huge it is:

That was sitting in the levee for years before Katrina struck.

Another deadline missed

2011-08-19T06:40:00.000-05:00

Back in June, when the Corps had the press out to the West Closure Complex for their pump demonstration (when they turned on 8 of the 11 pumps), they claimed all 11 would be ready by the peak of hurricane season, defined as the period between Augst 15 and October 1:

"[Corps project manager Kevin] Wagner said the remaining three pumps are expected to be operational by the peak of the hurricane season."

As of today, according to SLFPA-W inspector Danny Caluda, 10 of the 11 pumps (nos. 4 through 13) have been wet and dry tested, but pump number 3 is still not ready, and is possibly not going to be for quite a while. It kind of makes one wonder about Mr. Wagner's designation by the Corps as Project Manager of the Year, especially the fact it was given before the project was even close to finished.

What are the consequences of this? Looking at the results of the modeling done by the Corps in connection with the consequences of the pumping shortfall from the Interim Standing Instructions attached to the WCC Water Control Plan:

Water elevations behind the stations are shown in the left hand column. The reason they stop at 8.2 feet is that is the elevation when stormwater - pumped from local pump stations into the the detention basin formed by the Harvey and Algiers canals - would overflow the floodwalls and levees along those canals. This table assumes two things:

1) The floodgates at the WCC are closed, necessitating turning on the WCC pumps to remove the stormwater from the detention basin

2) The local pumping stations are pumping continuously the whole time, a circumstance the Corps calls "unlikely," but we all know what happens with events the Corps calls "unlikely."

Along the bottom row of the table is the total amount of time for that overflow to occur. It happens because the WCC has no spare capacity built it; it can only keep up with the local pumping stations with all 11 of its pumps on. As you can see, with only 8 pumps it would take a little over 17 hours for the local pump stations to overwhelm the WCC pumps and send water into neighborhoods (either over the floodwalls or because local pump stations would be shut down). There are now ten pumps in place, meaning the WCC can operate for 50 hours continuously until the walls are overtopped or the local pump stations must shut down.

Note that assumes all ten pumps don't break down at any point during that time.

Inspections from the other side

2011-08-14T18:50:00.001-05:00

The inspection reports from the state-contracted inspectors on loan to the Southeast Louisiana Flood Protection Authority - West (SLFPA-West) have been invaluable in keeping track of the various storylines unfolding at the Corps projects on the west bank. I've personally spun out a dozen posts based on them.

The question has always been, "Why is it only the west bank flood protection authority has these inspection reports? What's the SLFPA-East been doing?"

Wonder no more. I've received the SLFPA-East inspection reports from January, 2010 through June, 2011. They are all bundled up in a single zipped file here. The file is huge - almost 800 megs - and contains about 1360 individual inspection reports and a handful of other related documents. The reports are organized by day and by inspector, with each day having up to five individual reports, each from a different inspector. If you can organize by the file path within your zipped file browser, it'll make it easier to view the reports in order. The inspectors seem to cover set geographical areas (St Bernard Parish, Orleans lekfront, etc), so each report contains information on multiple Corps projects. The inspectors were provided by PBS&J.

I have not reviewed all these files yet, but I wanted to make them available as soon as I could. Enjoy.

Decison day? Yes

2011-08-04T06:10:00.009-05:00

According to the GAO bid protest docket website (plug in "W912P8" in the "Solicitation Number" field to get all Corps New Orleans District protests, including the six actions on the permanent pumps solicitation of W912P8-09-R-0013), today is the deadline for the first of the permanent pump station protests to be decided. Of course, that might just be an artifact of the GAO's reporting, and the decision might not come for a couple of months. Still, I did want to put the news out there.

Update - later in the day

Maybe today wasn't the deadline, because no decision was recorded on the GAO's webpage. Perhaps the actual deadline is in October, as I surmised earlier. Hits on my blog today indicated keen interest, so maybe something happened and the webpage just didn't get updated.

Update - the next day

Huge news - the GAO upheld all the bidders' protests.

They updated their webpage today, indicating the protests were sustained. They also released a statement on the decision to the media, but not to the public yet. The Times-Picayune wrote it up with the lede:

"The process the Army Corps of Engineers used to award a $675-million contract for pump stations to a New Orleans firm that had hired an official who formerly worked for the agency was flawed, according to a decision from Congress's investigative arm."

This refers to the lead firm in the winning coalition - CDM - hiring Richmond Kendrick, the Corps' former Chief of Program Execution for the just shuttered Hurricane Protection Office (HPO).

It's remarkable that the GAO actually rejected a bid award because of the infamous revolving door between the Corps and their contractors.

There was some other stuff too:

"The GAO's decision noted several flaws in the process used to award the contract. In particular, the Corps failed to properly evaluate CBY's technical proposal for pump station operation and that bidders 'may have been misled about the role of price in the evaluation.'"

Corps stenographer Engineering News-Record also had a write-up, adding an extra detail:

"Related to pump station operation, GAO also notes a technical discrepancy, says White. "The Corps evaluation was flawed because the agency's requirements for withstanding lateral loads," he says."

I'll keep an eye out for any indication the revised bid specs have been circulated to the final five bidders, assuming all of them bid again. It's hard to see how CDM will succeed unless they remove the problem that led to the decision, but we'll see.

And of course, this means the rusty pumps will stay out there much longer. Even if they all have their carbon steel parts replaced with stainless steel, there's still unaddressed design flaws. And the testing they do - ten minutes every six or eight weeks (or longer - it's kind of whenever they get around to it) - doesn't prove anything other than they can turn on the engines.

Debris Part 8

2011-07-26T15:30:00.015-05:00

Part 1
Part 2
Part 3
Part 4
Part 5
Part 6
Part 7
Part 9
Part 10
Part 11

All of a sudden, the debris story grew legs over the last three days, with multiple reports in multiple outlets. In part it was due to concern over stuff like this:

We'll return to that a little later.

Media interest before SLFPA-W July board meeting

The spark for the sudden jump in interest was the Southeast Louisiana Flood Protection Authority - West's (SLFPA-W) July 25, 2011 regular board meeting, where the Corps was called to account for all the unresolved debris problems. The SLFPA-W board appears beyond fed up, as they released multiple documents to the media in an effort to overcome any Corps spin. By all accounts, they may have succeeded.

Before the meeting there were two reports, one from WWL-TV and one from Fox 8 (WVUE).

WWL's July 25th report included video from the SLFPAW of a large chunk of debris getting dug out of a stretch of extremely troubled project WBV-14c.2 (contractor: Phylway; borrow source: River Birch):

The video also includes a map of WBV-14c.2, seemingly taken from the SLFPA-W, showing the entire project is of concern:

The WWL report includes a number of quotes from a clearly fed up SLFPA-W President Susan Maclay, such as this one characterizing the debris:

"'... you have large pieces of bricks and concrete and rocks and steel rods and wood, almost tree trunks,' Maclay said. 'This is going to undermine the levee.'"

The WWL story also reveals (again through documents released by SLFPA-W) that some Corps Quality Assurance (QA) inspectors claimed they alerted Corps upper-ups about the problem:

"Even more disturbing, she said, is that the corps' quality assurance reports reveal that its own inspectors have also alerted corps management about the problem.

One entry says, '...again mentioned to CQC the importance of removing objectionable materials.'

'But management has not done anything about it,' Maclay said.

She said so far the corps' management has only responded with what she calls excuses, with the corps saying the amount of debris in the levees is within specifications.

'But if you read the specifications, clearly it's not,' Maclay said."

Based on the level of antipathy by the Corps toward the state inspectors we've seen recorded in the state inspector's reports, I think such excerpts from Corps QA folks need to be taken in perspective, but they are enlightening.

One other thing: while the WWL report indicates "CQC" in the quote above stands for "Corps Quality Control," that's wrong. It actually stands for "Contractor Quality Control." It is the contractor, not the Corps, which is responsible for quality control - like ensuring there's no debris in the material - on Corps jobs. Nevertheless, Corps upper management has long been aware of these problems.

Fox 8's first report aired a day earlier, on Sunday, July 24, 2011 (video can be found at link. I'm not embedding it because Fox 8's embedding doesn't seem to work right).

It highlights a July 15, 2011 letter from SLFPA-W President Maclay to Corps New Orleans District commander Ed Fleming (linked here), which says, among other things "the use of unsuitable material in the construction of this project has been pervasive and continues unabated." The letter concentrates only on three projects: WBV-14c.2, WBV-15a.2, and WBV-3b. The problems at WBV-14c.2 are well documented, as are those at WBV-15a.2, though notably the letter lists many other concerns at WBV-15a.2 besides debris.

WBV-3b seems to be a particular focus of the SLFPA-W. The inspection records available online do not give the full picture which undergirds that concern, but records recently released by the SLFPA-W do. How about this:

That is a huge log that was pulled out of the finished levee at WBV-3b on June 15th during part of the pre-final inspection of that project. But that's not the worst, because look what happens when the camera pulls back:

We can see giant chunks of concrete, what appear to be logs even bigger than the one in the detail above, and an assortment of junk that should make everyone at the Corps wonder what exactly their commanders were crowing about two weeks earlier on June 1st of this year, when they claimed the system was ready for a 100 year storm. And Maclay's July 15th letter indicates there was more:

"Because our inspectors reported finding large logs and other unsuitable material in this levee section [WBV-3b], the contractor [Shavers-Whittle] and USACE's construction division visted the site and removed utility trailers filled with unsuitable material."

That's "trailers," as in more than one. Yikes.

Interestingly, the Fox 8 report includes maps of the projects listed in the letter which appear to go beyond the projects highlighted. Specifically, this map:

The voiceover during the report says this shows the section of levee from the new Westwego pump station to Orleans Village. That description corresponds with the "official" description of only WBV-14c.2, which is located here, in the northern half of the highlighted section on the Fox 8 map:

The highlighted section on the Fox 8 map actually goes much further than the eastern end of WBV-14c.2, tracking all the way down to the end of another project with debris problems, WBV-14b.2:

I don't know what the source document for the Fox 8 graphic was. The July 15, 2011 letter from President Maclay does not explicitly reference WBV-14b.2, and the Fox 8 graphic misplaces the location of Orleans Village:

More than likely, this was simply a graphics error by Fox 8, even though we know WBV-14b.2 has also had debris problems.

Media reporting of the meeting

Anyhow, after the SLFPA-W board meeting, there was another flurry of stories. Again, WWL-TV and Fox 8 had reports (I'm only embedding WWL's video; Fox 8's can be found at the link because their embedding doesn't work too well):

The Times-Picayune also chimed in.

All three reports hit mostly the same notes. They all reported the same platitudes ("public safety is out number one priority" and such) and excuse-making by the Corps. From WWL's report:

"Monday, corps officials pointed out, the Westwego levee [specifically WBV-14c.2] is still under construction, and some areas analyzed could have been cleared of debris after inspectors left.

They said spot inspections often lack proper context."

This is pure BS. There are literally dozens of inspection reports, some on a weekly basis, with hundreds of photos taken along lengthy stretches of multiple projects showing debris all over the place. Indeed, we've seen the "we'll clean it up" excuse from the Corps' QA guys too many times to count. And the tone of such comments undermines the further BS about the Corps' work being a partnership with SLFPA-W and the state.

Perhaps it is then fitting that the New Orleans District is being shoved aside in part of their QA duties, as WWL goes on to report:

"[Fleming] said a new inspection team will be on the ground by next week, the number of inspections will be increasing and crews will trench the levees in some spots to determine how deep inside debris is located.

[...]

Corps officials said the new inspection team will come in from outside the New Orleans District."

Fox 8 says the outside QA team will be overseeing only the trenching:

"Also, a quality assurance team will arrive in town soon to oversee the trenching work."

And unfortunately, it doesn't appear that team will have much autonomy, as the T-P reports:

"Overseeing the efforts will be senior project manager Kevin Wagner."

Wagner is an old hand at the New Orleans District.

Of course, the trenching on the new levee is also a major development. We've already seen trenching occur on the existing levee at WBV-14c.2, when it was being used to build up the new levee:

It was found to contain a ton of debris, just like the new incoming dirt. The Corps tried to discount the findings of those excavations; fortunately the state did their own analysis which confirmed the excessive debris in the existing levee.

I personally think comments from SLFPA-W president Maclay in the first WWL story are the most on point:

"Maclay said it may well be necessary to tear down the levee and rebuild it from scratch."

Based on the reports, that sounds about right.

Rusty pumps - the summing up

2011-07-25T09:50:00.013-05:00

[Update, September 3, 2011: A pump at the Orleans Avenue structure - E3 - sprung a corrosion-related oil leak on August 10, 2011 and was pulled out shortly afterward. Details can be found at this subsequent entry.]

There's lots of matters the Corps of Engineers' New Orleans District would rather the public didn't know about, or at least pay attention to. Many times, it's the details of things. They say on June 1, 2011 they've met their goal of hundred year protection, but you find out there's dozens of unfinished projects. They point to how great the West Closure Complex is, but then you learn there were only 8 of 11 pumps ready at the beginning of hurricane season and an early season storm could force local pump stations to shut down. They say the levee system is better than ever, and then you find out it's filled with concrete, wood, steel, and other flotsam.

So it goes with the 54 hydraulic pumps installed at the three outfall canal sites in New Orleans - 32 at 17th Street, 10 at Orleans Avenue and 12 at London Avenue. 40 of them are the 60" type, while the other 14 - all installed at 17th Street - are the smaller 42" version (this post focuses on the 60" pumps, due to a severe lack of contracting information for the repairs to the 42" pumps). The Corps would rather the public didn't bother with such details as whether the pumps work under storm conditions, or if they have crippling design flaws, or if the salt water in Lake Pontchartrain is turning them into rusty heaps. That's because they know if the public were privy to all those pesky details, they might freak out over how scary the story is.

Thank goodness for the Freedom of Information Act, though, because it allows us to push past the well financed Public Affairs curtain and see the entire rusty pump story for what it is: a fiasco five years in the making.

The first signs - summer, 2006

It started nearly instantly after the pumps first went in the canals in June, 2006, with corrosion appearing on the hose connections from the hard piping to the pumps. Hydraulic oil is driven from hydraulic power units onshore through the piping and hoses to the pumps which sit in the canals. Here's a picture of those rusty hose connections: (via SCPR Flickr):

And here's a closeup of one of those rusty hose connections (via SCPR Flickr):

This was a hint that the environment around the waters of Lake Pontchartrain was very salty, and materials susceptible to corrosion - such as the carbon steel making up the pumps - would perform quite poorly. This shouldn't have surprised anyone, since Lake Pontchartrain is the second largest salt water body in the country.

Missing the big picture - 2006 through 2008

But over the next few months, the Corps and its paid consultants (many of which were former Corps themselves) would not look at the systemwide problem augered by that early rust, but would instead focus narrowly on a tiny sliver of the corrosion problem. They concentrated on the hose connections to the pumps which sat beneath or near the waterline as the pumps rested in the canals (these hose connections were at the opposite end of the hoses from those shown in the June, 2006 photos above):

Remarkably, their "solution" to preventing the rusting of those connections in early 2007 was the addition of more carbon steel to the pumps, in the form of piping extensions:

This raised the lower hose connections above the waterline, but also had the effect of giving much more surface area for rust and marine organisms to attack. It was like discovering your car's engine was on fire, and deciding to check the spark plugs - while wearing a suit covered in gasoline!

With all that yummy carbon steel waiting like a buffet, the marine organisms in the Lake Pontchartrain water started chomping away at the coar tar epoxy paint, exposing the bare metal below in no time. Corrosion of the steel quickly followed. An oil spill from pump E5 at the 17th Street site on June 26, 2008 was a hint of things to come. The Corps paid over $10,000 to clean it up, but never bothered to report it to the federal or state authorities.

A month later a solicitation for comprehensive pump maintenance for pumps across all three sites was issued. It was a small business Section 8(A) set aside, which is a rather odd choice for such a critical function as maintenance of pumps upon which the residents of the City of New Orleans relied for protection of life and property. Nonetheless, the contract was awarded August 28, 2008 to small business Healtheon.

Early self-repair efforts prove inadequate - Spring, 2009

By early 2009, the hydraulic oil which powered the pumps and which was contained within the pipes was seeping regularly from multiple pumps, and sometimes gushing into the canals. A halting repair effort began in March, at first concentrating only on pumps with known leaks.

The Corps attempted to repair them on their own, sometimes going to interesting lengths to conceal their efforts from the public. But after six pumps had to be pulled out in three months, accompanied by five oil spills (four of which went unreported) costing over $30,000 to clean up, it was clear professional help was needed.

Professionals are brought in - summer, 2009

Task orders started going out on the Healtheon contract in July, 2009. Healtheon hired Conhagen, Inc of Kenner, LA to perform the repairs. Conhagen had been performing similar services for the New Orleans Sewerage & Water Board for many years, and indeed it is unclear why the Corps simply didn't hire them directly.

The first task order, on July 21, 2009, called for the partial repair of four 60" pumps at the 17th Street site. New and more plentiful zinc anodes would be attached to the pumps to fight future corrosion, and bits of the pump housings eaten away by rust would be weld repaired.

Also, all the carbon steel piping on the pumps would be removed and replaced. However, only some of it - the piping inside the pump bore - would be upgraded to more corrosion resistant 316L stainless steel. The piping and oil coolers on the outside of the pumps, which sit in the same salty water as that inside the pumps, would remain carbon steel. Critical moving parts inside the pumps, like bearings and the Rineer hydraulic motors, would also not be replaced on a comprehensive basis. The pumps the Corps had pulled themselves earlier in 2009 also received this rather limited set of repairs, including two other 60" pumps. In sum, this work only addressed about a third of corrosion damage with upgrades, replacing the rest of the rusted or damaged pieces in kind with exactly the same materials.

Fortunately, one of the contractual requirements was generation of reports by the contractor on what they found on these pumps when they came out of the water. The pictures in those reports were disturbing:

Here's a section of pipe removed from the pump sitting on the table at Conhagen (the fitting at the bottom indicates it may be from the inlet or outlet of one of the hydraulic coolers)

It appears that a large chunk of the pipe is completely gone in this photo. Also note the rusty hose fitting on the table:

Here's one of the oil coolers mounted to the sides of the pumps, with a generous coating of marine growth:

These coolers are meant to lower the temperature of still-hot hydraulic oil passing through after leaving the Rineer hydraulic motor but before returning to the reservoir on the engine skid back on the landside of the canal. The residual heat is supposed to be transferred to the canal water, but such severe fouling would undoubtedly hamper that from happening.

Here's some of the piping extensions installed over the winter of 2006-07, specifically the return piping (two larger lines) and the smaller case drain line:

It was clear there was an urgent problem facing the Corps - systemwide corrosion of all 54 pumps within two years of their acceptance for service. Indeed, Conhagen said so in their summary of the repairs, terming the corrosion-related failure of the Corps' other hydraulic pumps "imminent." However, the Corps continued on a rather slow pace of repairs, only contracting with a single repair shop and never sending more than four pumps out at any given time, even outside of hurricane season.

Repair scope widens - spring, 2010

Indeed, because of the Corps' slowness to respond - despite the dire warning from Conhagen - nearly seven months went by before any more pump repairs took place. In February, 2010, a task order for further repairs was finally issued, calling for two more pumps to be pulled out at the 17th Street site. They would receive a greatly enhanced set of repairs compared to the pumps fixed up in 2009. All the carbon steel piping - inside and out - would be replaced with stainless steel. Rotating parts like the Rineer motors and the bearings would also be replaced. Seals would be replaced, and new gland plates holding those seals would be manufactured of sturdier materials. All fasteners would also be upgraded to stainless steel. Essentially, the pumps would be completely rebuilt, inside and out.

It's a good thing the Corps finally expanded their focus to include the external piping, because here's one of the oil coolers on those first two pumps pulled in 2010:

It's also a good thing they started systematically replacing bearings, seals, and Rineer motors, because they were taking serious damage. Here's one of the Rineers from that same pair of pumps:

And here is the area around the shaft and bearings, showing obvious evidence of water infiltration:

From the amount of rust inside the bearing housing, as well as the significant discoloration along the shaft, it is clear the seals completely failed. This would have exposed the bearings and shaft to salt water, causing failure of the pump. And there is no doubt similar damage had been done to the seals on the Rineer motor, damaging it as well.

The pace of repairs would increase throughout the spring of 2010, with eight more 60" pumps going to Conhagen's shop for the upgraded rebuilds spread over two additional task orders. The pictures from the repair reports continued to show extensive corrosion of all pumps. Rust covered the outside of the pumps, such as 17th Street pump E2...

...as well as the interior, as seen inside London Avenue pump E4:

As with 17th Street pumps W5 and W6 earlier in 2010, there was also evidence of water infiltration into the parts of the mechanism that were supposed to be sealed. Here's part of the drive of 17th Street pump E6 showing corrosion on surfaces supposedly protected by seals:

This should not be showing any signs of rust, but there they are.

There was substantial corrosion damage to the main pump housings as well. At the very bottom of the pump is the suction bell, the tapered portion of the pump which funnels water up to the impeller. The funnelling effect is assisted by tapered vanes. The suction bell is always submerged, and the brackish water took its toll. Here's 17th Street pump E2's suction bell:

and 17th Street pump E6's suction bell:

Conhagen addressed all these areas of concern in their repairs that spring.

Work slows but corrosion continues - summer & fall 2010

The work on these 8 pumps was performed under a new contract given to Healtheon, this one with a scope specifically tailored to the corrosion repairs. But its funding - a potential value of just under $3 million - was horribly inadequate to deal with the entire problem. As the 2010 hurricane season dawned on June 1, there were still 30 of the 60" pumps in the water with either partial (5 of them) or no repairs to arrest the ongoing corrosion damage, and the Corps had already spent half of the $3 million repairing just 8 pumps. Fixing the other 30 would take at least $4.8 million, meaning a shortfall of at least $3 million. The effects of that lassitude would be seen immediately.

The next day, June 2, 2010, pump W1 at the 17th Street site would spring a leak, dumping 15 gallons of hydraulic fluid in the canal. It would be pulled out two weeks later and sent to Conhagen's shop for a rebuild. The leak should not have been a surprise in light of the extreme corrosion found on the pump's external piping:

The next month, pump W1 at London Avenue came out, and pump W4 at Orleans Avenue was sent to the shop in October. Both showed typical corrosion damage. For example, the Rineer motor in Orleans pump W4 looked particularly rusty:

While there were no spill reports for these two pumps, I bet the Corps pulled them out because they found oil in the water and didn't report it.

These three were the only 60" pump repairs during the 2010 hurricane season, since the Corps shut down their scheduled repair effort when the season started in June, another sign of a lack of urgency. In this case, it was brought about by another poor Corps decision, one made in 2006 and 2007 during the construction of the interim closure sturctures. The Corps did not install any spare pumping capacity at the structures, meaning they are unable to repair pumps while providing the required ability to drain the canals when the gates are closed against a storm surge. They have duplicated this error at the massive West Closure Complex.

Repairs restart in earnest - fall, 2010 through spring, 2011

After the end of the 2010 hurricane season, more scheduled repairs with the enhanced scope were tasked to Healtheon. Two pumps were pulled out at the London Avenue site in December, an event accompanied by not one, but two oil spills, totalling 125 gallons. Again, there was major corrosion found on the external piping:

With the Corps approaching the cost ceiling on their second pump repair contract, they solicited for a third one in October, 2010 and awarded it in February, 2011. Once again, Healtheon was the winner, with Conhagen doing the actual repairs. This time, the cost ceiling was $6.75 million, over double the potential value of the prior contract. This should be more than enough to finish off the repairs (they average about $160,000 per pump) and have about $3 million left over until the permanent pumps come online in 2015. Why did it take them over two years to figure out they needed more cash for these repairs? I don't know.

This new contract was put to use immediately, with eight more pumps getting pulled out under two task orders on it in the spring of 2011. Again, the pictures in the Conhagen reports demonstrated these pumps should have been repaired years earlier:

And while these sorts of pictures and repairs were becoming depressingly typical, there was a twist in one of those spring 2011 task orders.

One of the pumps pulled out in March, 2011 was E1 at the London Avenue site. When the pictures from the repair report for that pump were examined, it was revealed that London Avenue E1 had already been out for repairs two years earlier, in the spring of 2009:

This showed the Corps did not have confidence in the 2009 partial repairs, and considered the pumps fixed that year as just as vulnerable to corrosion-related failure as those with no repairs. That would be proven just months later.

Earlier repairs prove inadequate - summer, 2011

As in 2010, the Corps shut down their scheduled pump repairs just before the beginning of the 2011 hurricane season. At that point 17 of the forty 60" pumps either had partial or no repairs. And just as in 2010, some of those pumps started spewing oil nearly immediately.

It is tough to nail down the exact date due to lack of federally required reporting, but sometime in May or June, 2011, pump W8 at the 17th Street site was leaking oil. It got yanked and sent to Conhagen's shop for a rebuild. W8 was a pump that had been part of the incomplete 2009 repairs, proving those repairs were effectively money flushed down the drain. While the parts of the pumps which received upgrades in 2009, like the interior piping, appeared to do well...

...the other components which were replaced in kind in 2009, like the oil coolers made of carbon steel, didn't do so well:

The total expense for both times this pump was pulled out would come to approximately $227,000. The average expense for these pump rebuildings has been around $160,000, meaning about $67,000 was wasted on this pump.

This story would be repeated just a month later, as pump W9 at 17th Street would also start leaking (the Corps reported that one) and get sent to Conhagen for a rebuilt. It too was a 2009 partial repair. Three more 60" pumps "fixed" in 2009 remain in service at 17th Street, along with an unknown number of 42" pumps.

Where we stand - hurricane season 2011

Once 17th Street pump W9 goes back in the water near the end of this month, the Corps will have rebuilt 25 of the 60" hydraulic pumps over a span of about two years (three of them twice), leaving 15 of them effectively in jeopardy of failure during a storm due to leakage of hydraulic oil, which is used to power the pumps. Here's the individual statuses at each site as of the most recent confirmed repairs (17th Street's W9 will not be included until its repair is confirmed through documentation):

Remember they were aware of this problem within a month of installing the pumps in 2006 but only started addressing it in 2009. And a very long two years after that, 38% of the large pumps (15 out of 40) remain in imminent danger of failure when called upon in a storm because of corrosion. Along the way there have been at least nine oil spills totalling hundreds of gallons which have gone unreported to federal authorities, as required by law.

Then there's the dollars. The Corps have spent over $4.3 million on these repairs, and that's just on the 60" pumps. There's likely another $2.2 million to be spent rebuilding the remaining 15 60" pumps. That's on top of the $37.8 million they spent originally on those pumps buying them and trying to make them work in 2006 and 2007, which included a $4.7 million delivery incentive.

That makes the total spent on each rebuilt 60" pump an average of $1.1 million. The original contract price for each pump - before all the modifications on the original contract and all the repairs over the last two years - was $750,000. Thus - on average - the Corps will have spent, once the repairs are done, nearly an additional 50% of the original purchase price since 2006 on each of the 60" pumps.

This repair "effort" has seen zero urgency, inadequate funding, a remarkable ignorance of the Clean Water Act, and months of wasted work in 2009 which must now be redone at extra expense. From beginning to end, the Corps has shown a deep reluctance to do the right thing: yank all the pumps out and send them to wherever they need to go to be repaired. Once they started fixing them in 2009, everything should have been done before June 1, 2010. Instead, at the lazy pace they've set, all the pumps will not be rebuilt until just before the 2013 hurricane season, two years from now and seven years after the initial identification of corrosion as a serious problem. This is unacceptable.

Many but not all of the prior entries in this series are linked throughout the report above. For convenience, here's the complete set:

Imminent, originally posted May 13, 2010, covers the 2009 repairs

How did the pumps get from..., originally posted May 27, 2010, covers the original 2006-07 decisions on corrosion

This year's scramble, originally posted June 3, 2010, covers the spring 2010 repairs

Worse than previously known, originally posted June 11, 2010, also covers the spring 2010 repairs

Corps of oil, Part 1, originally posted June 20, 2010, covers oil spills from 2006 and early 2007.

Corps of oil, Part 2, originally posted June 21, 2010, covers oil spills in later 2007

Corps of oil, Part 3, originally posted June 22, 2010, covers oil spills from 2008 through 2010

No urgency, originally posted August 18, 2010, reports on repairs during summer, 2010

Quick update (with pretty new trees!), originally posted August 30, 2010, gives a summary of corrosion repair events to date at the fifth anniversary of Katrina, as well as revealing the Corps' wasteful spending on tree planting around the closure sites

The latest on lakefront pump repairs, originally posted March 23, 2011, looks at the repairs from summer 2010 through early 2011

The 2011 pump rebuild scramble, originally posted May 9, 2011, covers the spring, 2011 repairs

Brief update on London Avenue pumps, originally posted June 24, 2011, also covers the spring, 2011 repairs

Finally, originally posted July 13, 2011, provides an update on receipt of FOIA-requested documents

No surprise, originally posted July 15, 2011, looks at the summer, 2011 re-repairs of pumps also pulled out in 2009

No surprise, Part 2, originally posted July 20, 2011, covers possible contamination of the pumps' hydraulic fluid with water

No surprise, Part 2

2011-07-20T13:12:00.003-05:00

Over the last two years, we've learned many, many of the Corps' 54 hydraulic pumps have been leaking oil into Lake Pontchartrain as they rust away. Sometimes the Corps reports these spills - as they must under federal law - and sometimes they don't. And sometimes pumps are scheduled to come out for rebuilding, while other times pumps are yanked as a result of spilling oil.

The places oil can leak out on these pumps are numerous. They include a slew of different pipes on each pump through which hydraulic oil (which powers the pumps) flows; most of that piping sits in the brackish water of the lake 24 hours a day. When the pumps aren't running, the oil sits in the piping on the pumps and in the hundreds of feet of pipes leading away and from the pumps, all the way back to the reservoirs on the hydraulic power units (HPU's) located on the land sides of the canals.

The ongoing corrosion of these completely or partially immersed pump-mounted pipes, which were carbon steel when originally installed, leads directly to leaks. While the Corps has had their contractor yank out 25 of the 40 major 60" pumps located across all three sites (three of them twice) and replaced that carbon steel piping with stainless steel, the remaining 15 60" pumps still have the rust-friendly carbon steel piping on them. As we've seen in earlier posts, the salt water does a number on this piping:

That photo is of an oil cooler on pump W5 or W6 from the 17th Street site. It was taken a mere two and half years after these pumps were accepted for service by the Corps. In actuality, the pumps started spending time in the lake waters in mid-2006, but they were taken out and put back in so many times over the following year while dealing with deep-seated design issues (some of which remain unresolved to this day) that the corrosion clock only started started running consistently in the latter half of 2007.

There are many, many more pictures of rusty pipes like this from the Conhagen repair reports:

With all this rust causing all these oil leaks, it makes sense to ask, "If the oil is leaking out of the pumps, doesn't that mean water is leaking into them?"

The answer is "Yes." When the pumps are not running, the oil is not pressurized by the HPU's. So oil can seep out of leak points and water can come in. If the Corps turns on the pumps with those leaks and starts circulating all this oil mixed with water through the entire hydraulic oil loop, the water gets into the delicate parts on the HPU and can do serious damage.

That's why it should be a standard part of the repair to completely flush out the entire hydraulic loop for any pump that get's pulled out for repairs. That would include all the connecting piping from the pump back to the onshore HPU, as well as the HPU itself. Natually, though, the Corps doesn't do this. It's too expensive (for them, not for normal engineering organizations). When the pumps were put in back in 2006, there was pickling fluid remaining in the pipes when they were intially filled with oil. The resulting jello-like muck took two flushings to remove, at an expense of hundreds of thousands of dollars paid for by the U.S. taxpayer, despite the problem totally coming out the vendor's poor performance. I imagine that expense is what keeps the Corps from cleaning out the connecting pipes every time they yank out a pump for rebuilding.

So that just adds one more question about the performance of these things if they're ever needed in a major storm (earlier storms when they were turned on were absolute pea shooters). And it should come as no surprise that problems have cropped up directly related to contaminated oil.

On December 9, 2010, the Corps issued task order #7 on the second Healtheon/Conhagen pump repair contract (there have been three such contracts - more evidence of the Corps inability to plan ahead). The task order called for the rebuilding of London Avenue pumps E2 and E6.

I detailed the facts surrounding this set of repairs in my earlier post, "The latest on lakefront pump repairs." While there were some bumps along the way during these repairs, including apparently two oil spills totalling 125 gallons, the repairs themselves seemed to go somewhat smoothly (except for a distinct lack of information on the Rineer hydraulic motors). As expected, the pumps were in terrible condition when pulled out. In fact, one of the sample pictures above of really bad rust was taken from the Conhagen repair report for these two pumps:

The pumps were put back in the canal January 24, 2011.

However, that wasn't the end of the story. Just about a week after the pumps returned to service, the Corps issued $22,188.68 modification 2 to task order #7, increasing the cost for the repair of these two pumps to $304,898.07. The scope for the modification, knowing what we know about the oil leaks on all these pumps, should come as no surprise:

"- Take oil sample out of reservoir have analysis performed.
- Drain and pump oil out of reservoir, hoses, piping, etc. and flush to remove all contaminated oil.
- Remove the hydraulic pump, motor, coolers, and piping and flush to remove all contaminated oil.
- Dispose of contaminated oil
- Furnish new replacement hydraulic oil
- Refill reservoir on test unit
- Provide standby crane for this additional work.

Completion Date:
Work shall be completed within one week from the modification effective date."

The verbiage of the task order, as well as its price, lend itself to a couple of interpretations:

1) The Corps discovered the oil in either the E2 or E6 hydraulic loops at the London Avenue site was contaminated and had Conhagen remedy the situation. The fact that this modification was issued a week after the pumps were acceptance tested argues for this, although admittedly the Corps has a long history of post-dating their contract actions.

2) Conhagen, while they were pressure testing the pumps at their facility in Kenner, LA with one of the Corps' spare hydraulic power units, encountered a problem with contaminated oil and remedied it. The mention of the reservoir on the "test unit" argues for this. We have a picture of what a typical oil pressure test at Conhagen's site looks like, from the repair report for 17th Street pumps W4 and W7:

Whether the contaminated oil was encountered at Conhagen's shop, or out at the site, the larger point about likely widespread oil contamination still stands. and adds just one more question mark above the potential future performance of these pumps.

No surprise

2011-07-15T06:56:00.015-05:00

This post is one in a series I've been running since May, 2010 about the corrosion-related repairs to the 54 hydraulic pumps at the Corps' three New Orleans outfall canal sites. I've only recently started to receive contract documents and reports on these repairs via the Frreedom of Information Act.

A post from earlier this week provides links to the earlier articles in the series. Since the documents released date back almost two years, many of those earlier articles have been heavily revised to reflect the newly gained knowledge from the documents.

But the released documents also included information on very recent events surrounding these pumps...

In the last post, I noted that two pumps at 17th Street (W8 and W9) had been pulled out in the last couple of months. Those two pumps were already pulled out in 2009 for repairs, yet here they are coming out again. Why is that happening?

Because the parts that weren't overhauled in the 2009 repairs - which make up most of the pumps - gave out.

Just as a reminder, the repairs to pumps in 2009 (covered in earlier posts here and here) only concerned themselves with replacing carbon steel piping on the inside of the pumps with stainless steel:

The piping on the outside of the pumps was replaced, but with the same carbon steel material that had rusted in the first place. Then it was repainted with the same paint as before as well:

Had there been new stainless steel piping on the outside of the pumps, it would appear the same as the silvery stainless steel piping shown in the picture of the inside of the pump above.

In addition, all the moving parts like the bearings and the Rineer motors were also put back.

So essentially, about a third of what needed to be fixed was fixed, and the pumps were put back in the brackish Lake Pontchartrain waters to start corroding again. Six 60" pumps - five at 17th Street and one at London Avenue - as well as an unknown number of 42" pumps at 17th Street received this suite of repairs. It is no surprise that these pumps would be subject to the same corrosion damage as any other, mostly-carbon steel piece of equipment sitting in salty water.

We saw the first evidence of this earlier this spring, when pump E1 at the London Avenue site was removed as part of a planned pump pull. I only recently figured out that this pump was repaired in 2009 in addition to the five 60" pumps at 17th Street.

The London E1 stainless steel interior pipes, installed in 2009 and seen in the photo above, did not appear severely rusted in spring, 2011...

...while the carbon steel external piping and coolers looked like they took some serious corrosion damage since June, 2009:

[More details on the 2011 repairs to this pump are in my earlier post, "The 2011 pump rebuild scramble."]

That this pump was taken out as part of a planned complete overhaul in the spring of 2011 is evidence the Corps recognized the 2009 repairs were inadequate. But with many other pumps completely untouched since 2007, they triaged and focused on the more damaged ones rather than the other five pumps repaired in 2009.

However, it was only a matter of time before the pumps partially fixed in 2009 also started breaking. That's what happened with 17th Street pumps W8 and W9 in the last two months.

The first failure appears to have occurred in May or June of this year. We learned this from $89,489.55 task order #8 on the second Healtheon contract, issued on June 7, 2011. Prior to this, the second Healtheon contract had lain fallow since the issuance of task order #7 and its immediate follow-up modification on December 9th and 10th, 2010. After that, the 60" hydraulic pump repair work had been moved over to the third Healtheon contract, which was inked in January, 2011. Issuance of a task order on a contract previously thought ended was a little odd, but not unprecedented. The first Healtheon contract has continued to get task orders issued on it as well.

Anyhow, the body of task order #8 reads,

"Remove pump # 8W at 17th St. Canal to inspect, recommend what need to be repaired and reinstall it.

Completion of this work will be six weeks after notice of award.

Note:
1. Pump # 8W was overhauled in Aug 2009 and is currently leaked [sic] outside of the pump.
2. After the contractor inspects and finds out what the pump problems are, we will provide Contracting Division a separate scope of work for repair."

So the Corps had discovered an oil leak on pump W8. Did they report it to the National Response Center, as required under federal law? Apparently not, because searches on the NRC website show no reports for such an incident prior to June 7, 2011. That's really no surprise - the Corps has little regard for the law on reporting oil spills.

Pump W8 was pulled out of the 17th Street site and sent to Conhagen's shop. A $30,178.32 modification to task order #8 was issued June 20, 2011:

"The following scope of work was found necessary upon removal and inspection of Pump #8W at the 17th St. ICS:
- Remove hydraulic motor and motor adapter. Replace motor with gov-furnished new one.
- Build up weld eroded and corroded areas on motor adapter and other pump surfaces.
- Replace oil coolers and connected piping. Spot prep minor corrosion on other exterior piping.
- Replace small bypass hose for motor case drain. Replace exterior pipe u-bolts.
- Provide a full coat of epoxy paint on pump assembly in addition to touch up of original scope.
- Have mechanical seal reconditioned at subcontractor’s shop (to be returned to gov). Install new gov-furnished seal."

I would have expected the scope to simply be a restatement of the normal scope of work, with perhaps the exception of replacement of the interior piping. Instead, we find this oddly worded scope, which is rather lacking in detail. Nevertheless, it appears to cover much of the work which would be expected.

Fortunately, Conhagen had a better grasp on what needed to be done. The Conhangen repair report for pump W8's second time in their shop gives a detailed scope which covers everything needed to bring this pump up to same status as other completely rebuilt units.

That report shows the areas which one would expect to be the most damaged - those not upgraded in the 2009 repairs - were indeed the most damaged. The carbon steel oil coolers installed in 2009 had unsurprisingly accumulated a great deal of marine growth and had rusted:

Admittedly, the damage wasn't as great as we have seen on other pumps, but it is evident. Here's a similar area on one of the pumps after it was repaired in 2009:

The Rineer motor and its mount were also rusty:

Here's the "before" picture of that area in 2009:

Conhagen fixed up the pump, leak tested it, and put it back in the water on June 6, 2011, according to the acceptance testing record included in Conhagen's report.

Wait, what?

The original task order for the removal of this pump was dated June 7, 2011, and the follow-up modification with the scope of repairs was dated June 20, 2011. So how did the pump go back in the water before there was any contract for repairs?

It appears this is once again the Corps' Contracting Division issuing after-the-fact task orders and modifications for work already completed. In fact, the cover page of the Conhagen repair report indicates their work on the pump actually started three weeks before they received their first contractual documents:

One has to wonder how exactly contracts are run at the New Orleans District, when we find stuff like this.

So apart from the contracting slight-of-hand, I think we can take the Conhagen photos of W8's repairs in May and June of 2011 as proof that it has been fully rebuilt, meaning the 17th Street pump status can be updated again. However, the fact that W8 after previously being partially repaired still sprung a leak means that the remaining 60" pumps fixed in 2009 are now not just questionable, but effectively unrepaired. So the revised 17th Street status is:

This now means there are sixteen 60" hydraulic pumps across all three sites which cannot be trusted to be free from corrosion-related failures in the future:

17th Street: E5, E7, W2, W3, W9, W10 (W2 & W3 never previously repaired)
Orleans Avenue: E1, E2, E3, E4, E5, W1, W2, W3, W5 (all never previously repaired)
London Avenue: W2 (not previously repaired)

Indeed, this assertion was confirmed just last week, as 17th Street pump W9, another of the pumps only partially repaired in 2009, also sprung a leak.

This time, there was an oil spill report to the National Response Center, on July 5, 2011:

"Official Material Name: HYDRAULIC OIL
Qty Released: 8 GALLON(S)
Qty in Water: 8 GALLON(S)
CALLER IS REPORTING A DISCHARGE OF HYDRAULIC OIL FROM A HYDRAULIC PUMP DUE TO UNKNOWN CAUSES.
REMEDIAL ACTIONS: CLEANING UP SPILL WITH ABSORBENT PADS WITHIN THE BOOM SYSTEM."

Two days later, on July 7, 2011, the Corps issued $89,489.55 task order #9 on the second Healtheon contract. That dollar amount is exactly the same as the initial task order (#8) on 17th Street pump W8 in June. This time, the task order reads:

"Remove pump # 9W at 17th St. Canal to inspect, recommend what need to be repaired and reinstall it.

Note:
1. Pump # 9W was overhauled in Aug 2009 and is currently leaked [sic] outside of the pump.
2. After the contractor inspects and finds out what the pump problems are, we will provide Contracting Division a separate scope of work for repair.
3. Estimated date to start is 7/8/ 2011 and overtime for the contractors to work on Saturday 7/9/11 and Sunday 7/10/11 should be authorized."

Like the dollar amount, this is a copy and paste from the initial task order for pump W8, down to the typo on the work "leaked."

Since my receipt of documents via FOIA only covered stuff through last week, I don't have the task order modification which covers the actual repairs of 17th Street pump W9, but I am certain it looks nearly identical to the corresponding modification for pump W8. Right now, pump W9 is sitting on Conhagen's shop floor, and it will probably go back in the water before July 31.

The pattern of pump repairs over the past two years has been for scheduled rebuild work to only be undertaken outside hurricane season, meaning from December 1 to May 31st. If individual pumps fail during or close to hurricane season, as 17th Street pumps W8 and W9 have in the past couple of months - or as three other pumps did during the 2010 hurricane season - they are yanked out and rebuilt. Everything else - including pumps that have been rusting for nearly five years now - is left in the canals. This is a remnant of a foolish Corps decision made after Katrina - and carried out for two years of engineering and construction in 2006 and 2007 - not to supply any spare pumping capacity at their closure structures.

At the pace demonstrated by the repair contractor over the past two years, it seems impossible all 16 remaining 60" pumps will be completely rebuilt by June 1, 2012. Only ten pumps each were rebuilt in the 2009-10 and 2010-11 hurricane off seasons. So it seems likely - barring further (admittedly very likely) oil spills from unrebuilt pumps - there will still be rusty pumps in the canals all the way through the 2012 hurricane season.

Finally

2011-07-13T11:25:00.004-05:00

On July 8, 2011, the Corps finally started turning over the records on the rebuilding of the hydraulic pumps at all three outfall canal sites. I had requested these documents through the Freedom of Information Act over a year before. In the interim, there were a number of additional repairs, and many documents associated with those actions were also given to me.

As a result, I am currently heavily editing my previous entries about the pump fixes, adding in pictures and descriptions of the repairs taken from Corps contract actions and reports written by the pump repair shop, Conhagen, Inc, located in Kenner, LA.

Previously, I had only received a single Conhagen report which covered the repair of two pumps. I now have an additional nine reports documenting the repairs of 15 other pumps.

Here's links to the reports:

17th Street pump W10 and bridge pumps #1 and #6 (May-July, 2009) - no Conhagen report generated because the Corps did the work themselves

17th Street pumps E5, E7, W8 and W9 (July-September, 2009) (report received previously)

17th Street pumps W5 and W6 (February-March, 2010)

17th Street pumps E1 and E2 (March-April, 2010)

London Avenue pumps E3 and E4 (March-April, 2010)

17th Street pumps E6 and E8 (April-June, 2010)

London Avenue pumps W3 and W4 (April-June, 2010)

17th Street pump W1 (June-July, 2010)

London Avenue pump W1 (July-August, 2010)

Orleans Avenue pump W4 (October-November, 2010)

London Avenue pumps E2 and E6 (December, 2010 - January, 2011)

London Avenue pumps E1 and E5 (March-April, 2011)

London Avenue pumps W5 and W6 (April-May, 2011)

17th Street pumps W4 and W7 (March-May, 2011)

17th Street pumps E3 and E4 (May-June, 2011)

17th Street pump W8 (May-June, 2011)

17th Street pump W9 (July, 2011 - ?) - pump currently under repair.

Sharp eyed readers will notice the last two pumps, W8 and W9 at 17th Street, had supposedly already been rebuilt in 2009. However, those rebuilds only replaced a few of the carbon steel pipes, not all of them. They also didn't touch the bearings, Rineer motors or any moving parts. Unsurprisingly, these pumps have failed again, this time from the unrepaired parts.

The entries I'm updating or have already updated include:

Imminent, originally posted May 13, 2010

This year's scramble, originally posted June 3, 2010

Worse than previously known, originally posted June 11, 2010

The latest on lakefront pump repairs, originally posted March 23, 2011

The 2011 pump rebuild scramble, originally posted May 9, 2011

Brief update on London Avenue pumps, originally posted June 24, 2011

Other posts might also get updated or revised. And there will be coming posts on the current repairs, or "re-repairs" as it were.

In the meantime, here's some photos from the newly received reports...

A hole rusted straight through one of the oil coolers on pump W5 or W6 at 17th Street (report here):

Rust inside the supposedly sealed bearing housing on pump W5 or W6 at 17th Street (report here):

Extremely severe rust on London Avenue pump E4 oil cooler and attached piping (report here):

Identical corrosion on 17th Street pump W1 (report here):

Chunks missing from inlet suction bell on Orleans Avenue pump W4, due to corrosion (report here):

Other chunks missing from inlet suction bell on 17th Street pump E6 (report here):

More chunks missing from vanes inside suction bell on 17th Street pump E2 (report here):

Rusty Rineer hydraulic motor and attached piping on London Avenue pump E2 (report here):

Similarly rusty Rineer motor and piping on Orleans Avenue pump W4 (report here):

There's much more where that came from. But the important thing to remember is this: there are currently at least 16 pumps in the water which have had inadequate or no repairs to address these problems, which were evident after just two years in the canals. That includes 9 of the 10 pumps at the Orleans Avenue gates. Those 18 pumps have been in the canals for over four years now, and the only reason the Corps pulls them out is if they spring an oil leak.

Permanent pumps slip to Katrina 10th anniversary

2011-07-01T10:23:00.025-05:00

[Updated July 12, 2011 - see bottom of post]

[Corps confirms 2015 permanent pump completion. See July 16, 2011 update at bottom of post.]

In my last post, I mentioned the outfall canal updates the Corps gives the Southeast Louisiana Flood Protection Authority - East. In the most recent update, from early June, there was also a slide about the permanent pump station project:

It indicates that the completion date for the permanent pumps will likely slip to 2015, due in part to the bid protests filed by losing bidders (but really due to the Corps' lassitude and intransigence in getting the project underway from 2006 through 2010). These stations were supposed to have been in service in 2010.

The permanent pump stations project was awarded April 13, 2011 to a coalition of companies calling themselves "CBY Builders." It includes CDM, Brasfield & Gorrie, and Yates Construction. It immediately garnered bid protests from two of the losing bidders. That's not a surprise, since the job value is $674,998,555.36. Notably, that's over $129 million less than the $804 million congressional appropriation for the job.

Who are the protestors? Who were the other bidders?

The Corps used a two stage bidding process. In the first stage, they accepted bids from anyone. They also made public the technical specification. Then, in the second and much more secret stage, they narrowed the list to the five top players (i.e. "the list that counts"). In this second stage the Corps didn't let anyone know who was bidding, they didn't release any specifications or drawings, and they didn't put out any press releases. It was remarkably hush-hush for a project worth over half a billion dollars.

We already know the winner. Here's the other four:

Bechtel team (protestor)

Bechtel partnered with New Orleans-area engineering firms Waldemar S. Nelson; Brown, Cunningham & Gannuch (aka "BCG"); N-Y Associates; and Eustis Engineering. During the bidding process, Bechtel had a website up at http://www.nops-pccp.com, but it went away after the contract was awarded.

PCCP Constructors (protestor)

PCCP Constructors is made up of Kiewit, Traylor Brothers, and the New Orleans-area based M.R. Pittman group. This is the same set of companies that make up the joint venture "Gulf Intracoastal Constructors," who has the prime contract for the West Closure Complex.

Louisiana Canal Constructors (no protest)

"Louisiana Canal Constructors" appears to be Alberici and Archer-Western (a subsidiary of Chicago-based Walsh Group), the same team that worked levee project LPV-111 along the Gulf Intracoastal Waterway in New Orleans East. Parsons was also involved. We know this because all three firms are listed on the Louisiana Secretary of State entry on LCC. Incidentally, that entry shows that LCC is actually not in good standing due to failure to file an annual report this year. That's not surprising, since their lack of bid protest activity says the LCC team has decided to call it a day on this project.

Weston group (no protest)

The Weston group also includes engineering heavyweight HNTB. Weston was so interested in this project last year that they actually lobbied congress on it. A lobbying disclosure form for Weston's chief lobbyist, Sean McGraw, notes the New Orleans Permanent Pump Stations as a target for their lobbying during debates over the FY2010 Energy & Water bill

Other companies floating around the bidding process included Odebrecht, Arcadis, Shaw (who had advertisements on job boards for work on this project two years ago) and Volkert, though I don't know who they were partnered with. All these names are rather heavy hitters in the engineering industry, which is no surprise considering the mammoth size of this prize.

Corps helps out finalists

Part of the reason we know who's who among the bidders is because of solicitations for small business subcontractors each team issued last fall. These sorts of subcontracting gatherings are common for bigger projects; they're useful for signing up small businesses to meet the Corps' small business goals. Sometimes the Corps puts them on, but in this case, each firm - each privately-owned firm - staged them. PCCP Constructors' event was on October 19th, 2010 at the University of New Orleans (UNO):

Bechtel's was on October 6th at the Best Western on Severn in Metairie:

Louisiana Canal Constructors' event was on September 23rd at Southern University:

Weston's team had an event on October 20th at UNO:

I can't find an announcement for an event hosted by the winning bidder, the CDM team.

Here's the twist: all these announcements for events put on by private companies were - and are - residing on the Corps' computers, specifically the subdirectory associated with the Corps' Small Business office. PCCP Constructors' is here, Bechtel's is here, LCC's is here, and Weston's is here.

At one point, the Corps provided links to the Weston and PCCP Constructors flyers on their Small Business Programs page:

[Note to Corps: I've already got Google screenshots showing each of these flyers on your servers, so don't bother taking them down.]

If you think having a government agency advertise for private companies looking to do business with that agency is unseemly, you're right. But it's par for the course for the New Orleans District, which enjoys cozy ties to many large firms (see the details on Kiewit's protest below). One of the flyers notes that two officials from the Small Business Administration (Jo Lawrence and actually-Corps-guy Randy Marchiafava) were confirmed attendees, and two Corps officials (Ned Foley and Michael H Bloom, both Corps small business guys) were invited, so the government was out there appearing at events held by private firms to help them improve their bids.

Details on the protests

It is noteworthy that there's not just one protest for each of the two protesting bidders. In fact, each of them have filed three protests since the contract award. According to the General Accountability Office's bid protest docket, Bechtel filed protests on April 26th, May 2nd, and June 6th. Kiewit's group filed protests on April 26th, June 6th, and June 24th.

I'm sure the April 26th protests by each firm were probably already prepared before the contract was even awarded. The June 6th protests appear to be protestors' responses to the Corps' initial report to the GAO, if the schedule above is to be believed.

[Update July 12, 2011

The July outfall canal update indicates the June protests by the bidders were "supplementals," whatever that means. See complete update at the bottom of this post.]

Back before all these protests started rolling in, the Corps' schedule (found on the last page of the April, 2011 Orleans Levee District status update) naively called for absolutely no impact by them, and a project completion date of October, 2014:

With Kiewit's protest filed on June 24th, the GAO's latest response date is now October 3, 2011 (the GAO has 100 days to decide bid protests). Note this is later than what early June schedule above shows, since the slide was produced before the latest protests were filed. Comparing the two schedules and the GAO protest dates, it is easy to conclude about six months of schedule slippage, pushing the project completion date to approximately March, 2015. And that assumes the project doesn't get rebid as a result of the protests, which is what one of the protestors is calling for.

[Update July 12, 2011

It is possible the protests are all part of one big file, with an August deadline still in effect. The Corps is playing the project delay card close to their vest. See the complete update at the bottom of this post.]

Kiewit goes public
Alleges Corps revolving door killed their bid

Kiewit went to the local media about their protest soon after filing in April. On May 4, 2011, WVUE-TV ran a story titled "Former employee accused of giving inside info to competing company."

The story reports:

"PCCP Constructors believes a senior level Corps employee provided inside information to the company that was ultimately chosen for the project, CBY Builders.

'It’s something that needs to be looked at and examined so that we know exactly what was said when and what kind of inside information may or may not have been disclosed that would benefit one party over another,' PCCP spokesman Kent Grisham explained. [note: Grisham actually works for Kiewit]

Grisham is talking about Richmond Kendrick. Kendrick was the chief of program execution for the Corps’ pump stations project. But in August 2010, Kendrick left the Corps and went to work for a company named CDM.

CDM is a partner of CBY Builders."

That's a pretty interesting charge. Of course, the revolving door between the Corps and contractors is well known within construction circles, but less so to the general public. Many of the most prominent, and some of the less prominent, figures involved in the Corps' post-Katrina work have left for contracting jobs which brought them right back into the Corps' offices working on that same post-Katrina work. In fact, here's a 2009 article in an industry trade organ featuring two of them: Dan Hitchings (former Task Force Hope leader and now working for Arcadis on Corps hurricane protection contracts) and Mike Rossi (who wasn't directly involved in the post-K work, but who now works for Vali Cooper International, the New Orleans-area company started by former Hurricane Protection Office leader Jeff Bedey and which claims to have Corps hurricane protection contracts, though they're more likely subcontracts). Ironically, the article also features an interview of Richmond Kendrick in his now-former capacity as deputy of program execution for the Corps’ Hurricane Protection Office.

Anyhow, it's unusual for the revolving door to get publicized in this fashion, but when there's hundreds of millions of dollars on the table...

The WVUE-TV report continues with an actual quote from one of Kiewit's protests. Given the May 4th date of the story, it could only be their first one, filed on April 26th:

"The protest filed by PCCP to the Government Accountability Office states, 'The most critical events in selecting the winning offeror for the PCCP (Permanent Canal Closure and Pumping) contract occurred after Mr. Kendrick took his inside knowledge concerning this procurement to CDM.'

Grisham says the company’s goal is to have the Army Corps re-open the bid process, now that Kendrick has left the government agency."

That would certainly be remarkable, though given the size of the contract, not unprecedented.

I'll stick close to this $674,998,555.36 story.

Update, July 12, 2011

The latest outfall canal remediation project update - issued in the first week of July - also contains a small update on the permanent pumps:

This update seemingly still calls for an August 4, 2011 response to all the protests. It calls the latest round of protests from June "supplemental protests." It also notably removes any reference to the start or completion date of the project.

I'm not sure what to make of the Corps insisting the GAO will come back August 4th with a decision. That date is based on the date of the original protests in April. The GAO's own docket indicates that each protest restarts the 100-day clock for them, so that while the earliest protest could be resolved in August, the later June ones might not get a decision until October.

Of course, we could just be seeing a limitation of the GAO's website, in that all three protests for each firm might indeed be part of a whole, but the website doesn't have a way to show that. Indeed, ther eis some support for that. The protest numbers given to PCCP Constructors' (nee Kiewit's) protest are B-405036.1, B-405036.4, and B-405036.6, while those assigned to Bechtel's protest are B-405036.2, B-405036.3, and B-405036.5. That would seem to indicate that they are simply part of an overall file numbered B-405036.

Two other shreds of information come out of this latest stab at a schedule by the Corps:

1) There awas apparently some kind of hearing the last week of June in Washington.

2) That hearing was followed up by a briefing (presumably by GAO to the Corps) on July 7th, last Thursday.

Again, I'll try to stay on top of this.

Update, July 16, 2011

The Corps has confirmed the 2015 completion date for the permanent pumps.

In a July 12, 2011 presentation given to FEMA about the hurricane protection system by the Corps' Mississippi Valley Division (one step above the New Orleans District), the 2015 date was made explicit on this slide of completion dates for incomplete projects:

Incidentally, that is slide 1 of a pair of slides listing 30 incomplete 100 year protection projects:

The 16 projects with blue rows are those only at the 100 year level of protection if the Corps brings in Hesco baskets and sandbags immediately before a storm. The 13 projects with yellow rows actually have something built (like sheetpile or the like) to give interim 100 year protection. Remarkably, there's a still a project in St Charles Parish which hasn't even been awarded. So much for breaking their backs to meet the June 1st, 2011 deadline.

And finally, it turns out the Corps actually confirmed the 2015 date the same day this post first appeared. In a July 1, 2011 press release trumpeting their piecemeal, inadequate remediation work along the outfall canals, they included this:

"The 17th Street, Orleans Avenue and London Avenue outfall canals extend from interior pump stations to Lake Pontchartrain in metro New Orleans. Interim Closure Structures currently block 100-year storm surge from entering the canals; they will be replaced with permanent canal closures and pump stations that are scheduled to be complete in 2015. Because of the canal closures, the outfall canals’ primary purpose is to remove rainwater from the city. The floodwalls and levees lining the canals are no longer the primary storm surge defenses for New Orleans."

Making it up as they go along

2011-06-30T11:00:00.009-05:00

The Corps gives updates on the outfall canal remediation projects (those are the projects to raise the Safe Water Elevation, or SWE, along all three outfall canals to 8 feet) every month to the Southeast Louisiana Flood Protection Authority - East (SLFPA-E). SLFPA-E has been good enough to upload the slides from those presentations to their website. Here's links for the ones I've seen thus far:

late March, 2011 (at end of report)
late April, 2011
early June, 2011

The June update includes this information on the 17th Street project:

"Reach 16 Surveys being conducted to determine if improvements are needed"

and

"Reach 16 may need 6” to 18” of embankment at the toe of the levee, determination will be made after the survey is analyzed."

When I first read that, I thought it was good, because I had remembered there was a sub-8 foot SWE reach along the 17th Street canal that wasn't included in the original remediation plans. Then I went back and looked and realized it was actually Reach 30:

So where is Reach 16?

It's across the canal from Reach 30, down south of the I-10 bridges:

The Corps included pictures of it in the presentation:

Reach 16 was not included in the 17th Street canal remediation project when the contract was awarded in December, 2010. But now the Corps is saying extra dirt might be required at the toe of the levee (the part of the levee at its bottom where it goes from sloped to flat), and is even including pictures of the Reach 16 toe.

The changing Safe Water Elevation of Reach 16

So what's going on with Reach 16? The main chart of Safe Water Elevations on page 12 of the October, 2010 17th Street SWE report (report and all appendices here) shows the Safe Water Elevation for Reach 16 was the bare minimum of 8 feet, meaning no remediation was required:

So if the SWE was 8 feet, based on seepage considerations, what's the problem?

A clue comes from further inside the report. These types of reports are written from the inside out, with the individual results sections compiled first, and then the introductory and conclusory sections written around them. Those "outer" sections are the ones written last and which have the heaviest attention paid to them during editing, because they are the sections most likely to be read (who except crazies like me reads the whole report?).

So when one reads the introductory and conclusory sections, one finds this unambiguous statement reinforcing the main chart:

"Seepage was not found to be a concern on any reach of the 17th Street canal for an SWE below +8.0 feet."

But when we look at the individual section on the seepage results, we find this chart on page 124:

The actual calculations originally resulted in a seepage SWE of 7.5 feet, 6 inches below the 8 foot target. A SWE of 7.5 feet may have meant more remediation along the canal (theoretically, though we've seen the Corps ignore other sub-8-foot calculations). Reach 16 is 1615 feet long, so that's a lot of remediation.

So how did the SWE jump from an unacceptable value of 7.5 feet to the barely acceptable value of 8 feet? The report itself provides the rather bizarre answer, but first there's a brief detour to explain (in my rudimentary understanding) how seepage works.

The (very) basics of seepage

To help, here's a diagram of the layers of various soil along Reach 16:

This diagram is taken from the calculations for "slope stability," not seepage. The diagram for seepage doesn't include such clear delineations of the different soil types, so I grabbed this one.

As you can see, there's a thick layer of sand - shown in green - underlying the whole thing. I've noted that the bottom of the canal is at -9.4 feet, while the sand starts at -9 feet.

On the "protected" side - i.e. the non-canal side in Jefferson Parish - there's stronger, less permeable soil overlying this sand, both within the levee and next to it. That's the stuff shown in yellow and orange, labeled either "Marsh," "Embankment Fill," or simply "Fill." Below the sand is a layer of pretty impermeable clay, shown in red and brown. And at the top of the diagram is the I-wall, which penetrates down to -1.5 feet, or 7.5 feet short of the top of the sand layer and 44.5 feet short of the clay layer.

For seepage to occur, water from the canal must contact the sand surface directly. It must then travel through the sand to the protected side. For it to truly be a problem, though, it must exert an upward force against the protected-side soils which is greater than the force exerted downward by the weight of those soils. If it does this, then a connection like a pipe is made between the canal and the land side, and the foundation underneath the levee and floodwall can be quickly undermined.

In Reach 16's case, it would seem the first part of a potential seepage problem - a canal bottom lower than the top of the sand - is in place. The bottom of the canal is 0.4 feet into the sand layer. This condition is referred to by the Corps' consultants as an "open bottom," and Reach 16 is not the only reach along the 17th Street canal to exhibit it. Indeed, Black & Veatch analyzed five other reaches with survey data indicating an open bottom: 15 (later split into 15A and 15B), 31, 32, and 33. All these are reaches near the I-10 or Veterans bridges:

These calculations, while quite complicated in their guts, are simple in concept. The canal is modeled from known survey data. Water is placed in the canal up to the target SWE of +8 feet and to a maximum of +10 feet (the Corps limited SWE calculations to 10 feet of water, even though the walls are much higher). If the resultant calculated safety factor is above the prescribed safety factor for the particular type of failure under study (global stability, slope stability, gap, seepage, etc, each of which has its own safety factor), then the reach under consideration passes.

In the case of all those other "open bottom" reaches, Black & Veatch was able to load the canal up with water to an elevation of +10 feet and not fall below the Corps' seepage safety factor of 1.6.

But for Reach 16, Black & Veatch could only load the canal up with water to an elevation of +7.5 feet before they fell below the 1.6 safety factor. Just for reference's sake, the canal was able to be loaded up to +10 feet on Reach 16 for all the other types of failures Black & Veatch was looking at. So seepage was what is called the "controlling" criterion for Reach 16.

Sediment

Now, there's one other thing that can mitigate against seepage. That is stopping the canal water from getting to the sand in the first place. All three outfall canals have sediment along their bottoms, carried there from the city's streets and drainage tunnels and pipes. It is deposited there constantly, and especially when there's a rainstorm and water is pumped along the canals out to the lake.

I wrote about this sediment in 2009, when I based an entry on the 2007 iteration of the 17th Street SWE report. That report indicated sediment was quite thick along the entire canal. However, those thickness readings were based on soundings, not direct borings into the canal bottom. In addition, the 2007 report did not characterize the sediment at all, including its permeability or how much of it was mixed in with sand. The October, 2010 report notes (and this is quite important) that the Corps has still not measured the sediment, referred to as "fine grained materials:"

"[T]here are no borings in the canal to confirm the thickness of the fine grain materials over the beach deposits."

This is a yawning gap in the data required to completely analyze seepage.

As noted above, a number of reaches were analyzed as "open bottom" reaches. With regard to sediment, that likely means that Black & Veatch and the Corps made a judgement that the sediment blanket along those stretches was too thin to stop the water flowing through the direct connection exposed by the top of the sand layer being above the bottom of the canal. In addition, Black & Veatch also noted six other reaches - 14, 17, 18, 29, 30, and 34 - which had thin sediment layers, but whose canal bottoms were apparently not below the top of the sand layer. These weren't analyzed as open bottom reaches.

So they were making judgements about the thickness of the sediment layer throughout the preparation of the report. But how did they do so when they had no detailed canal borings to confirm those thicknesses? Where was the hard data?

Detailed borings of the sediment along the London Avenue canal bottom in 2010 led to a radical recasting of SWE calculations there, essentially restarting the engineering effort and eventually leading to the inclusion of many reaches in the remediation project along that canal which had previously been thought safe. Yet, no similar effort was undertaken along the 17th Street canal (or the Orleans Avenue canal for that matter).

So with no detailed information on the sediment overlying the beach sand at the bottom of the canal along Reach 16, and with a calculation showing a sub-8-foot SWE along that reach from seepage due to an open bottom, how then did the SWE jump 6 inches to 8 feet? What was that judgement based upon?

1983

This is where it gets bizarre.

Instead of either:

a) using the lack of detailed sediment data to take the conservative course - assuming an open bottom along Reach 16 and sticking with with the 7.5 foot SWE within the seepage section of the report, or

b) revealing the source of their heretofore undisclosed sediment thickness data,

Black & Veatch and the Corps chose a third course. They startlingly reached back almost 30 years to rebut their own seepage calculation. The report starts describing a December, 1983 test along the 17th Street canal's Reach 16 to determine whether a direct sand connection existed at the canal bottom. This test took place ahead of 1984-85 dredging of the canal by the Sewerage & Water Board, before the later walls were designed and built. The test was used to help get a permit from the Corps for that dredging. Presumably, reassuring the Corps that the canal would not be undermined by seepage during the dredging would go a long way toward getting the dredging permit. Since the dredging happened, it clearly did the trick.

In fact, seven years later, as part of the now-discredited 1990 17th Street canal design document which led to the placement of the I-walls destined to fail in 2005, the Corps themselves used the report of this test in the same way: to rebut concerns about seepage. Thus, it is beyond surprising to see it resurrected here to do the same thing a third time - rebut concerns about seepage - as if nothing had happened in the intervening decades.

Nonetheless, two whole pages of the 2010 report are devoted to blithely describing the 1983 test, recommended by Eustis Engineering to the then-consulting engineers on the dredging, Modjeski & Masters (Eustis still does millions of dollars of geotechnical work for the New Orleans District). The complete report on the 1983 test can be found within volume 2 of the 1990 Design Memorandum 20 (or DM-20), the base design document for the 17th Street canal walls. Scroll to page 78.

In 1983, at Eustis' recommendation, a 100' by 100' square area in the middle of the canal was excavated to expose the beach sand. Also, at Eustis' recommendation, piezometers were installed on the land side of the Reach 16 levee, just south of I-10. These piezometers sensed the water elevation in the soil below. If there were a seepage connection between the water in the canal and the piezometers outside the canal, changes in the canal water elevation would be reflected in the piezometer readings. Similar arrangements were a part of the 2007 London Avenue canal load test.

According to the 2010 report, during the 1983 test the water in the canal went up 0.41 feet while the sand was exposed on December 16, 1983, but the piezometers outside the canal either went down or remained unchanged. The 1983 test authors also noted that the test location was partially covered in sediment on the 16th, and that the entire location was covered in sediment three days later. To Eustis, this demonstrated that there was no seepage connection, either from the sediment cover or because there just wasn't one. Frankly, their reasoning is quite muddled.

However, Black & Veatch accepted the "no seepage" finding of the 1983 test, while including a token suggestion that before the SWE for Reach 16 is finalized, "a sampling program be performed to determine whether a [sediment] blanket exists at this location." But nowhere in the 2010 report does it indicate such a sampling program took place before the SWE for Reach 16 was raised to 8 feet in that same report. Indeed, there was no such sampling program during the 1983 test either. It appears that the 8 foot number came about directly - and only - as a result of Black & Veatch and the Corps accepting the findings of the 1983 test as gospel. It's like they just willed it into being. This is supported by the footnote on the master SWE chart on page 12 of the report:

...as well as the plain text of the 8 foot SWE in the main chart itself. This was stunning for me to read - a consultant in 2010, using the Corps' incomplete, muddled pre-Katrina design data to support a conclusion not supported by the post-Katrina data record, a data record also lacking key pieces of information. It's almost like they were looking for any shred of evidence that might support their pre-made conclusion that seepage could not possibly be a problem, their own calculations be damned.

I was not the only one to notice this.

Behind the scenes

On a consultant-written report such as this one, after the report is drafted it is sent to lots of people outside and inside the Corps for comments. They use a system called "Dr Checks" to log those comments, the responses, and the responses to the responses. Thus, the Dr Checks printout is the sort of document the public almost never sees - the official record of the internal engineering debates that go into the formation of these reports.

When I got the October, 2010 17th Street and Orleans Avenue reports, I was stunned to find the Dr Checks printouts were included as appendices (the London Avenue report didn't have one). The Orleans Avenue one runs 159 pages (complete Orleans Avenue report and appendices here). The 17th Street one is significantly shorter - just 25 pages long.

That doesn't mean the 17th Street Dr Checks printout is any less valuable. In fact it contains a brief but vigorous debate over seepage along Reach 16.

Keep in mind the Dr Checks comments are based upon draft versions of the report which we don't have. Thus, sometimes the comments mention a criticism which - when one reads the final report - it is clear eventually got addressed. Nonetheless, this particular chain of comments is extremely enlightening.

The comment chain starts with Noah Vroman, a Corps seepage expert working at the Corps' Engineer Research & Development Center (ERDC) in Vicksburg, Mississippi. Vroman's name has popped up often in the past few years in the many reports generated on seepage concerns along the London Avenue canal.

On July 23, 2010, Vroman says of Reach 16,

"I calculated a FOS [Factor of Safety] of 1.2 at a WL [Water Level] of 10.0"

He is noting this because the acceptable seepage safety factor is 1.6.

Black & Veatch geotechnical engineer John Koontz is the first to reply to Vroman, on August 7th:

"This reach was reviewed. A full scale canal seepage test was performed by excavating the beach deposits. No response was observed at the toe. The report is in DM-20 Vol. II. Discussion of this test has been added to the report."

As noted earlier, the "toe" is the section of the levee at the bottom, where it goes from sloped to flat. It is generally the most vulnerable section, because there's less soil there than at the top.

Also as noted above, "DM-20" is the 17th Street Design Memorandum, or the official design document underpinning the floodwalls and levees which failed in 2005.

As to the substance of Koontz's comment, we can surmise that the description of the 1983 test had not been included in the draft version Vroman was reviewing in July.

Vroman responded on September 23rd with one of the longest Dr Checks comments I've seen in these documents. From the date and the context of the comment, I surmise this September 23rd comment is based upon a subsequent draft, which we don't have. Note that I've added paragraph breaks to Vroman's comment to make it more readable:

"Do not agree with response.

I believe key elements of the results of the test section are not adequately addressed in the revised SWE report. The test section that is reference[d] was conducted in 1983 due [to] concerns with dredging operations in the canal that could potentially expose beach deposits in the canal. Eustis Engineering Company provided expertise and guidance for the test. They concluded from the test that "the possibility of blow out during high-water conditions in the canal is probably slight" and additional monitoring during dredging operations should be conducted. It is obvious that Eustis Engineering was concerned after the test that a "blow out" condition could still occur."

Let me pause here and point out that additional monitoring was conducted during that dredging. Piezometers were installed outside the canal as the dredging advanced along the canal between December, 1984 and April, 1985. They didn't really show any evidence of seepage. However, there was no characterization of the canal bottom as part of that monitoring.

Returning to Vroman's comment:

"The key elements that should be included in the SWE report concerning seepage and the canal test are:

- This was not a full canal test but merely monitoring an area where the possibly [sic] of exposing the beach sand was likely.

- after excavation of the test section, a boring was taken just three days later and there was 2.7 ft to 4 ft of sedimentation above the sand. The authors of the test section report indicated that this sediment was the likely reason the piezometers did not respond to changes in canal water elevations.

In addition, the authors of the test section report also indicated that the beach sand at the bottom of the canal was possibly contaminated or mixed with fines and "sealed" the bottom of the canal."

Based on the final text of the 2010 report, it appears this information was not in the September draft, but was included in the final report as a result of this comment.

Vroman continues:

"Thus it is important for the SWE evaluation analysis to consider whether this sedimentation layer exists within this reach. The same diligence used on the London Avenue SWE report for seepage concerns and sedimentation in the canal should be used in this reach.

Regarding this SWE report and the condition of the canal in 2010 and beyond, does a reliable sedimentation layer exist above the beach sand that will prevent adverse seepage conditions?"

And there it is - the critical question. Essentially, "what hard data do we have on the sediment?" And he's not asking for what was known in 1983 about a 100' by 100' section, but what is known along the entire canal - today. As was demonstrated on the London Avenue SWE effort, complete characterization of the sediment is unbelievably critical to understanding seepage. Since there has not been any comprehensive effort to do so along the 17th Street canal, it is only conservative to stick with the results from the "open bottom" calculations, which gave the 7.5 foot SWE for Reach 16. And, more importantly, the lack of that current, comprehensive sediment data brings into question how correct the seepage calculations are for the entire canal.

Vroman didn't just bring this up on the 17th Street report. We find him asking the same questions about sediment (also called "silt") on the Orleans Avenue report, because - like along the 17th Street canal - there was no comprehensive characterization of canal sediment at Orleans either. From a comment on the Orleans Avenue Dr Checks printout marked "this item flagged as a critical issue:"

"Different approach was used in Orleans canal as opposed to London.

For the London analysis, silt layer in the canal was critical feature for seepage and was deemed nonexistent if silt is less than 2 feet thick. For Orleans no canal side investigation was performed to determine the thickness of any impervious layers above the sand. This aspect is very critical to the analyses.

For Reach 1, there is only about 1.6 feet of marsh shown above the sand. This assumption was made purely on survey data and assuming the [base of the] marsh is constant between borings 200 feet apart (east bank and west bank). I think using this approach would warrant more conservatism than used for London. Anything less than 3 feet thick should be assumed to be nonexistent and direct connection to the bottom of the canal could be possible."

Vroman's comments on the Orleans Avenue report appear to have won the day, as his concerns about seepage led to the seepage calculations getting redone to include consideration of an "open bottom" on Reach 1 and other places. And on Reach 1, seepage was found to be to be the controlling criterion. In fact, sheet pile is being driven there to completely cut off the seepage path as part of the Orleans Avenue remediation project, as shown in this photo from the June update:

Of course, without a comprehensive characterization of the sediment along the Orleans Avenue canal, doubt remains about many of the other seepage calculations.

The outcome on the 17th Street canal report was quite different. The Black & Veatch SWE project manager, Larry Almaleh, responded to Vroman's 17th Street comment on October 1, 2010, just before - or perhaps the day of - the report's final issuance:

"The SWE for a FOS of 1.6 for an open bottom condition at this location as identified in the report is elevation +7.5 [feet]. Based on the identification of a significant sedimentation at the base of the canal and the lack of response in the piezometers raising of the SWE to +8 is not unreasonable. Note in multiple meetings and corresondence B&V has been told there is no seepage concerns in the 17th street canal by the HPO and the external review group."

Nowhere in the report is a precise identification of "significant sedimentation" outside of the 1983 test actually named or described.

But most telling is that the Corps ("HPO" is the Corps' Hurricane Protection Office, co-located with the New Orleans District) had apparently told Black & Veatch the result of the seepage calculations before they were done: "B&V has been told there is no seepage concerns in the 17th street canal by the HPO..."

I thought Black & Veatch was supposed to be reporting to the Corps on seepage concerns, not the other way around?

Handwaiving

So stepping back from all this, it looks to me like there was a ton of handwaiving going on around the seepage concerns at the 17th Street canal's Reach 16 specifically, and along the entire 17th Street canal generally, possibly on orders from the Corps Hurricane Protection Office in New Orleans. And while Vroman's comments apparently got an expanded description of the 1983 test added to the final report, it didn't change the underlying fact that Black & Veatch and the Corps were solely relying on a nearly 30-year-old test from a discredited document to support their pre-reached conclusion that seepage was nothing to worry about along the 17th Street canal.

So not surprisingly, despite calculations and the Corps' own non-New Orleans-based experts pointing to the absolute need to fully characterize the depth and composition of the sediment along the canal bottom, not to mention possibly remediate the 1615 feet of Reach 16 because of seepage, Black & Veatch issued their report in October, 2010 with the conclusive statement,

"Seepage was not found to be a concern on any reach of the 17th Street canal for an SWE below +8.0 feet."

That conclusion then flowed directly into the scope of the remediation project along the 17th Street canal. Until this most recent project update nine months after the SWE report was issued and seven months after the remediation contract was signed, there has been no effort to address seepage along the canal, including at Reach 16. All the remediation efforts are geared to other concerns like overall stability.

The way the Corps and Black & Veatch got to that outcome - even if they are possibly correcting it now - should make people very queasy. How could anyone in good conscience use nearly 30-year-old incomplete 17th Street design data and ignore post-Katrina data collection and calculations to support conclusions about the canal's safety in the present day? After all that has transpired, and after all the Corps' pronouncements that they've changed, it looks like more of the same over five years later.

Reach 16 to be fixed anyway?

That brings us full circle to the June remediation project update, which mentions that the Corps is now looking at putting 6 to 18 inches of dirt at the toe of the levee along Reach 16, based on surveys of the reach. That toe is undoubtedly the point found vulnerable in the Black & Veatch seepage calculation that found the 7.5 foot SWE. Thus, if more soil is piled over it, presumably water carried through the sand by seepage would not be able to push up and cause a failure.

Of course, the only definitive solution to stopping seepage is not piling more dirt on the landside, but driving of 60 foot steel sheet piling into the clay layer along the entire reach. In fact, that has been how seepage has been controlled everywhere else, including on both the Orleans Avenue and London Avenue remediation projects. I assume the Corps thinks that's too expensive or time-consuming or embarassing to do so along the 17th Street canal, so they are proposing a band-aid of a slightly thicker levee toe. It's a half-measure that doesn't fully address the problem.

The June update does not make clear whether the survey is of the canal bottom or the levee. We can hope it is of the canal bottom. Perhaps this is the "sampling program" given a token mention in the 2010 report? Indeed, we can hope there is a comprehensive effort underway to characterize the sedimentation along the canal bottom, and that effort will lead to a recasting of the engineering similar to what happened along the London Avenue canal last year. Since this update is about a month old, we should know whether those hopes are just that within a couple of weeks.

Debris Part 7

2011-06-25T13:15:00.006-05:00

Part 1
Part 2
Part 3
Part 4
Part 5
Part 6
Part 8
Part 9
Part 10
Part 11

One would think after having massive debris problems exposed on the front page of the paper, as happened on June 12, 2011, the Corps and their contractors would be moving heaven and earth to eliminate those problems before the next time the Southeast Louisiana Flood Protection Authority - West inspector came knocking. One would be wrong.

This week we got a look at the report for the first SLFPA-W inspection to occur after that article appeared (inspection date: June 17, 2011) at deeply troubled project WBV-14c.2 (contractor - Phylway, dirt source - River Birch). The debris has been so bad there that I've devoted two entire entries to it (Parts 1 and 4) and the Times-Picayune's Paul Rioux had it in a featured role in his article.

Despite all that attention, nothing has changed at this project, and that includes the bizarre Corps paranoia during SLFPA-W inspections. The report begins:

"Check-In:
- SLFPA-W reps met with Jeremy George (USACE) at the construction trailers at 7:11am. Work progress was reviewed and USACE conducted site visit. Chad Thibodeaux (USACE) arrived on site at 7:43am, and remained 50yds behind other participants of site visit."

There's even pictures of this behavior following this description:

I don't know if the guy in the top picture is Jeremy George or Chad Thibodeaux, but it doesn't really matter. Their behavior during this inspection appears to have been only slightly less looney than during the June 10, 2011 inspection:

"SLFPAW reps arrived at the construction trailers at 10:22am and met with Jeremy George (USACE), Chad Thibodeux (USACE), Lauren Fagerholm (USACE), and Susan Poag (Times-Picayune). USACE conducted site visit while remaining 50 yds behind SLFPAW reps. USACE also video-recorded SLFPAW reps throughout site visit."

So, moving on from the Corps melodrama, here's what the June 17th report says about the project:

"Levee:
- East end of project (Sta.182+00 to Sta.120+00):
The levee in this area is up to elevation 13.0' on the ends of this section, and elevation 12.0' in the middle. A considerable amount of concrete and wood debris was found in the levee section during site visit. Pickers have not yet picked this area.

A 6in lift has been placed from Sta.120+00 to 157+00 on the protected side berm in this area. A considerable amount of wood debris was found in this area during site visit. This area had not yet been picked. Sta.161+00 to Sta.171+00 of the protected side berm had been picked and compacted by the contractor earlier in the week.

- North/South Levee and protected side berm (Sta.109+00 to Sta.70+00):
Sta.109+00 to Sta.89+00 of the levee in this area had been fine-graded and is acceptable. A few stones and wood debris were found throughout the remaining levee section and the protected side berm of this area. Jeremy George (USACE) removed most of debris immediately.

- West end of project (Sta.55+00 to Sta.0+00)
A 6in lift has been placed on the protected side berm of this project from Sta.29+00 to Sta.11+00. Some small stones and wood debris were found in this area. This area has not yet been picked. No further degrading of the levee has taken place in this area."

Put simply, there's still debris throughout the site. There's no excuse for this when other projects are able to report no debris problems.

As is usual with reports from WBV-14c.2, there's mounds of debris pictures. This time around, they take up 22 pages. Here's a sample:

This project just needs to be shut down. After nearly nine months of unrelenting reports of debris coming from it, it's appears the Corps and the contractor aren't interested in addressing the problem.

The debris problems continue elsewhere. In Part 5 we noted that WBV-14a.2 (contractor: Purnell), a 3 mile long levee project on the west bank of the Harvey canal, has had its share of debris problems. They continue, as documented in the June 6, 2011 report of the SLFPA-W inspection on June 3, 2011:

"SLFPA-W began site visit at approx. Sta.964+50 where geotextile was being installed in this area at elevation 1.0'. Lift 8 is being placed on top of the geotextile. A 12 man picking crew was out picking debris from the material that was being hauled in."

A twelve man crew sounds like progress. However,

"No work has been done since last report [on May 23, 2011] from Sta.935+00 to Sta.951+00. The debris in this area will be picked before the next lift is placed."

"SLFPAW expressed their concerns about the debris being hauled into the jobsite. USACE rep assured SLFPAW that all material will be picked with the same 12-man crew."

"We know we've said a gajillion times the debris will be picked up before the next inspection, but this time we really mean it! So don't pay attention to this:"

And that's what we can see on the surface.

The quote above mentions the prior WBV-14a.2 report. So what did that report, from May 23, 2011, say about this area?

"3rd lift was being placed and disked between approximate stations 935+0 and 941+00. The crew was working ahead of the material being placed and appeared to be removing all of the debris. SLFPA-W and USACE reviewed area between the picking crew and the material being placed and found it free of debris."

So debris showed up on June 6th in an area that was being picked and appeared clean on May 23rd. Is it possible the Corps and their contractors have been playing a shell game with debris picking? That is, when the SLFPA-W inspectors are out, do the Corps and their contractors deploy debris picking crews (sometimes), or claim the area will be picked by the next visit? I ask because when the inspectors do their due diligence at the next visit and check the claims of debris picking, they find it was all a show.

The following inspection of WBV-14a.2 (on June 14, 2011) also included an appearance of a picking crew:

"Sta.973+00 to Sta.960+00. Contractor is currently picking this area and preparing for another lift."

Here's the picking crew at work:

The report also says:

"Sta.973+00 to Sta.960+00 was examined by SLFPA-W reps and is clean enough for the next lift to be placed on top of this material."

Due to the brevity of this report, it is unclear if the inspectors revisited the section of the project noted for debris in the June 6th inspection report (Sta.935+00 to Sta.951+00); it only notes no work had taken place there since the last inspection. Also there aren't any debris photos in this report. Both of those appear to be because SLFPA-W inspector David Roark was not along on this inspection. He tends to write very detailed reports and take lots of pictures.

So it appears the debris problems, while perhaps lessening at some projects, remain severe at others. This does not bode well.

WCC pumping shortfall alleviated slightly

2011-06-24T13:45:00.005-05:00

[Updated July 7, 2011. See end of post]

As I reported earlier this year, the Corps and their contractor on the West Closure Complex (Gulf Intracoastal Constructors, or GIC) were unable to have all 11 of the pumps installed and running on June 1, 2011 (see the earlier post for the basics of the West Closure Complex, or WCC). We know from press coverage and the SLFPA-W reports filed by Danny Caluda that 8 of the pumps passed their wet and dry testing just before the deadline. Fortunately, it appears the clause within the specifications that would have allowed the wet tests to be waived has not been activated.

The pumps are numbered 3 through 13. Pumps 1 and 2 were eliminated from the design as a cost saving measure, and the remaining 11 pumps were redesigned to make up the flowrates of the removed pumps.

Pumps 6 through 13 were the ones declared ready on June 1, 2011 after their tests. Now pump 5 has passed its wet testing.

Mr. Caluda tells me the successful 4 hour wet test took place on June 14, 2011, meaning there are now 9 of 11 pumps available for stormwater removal on the west bank.

The modeling done by the Corps in connection with the consequences of the pumping shortfall is summarized in this table from the Interim Standing Instructions attached to the WCC Water Control Plan:

Water elevations behind the stations are shown in the left hand column. The reason they stop at 8.2 feet is that is the elevation when stormwater - pumped from local pump stations into the the detention basin formed by the Harvey and Algiers canals - would overflow the floodwalls and levees along those canals. This table assumes two things:

1) The floodgates at the WCC are closed, necessitating turning on the WCC pumps to remove the stormwater from the detention basin

2) The local pumping stations are pumping continuously the whole time, a circumstance the Corps calls "unlikely," but we all know what happens with events the Corps calls "unlikely."

Along the bottom row of the table is the total amount of time for that overflow to occur. It happens because the WCC has no spare capacity built it; it can only keep up with the local pumping stations with all 11 of its pumps on. As you can see, with only 8 pumps it would take a little over 17 hours for the local pump stations to overwhelm the WCC pumps and send water into neighborhoods (either over the floodwalls or because local pump stations would be shut down). The addition of a ninth WCC pump buys over 8 more hours.

Mr. Caluda says pump no. 4 is next up, but not for at least two more weeks.

[Update - July 7, 2011

According to Mr. Caluda, Pump 5 is due to dry tested July 7, 2011. Details on the specifics of dry and wet testing can be found here.

After that, pump 4 is anticipated to be wet tested on July 19, 2011. Testing dates for pump 3 are currently unknown.]

Brief update on London Avenue pumps

2011-06-24T13:15:00.003-05:00

For much, much, more information regarding these pumps, two others at 17th Street out at the same time, and the contracting and repair activities surrounding them - including pictures from Conhagen repair reports - see the previous post, "The 2011 pump rebuild scramble."

That post includes all the information on the last eight pumps to be repaired before June 1, 2011.

As I reported in May of this year, hydraulic pumps W5 and W6 at the London Avenue closure site were removed for complete rebuilding in late April, likely on the 26th. Here's the empty slots:

We can tell these pumps were pulled out because we can see the bottom flanges of the elbows unattached to any pumps below them. We also don't see the distinctive piping extensions added to the pumps in 2007, as we could on the adjacent pump W4.

I can report that W5 and W6 went back in at the end of May:

This means that 11 of the 12 hydraulic pumps at the London Avenue site have been overhauled due to severe corrosion of all their internals:

None of the contract documents received from the Corps as of July 13, 2011 indicate pump W2 was ever pulled out.

Also, it must be emphasized that the "reliability" shown in the graphic above only refers to the corrosion status of the pumps. It does not address any of the issues uncovered by Ms. Maria Garzino or myself.

Steve Beatty of The Lens contributed to this report.

Debris Part 6

2011-06-16T15:00:00.012-05:00

Part 1
Part 2
Part 3
Part 4
Part 5
Part 7
Part 8
Part 9
Part 10
Part 11

In the last part of this series, I noted that at least 8 Corps hurricane protection projects spanning 21.9 miles of west bank levee had been found to be contaminated with disturbingly large amounts of debris, including wood, rocks, concrete, bricks, and steel:

This information is thanks to SLFPA-W inspectors (provided by the state of Louisiana) who have been scrupulously documenting the problem for months.

On Sunday, June 12, 2011, the Times-Picayune's Paul Rioux had an excellent article on the front page about the whole debris issue. Titled "West Bank levee work faulted for excessive debris in dirt," the article hit all the high points this series has covered over the previous five parts, while filling in a number of holes. The best part is it did so much more concisely than I have. The T-P's editorial board followed up with an editorial two days later emphasizing the importance of overcoming the persistent debris problems across the west bank.

The article began with the unveiling of debris project number 9: WBV-3b (contractor: Shavers-Whittle). It's a little over a mile long on the east bank of the Harvey Canal, as shown on this map:

The project is substantially complete, so the 2011 inspection record has been understandably scanty. Most of the inspections took place during construction in 2009 and 2010. The last inspection in 2010 noted debris in the completed levee. From the July 13, 2010 report of the July 8, 2010 inspection:

"Ruts, organic matter, concrete/rocks and grass issues. (south end to gas line)"

"Ruts, organic matter, concrete/rocks and grass issues. (gas line to levee/flood wall transition)"

A few debris pictures were included:

The T-P article describes a piece of debris dug out by SLFPA-W employees on Monday, June 6, 2011:

"While inspecting a nearly finished levee south of Harvey on Monday, West Bank levee authority officials spotted a chunk of wood protruding near the levee’s crown, so they grabbed some shovels and started digging and digging and digging.

Twenty minutes and two broken shovels later, they had unearthed a log the size of a typical suitcase.

'If they’ve got a 200-pound log buried in that levee, it makes you wonder what else is buried there,' said Giuseppe Miserendino, the levee authority’s regional director. 'How does your quality-control person miss something that big? What else didn’t they catch?'"

Indeed.

So even though the publicly available record doesn't indicate debris problems, I'm still going to add WBV-3b to the map:

[Note: On June 15, 2011, just three days after the Times-Picayune published their article, SLFPA-W inspectors returned to WBV-3b to continue the pre-final inspection they started on June 6, 2011, when the log described above was unearthed. From a letter sent by SLFPA-W President Susan Maclay to the Corps on July 15, 2011, we learn that there were multiple trailers' worth of debris dug out of the WBV-3b levee that day. SLFPA-W inspectors got this photo of a log in one of those trailers:

That looks much bigger than the one featured in the Times-Picayune.

But what is way worse is what we can see when the camera pulls back:

Clearly, there are substantial debris problems with the completed levee at WBV-3b. There's more on this in Part 8 of this series.]

Regarding quality, Rioux got a quote from Julie Vignes, the Corps' top person on the west bank:

"'We will make sure everyone is comfortable that the levees meet the quality mark that we all want,' [Vignes] said. 'We have a very good working relationship with the levee authority inspectors. They find things, we find things, and we work together to resolve them.'"

We'll discuss the considerable issues with quality and the Corps' skewed view of it later. But it is worth pointing out how that "very good working relationship" gets expressed on the ground. Exhibit A comes from the latest SLFPA-W inspection report of deeply troubled project WBV-14c.2. This inspection is the one the Times-Picayune's Susan Poag photographed for the article on June 10, 2011. The report begins:

"SLFPAW reps arrived at the construction trailers at 10:22am and met with Jeremy George (USACE), Chad Thibodeux (USACE), Lauren Fagerholm (USACE), and Susan Poag (Times-Picayune). USACE conducted site visit while remaining 50 yds behind SLFPAW reps. USACE also video-recorded SLFPAW reps throughout site visit."

Susan Poag's photo of this inspection indeed shows one of the Corps personnel holding a video camera.

I guess the Corps is so high on their working relationship with SLFPA-W, they wanted to record it for all posterity and replay it when they were feeling blue.

But seriously, we've seen this sort of paranoia before. Remember this Corps employee photographing The Lens' Steve Beatty on October 12, 2010?

Mr. Beatty was photographing a rusty hydraulic pump being pulled out at the Orleans Avenue closure structure. The Corps doesn't like people watching them, and their first creepy instinct seems to be to photograph the watchers to get - I don't know - something incriminating? I don't get it.

Anyhow, WBV-14c.2 plays a feature role in Mr. Rioux's article. It is one of the worst projects for debris, with multiple avenues for non-clay material to find its way into the new levee, and constant reports dating back to last fall documenting many debris issues. The T-P article focuses on just one of those issues - wood chips.

We looked at the SLFPA-W reports on wood chips from March and April of this year in Part 1. The T-P refers to the March 24, 2011 inspection record:

"SLFPAW representatives expressed their concerns about the debris in the material in the levee, and the material that is being hauled in to Jeremy George (USACE). The current lift is being placed on top of a questionable amount of debris before the problem has been resolved; this may raise concerns of acceptability."

That report includes pictures like this:

Rioux's article notes:

"Corps specifications permit isolated pieces of wood as long as they are shorter than 12 inches, have a cross section less than 4 square inches and comprise no more than 1 percent of the levee material’s volume.

Miserendino said corps officials assured him the wood chips did not exceed the 1 percent threshold. But he said a team of 10 state geotechnicians determined April 1 that the chips amounted to 1.8 percent of the material in some spots, nearly twice the corps’ limit."

The state wisely did not trust the Corps to do their own analysis. I have a copy of the final report done by the state's consultant here.

Here's the Corps' Ms. Vignes' response from the T-P article:

"Vignes said that the overall average was less than 1 percent and that the tests indicated there was a 'very small' chance the wood chips would cause the levee to settle prematurely."

Slabbed had an interesting reaction to this quote:

"Julie hon, if we stuck one end of you in an oven at 500 degrees and the other half of you in a vacuum chamber at -400 degrees your overall average temperature is a comfy cool 50 degrees. Levees, like your body need complete continuity (in the case of levees that of quality construction) to survive intact and you of all people should know that."

My response is a little less heated. Taking a look at the actual report, we find the state's consultant, GeoEngineers of Baton Rouge, took six samples on April 20th (it appears the Times-Picayune got the date wrong) and analyzed them for wood content. their findings on the numbers are unambiguous:

"Wood content ranged from 0.1% to 1.8% by volume and 0.01% to 0.03% by weight, based on total volume and weight (soil + wood)"

There was no "overall average" reported. Ms. Vignes was totally just making that up.

In the T-P article, Vignes continued:

"Noting that the levee is being raised in layers up to a foot deep, she said the inspectors arrived before workers had picked debris from a new layer of dirt."

"Vignes said the corps determined the material was from a new pit that, as is often the case, had roots and other woody debris near the surface. She said the pit had been excavated deep enough that the debris was no longer an issue, so the contractor continued using it."

So here's the Corps explanations:

a) Yes the amount of debris is over the 1% specification, but only in a relatively tiny portion of the levee. If you look at the whole 3.5 miles, it's fine! This is known as the "Tony Hayward" defense.

b) Also, SLFPA-W came early. They didn't give us a chance to clean the place up!

c) Finally, the borrow pits had wood chips, but they don't now. So ignore all those reports and pictures of other wood, rocks, concrete, bricks, and steel before, during, and since this particular wood chip problem.

I don't think Corps Public Affairs will be putting Ms. Vignes in front of any more reporters. Also, if her quotes indicate how the Corps gets the levees to "meet the quality mark that we all want," it seems clear that "quality" desn't mean the same thing in Corps-speak as it does in the real world.

Mr. Miserendino ably rebuts Ms. Vignes' last two arguments in the T-P article:

"Miserendino said wood chips were present at each of the authority’s subsequent weekly inspections. "Every time we questioned the amount of debris, the response was always the same, 'Oh, we haven’t picked it yet,'" he said."

Indeed, the next SLFPA-W inspection report, from the March 31, 2011 inspection says:

"Debris in the material that is being hauled in from Sta.55+00 to Sta 25+00. Area has not been picked yet but USACE assured SLFPAW representatives that this area would be picked before the next levee lift was placed."

But the wood chips persisted. Two weeks after the March 24th inspection, they were widely distributed. The report on the April 7, 2011 inspection noted:

"Site visit began on the floodside of the levee between Sta.0+44 to Sta 55+00. The material that is being hauled in contains a considerable amount of debris and is being picked out. Pickers were at approx. Sta.15+00 during site visit. Levee lifts are being placed on top of an abundant amount of small wood chips (within spec size) in this area. Phylway reps informed SLFPAW that USACE stopped hauling of material to this area until the problem is further investigated."

The pictures tell the tale:

This report further undercuts Ms. Vignes' reasurances. Even when some debris was being picked up, most of it was still getting buried into the levee.

Mr. Rioux's article finally notes that no matter what the Corps says, the debris problems have continued unabated at WBV-14c.2, noting the June 2nd report says "debris throughout the jobsite continues to be an issue."

After the article was published, an even more recent report was made available. It is from June 10th and like the June 2nd report, it's more of the same. In addition to the palpable mistrust between the Corps and SLFPA-W ("USACE also video-recorded SLFPAW reps throughout site visit"), the same old debris concerns are there:

"Levee:
- East end of project (Sta.182+00 to 162+00):
A noticeable amount of debris was found on the protected side berm and on the levee in this area. USACE informed SLFPAW reps that this area had not been picked yet.
- East end of project (Sta.162+00 to Sta.120+00):
A noticeable amount of debris was also found on the protected side berm in this area. Sta.150+00 to Sta.162+00 had been picked earlier in the week. Sta.150+00 to Sta.120+00 had been picked a few weeks prior to inspection.
- North/South levee (Sta.70+00 to Sta.112+00):
A fair amount of debris was found on the levee section in this area.
- West end of project (Sta.15+00 to Sta.0+00):
Inspected area where contractor had stopped degrade operation. A small amount of debris was found was found on top of the levee in this area."

Just as Mr. Miserendino predicted, the Corps claimed the "area had not been picked yet." To which I - and likely the SLFPA-W inspectors after all the other times they had heard that - would ask, "Why the hell not?"

You could almost hear the SLFPA-W inspector sigh as he typed the wrap up of the visit:

"Debris throughout the jobsite continues to be an issue. East end of project (Sta.182+00 to 120+00) will need to be picked again; including areas that were picked earlier in the week. North/South levee (Sta.70+00 to Sta.112+00) will need to be repicked prior to seeding of the levee in this area. West end of the project (Sta.15+00 to Sta.0+00) where degrade operation had stopped will need to be re-picked also."

The SLFPA-W inspector reinforces the point with 21 pages of pictures of rocks, wood, and other detritus like these:

I'm sure the Corps got great video of those pictures being taken.

Further details on the copious problems at WBV-14c.2 can be found in parts 1 and 4 of this series. Those problems would seem to be tightly wound up with the contractor on the project (Phylway) and the dirt supplier (River Birch).

Rioux's article provides confirmation of the link between the River Birch pits and the multiple debris-laden projects in which Phylway has been involved (see Part 5 for specifics on those projects), while providing more detail:

"Officials with Phylway Construction of Thibodaux, which has a $28.8 million contract to raise the [WBV-14c.2] levee, did not return a message seeking comment.

Vic Culpepper, River Birch’s technical director, emphasized that the corps approved all three of River Birch’s pits based on soil tests.

"In the past five years, we’ve supplied more than 2 million cubic yards of clay to more than 20 levee projects on the West Bank and never had an issue," Culpepper said. "That tells me the clay isn’t the problem."

He said River Birch simply leased a portion of one of its pits to Phylway, which excavated the dirt itself.

"How they process or handle it is up to them," Culpepper said. "I’m not saying they did it right or wrong, just that it wasn’t our responsibility."

Having seen the amount of debris in the dirt coming out of the River Birch pits, the argument they should be trusted because they have supplied 20 projects does not give a warm and fuzzy feeling.

Also, the mention of the lease between Phylway and River Birch firms up my suspicions about a contractual relationship between the two firms, but that doesn't seem to matter as River Birch's Culpepper seems more than willing to throw Phylway under the bus at the first question from a reporter.

Finally, River Birch's claim of "no issues" to a reporter, when it has taken subpoenas and FBI raids to get information out of River Birch in the federal investigation, is absurd in the extreme. I doubt they would reveal additional unpublicized problems to a reporter writing an article documenting the problems that have already been publicized through numerous inspection reports available on the internet.

Rioux's article notes the power of those inspection reports by recounting an incident involving the May 10, 2011 SLFPA-W inspection of project WBV-17b.2:

"Inspector David Roark concluded that the levee authority’s 'concerns about the debris appear to have been disregarded to meet the June mandate' to provide protection from a 100-year storm by the beginning of the just-begun hurricane season.

Miserendino said the scathing report elicited a series of calls from the corps official overseeing the project to try to rectify the problem.

'After we called them out, the contractor hired more people to pick debris,' Miserendino said. 'They handled that quite well.'"

I must say that story sounds good. The WBV-17b.2 inspection reports show a slightly more nuanced set of events. After the May 10th report, there were actually more pictures of debris in the next report (30 vs. 29), though the debris in the subsequent report does seem to be smaller.

As noted in Part 5, that following WBV-17b.2 inspection, on May 23, 2011, featured the Corps inspector - Larry Pryor - pushing to finish the levee in advance of the June 1 deadline, and the SLFPA-W (represented by David Roark) insisting (rightly) that it's not acceptable.

"Overall review of site visit:
Levee is completed to construction elevation with the exception of two areas. The first at the east entrance levee construction crossing for WBV-72 and the second at the discharge pipe construction crossing for WBV-76.

USACE inspector expressed that the contractor believes the levee has been constructed to plans and specs and is ready for seeding; after reviewing the levee it appears to still have high and low areas along the top and sides of the levee. Also, debris was found along the top and sides of the levee.

Debriefing:
Before seeding, the levee needs to be graded, compacted and the debris removed (wood and concrete)."

So debris was still in the levee, even after the Corps sent more folks out there to clean it up:

As I mentioned above, these appear to be somewhat smaller debris pieces than the ones shown in the May 10th report, perhaps pointing to the increased efforts to pick debris reported by Mr. Miserendino. However, the effect of increased attention to debris was undercut by obvious wavyness of the levee:

Since the T-P article appeared, another WBV-17b.2 report has been made available. The report on the June 9, 2011 inspection, shows that the impact of the May 10th report was probably temporary. The report says:

"- The crown and side slopes of the levee still contains high and low areas which will need to be corrected. Unless the contractor believes the levee is construction to plans and specs there should be grade stakes along the project for control points for final grade and compaction.
- Rocks/concrete and wood debris still exist; need to remove before seeding."

"Debris, mostly rocks/concrete was noted during today's visit. SLFPA-W and the USACE representative picked up and placed most of the found debris in small piles along the levee."

The debris photos (17 of them, admittedly fewer than before) show the rocks and concrete:

And the wavyness in the levee remains as well:

Based on the reports since May 10th, it appears the Corps and the contractor (Healtheon) remain ultimately unresponsive to not only debris concerns, but also other major issues of quality.

Mr. Miserendino and the SLFPA-W insepctors have good reason to worry about such quality issues. Mr. Rioux's article notes:

"Miserendino emphasized that the debris problem isn’t so severe that he fears a catastrophic levee failure could put people’s lives and homes at risk. Instead, he’s worried the unsuitable material will cause the levees to subside prematurely, sending the authority’s maintenance costs skyrocketing.

'Do I think these levees are going to blow out? Do I think they’re going to fail? No,' he said. 'But do I think we could face a significant maintenance nightmare that will cost Jefferson Parish taxpayers a lot of money? Yes.'"

Mr. Miserendino's concerns about maintenance dollars are legitimate, since the SLFPA-W is currently looking at future bills for maintenance of this system which they know they cannot afford. Any additional problems would distract from keeping the good stuff maintained.

Also, I have to note that Mr. Miserendino may be slightly optimistic, since we don't really know what else is in these levees. According to their reports, the SLFPA-W inspectors only started clamping down on debris in the last half of 2010, but levee construction has been going nearly continuously since 2005. Indeed, on troubled project WBV-14c.2, the existing levee - which was being used to build the new levee - was so contaminated with debris that the dirt from it was ripped off the project and thrown away. How many other spots like that - of which Mr. Miserendino and the public are unaware - lie beneath the miles of levee under construction the last six years?

Put this together with the Corps videotaping the SLFPA-W inspectors, the unending problems with debris on multiple projects, the blatant ignoring of SLFPA-W inspectors' documented concerns, the need for the SLFPA-W to go to the paper to get their concerns heard, and the Corps' relentless, ultimately failed drive to finish on June 1, 2011. Despite what the Corps claims, it is clear the relationship between the Corps and the SLFPA-W is toxic, and that the Corps' side is powered not by a desire to do things right - or even according to their own specifications - but to just do things fast with little regard to quality and lots of deference to contractors' wishes over those of the SLFPA-W. I know no one likes having their mistakes pointed out, but Mr. Miserendino says it best at the end of the T-P article:

"'We’re not going to get another bite at the apple. We’re not going to have another $14 billion dropped on us,' he said, referring to the total cost of post-Hurricane Katrina flood-protection improvements across the New Orleans area. 'We’ve got one shot at this, and we’ve got to get it right.'"

Still to come

2011-06-07T12:15:00.006-05:00

A while back I reported briefly on what I believe was a massive outage of the Corps' control and monitoring systems (called "SCADA" systems) along all three New Orleans outfall canals in December, 2009.

The entry was based on a massive amount of SCADA data (multiple gigabytes) I received through FOIA requests of the Corps New Orleans District. The data covers about two years' worth of readouts from every level gauge and every sensor on every pump along the the London Avenue canal. It also covers about a year's worth of the equivalent data on the 17th Street and Orleans Avenue canals. The London Avenue data runs from mid-2008 through mid-2010, while the 17th Street and Orleans Avenue data runs from mid-2009 through mid-2010. The data is logged every ten seconds.

Due to how the Corps preserves and overwrites this data, I believe I am probably the only person that now has this information. It includes run data for four of the six storm-related closures of the Interim Closure Structures since 2006. That is, I've got data every 10 seconds during the two closures at the London Avenue canal in September and November, 2009, as well as the two 2008 closures at London Avenue (Gustav and Ike). The data doesn't include the 17th Street closures during Gustav and Ike. I've also got data on every time the pumps were turned on during the mid-2008 or mid-2009 (depending on canal) through mid-2010 period noted above. This likely represents the only true data on how the Corps' lakefront pumps actually run. I should be able to match it up with the pump logs I've obtained from the New Orleans Sewerage & Water Board, showing how much water was pouring into the canals when the Corps pumps were supposed to be pulling it out.

Prior to this, the only other run data released to the public was a highly redacted set of partial data for the 17th Street and London Avenue canals during Gustav and Ike, sent to Molly Peterson as part of a FOIA request for her 2009 "Pumps Under Pressure" report. With my SCADA data, I'll be able to see if the data sent to her was valid.

I'll be bringing out some analysis around this soon. It's a lot of data to crunch, so it takes a while. But maybe this will answer some questions that have lingered for a very long time...

Debris, Part 5

2011-06-03T11:05:00.031-05:00

Part 1
Part 2
Part 3
Part 4
Part 6
Part 7
Part 8
Part 9
Part 10
Part 11

The record in dealng with debris has been mixed throughout the Corps' projects. Let's look through the system, courtesy of the inspection reports prepared for the Southeast Louisiana Flood Protection Authority - West.

WBV-12

Some projects appear to have moved past the problem. Earlier, we noted WBV-12 (a project titularly awarded as task order to the much larger West Closure Complex contract. The WCC prime contractor is Gulf Intracoastal Constructors, which is actually Traylor Brothers and Kiewit. However, WBV-12 was actually immediately subcontracted to Phylway, who we will find out is behind many of the debris-laden projects.) having debris problems dating back to the beginning of the project last fall all the way through March of this year. From the April 4th report on March 28, 2011 inspection:

"Debris was noted in several areas between the sheet pile driving operation and the first location of embankment work. As debris was found the USACE inspector placed debris in the back of his pickup truck. Between approximate stations 170+00 and 175+00 SLFPA-W noted debris in surface being covered by current lift and in material being placed, no personnel was monitoring area for debris. SLFPA-W continued to approximate station 138+00 were the second location of embankment work is ongoing. Debris was noted in the surface being covered by current lift as at the previous area of embankment work. The truck spotter did remove debris as SLFPA-W began taking pictures.
Overall I believe that since the current lifts are being placed on a surface containing debris, then how much debris was not removed?"

The debriefing summary also noted:

"At the second location the truck spotter began picking up debris when SLFPA-W was photographing debris."

Subsequent inspection reports have better news:
May 3, 2011 report of April 25, 2011 inspection:

"The debris found today was excavated from the flood side along the slurry trench installed. This material is to be hauled off and replaced with compacted material. Overall debris issue has diminished to nil. Truck spotters still utilized to remove during dumping operation. USACE inspectors to insure that the current method of removing debris continues."

And the May 23, 2011 report of the May 16, 2011 inspection:

"Debriefing:
No issues at this time. Debris no longer an issue, minimum amount was found during today's visit."

That's good. But the question asked in the report on the March 28th inspection is still hanging out there:

Overall I believe that since the current lifts are being placed on a surface containing debris, then how much debris was not removed?"

Moving along, we find other projects are still dealing with debris.

WBV-14a.2

WBV-14a.2 is a 3 mile long hundred year levee project on the west bank of the Harvey Canal:

The lead contractor is Purnell Construction, though a company named Hill Brothers also claims it. It's likely Hill Brothers is subcontracted to Purnell.

Before moving on to the debris problems, I must admit to some confusion regarding the scope of this project. Why is the Corps building a 100-year project behind the West Closure Complex (WCC), when they claim that the WCC is supposed to take all these floodwalls and levees out of the line of fire of a hundred year storm? In fact, there are two other 100-year projects immediately to the north and south of WBV-14a.2 (WBV-38.2 and WBV-23), both of which also lie behind the WCC. This seems like money that could have been better spent elsewhere, using the Corps' reasoning. After all, many of the other projects behind the WCC - including the projects just across the Harvey Canal from these - are not up to the 100 year standard, so why do these three get that treatment?

Anyway...

WBV-14a.2 has had reports of debris problems through May. During a May 5th inspection documented in the May 9, 2011 report, there was this:

"Debris in current material being placed"

This is followed by 3 pages of debris pictures:

The report also notes:

"Debris in material previously placed and being topped by current lift. SLFPA-W noted two man crew picking up debris."

So debris was getting buried. That's very bad. Also, a two man crew is too small for such work.

The above comment was followed by another 7 pages of debris pictures:

From the pictures, it is clear the debris is not just organic, but also includes rocks or some other inorganic material.

Understandably, SLFPA-W came back just about a week later. On May 13th (documented in the May 16, 2011 report), this is what they found:

Overall review of site visit:
USACE inspector [Wade Lucas] expressed that the contractor has stepped up debris removal from a two man crew to a seven man crew along with a crew removing debris at the pit (River Birch). Once the material is on site the material is picked through three times before the next lift is installed, first time during dumping and spreading operation, second time during the disking process and compaction and the third time during disking before placing the next lift."

As I said, a two man crew was too small.

Also note that a River Birch pit is mentioned again. We heard about debris coming out of one of the River Birch pits on WBV-14c.2 as well.

The report continues:

"Reviewed area between approximate stations 959+00 and 972+50; 6th lift has been placed. The amount of debris is considerably less in the 6th lift than noted previously during the placement of the 5th lift.

Reviewed area between approximate stations 935+00 and 951+00; 1st lift has been placed. Debris crew picking up debris in this area during visit. Several large pieces of wood was noted along with one large rock. Area also contains a considerable amount pieces of wood. The amount and size of debris found is substantial to warrant concerns that the contractor should put forth more effort in the removal of debris in this area."

Let's stop right here and read between the lines. The report starts with the Corps inspector's rather glowing description of the contractor's (that's Purnell or Hill Brothers) efforts at debris removal. Then it moves on to the reality that shows those efforts aren't worth very much.

If SLFPA-W inspectors are finding all this during weekly inspections, what exactly are the Corps folks doing when the SLFPA-W isn't there, especially during the months when SLFPA-W wasn't devoting its limited resources to stepped-up scrutiny of such projects? We've got dozens of reports of SLFPA-W inspectors finding large bunches of debris through many projects and over many months. How is this possible? Isn't the Corps paying its contractors and its Quality Assurance/Quality Control folks to keep tabs on this? Are the Corps folks and the contractors really acting on behalf of the public, or strictly on behalf of the contractor?

The SLFPA-W has similar concerns, as the report notes:

"Although the contractor appears to have stepped up debris removal operations, SLFPA-W has concerns with lifts 1 through 4 in area placed before the additional debris removal efforts were in place between approximate stations 959+00 and 972+50. A substantial amount of debris was noted in the 4th lift in the last report, which in may still exist.
SLFPA-W has concerns that with 250 plus trucks per day bringing in material and the amount noted today; the contractor needs to put forth a greater effort in monitoring and removing of debris."

Translation: SLFPA-W believes this entire new levee may be shot through with debris, and it is the fault of the contractor and the Corps. The SLFPA-W inspector includes 6 pages of debris pictures to make his point.

That brings us to the most recent inspection of WBV-14a.2, which took place May 23, 2011 and was documented in the May 30, 2011 report:

"SLFPA-W observed during today's visit that the contractor's nine man crew with supervision removing debris from material on site. The crew was working ahead of the material being placed and appeared to be removing all of the debris. SLFPA-W and USACE reviewed area between the picking crew and the material being placed and found it free of debris."

So they went from two men, to seven men, to nine men. What were they doing for the months before the SLFPA-W started zeroing in on them? Likely nothing.

Continuing with the report:

"Reviewed area between approximate stations 959+00 and 972+50; lift 7 has been placed to an elevation 1.0'. SLFPA-W and USACE reviewed approximately 600' of this area and found large pieces of wood debris. Later the contractor informed SLFPAW that the area has been disked but the debris in this area has not been removed.

Reviewed area between approximate stations 935+00 and 951+00; the 3rd lift was being placed and disked between approximate stations 935+00 and 941+00. This is in the area that the debris crew was observed picking up debris. The current material appeared to contain less debris but several large pieces were noted and the contractor removed them immediately."

The SLFPA-W wraps it up with this:

"SLFPA-W reiterated has concerns between approximate stations 959+00 and 972+50 with lifts 1 through 4 that were placed before the additional debris removal efforts were in place. A substantial amount of debris was noted in the 4th lift during placement of the 5th lift in the 05-May-2011 report, SLFPA-W needs verification that this debris has been removed."

I seriously doubt they will get such verification without tearing the levee apart.

Here's a sample of the debris SLFPA-W found on May 23rd:

By the way, this project doesn't look anywhere near complete, despite the Corps saying it was 100-year ready.

WBV-14b.2

WBV-14b.2 is a 3 mile long stretch of levee between an area near Lapalco Blvd to a point near where Highway 45 runs alongside the west bank levees. Here's the exact location:

It's a zig-zag section of levee. The project involved mostly shifting the centerline of the new levee toward the protected (land) side from the existing levee. Some sections of the new levee would simply be built directly over the existing levee. The new levee would be built with dirt from both borrow pits the existing levee, the same as adjacent project WBV-14c.2.

The borrow apparently came from one of the River Birch pits. The SLFPA-W inspector noted in the January 18, 2010 report on his January 13, 2010 inspection:

"The contractor is using River Birch as their borrow source."

This report was long before SLFPA-W became concerned about debris. However, about a year later the situation would be far different. With the SLFPA-W noticing copious amounts of debris on projects all over the west bank, they arrived at the WBV-14b.2 worksite on December 9, 2010 to find this:

"Debris was found throughout the flood side of levee and heavy in the berm area. During inspection the contractor began pushing embankment material from the protected side over the levee to the flood side. SFLPA-W representative noticed that dozer working material to the flood side was burying debris, as noted in issue #8. The protected side contains some debris but not as heavy as the flood side. Contractor had no one picking up debris in "area of work".

That's about as bad as it gets. The report contains 26 pages of debris photos like these:

We don't know if the debris came from the existing levee or from the River Birch pits. Either way it's not good.

The report also contains pictures of the contractor burying the debris, as described above. The debris appears to be large sheets of rusty steel:

Considering the contractor (Creek Services) was willing to do this in full sight of the inspectors, it's a good bet this had been going on the entire year the project was under construction up to that time. It's too bad SLFPA-W only concentrated on the debris problem in December, though, because the project was nearly done at that point.

Subsequent reports as the project wound down noted some progress, albeit mixed. The December 28th report of the December 21, 2010 inspection said:

"Very little debris was found on flood side of levee from Sta.0+35 to Sta.20+35. A 3 man crew with a bobcat was walking the entire levee area and picking up unsuitable material. Corps inspector assured us that this type of action would be continued throughout entire job."

After reading as many of these reports as I have, I have learned to be very skeptical when I see the words "Corps inspector" and "assured" in proximity. And as expected, an inspection on February 23, 2011 of the mostly finished levees turned up debris in the final product:

"23-Feb-2011 - The first day of inspection between approximate stations 164+92 to 74+65 revealed debris, ruts and low areas. It was apparent that some of the ruts were caused by others.
Contractor called out a crew to begin picking up debris. Crew picked up debris that SLFPA‐W crew flagging; no other debris was picked up by crew. SLFPA‐W (Roark, Muscarello), USACE (Major Giles, Channing) picked up debris in areas of flags and placed at flags as depicted in pictures."

There was still debris during another inspection on March 29, 2011 and April 1, 2011, but it was minimal:

"Overall debris found was within limits of specifications. Only one piece of wood was found of a substantial size and a minimal amount (1/2 dozen) of small concrete pieces were found, contractor removed all debris immediately. During inspection SLFPA-W noted contractor was picking up debris that was less than size limits set in specifications."

It's probably too little, too late.

WBV-15a.2

Debris problems on this 3.75 mile long levee project (contractor: Phylway) near Lake Cataouatche date back months, and possibly longer. In October of last year, SLFPA-W inspectors noticed rocks and other debris in completed sections of the levee. During an October 19, 2010 inspection, they noted:

"Issue: Unsuitable material found on levee from Sta.320+00 to Sta.330+0. Rocks are localized on floodside of levee."

SLFPA-W inspectors continued to find debris during the October 25, 2010, November 4, 2010, and November 8, 2010.

On December 21 and 22, 2010, things seem to get more serious. The inspection report for this two day visit indicates it was the first time SLFPA-W inspector David Roark had come to the site. His inspection report notes the debris problem in much more complete terms:

"21-Dec-2010 - SLFPA-W met Coe representative (Paul Williams) at the Coe site trailer. Reviewed project progress and both parties proceeded to site.
Overall review of inspection, some debris was found and Coe representative called contractor to site area of debris. Continuing with project inspection the contractor joined us; debris was immediately picked up as found during the inspection tour. Both the Coe representative and the contractor were concerned with the amount of debris found in area of finished grade berm (protected side) which basically filled the back of the Coe representative's truck (as depicted in photo P1000040)."

The Corps inspector and the contractor were concerned. Right. More like finally confronted. Here's the photo of the back of the Corps inspector's truck, filled with junk pulled out of the levee:

I'm not exactly sure of the workflow on a levee project. However, I believe dirt is initially delivered to piles in a "stock yard" on the site. Then I think it is taken to a flat area where it is laid out and "processed" before it is placed on the levee. Theoretically, the "processing" step should get the debris out, but it really shouldn't be in there in the first place.

This report includes pictures of the stock yard in a section noted as an "Issue." Specifically, the report says, "Unsuitable material in dirt needs to be picked out after dirt is spread," implying the debris in these photos is destined for the levee. Take a look at all the junk in this material:

If this is what is coming out of the borrow pits, and if the contractor and Corps inspectors are as slack about removing debris as the SLFPA-W inspectors lead us to believe they are, it's no wonder junk is making it into the levees. These stockpiles almost look like more rock than dirt.

Back to the inspections. Debris was again found during the January 4, 2011 inspection. That seemed to have been the final straw, because the next inspection - on January 20, 2011, the SLFPA-W sent three SLFPA-W inspectors and the Corps sent a Major. The very throrough inspection generated another truck bed full of stuff:

Afterwards, everyone seemed to think things would be okay:

"Debriefing: It was agreed that the debris concerns still exist, but the actions that the USACE and contractor have taken appear to be resolving the debris concerns.
Actions taken:
- Truck spotter is removing debris as trucks dump.
- Contractor's crew inspects for debris again, photos and removes any debris found.
- USACE inspects area again for any debris, debris found contractor removes."

But - as with so many other projects - the debris kept showing up. From the February 21, 2011 inspection:

"Reviewed areas of currently placed fill between approximate stations 465+00 and 468+00 along with section between stations 340+00 and 325+00. Some debris was found between stations 340+00 and 325+00 on the protected side as noted later in report by photos. Contractor stated lift was just placed and assured that the next step is to remove debris."

More assurances that debris would be removed. After a while, they just don't wash.

So what was reported during the April 6, 2011 inspection should not come as a surprise:

"Reviewed flood side of project between approximate stations 317+00 and 377+00. Debris was numerous through out this area in the berm. Most debris found was rocks and concrete; all of a considerable size."

And the SLFPA-W inspector's view should also be no surprise:

"Debriefing was held with USACE and the contractor. The amount of debris found along the flood side berm and haul road is still a considerable amount. During the past several site visits debris along the flood side berm has been located and removed. SLFPA-W believes the amount of debris found should have decreased in this amount of time and has requested that the contractor focus on debris removal before the next SLFPA-W site visit. Contractor agreed and is also attempting to schedule to begin seeding operation at the end of two weeks or the beginning of the third week from today. Seeding operation will cover the protected side berm along with both sides and top of the levee. Contractor to harrow areas before seeding and follow with a crew to remove any debris. USACE will notify SLFPA-W once harrowing begins."

How many times do the contractor and the Corps inspectors have to agree that debris is a problem, and then have more debris pointed out during the very next inspection, before something serious is done?

Unfortunately, this is the last report we have from this project. However, I think you get the point.

River Birch pits (+ Phylway?) = debris

Something starts to emerge from these reports: the River Birch pits appear very, very bad. We've seen the possibility of it in the WBV-14b.2 reports above, we've seen it mentioned definitely in the WBV-14a.2 reports - also above - and we've seen it mentioned in the WBV-14c.2 reports earlier. Now it shows up in the most recent inspection report from project WBV-14e.2.

WBV-14e.2 is a levee project across from the West Closure Complex, on the Gulf Intracoastal Waterway:

In the May 27th report on the May 26, 2011 inspection of WBV-14e.2, we read this:

"- Reach 3C (approximate stations 768+50 to 799+35), existing levee has been capped to elevation 11.0'. Contractor was placing embankment on protected side berm at approximate Sta.787+50. Contractor began hauling in material from the River Birch pit on Thursday 5/25/11.

Debriefing:
- SLFPAW reps expressed their concerns about the contractor changing from the Willowbend to RiverBirch pits."

The contractor is Phylway, the same one as on WBV-14c.2.

What exactly is happening here? We know River Birch has some sort of hold on certain members of the Jefferson Parish political establishment, but do they also have a similar influence on the Corps and their contractors? And what would make the contractor switch from a pit which is known to be clean to one that is known to be debris-laden? After all, the SLFPA-W reports have repeatedly commended the Willow Bend pit as quite acceptable. In their May 23, 2011 report on a May 18, 2011 inspection to WBV-14e.2, they specifically noted:

"No issues at this time. Material is hauled from the Willow bend pit and is very clean."

A possible connection is the contractor Phylway. Their name, along with River Birch's, keeps popping up on projects where debris is a problem. They are the contractor on what appears to be the worst project, WBV-14c.2, as well as three other projects with reported debris problems: WBV-12 (covered in Part 3), WBV-14e.2 (covered above), and WBV-15a.2 (also covered above).

I should also note that WBV-14e.2 is yet another project that was not actually at the 100 year level of protection at the end of May, despite the Corps saying it was. From the May 27th report on the May 26, 2011 inspection of WBV-14e.2:

"This project involves raising the elevation of approximately 3.5 miles of existing earthen levee to the 100-year level of protection varying between elevations +13.0 to +14.0."

"- Reach 1 (approximate Sta.649+54 to 661+50) has been constructed to elevation 12.5'. Final construction grade will be elevation 14.0'. No work was being done in this area during site visit.
- Reach 2 (approximate Sta.662+00 to 741+50), existing levee has been capped to elevation 11.0'.
- Reach 3A and 3B (approximate Sta.742+00 to 768+00) has been constructed to elevation 13.5' and has been seeded.
- Reach 3C (approximate stations 768+50 to 799+35), existing levee has been capped to elevation 11.0'. Contractor was placing embankment on protected side berm at approximate Sta.787+50. Contractor began hauling in material from the River Birch pit on Thursday 5/25/11."

How many projects are there like this?

WBV-18.2

We looked at the March 23, 2011 inspection of WBV-18.2 (contractor: Circle Construction) in part 3 of this series. At that time, there were debris problems. Since there, there's been two more inspections. An April 4, 2011 inspection revealed:

"Contractor placing only material to be processed and sealing in preparation of forecasted storm. Reviewed project from approximate station 245+00 to 160+00 along the protected side berm. Debris noted today was localized in the original berm area on the protected side between approximate stations 180+00 and 178+00. Overall material being placed for processing appears to be free of debris."

We get a hint of why the material looks cleaner in the report for the May 10, 2011 inspection:

"Contractor processing material; overall material placed for processing appears to be free of debris. Embankment material currently being placed on protected side berm at approximate station 211+00.

Very little debris was noted in review of levee and berm; debris found was removed and tossed to toe of levee (protected side) to be picked up by contractor later today.

Debriefing:
No issues, excavation was halted at pit were debris was coming from. Current material appears to be free of debris."

So this contractor (Circle Construction) stopped using a pit that was putting out debris. It's unfortunate the pit is not identified, but it does lend more credence to a theory that certain pits are contaminating levees all over the west bank with debris. However, I should note that this project was using borrow from pits located on the site, so it's not clear if the inspector was referring to those pits, or pits elsewhere. Nevertheless, the debris problem was there.

WBV-17b.2

We also looked at this project (contractor: Healtheon) in an earlier post. It too was having debris problems. And like so many others, they have continued.

Except there is one difference. The SLFPA-W have explicitly linked the debris issue to the Corps' drive to build stuff before June 1, 2011. This is shocking stuff to read.

From the May 16th report on the May 10, 2011 inspection report of WBV-17b.2:

"Overall review of site visit:
Levee is completed to construction elevation with the exception of two areas. The first at the east entrance levee crossing for WBV-72 and the second at the discharge pipe crossing for WBV-76.
The last lift placed atop the levee is compacted but not graded level; contains high and low areas (not sure which area is to grade). The sides of the levee where the last lift was placed appear not to have been graded or compacted. A considerable amount of debris was found through out the top and sides of the levee along with a small amount in the flood side berm."

So not only is there debris through this supposedly finished levee, it's also not really finished.

But here's the gauntlet being laid down:

"Debriefing:
The amount of debris found in the material placed in the levee is considered unacceptable. SLFPA-W noted no crew on site picking up debris today.
As reported in previous report; SLFPA-W has concerns that some debris may not have been removed due to the rate of 38 trucks per hour dumping material and no personnel was noted picking up debris.
Previous SLFPA-W concerns about the debris appear to have been disregarded to meet the June mandate."

Wow. That's really, truly laying it on the line. They (David Roark representing the SLFPA-W and David St. Marie representing the state of Louisiana) follow up with 8 pages of debris photos, and even include direct quotes about debris from the specifications. It is as close to an out-and-out argument as you will see in such a document.

The next inspection (May 23, 2011) shows the Corps inspector - Larry Pryor - pushing to finish the levee in advance of the June 1 deadline, and the SLFPA-W (represented by David Roark) insisting (rightly) that it's not acceptable.

"Overall review of site visit:
Levee is completed to construction elevation with the exception of two areas. The first at the east entrance levee construction crossing for WBV-72 and the second at the discharge pipe construction crossing for WBV-76.

USACE inspector expressed that the contractor believes the levee has been constructed to plans and specs and is ready for seeding; after reviewing the levee it appears to still have high and low areas along the top and sides of the levee. Also, debris was found along the top and sides of the levee.

Debriefing:
Before seeding, the levee needs to be graded, compacted and the debris removed (wood and concrete)."

Who exactly is Mr. Pryor working for here? Along with debris photos, the SLFPA-W inspection report includes page after page of photos of depressions and debris in the levee, but the Corps wants to say it's done? Here's the best picture (among many) of the rather obvious high and low areas along the top of the WBV-17b.2 levee:

There is something deeply wrong going on.

Conclusion

A few trends emerge from reading enough of these reports:

1) The debris problem is widespread

Here's a map showing the projects where SLFPA-W inspections have documented debris problems:

All those red lines represent eight levee projects covering 21.9 miles, 18.9 miles of it on the front lines of a hurricane. It looks like the majority of the 100-year hurricane protection levee system along the west bank, excepting the Mississippi River levees. This is not an isolated problem. This is a systemic threat to the integrity of the storm protection around New Orleans.

2) The Corps does not like the SLFPA-W

SLFPA-W inspectors do not appear to be welcome on the project sites by Corps inspectors. Some Corps inspectors explicitly ignore them, while others insist projects are acceptable when they clearly, obviously are not. As a result, levees appear to be getting built improperly, despite SLFPA-W objections.

3) Some borrow pits are debris-laden

Numerous SLFPA-W inspection reports point to borrow pits owned by River Birch as the culprit in the excessive amounts of debris being placed into miles of new levees around the west bank. Certain projects' contractors and Corps inspectors (and higher Corps officials) are well aware of this, yet continued - and continue - to use these pits, sometimes even switching to them from cleaner pits (this happened the last week of May on one project). This is a serious, systematic problem that screams out for investigation, especially in light of the ongoing federal investigation into River Birch and their dealings with Jefferson Parish politicos.

4) The Corps' race to June 1st has materially affected the levee work, resulting in unacceptable work.

The SLFPA-W says it explicitly and implicitly throughout their reports: the Corps' hurrying has damaged their levee work. Guess who eventually pays? The people living behind those levees.

5) The Corps lied about projects being 100-year ready on June 1, 2011

SLFPA-W photos and inspection reports from the last week of May document numerous projects still well below 100-year elevations, despite Corps public pronoucements claiming the opposite. This appears to be lying by the Corps.

In the next part, the debris problem breaks through to the print media.

Debris, Part 4

2011-06-03T11:00:00.007-05:00

Part 1
Part 2
Part 3
Part 5
Part 6
Part 7
Part 8
Part 9
Part 10
Part 11

We last looked in on the debris-laden levees lining the west bank of the Mississippi a little while ago (earlier parts linked above). It still doesn't look good.

As before, we'll start with troubled project WBV-14c.2. The last update we had was from late April, and had occurred after the Corps had actually had their contractor take samples of much of the project to determine how much debris was within the levees.

The Southeast Louisiana Flood Protection Authority - West has posted four more inspection reports for WBV-14c.2 since then, evidence they are keeping a close eye on this project.

The May 6, 2011 report on an inspection conducted May 3, 2011 indicates debris was still a problem:

"Overall review of site visit:
- West end of job (Sta.0+44 to Sta.55+00). No material was being hauled into area during time of inspection. Area is up to elev. +10'. Degrade of existing levee is scheduled to begin next week. Much less debris is being found in the levee section.
- All sheet piling on the west side of Westminister Pump Station has been driven. Hpiles are currently being driven at monolith G.
- On the east end of the job (Sta.119+00 to Sta.182+00) material is being hauled in to fill the protected side berm. Some debris is being found on the surface in this area.

Debriefing:
- All issues and concerns were discussed between SLFPAW and USACE representatives that day. Debris throughout the levee system remains to be an issue, although the material being hauled in is cleaner than previous material."

Only three pages of debris photos were included with this update. Here's a sample:

The next inspection took place May 12, 2011, as documented in the May 13, 2011 report:

"Overall review of site visit:
- West end of job (Sta.0+44 to Sta.55+00). The new levee is up to elevation 10 in this area and the degrade of the existing levee is in progress. A large amount of concrete rocks is being found in the existing levee. Contractor will need to go through this area and pick out unsuitables."

Concrete in the existing levee, which is then being put into the new levee. Nice.

Just to remind you, the new levee is being built, in part, using the existing levee. When the new levee reaches an elevation of 10 feet, the contractor is to start "degrading" (i.e. taking apart) the existing levee and use that material to build the new levee up to the final elevation of 13.5 feet:

The problem is that the existing levee is obviously shot through with debris, as the Corps and the contractor, Phylway, have known for months.

The report continues

"- North/South levee (Sta.68+00 to Sta.112+00). Contractor was hauling in material to the protected side berm during site visit (approx. Sta.95+00). Material is much cleaner than previous material that had been hauled into job site but the area will still need to be picked. Pickers were at approx. Sta.105+00 during site visit.
- East end of the job (Sta.119+00 to Sta.182+00). No active work was being performed on this end of the job, but some unsuitables were found on the protected side berm.

Debriefing:
- SLFPA-W reps expressed their concerns to Jeremy George (USACE) about the concrete being found in the existing levee that is being degraded (approx. Sta.0+44 to Sta.55+00). Debris throughout the levee system remains to be an issue, although the material being hauled in is cleaner than previous material."

There's three pages of pictures of the concrete from the existing levee on the west end of the project:

And three pages of the miscellaneous debris from the east end:

Things got worse the next week. Here's what happened during the May 18, 2011 inspection, documented in the May 23, 2011 report:

"Levee:
- West end of project (Sta.0+44 to Sta.55+00):
The new levee is up to elevation 10.0' in this area and the degrading of the existing levee is in progress. Material is being placed and compacted atop the new levee. Material from degraded levee contains concrete, bricks and wood debris. SLFPA-W reviewed top of new levee ahead of placement operation and noted some debris. Approximately a 1' lift has been placed on the flood side berm between approximate stations 0+44 and 10+00. Material appears not to have been disked, cleaned of debris or graded to final grade; contains debris."

Concrete, brick, and wood now. Not good. the description continues:

"- North/South levee (Sta.68+00 to Sta.112+00):
Contractor wetting and compacting crown of levee between approximate stations 95+00 and 110+00. SLFPA-W noted concrete debris in crown and slope (protected side) of levee.
- East end of the project (Sta.119+00 to Sta.182+00):
Contractor placing material along the protected side berm. SLFPA-W reviewed several areas and noted that the debris found was large in size but not as abundant as previous material. USACE inspector stated that debris will be removed during disking operation before final grading.

Debriefing:
Reviewed areas containing debris with USACE inspectors.
West end of project approximate stations 0+44 to 55+00:
- Debris in material from degraded levee, material to be disked and picked before next lift is placed.
- Debris in material placed along flood side berm between approximate stations 0+44 and 10+00. Material to be disked and debris removed before final grading.

North/South levee (Sta.68+00 to Sta.112+00):
USACE to review area again for debris.

East end of the project (Sta.119+00 to Sta.182+00):
Debris in material placed along protected side berm. Contractor to disk and remove debris before final grading."

With this update, we return to the many, many pages of debris photos seen earlier. There's 11 pages this time around. Here's some of the photos of the bricks, concrete and wood coming out of the existing levee and being placed into the new levee:

The debris problems continued with the latest inspection, just before the Corps' June 1st deadline. The May 27, 2011 report of the inspection held that same day says:

"Levee:
- West end of project (Sta.0+44 to Sta.55+00):
The new levee is up to elevation 11.0' in this area and the degrading of the existing levee is complete. Material from degraded levee contains concrete, bricks, and wood debris. Material needs to be disked and picked before another lift is placed in this area."

I just can't believe that they got everything out. The new levee likely has a ton of debris in it, both from the old levee and from the borrow pit.

The report goes on:

"- North/South levee (Sta.68+00 to Sta.112+00):
Final 6" lift is being installed in this area. Pickers have picked the entire north/south levee this week. Contractor plans to seed this levee within the next few weeks. SLFPAW will be notified before the seeding of this levee for a final inspection.
- East end of the project (Sta.119+00 to Sta.182+00):
Contractor is placing material along the protected side berm at approx Sta.170+00. Throughout the levee and berm of this area concrete and wood debris was found.

Debriefing held between SLFPAW reps and USACE:
- West end of project approximate stations (0+44 to 55+00):
Debris in material from degraded levee, material to be disked and picked before next lift is placed.
- North/South levee (Sta.68+00 to Sta.112+00):
SLFPAW reps will be notified before seeding of this levee is to take place.
- East end of the project (Sta.119+00 to Sta.182+00):
Debris in material being placed and throughout levee section. Contractor to disk and remove debris before final grading."

I'll skip the normal pictures of debris and concentrate on another issue.

At the same time this inspection was taking place, the Corps was claiming they had hit their goal for providing 100 year protection for the greater New Orleans area by June 1, 2011. Those statements included some caveats about a few projects they had "temporary measures," but for the most part, they claimed their construction was up to the 100 year level of protection (LOP). They included WBV-14c.2 as built up to the 100 year LOP on this map.

According to the plans and specifications, the 100 year design height for this project was to be 13.5 feet (notably half a foot short of the official design height established in 2007). This May 27th report explicitly says sections of the project were at 11 feet that day. In addition, there were some small sections of steel and concrete floodwall being built. Look at the May 27th photos of those areas:

Does that look like a project ready to confront a 100 year flood? Yet the Corps claims it's ready. This appears to be more than stretching the truth.

So that's where WBV-14c.2 is at the end of May. But it is not the only project with debris problems. In the next part of this series, it becomes clear the problem is still widespread, and the SLFPA-W makes explicit what they feel is causing it.

Pumps ... fixed?

2011-06-03T06:30:00.017-05:00

[Updated June 4, 2011, see bottom of post]

When we last looked in on the pumps at the Corps' West Closure Complex (WCC), they were all getting yanked out because all their bearings were overheating after just a few minutes' run time. How'd that work out?

Apparently pretty well. The Southeast Louisiana Flood Protection Authority - West has published their latest progress report on the WCC. It covers most of May, 2011 and it includes an update on the pumping situation:

"PUMP TRAIN:
During the past two (2) weeks, Pumps 6, 7, 8, 9, 10, 11, 12 and 13 were reassembled, realigned, break-in tested and wet tested for four (4) hours. There were some minor issues regarding fuel delivery and engine speed control but all issues were corrected and the four (4) hour tests were completed as specified. Test results have been submitted and are being reviewed, but preliminary vibration testing indicates that most recorded vibrations were in the 0.060 inches per second range and the allowable vibration according to Hydraulics Institute Standards ANSI/HI 9.64 is 0.28 inches per second RMS for this pump speed. All pump bearing temperatures stabilized well below the 200 deg. F maximum limit with 134 deg. F being the maximum bearing temperature recorded on a non-flooded upper enclosing tube bearing.
This brings the pump count to eight (8) that have been officially wet tested as required for the 01 June 2011 Interim Protection deadline. The Contractor intends to complete an additional pump next week and two (2) others in the next several weeks."

Sounds like good news.

"the four hour [wet] tests were completed as specified." Let's look at the details of those tests. We looked at the specifications of the wet tests earlier. Here they are again:

"Each pump unit shall be given a test with water under load, at or near normal operating conditions, for at least 4 hours or as directed by the Contracting Officer. The test shall be conducted by the Contractor and will be witnessed by the Government. All supplies and equipment required to conduct the test shall be provided by the Contractor. During the test the operation of the pumps will be observed and measurements of sound, vibration and bearing temperatures shall be taken and recorded. Without additional costs to the Government, the Contractor shall make all changes and correct any errors for which the Contractor is responsible. The Contracting Officer may waive or postpone the test if sufficient water is not available. Appropriate changes will then be made to the contract.

Vibration and bearing temperature records shall be kept by the Contractor and turned over to the Contracting Officer at the conclusion of the contract."

So there are four basic parameters involved here (besides the four hour duration), though I'm sure there's more detailed monitoring going on for the test. The basics are:

1) Sound
2) Vibration measurements
3) Bearing temperature measurements
4) "Normal operating conditions"

The first three are to be measured at the fourth. Thanks to this latest SLFPA-W update, we know a little about the last three of these parameters. The update is (ahem) silent on the sound measurements.

Vibration is supposed to fall within limits set by ANSI/HI 9.6.4, "Rotodynamic Pumps for Vibration Measurement and Allowable Values" for a given speed. It's good to see this standard number in print, because it is the latest standard available (unlike the list of standards in the actual specification, which was obviously not checked before it was issued). It's from 2009.

Bearing temperatures are apparently not supposed to top out above 200 degrees F, and during the tests never exceeded 134 degrees F.

The other parameter mentioned in the wet test specifications is "normal conditions." That refers to two numbers: the height of the incoming water and the speed at which the pumps spin. The Corps doesn't control how much water is at the inlets to these pumps. It can range anywhere from 0 feet to 7 feet, and is dependent on the tides and if there was rainfall. But they can change the speed, which directly affects the flowrate they are shooting for - around 1740 cubic feet per second (cfs) per pump. The exact speed-flowrate correlation was determined during model testing by the manufacturer, Fairbanks-Morse.

I have learned from the author of the SLFPA-W update, Mr. Danny Caluda, that the 5400 horsepower C280-12 Caterpillar pump engines are programmed to spin at two speeds: 990 RPM and 1010 RPM. After the power is transmitted from the engine through the Lufkin Industries gearbox, the two engine speeds correspond to a low pump speed of 171.7 RPM and a high pump speed of 175.2 RPM. The lower speed is for lower inlet water elevations (anything below 2 feet). The higher speed is for the higher elevations (2 feet and above). The WCC Water Control Plan and the more detailed Interim Standing Instructions call for the inlet height to be kept below 4 feet, with hopes that it never gets to 7 feet. If water does get that high, that probably means the local pump stations feeding the WCC will be shutting down in order to prevent levee breaches, but causing neighborhood flooding. More details regarding this scenario, which is more likely with the 8 pumps currently in service as opposed to the full complement of 11 pumps, can be found here and here.

Note the rated - and specified maximum - speed of the engines is 1000 RPM, but the top speed in practice is 1010 RPM. Mr. Caluda tells me that at the 1010 RPM speed, the engines do not exceed their continuous duty horsepower rating, which is good. The increase above the original specification came about when the number of pumps was reduced from 13 to 11 in a cost-cutting move. In order to keep the total flowrate of the 11-pump arrangement equal or greater to that of the 13-pump arrangement, it was decided to run the pumps a little faster at higher inlet elevations. The speed increase from 990 to 1010 RPM allows for an additional 45 cubic feet per second (cfs) of flow for each pump, or an overall increase of 495 cfs, something which will be useful when the water is higher during a storm.

Since the water was so low in the canals during these May tests (around 0), the pumps were only tested at the lower speed, 990 RPM (as noted on the photos below). The difference between the two settings, 20 rpm on the engine or 3.5 rpm on the pump, is not particularly substantial, so running these tests at 990 RPM would appear to satisfy the specification requirements for testing "at or near normal operating conditions." In addition, Mr. Caluda informs me that when the water does get above 2 feet (after a rainstorm causes stormwater to be pumped into the canals upstream of the WCC), the Corps and SLFPA-W hope to have tests done at the higher speed.

Mr. Caluda included pictures of from the tests of four pumps - 6, 8, 11, and 13. There are no pictures from the tests of pumps 7, 9, 10, or 12.

On May 18, 2011, pump 11 was tested:

Pumps 13 and 8 were tested on May 21st:

Pump 8 was tested again on May 23:

Pump 6 was tested May 25th:

The only press video I can find around the pump testing was a video shot by the Times-Picayune newspaper on May 24th:

The date of the video, May 24th, is confirmed by this Corps photo:

What's interesting about this video, besides the fact that it appears to be the only one of the pump testing, is that it shows another test of pump number 8. That would make at least three different days pump 8 was turned on for testing - May 21st, May 23rd, and May 24th.

I've learned that the actual final 4 hour test of pump 8 did not occur until May 31st. Apparently, there was an issue with the Caterpillar engine speed control during the intial tests. An engine control module was replaced and the pump was retested, but the problem persisted. After replacing a wire from the engine control panel to the governor, the problem was repaired and the pump made it through its four hour test on the 31st. These kind of hiccups are not unusual during testing, though it is notable that the engine had previously been tested by Caterpillar last summer before installation.

However, this does bring into question Corps public statements about the testing on May 24th. Take a look at this screenshot of another Corps photo taken that day:

There are different flavors of "success." While the pump may have pumped the correct amount of water that day, its engine was still not meeting specifications, and the Corps knew that. It's a small thing, admittedly, but would it have really hurt them to exclude the word "successful" from the caption on this photo (and this one, and this one)? Such slipperiness with the truth does not help the Corps' case.

As noted in the SLFPA-W update, a ninth WCC pump is due to be tested within the week, and the remaining two pumps are due to be tested by the end of June. I will be sure to pass along the results of that testing.

[Update, June 4, 2011]

With the four hour wet tests apparently out of the way, the Corps felt confident enough to have the cameras out for a photo op, which they portrayed as a test.

WWL-TV was there:

As was Fox 8:

Both pretty much regurgitated Corps press release materials ("can fill a swimming pool in blah blah seconds," etc).

The Times-Picayune was also there, and provided a little more depth, mentioning the bearing failures in April, along with excuse-making by the Corps:

"A bearing on one of the pumps overheated and cracked, prompting the contractor to remove all of the pumps and modify the bearings, said Kevin Wagner, a corps senior project manager who said the need for adjustments was not surprising.

'These are custom pumps,' he said. 'It’s not like we can put them in a lab and test them. They had to be adjusted in the field.'"

"Adjusting" is tweaking the engine speed controls like they did on pump number 8. Pulling every pump out a month before startup and remilling every bearing on them, after spending hundreds of thousands of dollars to rush to get most of them installed before the building's roof was put on? That's systemwide screwup, either by Fairbanks Morse in the bearing specifications and drawings, Bollinger Quick Repair in the assembly, or some subcontractor in the manufacture. Here's the list of those subcontractors, courtesy of Fairbanks-Morse:

Advance Manufacturing, Cleveland, OH
Advantage Machining, Aurora, IL
Baumann Coatings, Bessemer, AL
Bay Cast, Inc, Bay City, MI
Bollinger Quick Repair, Harvey, LA
Decatur Foundry, Decatur, IL
Girard Machine Co, Girard, OH
Hardie Tynes Co., Birmingham, AL
J&J Precision Machine Co, Cuyahoga Falls, OH
Louisiana Machinery, New Iberia, LA
Lubrication Systems Co, Baton Rouge, LA
Lufkin Industries, Lufkin, TX
Monarch Corp, Milwaukee, MI
Profile Finishing Systems, Waukesha, WI
RH Fastener Supply, Riverside, MO
RoMac Freight, Parkville, MO
St. Mary's Foundry, St. Mary's, OH
WJW Freight, Syracuse, NY

And as far as the pumps being "custom," it's not like Fairbanks Morse is new to this business. Their website, right next to a link about these very pumps, says:

"For over a century, Fairbanks Morse has been manufacturing a wide range of pumps for applications in municipal and industrial installations."

It's hard to believe anyone at the Corps New Orleans District still tries to gild the lily, but I guess old habits die hard.

Here we go again

2011-05-11T08:30:00.003-05:00

The last time we took a look at the pumps at the new West Closure Complex (WCC) on the west bank, across from New Orleans, we found that only 8 of the 11 pumps would be ready for June 1. No reason was given for the shortfall. In addition, we found there were very loose terms on the testing of the pumps. That testing was due to start in April.

We know the Southeast Louisiana Flood Protection Authority - West (SLFPA-W) has inspectors watching all the Corps' jobs on the West Bank. From their inspection reports, we were able to learn about systemwide problems with excessive debris being placed in the levees all over the west bank (Part 1, Part 2, Part 3). SLFPA-W's reports on the West Closure Complex have been similarly detailed, including descriptions of the first pump testing, which did occur last month. It did not go well.

The May 3, 2011 SLFPA-W report on their April 29, 2011 inspection visit tells the tale (note, while the report indicates an inspection on April 29th, it actually recounts events that happened over a multi-day span dating back to mid-April):

"During this reporting period the dry testing of pump no. 13’s engine, gear and pump began. There were several issues regarding gear oil pressure alarms and engine fuel delivery, but once those items were corrected the engine was started and the pump rotated with a dry impeller."

[Note: while there are currently 11 pumps at the WCC, the original design called for 13. It appears the original numbering has survived into the final design, thus the descriptions of pumps 12 and 13 in a station with only 11 pumps.]

So a few hiccups, but they got it going for the dry test. The terms of that test are laid out in the WCC specifications. Here's an excerpt:

"After pumping unit has been completely assembled, including all rotating elements and lubrication system, operate at full rated speed for a period of 4 hours or until the bearing temperatures have stabilized, whichever is longer, to assure proper alignment and satisfactory operation."

The dry test specs also call for monitoring of vibration in horizontal and vertical axes. They then describe exactly what is meant by stabilization of the bearing temperatures:

"Pumping unit shall be operated at rated speed and the bearing temperatures recorded at 5 minute intervals until the temperature rate of rise has stabilized for all bearings. Bearings' temperature shall be considered stabilized when the rate of rise does not exceed 1 degree Fahrenheit in five minutes."

If that doesn't happen?

"Dry test run shall be repeated if it is necessary to interrupt the test before all bearing temperatures have become stable. If after a run of reasonable duration the temperature rate of rise for any bearing has not stabilized, test shall be terminated until the cause of overheating is determined and corrections made. Then dry test run shall be repeated."

This is all pretty reasonable. And if they have problems that are the fault of the manufacturer or the pump assembler?

"Should tests reveal that there is a design deficiency or a manufacturing error in pumping unit components, the problem shall be promptly corrected by and at the expense of Contractor."

Good. The public shouldn't be on the hook for a manufacturer not meeting the specs. By the way, the manufacturer of these pumps is Fairbanks-Morse. They were assembled by local company Bollinger Quick Repair.

So how did all this work in practice? Back to the SLFPA-W report:

"The engine was brought up to operating speed and the test concluded with a failure of the upper bearing in the pump."

That's not good.

"Upon observation it was determined that the bronze sleeve type bearing had cracked and was seized to the shaft sleeve. Further analysis indicated that the bearing was installed such that it was not plumb with the shaft and the shaft loading on the bearing caused the failure."

SLFPA-W included pictures of the cracked upper bearing on pump 13:

Date stamps on these and other pictures of pump 13 in the report indicate the bearing failure probably occurred between April 17th and April 19th.

After a failure like that, the prudent thing to do would be to check the other pumps for similar defects. That's what was done, according to the SLFPA-W inspection:

"The remaining aligned pumps were checked and minor adjustments were made."

With the lack of detail on this checking procedure, it's difficult to ascertain how comprehensive the procedure was.

The next step was to try another pump:

"Next pump no. 12 was prepared for a dry pump test and the operation of that unit was discontinued when the lower pump bearing temperatures became elevated above the safe operating range."

Again, not good.

"The decision to test pump no. 11 under wet conditions was made and the pump produced flow for approximately five (5) minutes before the lower pump bearing RTD indicated excessive bearing temperatures."

This decision certainly raised my eyebrows. It's not told how the Corps jumped from dry testing to wet testing, which is a big leap. While the specifications do not explicitly call for the wet test to follow the dry test, it would certainly seem prudent to do so.

Anyhow, at this point it was clear to them there was a systemic problem with the bearings:

"With that issue it was concluded that the bearing clearance tolerances were too tight and that all bearings in each pump would be removed and re-bored to achieve a total bearing clearance of 0.025 inches."

Whoa! After spending months installing all these pumps (and showing them off "victory lap"-style to various groups, multiple bunches of fellow Corps employees, and assorted Pentagon bigwigs before anything had even run), they now had to yank all 11 of them out and completely disassemble them. Maybe the tours should have waited until they had a functioning facility?

So they started pulling them apart:

"The removal of the three (3) bearings in each pump began with pulling of pump no. 3. Once the bearings were removed from pump no. 3, they were delivered to Bollinger Marine for boring. While boring operations were ongoing, pump no. 12 was removed from its intake bay setting and the pump was disassembled."

A passage later in the report gives an idea of what was involved:

"The majority of the mechanical work performed for this period revolved around the testing and ultimate disassembly of pumps 3, 12 & 13. That work included removal of the intake air and exhaust piping, disconnecting the gear oil and coolant piping from the right angle gear, disconnecting engine to gear drive shafts, disconnecting gear to pump shaft couplings, removal of the right angle gear, removal of the right angle gear pedestal and pulling of the three pumps that had stop-logs already installed in the intake bays."

Here's pump 3 after it was lifted out but before it was taken apart:

And here it is after disassembly:

That is a ton of work, and they're going to do this on ten more pumps (all eleven pumps were already installed, as noted in the previous SLFPA-W report). I assume the Corps is having the contractors segregate the expenses associated with this work so the taxpayer doesn't end up on the hook.

The pump removal didn't exactly go as smoothly as the above paragraph would indicate. Returning to the earlier passage in the report:

"At the time of this report, pump no. 13 was being prepared for removal but there was some difficulty in removing the lower half of the pump to gear coupling. Nitrogen was being used to cool the upper pump shaft and the pump coupling was being heated with rose bud torches to provide the expansion needed to release the coupling from its interference fit."

Nitrogen! They had to bring in nitrogen to decouple the pump shaft!

So how did all this wind up? From the SLFPA-W report:

"Currently the bearings from pump no. 3 are being installed in pump no. 12 and the current scheduling indicates a possible test of that pump by 07 May 2011."

Presumably, the next SLFPA-W report will detail what happened during that test.

The 2011 pump rebuild scramble

2011-05-09T10:00:00.026-05:00

This entry was updated July 14, 2011 to bring it in line with revisions made elsewhere on this blog. These revisions came about due to a large pump repair document release in June and July, 2011 in response to FOIA requests. As such, the tense is an odd mix of tone, mixing voices from both the original date of composition on May 9, 2011 and the later revision date of July 14, 2011.

Last spring, we looked at a flurry of hydraulic pump repair activity at the New Orleans lakefront gates. This spring, there seems to be another such push.

Third Healtheon contract, task order #1

At the end of the last update, we noted that two pumps at the London Avenue site - E1 and E5 - had been removed and taken to Conhagen's shop for rebuilding around March 22, 2011, based on an oil spill report from that day. Here's pictures of the empty E1 and E5 slots in early April:

As of late April, E1 and E5 were back in at London Avenue:

This work was performed under $598,101.68 task order #1 to Healtheon contract W912P8-11-D-0015, the third pump repair contract. As described in the previous post, that task order also called for the repair of 17th Street pumps W4 and W7:

CLIN 1001A - Remove, recondition and reinstall pump #4W and #7W at 17th Street Canal @ $262,376.76.
CLIN1001BC - Remove, recondition and reinstall pump #1E and #5E at London Avenue Canal @ $334,724.92. Contractor may opt to use 240-ton crane at this site in lieu of 500-tonn crane as stated in contract."

Modification 1 to task order #1, issued for $12,433.26 on June 8, 2010 - after the work had been completed - accounted for the bearing and seal work that has become standard on all these repairs:

"Add the following additional work for all four (4) pumps:

Repair Mechanical Seals at subcontractor shop (4 Each)
Remove Pump Shaft Bearings
Replace Radial bearings Impeller End (4 Each)
Replace Radial Bearings Motor End (4 Each)
Replace Thrust Bearings (4 Sets)
Replace bearing Spacers (4 Each)
Inspect Shafts and Housing
Mount New Bearings
Clean All Parts for Assembly
Reinstall Pump Shafts"

It also allowed for a smaller crane at the London Avenue site.

The Conhagen repair reports for these four pumps (London Avenue E1 and E5 report, 17th Street W4 and W7 report) include the now-usual photos of very, very, very rusty pumps. And they also help solve a mystery.

The pictures of London Avenue pump E1 include this shot:

Notice the multiple zinc anodes, and how they are fastened to the pump's suction bell. The original pump order included only a single pair of anodes for each pump, and they were fastened in a different manner. Here's pictures of the pumps in 2006, with their original anodes:

And here's one of the 2009 Conhagen-repaired pumps going back in the water:

All this indicates this pump had previously been removed and partially repaired. This helps explain the May 22, 2009 spill report I noted over a year ago in Corps of oil, Part 3:

May 22, 2009 at London Avenue
Quantity released: 40 gallons (from QRI task order)
NRC report: No
LDEQ report: No
Description: From $5399.16 task order #4 to the second QRI contract: "Provide labor, material, and use of eqipment [sic] to clean up about 40 gallons of hydraulic fluid that spilled from pump 1E at London Avenue Canal."

At that time, I noted that E1 was not among the four pumps that had been pulled out at the London Avenue site before June, 2010. Indeed, I don't know of any contractual evidence of the repair of London pump E1, but it obviously happened.

It was probably repaired by the Corps themselves, before they started sending pumps to Conhagen through the first Healtheon contract. And since this happened so early on in the repairs (May, 2009), it also likely had the very limited scope of work involving just replacement of the carbon steel pipes inside the pump bore with stainless steel, and simple cleaning of the external piping and coolers followed by a new coat of paint.

Indeed, other pictures of E1 from the 2011 repair report back this up. The interior pipes are not rusted at all, indicating they're stainless:

While the external piping and coolers look like they've taken some serious corrosion damage since the first round of repairs in June, 2009:

Five other 60" pumps - all at 17th Street - also received this misguidedly small set of repairs before the Corps realized they needed to do more.

This makes London Avenue pump E1 the first 60" pump to be pulled out twice, doubly compounding the errors in corrosion judgement made in 2006 and 2007. This is a massive waste of money. It is extremely likely this pattern will be repeated on the 17th Street pumps repaired in 2009 as well.

Anyhow, the other three pumps rebuilt under this task order had all the normal rust damage to their Rineer motors and attached piping:

And to their oil coolers and external piping:

Conhagen completed all the repairs (it's unclear whether they got new or refitted Rineer motors) in April, 2011. London Avenue pumps E1 and E5 went back in the water April 27, 2011. 17th Street pumps W4 and W7 were reinstalled three days later, on April 30, 2011.

Third Healtheon contract, task order #2

Work continued this spring at London Avenue, with pumps W5 and W6 coming out:

We can tell they're out because we can see the bottom flanges of the elbows unattached to any pumps below them. We also don't see the distinctive piping extensions added to the pumps in 2007, as we can on the adjacent pump W4.

This work likely took place on April 26th, since there was a spill report from London Avenue that day:

"CALLER IS REPORTING A DISCHARGE OF HYDRAULIC OIL FROM A HYDRAULIC PUMPS OVER THE WATER AT THE PUMPING STATION DUE TO MAINTENANCE ACTIVITY (MAINTAINING THE PUMPS)."

The Corps claimed 3 gallons for this spill. Again, like the last spill report in March, I find this implausibly tiny, and a likely underreporting. However, at least it was reported, unlike dozens of others over the last five years.

There is another possibility for the source of this spill. It could have resulted from the reinstallation of London Avenue pumps E1 and E5. The acceptance testing for those pumps occurred on April 27th, the day after this spill, but they could have been getting put back in on the 26th. Without more detail, it's unfortunately impossible to know.

Meanwhile, there was also work at 17th Street, with E4 coming out on May 2nd:

The corrosion on the outer pump housing and the oil coolers is obvious:

The corrosion damage to the innards of the pump, including the Rineer motor, is likely similarly severe.

E3 was also due to come out that day, as evidenced by the missing flange bolts on its elbow:

E3 and E4 were untouched before this latest activity.

All four of these pumps came out under $523,862.74 task order #2 to the third Healtheon/Conhagen contract, issued April 25, 2011. The text of that task order reads,

"CLIIN 1001A: Removal, reconditioning and reinstallation of pump #3E and #4E at 17th St. Canal Interim Closure Structure.

CLIN 1001BD: Removal, reconditioning and reinstallation of pump #5W and #6W at London Ave. Canal Interim Closure Structure.

All work shall be completed by 17 June 2011."

However, as the Corps' self-imposed June 1st deadline for the completion of the 100 year protection system around the New Orleans area approached, apparently someone wanted results faster. Modification 1 to task order #2, issued after the fact on June 8, 2011 did two things:

1) It moved the deadline for the work on these four pumps up to May 31, 2011.

2) It added the now standard suite of repairs to bearings and seals:

"The Scope of Work is also modified to add the following additional work for all four (4) pumps:

Repair Mechanical Seals at subcontractor shop (4 Each)
Remove Pump Shaft Bearings
Replace Radial bearings Impeller End (4 Each)
Replace Radial Bearings Motor End (4 Each)
Replace Thrust Bearings (4 Sets)
Replace bearing Spacers (4 Each)
Inspect Shafts and Housing
Mount New Bearings
Clean All Parts for Assembly
Reinstall Pump Shafts"

It didn't do these things cheap. It cost $139,913.16, bringing the total value of this task order to $663,775.90.

My FOIA request also produced the Conhagen repair reports for both pairs of pumps (repair report for 17th Street pumps E3 and E4 and repair report for London Avenue pumps W5 and W6)

As usual, these reports are full of troubling photos. Here's a before and after of the seal flange on London Avenue pump W6:

Here's the seal flange on 17th Street pump E3:

I'm assuming this photo was taken to show the oil that had gotten past the seal and had spread. When they removed some of the outer housing, presumably this is what they found. This is bad, because big oil leaks like this mean lubrication for the bearings is gone, potentially causing pump failure.

Anyhow, Conhagen accelerated the repairs (hopefully completing them properly in the process). The London Avenue pumps went back in the water May 21, 2011. Here they are in place on June 1, 2011:

The 17th Street pumps were back in May 24, 2011, beating the Corps' artificial deadline.

[I realize this information and the charts below include information that occurred after the date of this post, but that's because I'm writing it after I got information that was missing at the time I originally composed it. It's timey-wimey.]

Stepping back a bit, this means the Corps has pulled ten pumps in five pairs for planned repairs (the planned repairs always occur in pairs) since the 2010 hurricane season:

London Avenue pumps E2 and E6
London Avenue pumps E1 and E5 (second round of repairs for E1)
London Avenue pumps W5 and W6
17th Street pumps W4 and W7
17th Street pumps E3 and E4

With the receipt of all the documentation on these pumps, we can update the pump status for the London Avenue site:

We can also update the 17th Street site. It is important to realize that the five 60" pumps reworked in 2009 with an incomplete set of repairs are extremely liable to oil spills.

The Orleans Avenue site has had nearly zero attention paid to it, with only a single pump pulled out in the last two years, and even that was only in response to an oil spill, not as a planned repair:

So, even after all the pumps we believe have gone for repairs, there will still be twelve 60" pumps still untouched - including 9 of the 10 at the Orleans Avenue site:

17th Street: W2, W3
London Avenue: W2
Orleans Avenue: E1, E2, E3, E4, E5, W1, W2, W3, W5

and another five with incomplete repairs:

17th Street: E5, E7, W8, W9, W10

which are likely to break again very soon.

And it's taken two years to get to this point. Since the Corps only pulls pumps out during hurricane season if they break, it's either unlikely (in the Corps' view) or likely (in view of all the photos of broken pumps I've published over the last year) we'll see more activity before the end of the 2011 hurricane season.

Matt Davis and Steve Beatty of The Lens contributed to this report.

Debris, Part 3

2011-05-05T10:00:00.020-05:00

Part 1
Part 2
Part 4
Part 5
Part 6
Part 7
Part 8
Part 9
Part 10
Part 11

In the previous two entries (Part 1, Part 2), we've been looking at the problem of debris within the levees the Corps of Engineers is building around New Orleans. If there's too much debris, organic or otherwise, within a levee, its integrity could be undermined over time, reducing the level of protection for the people behind it.

We've looked at one project where debris was a problem for 3 months before a plan was put in place. At another project, debris has been a continuing issue for over six months, but the Corps refuses to shut the job down to fix the problem. The Southeast Louisiana Flood Protection Authority - West (SLFPA-W) has said in its inspection reports that these debris problems could lead to them not accepting the projects from the Corps upon completion.

But that's just two projects out of dozens. How widespread is the debris problem?

Let's look at three more levee projects where the SLFPA-W has reported debris concerns, WBV-17b.2, WBV-18.2, and WBV-12:

WBV-17b.2

WBV-17b.2 is a relatively short stretch (just 1 mile) of levee on the Jefferson side of the Jefferson-St Charles Parish line. The project was awarded last November. There are only five SLFPA-W inspection reports thus far. The first, an early report from December, 2010, notes the contractor - Healtheon - hadn't yet mobilized. But the four more recent inspections show SLFPA-W ramping up inspections because of debris concerns.

From the March 21, 2011 report of a March 17, 2011 inspection:

"Debriefing was held on site with USACE inspector; surface debris was not abundant but the current 1' lift being placed contained debris. Due to the current spreading operation burying debris; SLFPA-W request a work plan on removing the current debris and controlling future debris hauled to site in fill material. Spoke with USACE representative, John Yanguba; he assured that the debris issue would be taken care of and requested another site visit later this week."

Of course, there's pictures:

The next inspection took place five days later on the 22nd. It's recounted in the March 28th report, with a healthy dose of skepticism by the SLFPA-W:

"An email was received for previously requested work plan on debris issue stated; 'We will reinforce and follow up with Ktor ["Ktor" = contractor] to have people spotting and removing debris as well as continue our inspection regimen which assures debris like this does not end up in final product." Some debris was found today of a considerable size, SLFPA-W expressed that debris of this size should not be found after contractor has worked area. USACE will meet with contractor again on debris issue."

So more promises. How did that pan out?

The SLFPA-W came out about a two weeks later, on April 4th to find out. The SLFPA-W didn't like what they found, as they wrote in their April 11, 2011 report:

"Contractor currently placing lift along protected side and top of levee between approximate station 157+00 and 155+00. Some debris was noted in current dumping and spreading operation; contractor personnel spotting trucks but not reviewing material for debris. No contractor personnel noted removing debris during today's visit. SLFPA-W and USACE reviewed remaining project along the protected side berm; debris was still found and some removed by USACE personnel."

Pictures:

And in the debriefing meeting at the end of the inspection, it's clear they feel things are getting worse, not better:

"Contractor currently dumping approximately 38 trucks per hour with no full time crew removing debris. The USACE inspector expressed that the contractor has three personnel assigned to picking up debris once in the morning and again in the afternoon. Several small piles of debris were noted during but no personnel picking up debris with the exception of the USACE inspector. SLFPA-W has concerns that some debris may not have been removed due to the rate of 38 trucks per hour dumping material and no personnel was noted picking up debris during today's visit. The debris issue has not improved since the last visit noting the size of debris found in the berm area."

38 trucks an hour, and no one spotting debris. For weeks at a stretch.

Most recently, though, things may have improved. From the April 25, 2011 report of the April 20th inspection:

"Reviewed material currently being placed at approximate station 132+50 (protected side) and at approximate station 130+00 (flood side); found no debris. Contractor is very cautious about not hauling in material full of debris to site. Material is processed and debris removed at the pit before hauling to site.
Levee between stations 160+00 to 133+00 is within the final lift or grading of final grade. The protected side berm between the above stations is to final grade.
SLFPA-W and the USACE inspector reviewed the levee and the protected side berm between stations 160+00 and 130+00; very little debris was found during todays visit.
The USACE inspector removed the debris that was found.
The degraded flood side of the existing levee was reviewed; a minimal amount of debris was found and removed by the USACE inspector."

However, that only accounts for what has happened recently. What about those days and weeks when the SLFPA-W wasn't there? And what exactly are the Corps folks doing that problems like this are even appearing, let alone alone appearing so often?

WBV-18.2

At WBV-18.2 (contractor: Circle Construction), a 2.65 mile long levee project underway for nearly two years, debris problems cropped up in March of this year. WBV-18.2 is located adjacent to WBV-17b.2 and is a much longer stretch of levee. From the March 28th report on an inspection on March 23, 2011:

"SLFPA-W reviewed embankment placement along the protected side berm. Current placement was noted in two areas; the first between approximate stations 236+00 and 230+00 along the protected side of the levee and the second area between approximate stations 225+00 and 222+00. SLFPA-W found debris in fill that appeared to be placed for processing between approximate stations 245+00 and 244+00. Various areas between approximate stations 224+50 and 219+00. Debris appeared to be in embankment fill currently being placed. Areas of previously place embankment appeared clean. Called the USACE inspector at 11:58am to inform about debris found. USACE inspector called back for specific areas that the debris was found, he and the contractor went out to areas to review debris. Also spoke with contractor on the location of the debris and was informed that some of the fill material was coming from the pit which may have had debris. SLFPA-W did not see any personnel picking up debris during today's visit."

By now you know what the pictures of debris look like, so I will dispense with them.

WBV-12

Meanwhile, over at the 2.2 mile-long levee project WBV-12 (a project titularly awarded as part of the West Closure Complex contract to Gulf Intracoastal Constructors, which is actually Traylor Brothers and Kiewit. However, it was actually subcontracted to Phylway, who we will find out is behind many of the debris-laden projects.), the April 4th report on March 28, 2011 inspection had this to say:

"Debris was noted in several areas between the sheet pile driving operation and the first location of embankment work. As debris was found the USACE inspector placed debris in the back of his pickup truck. Between approximate stations 170+00 and 175+00 SLFPA-W noted debris in surface being covered by current lift and in material being placed, no personnel was monitoring area for debris. SLFPA-W continued to approximate station 138+00 were the second location of embankment work is ongoing. Debris was noted in the surface being covered by current lift as at the previous area of embankment work. The truck spotter did remove debris as SLFPA-W began taking pictures.
Overall I believe that since the current lifts are being placed on a surface containing debris, then how much debris was not removed?"

Again, I'll dispense with examples from the 8 pages of debris photos, and simply say that this passage from the debriefing summary says it all:

"At the second location the truck spotter began picking up debris when SLFPA-W was photographing debris."

Debris problems at WBV-12 appear to go back right to the beginning of the project. Before construction had started, when the only activity happening on site was the processing of dirt, the October 6, 2010 inspection report, reporting on the September 29, 2010 visit, noted:

"Unsuitable material found in dirt. [SLFPA-W inspector] Chris [Muscarello] talked to Corps inspector Mr. [Julian] Hayman and he said they were picking out unsuitables as they see them."

Here's the pictures:

The following month, when clay started getting placed, SLFPA-W inspectors started getting very concerned about the debris at WBV-12. In the October 13, 2010 report on their October 12, 2010 visit, they found material along the entire length of the project. 11 of the 13 pages of the report are pictures of the debris:

The next time out, on October 26, 2010, SLFPA-W inspectors noted there was coodinated debris removal happening:

"Met with Corp inspector (Julian Hayman) on site. Reviewed embankment placement and debris pick up operations. Overall embankment clean of debris. Both embankment operations have personnel picking up debris as embankment is being placed. Debris is piled along work in progress area and then picked up."

However, debris - including steel - was still seeping into the project:

Inspections were less frequent over the following months. The report of a December 8, 2010 inspection didn't mention any debris. And while the material being trucked in during the February 14, 2011 inspection seemed to be clean on first blush...

"During dumping operation several trucks were watched while dumping and very little debris was noticed. The truck spotter was removing the debris as the trucks were dumping and during the spreading of the material."

in fact there was stuff in there - concrete:

"Further reviewed of the placement area revealed several pieces of concrete debris which the contractor failed to remove this was brought to the inspectors’ attention."

Which brings us back to the March inspection we started with.

Summary

That's five projects with miles of levees that have who knows what in them besides dirt:

This is a systemwide problem that continues to be addressed in a half-hearted way by the Corps and their contractors. Right now, it looks like the SLFPA-W are the only folks that care. That makes sense, since they're the ones who will be stuck with these things after the Corps finshes with them.

This must be addressed, and if it takes the Corps missing their precious June 1st deadline, then so be it. They were never going to make it in the first place anyway.

Unfortunately, as the next two parts reveal, the Corps did practitcally nothing to fix this problem in May. It just kept getting worse and more widespread, and the SLFPA-W got more and more alarmed.

Debris, Part 2

2011-05-05T09:15:00.014-05:00

Part 1
Part 3
Part 4
Part 5
Part 6
Part 7
Part 8
Part 9
Part 10
Part 11

In "Debris, Part 1" we looked at the serious problems at West Bank levee project WBV-14c.2. For months, the Corps and its contractor and subcontractors have been placing dirt with a lot of debris in it on the levee, all while representatives from the West Bank flood protection authority (the Southeast Louisiana Flood Protection Authority - West, or SLFPA-W) have been raising repeated and stern alarms.

We know about this from the voluminous and detailed reports filed by SLFPA-W inspectors. These inspectors are keeping track of dozens of projects and calling out problems when they see them. Right now, the problem of excessive debris in levee dirt is their number one concern.

Corps specifications call for no more than 1% organic debris by volume in their dirt. More than that could lead to holes in the levee forming as the organics break down over the years. SLFPA-W has already mentioned that they might not accept the Corps' work on WBV-14c.2 due to exceedance of this specification.

But the problem extends much further than this single project. In fact, it has struck at least 5 projects across the west bank, and likely more. It probably has quite a few folks freaking out. This post, like the last, will draw from the SLFPA-W inspection reports. These reports go back over two years and are meticulously detailed. They are also filled with pictures.

So let's start with WBV-72. That's a 2.8 mile long levee project on the west end of the WBV project, in St Charles Parish around Highway 90. The prime contractor is WRS Infrastructure. The official description of the project is (from the SLFPA-W inspection reports):

"Project consists of widening Highway 90, constructing two Waskey bridges, clearing and grubbing, dewatering, embankment of compacted and uncompacted fill, degrading sand cell levee embankment, Davis Pond Guide levee removal and opening, seeding and fertilizing, and all other incidental work thereto."

The exact location can be found on the Corps' April 15th construction progress map of 100 year projects:

We'll start with a passage from the December 13, 2010 WBV-72 report, documenting a visit on December 6, 2010:

"West end "Work in Progress": SFLPA-W representatives noticed several large pieces of wood at the west end of the project were the dozer was pushing fill. 12:28pm, walked to Coe [Corps of Engineers] representative (Larry Temple) sitting in truck and expressed that debris in area that dozer was pushing fill needed to be picked up before it was buried. Coe representative moved truck forward approximately 40ft, sat for a moment and drove off. SLFPA-W representatives watched, no one picked up debris and the dozer continued pushing fill."

Stay classy, Corps of Engineers!

It's too bad the inspectors didn't get pictures of this incident. However, they did get 33 pages of pictures of debris through the entire project. Here's just two of those pages:

Yes, that's a piece of pipe in the upper left of the second set of photos.

It wasn't like this problem appeared out of thin air. It had been going on for at least 3 months. The September 21, 2010 visit recapped in the September 24, 2010 WBV-72 report calls out debris:

"Issue: Unsuitbale [sic] material needs to be picked up."

And it came up again two weeks later during the October 5, 2010 inspection. From the October 8, 2010 WBV-72 report:

"Issue: Unsuitable material found in dirt. Chris tried to find CORPS inspector to tell him about this issue, but he was not found. Chris then told the contractor about this and the contractor assured us that they were going through and picking out unsuitable material in dirt."

And again on October 25th (from the 10/28/2010 WBV-72 report):

"Issue: Unsuitable material found in dirt at SM 100+00."

Once again on November 5th (from the 11/9/2010 WBV-72 report):

"Issue: Unsuitable material found in dirt at SM 100+00."

Yet again on November 8th (from 11/22/2010 WBV-72 report):

"Issue: Unsuitable material found in dirt."

And finally on November 23rd (from the November 30, 2010 report):

"Reviewed area of current material placement and noted several large pieces of debris in material, brought issues to Coe representatives attention. Coe rep expressed that material is being picked through at the borrow pit and on site. SLFPA-W noted a two man crew picking up debris as material was being dumped from trucks."

This pattern of inadequately sized crews for picking out debris, along with hollow reassurances from Corps representatives, is repeated throughout the SLFPA-W inspection reports for many, many projects. Fortunately, many of those reports include pages of debris pictures just like these:

This report includes 33 pages of debris photos. Clearly the two man crew and the debris pit folks (if they really existed) weren't doing their jobs.

Perhaps it was this November 23rd inspection that led to the Corps' Larry Temple's rudeness during the December 6, 2010 inspection described above. Let me just repeat that incident:

"West end "Work in Progress": SFLPA-W representatives noticed several large pieces of wood at the west end of the project were the dozer was pushing fill. 12:28pm, walked to Coe [Corps of Engineers] representative (Larry Temple) sitting in truck and expressed that debris in area that dozer was pushing fill needed to be picked up before it was buried. Coe representative moved truck forward approximately 40ft, sat for a moment and drove off. SLFPA-W representatives watched, no one picked up debris and the dozer continued pushing fill."

Let me also point out that after finding enough debris on November 23rd to fill 33 pages of their report with debris pictures, the SLFPA-W was able to duplicate the feat in reporting on the December 6th inspection, despite the Corps' assurances on November 23rd.

Perhaps after those embarassments, the Corps and the contractor eventually decided to do something about debris, though not for another couple of weeks after the incident with Mr. Temple.

First, though, there was the visit on December 13, 2010, documented in the December 14, 2010 WBV-72 report:

"Check-in: 9:33am, called Coe representatives, with no answer, left a message and continued with inspection. Coe representatives were then met out on jobsite at 12:08 pm. The issues of the previous reports were brought to their attention.
West end "Work in Progress": Trucks hauling in material and dozers working in material for levee lifts. Two men were on this end picking up debris but some was still found.
East end "Work in Progress": Very little debris was found in the material on this end. There was no active work going on at the time of this inspection."

They still only had two guys picking debris. So it is no surprise the SLFPA-W inspectors were able to get pictures like these:

There's 29 more pages of pictures like these.

The next week, the Corps and WRS finally seemed to have gotten the message. From the December 21, 2010 visit described in the December 28, 2010 WBV-72 report:

"Check-in: 8:48am, called Coe representative (David Trahan), met SLFPA-W onsite and was joined by second Coe representative and the contractor. Contractor expressed concern about correcting debris issue and proceeded explaining corrective messures:
1) A crew is assigned to the pit to remove debris from material before hauling to site.
2) A crew is designated to pick out debris as trucks dump material at site and dozer spreads.
3) As material is broken up and worked a crew will follow to remove any debris (everyone picked up debris yesterday till late in the afternoon).
4) Weekly a full labor crew will comb site for debris.
SLFPA-W representatives, COE representatives and WRS superindent continued on with site visit along with a crew to pick up any debris found.
Overall very little debris was found and debris found was immediately picked up, all parties assisted. Trucks were hauling in material at the west end of the project. SLFPA-W noted crew assigned to picking out debris from material being dumped and during dozer spreading operation. The main concern for this site visit was the debris issue and it appears the contractor is putting forth a plan to correct the debris issue. If this continues (according to debris found today) the debris issue should go away."

Of course, promises are easy to make. The contractor and the Corps made identical assurances back in October and November. Fortunately, at the SLFPA-W's next inspection on January 28, 2011, the stepped-up debris removal effort appeared to be working. From the January 31, 2011 WBV-72 report:

"Wet conditions has stopped hauling operations. Disking operation is ongoing trying to dry out material to allow hauling operations to resume. Contractor has two (2) crews removing debris on the west end, a total of ten (10) employees. Reviewed area that the crews are removing debris and found no significant amount or size of debris to note. Very little debris was found beyond were [sic] first crew was working (after the 3.5" of rain earlier in the week), contractor removing debris during visit. Noted to USACE and contractor that most of the debris was small, contractor stated that the employees were instructed to pick up all debris no matter what size. It appears that the contractor is still enforcing the corrective plan on the debris removal."

They had ten guys instead the meager two from before, which is interesting.

Here's further detail from the report:

"All parties agreed that the debris concerns continues to improve. During hauling operations the contractor still has a crew assigned to the pit to remove debris before loading and hauling. Once on site, during unloading and spreading operation, the contractor has a crew removing the debris. After the material has been spread it is disked and again the contractor removes any remaining debris. This operation is performed again after a rain."

There's been three more inspections since then. The SLFPA-W must feel everything is going well with this one, because they changed the frequency of inspection to monthly.

From the February 25, 2011 report on an inspection on February 22nd:

"Contractor and USCAE inspectors still enforcing debris plan set in motion. During hauling operations the contractor still has a crew assigned to the pit to remove debris before loading and hauling. During unloading and spreading operations the contractor has a crew removing the debris. After the material has been spread it is disked and again the contractor removes any remaining debris. Also, during and between the above sequences the USACE inspectors monitor the project for debris."

Yes, this is mostly copied and pasted from the January inspection report. That says the debris plan is continuing to work.

The April 4, 2011 report of the March 27, 2011 inspection said:

"Overall very little debris found. Contractor to set up labor force toward end of week (depending on weather) to concentrate on west end of project where degrading of existing levee is occurring and move eastward."

And the latest news, from the April 25, 2011 report about an inspection on April 21st:

"Overall very little debris found. Previous area of debris concern toward the west end at degrading of Dpeg levee has been cleaned of debris. The two large piles of grubbing and clearing debris at approximate station 149+50 is scheduled to be removed in May (weather will determine when in May)."

So things actually seem to be back to normal at WBV-72. However, we'll likely never know how much debris got put into the levee at times when the SLFPA-W wasn't looking over the Corps shoulder.

We'll take a look at more projects with reported debris problems in the next part.