Fix the pumps

Tuesday, July 26, 2011

Debris Part 8

Part 1
Part 2
Part 3
Part 4
Part 5
Part 6
Part 7
Part 9
Part 10
Part 11
Part 12
Part 13
Part 14

All of a sudden, the debris story grew legs over the last three days, with multiple reports in multiple outlets. In part it was due to concern over stuff like this:


We'll return to that a little later.

Media interest before SLFPA-W July board meeting

The spark for the sudden jump in interest was the Southeast Louisiana Flood Protection Authority - West's (SLFPA-W) July 25, 2011 regular board meeting, where the Corps was called to account for all the unresolved debris problems. The SLFPA-W board appears beyond fed up, as they released multiple documents to the media in an effort to overcome any Corps spin. By all accounts, they may have succeeded.

Before the meeting there were two reports, one from WWL-TV and one from Fox 8 (WVUE).

WWL's July 25th report included video from the SLFPAW of a large chunk of debris getting dug out of a stretch of extremely troubled project WBV-14c.2 (contractor: Phylway; borrow source: River Birch):


The video also includes a map of WBV-14c.2, seemingly taken from the SLFPA-W, showing the entire project is of concern:


The WWL report includes a number of quotes from a clearly fed up SLFPA-W President Susan Maclay, such as this one characterizing the debris:
"'... you have large pieces of bricks and concrete and rocks and steel rods and wood, almost tree trunks,' Maclay said. 'This is going to undermine the levee.'"

The WWL story also reveals (again through documents released by SLFPA-W) that some Corps Quality Assurance (QA) inspectors claimed they alerted Corps upper-ups about the problem:
"Even more disturbing, she said, is that the corps' quality assurance reports reveal that its own inspectors have also alerted corps management about the problem.

One entry says, '...again mentioned to CQC the importance of removing objectionable materials.'

'But management has not done anything about it,' Maclay said.

She said so far the corps' management has only responded with what she calls excuses, with the corps saying the amount of debris in the levees is within specifications.

'But if you read the specifications, clearly it's not,' Maclay said."

Based on the level of antipathy by the Corps toward the state inspectors we've seen recorded in the state inspector's reports, I think such excerpts from Corps QA folks need to be taken in perspective, but they are enlightening.

One other thing: while the WWL report indicates "CQC" in the quote above stands for "Corps Quality Control," that's wrong. It actually stands for "Contractor Quality Control." It is the contractor, not the Corps, which is responsible for quality control - like ensuring there's no debris in the material - on Corps jobs. Nevertheless, Corps upper management has long been aware of these problems.

Fox 8's first report aired a day earlier, on Sunday, July 24, 2011 (video can be found at link. I'm not embedding it because Fox 8's embedding doesn't seem to work right).

It highlights a July 15, 2011 letter from SLFPA-W President Maclay to Corps New Orleans District commander Ed Fleming (linked here), which says, among other things "the use of unsuitable material in the construction of this project has been pervasive and continues unabated." The letter concentrates only on three projects: WBV-14c.2, WBV-15a.2, and WBV-3b. The problems at WBV-14c.2 are well documented, as are those at WBV-15a.2, though notably the letter lists many other concerns at WBV-15a.2 besides debris.

WBV-3b seems to be a particular focus of the SLFPA-W. The inspection records available online do not give the full picture which undergirds that concern, but records recently released by the SLFPA-W do. How about this:



That is a huge log that was pulled out of the finished levee at WBV-3b on June 15th during part of the pre-final inspection of that project. But that's not the worst, because look what happens when the camera pulls back:


We can see giant chunks of concrete, what appear to be logs even bigger than the one in the detail above, and an assortment of junk that should make everyone at the Corps wonder what exactly their commanders were crowing about two weeks earlier on June 1st of this year, when they claimed the system was ready for a 100 year storm. And Maclay's July 15th letter indicates there was more:
"Because our inspectors reported finding large logs and other unsuitable material in this levee section [WBV-3b], the contractor [Shavers-Whittle] and USACE's construction division visted the site and removed utility trailers filled with unsuitable material."

That's "trailers," as in more than one. Yikes.

Interestingly, the Fox 8 report includes maps of the projects listed in the letter which appear to go beyond the projects highlighted. Specifically, this map:


The voiceover during the report says this shows the section of levee from the new Westwego pump station to Orleans Village. That description corresponds with the "official" description of only WBV-14c.2, which is located here, in the northern half of the highlighted section on the Fox 8 map:


The highlighted section on the Fox 8 map actually goes much further than the eastern end of WBV-14c.2, tracking all the way down to the end of another project with debris problems, WBV-14b.2:


I don't know what the source document for the Fox 8 graphic was. The July 15, 2011 letter from President Maclay does not explicitly reference WBV-14b.2, and the Fox 8 graphic misplaces the location of Orleans Village:


More than likely, this was simply a graphics error by Fox 8, even though we know WBV-14b.2 has also had debris problems.

Media reporting of the meeting

Anyhow, after the SLFPA-W board meeting, there was another flurry of stories. Again, WWL-TV and Fox 8 had reports (I'm only embedding WWL's video; Fox 8's can be found at the link because their embedding doesn't work too well):



The Times-Picayune also chimed in.

All three reports hit mostly the same notes. They all reported the same platitudes ("public safety is out number one priority" and such) and excuse-making by the Corps. From WWL's report:
"Monday, corps officials pointed out, the Westwego levee [specifically WBV-14c.2] is still under construction, and some areas analyzed could have been cleared of debris after inspectors left.

They said spot inspections often lack proper context."

This is pure BS. There are literally dozens of inspection reports, some on a weekly basis, with hundreds of photos taken along lengthy stretches of multiple projects showing debris all over the place. Indeed, we've seen the "we'll clean it up" excuse from the Corps' QA guys too many times to count. And the tone of such comments undermines the further BS about the Corps' work being a partnership with SLFPA-W and the state.

Perhaps it is then fitting that the New Orleans District is being shoved aside in part of their QA duties, as WWL goes on to report:
"[Fleming] said a new inspection team will be on the ground by next week, the number of inspections will be increasing and crews will trench the levees in some spots to determine how deep inside debris is located.

[...]

Corps officials said the new inspection team will come in from outside the New Orleans District."

Fox 8 says the outside QA team will be overseeing only the trenching:
"Also, a quality assurance team will arrive in town soon to oversee the trenching work."

And unfortunately, it doesn't appear that team will have much autonomy, as the T-P reports:
"Overseeing the efforts will be senior project manager Kevin Wagner."

Wagner is an old hand at the New Orleans District.

Of course, the trenching on the new levee is also a major development. We've already seen trenching occur on the existing levee at WBV-14c.2, when it was being used to build up the new levee:


It was found to contain a ton of debris, just like the new incoming dirt. The Corps tried to discount the findings of those excavations; fortunately the state did their own analysis which confirmed the excessive debris in the existing levee.

I personally think comments from SLFPA-W president Maclay in the first WWL story are the most on point:
"Maclay said it may well be necessary to tear down the levee and rebuild it from scratch."

Based on the reports, that sounds about right.

Labels:

Wednesday, July 20, 2011

No surprise, Part 2

Over the last two years, we've learned many, many of the Corps' 54 hydraulic pumps have been leaking oil into Lake Pontchartrain as they rust away. Sometimes the Corps reports these spills - as they must under federal law - and sometimes they don't. And sometimes pumps are scheduled to come out for rebuilding, while other times pumps are yanked as a result of spilling oil.

The places oil can leak out on these pumps are numerous. They include a slew of different pipes on each pump through which hydraulic oil (which powers the pumps) flows; most of that piping sits in the brackish water of the lake 24 hours a day. When the pumps aren't running, the oil sits in the piping on the pumps and in the hundreds of feet of pipes leading away and from the pumps, all the way back to the reservoirs on the hydraulic power units (HPU's) located on the land sides of the canals.

The ongoing corrosion of these completely or partially immersed pump-mounted pipes, which were carbon steel when originally installed, leads directly to leaks. While the Corps has had their contractor yank out 25 of the 40 major 60" pumps located across all three sites (three of them twice) and replaced that carbon steel piping with stainless steel, the remaining 15 60" pumps still have the rust-friendly carbon steel piping on them. As we've seen in earlier posts, the salt water does a number on this piping:


That photo is of an oil cooler on pump W5 or W6 from the 17th Street site. It was taken a mere two and half years after these pumps were accepted for service by the Corps. In actuality, the pumps started spending time in the lake waters in mid-2006, but they were taken out and put back in so many times over the following year while dealing with deep-seated design issues (some of which remain unresolved to this day) that the corrosion clock only started started running consistently in the latter half of 2007.

There are many, many more pictures of rusty pipes like this from the Conhagen repair reports:





With all this rust causing all these oil leaks, it makes sense to ask, "If the oil is leaking out of the pumps, doesn't that mean water is leaking into them?"

The answer is "Yes." When the pumps are not running, the oil is not pressurized by the HPU's. So oil can seep out of leak points and water can come in. If the Corps turns on the pumps with those leaks and starts circulating all this oil mixed with water through the entire hydraulic oil loop, the water gets into the delicate parts on the HPU and can do serious damage.

That's why it should be a standard part of the repair to completely flush out the entire hydraulic loop for any pump that get's pulled out for repairs. That would include all the connecting piping from the pump back to the onshore HPU, as well as the HPU itself. Natually, though, the Corps doesn't do this. It's too expensive (for them, not for normal engineering organizations). When the pumps were put in back in 2006, there was pickling fluid remaining in the pipes when they were intially filled with oil. The resulting jello-like muck took two flushings to remove, at an expense of hundreds of thousands of dollars paid for by the U.S. taxpayer, despite the problem totally coming out the vendor's poor performance. I imagine that expense is what keeps the Corps from cleaning out the connecting pipes every time they yank out a pump for rebuilding.

So that just adds one more question about the performance of these things if they're ever needed in a major storm (earlier storms when they were turned on were absolute pea shooters). And it should come as no surprise that problems have cropped up directly related to contaminated oil.

On December 9, 2010, the Corps issued task order #7 on the second Healtheon/Conhagen pump repair contract (there have been three such contracts - more evidence of the Corps inability to plan ahead). The task order called for the rebuilding of London Avenue pumps E2 and E6.

I detailed the facts surrounding this set of repairs in my earlier post, "The latest on lakefront pump repairs." While there were some bumps along the way during these repairs, including apparently two oil spills totalling 125 gallons, the repairs themselves seemed to go somewhat smoothly (except for a distinct lack of information on the Rineer hydraulic motors). As expected, the pumps were in terrible condition when pulled out. In fact, one of the sample pictures above of really bad rust was taken from the Conhagen repair report for these two pumps:


The pumps were put back in the canal January 24, 2011.

However, that wasn't the end of the story. Just about a week after the pumps returned to service, the Corps issued $22,188.68 modification 2 to task order #7, increasing the cost for the repair of these two pumps to $304,898.07. The scope for the modification, knowing what we know about the oil leaks on all these pumps, should come as no surprise:
"- Take oil sample out of reservoir have analysis performed.
- Drain and pump oil out of reservoir, hoses, piping, etc. and flush to remove all contaminated oil.
- Remove the hydraulic pump, motor, coolers, and piping and flush to remove all contaminated oil.
- Dispose of contaminated oil
- Furnish new replacement hydraulic oil
- Refill reservoir on test unit
- Provide standby crane for this additional work.

Completion Date:
Work shall be completed within one week from the modification effective date."

The verbiage of the task order, as well as its price, lend itself to a couple of interpretations:

1) The Corps discovered the oil in either the E2 or E6 hydraulic loops at the London Avenue site was contaminated and had Conhagen remedy the situation. The fact that this modification was issued a week after the pumps were acceptance tested argues for this, although admittedly the Corps has a long history of post-dating their contract actions.

2) Conhagen, while they were pressure testing the pumps at their facility in Kenner, LA with one of the Corps' spare hydraulic power units, encountered a problem with contaminated oil and remedied it. The mention of the reservoir on the "test unit" argues for this. We have a picture of what a typical oil pressure test at Conhagen's site looks like, from the repair report for 17th Street pumps W4 and W7:


Whether the contaminated oil was encountered at Conhagen's shop, or out at the site, the larger point about likely widespread oil contamination still stands. and adds just one more question mark above the potential future performance of these pumps.

Labels:

Friday, July 15, 2011

No surprise

This post is one in a series I've been running since May, 2010 about the corrosion-related repairs to the 54 hydraulic pumps at the Corps' three New Orleans outfall canal sites. I've only recently started to receive contract documents and reports on these repairs via the Frreedom of Information Act.

A post from earlier this week provides links to the earlier articles in the series. Since the documents released date back almost two years, many of those earlier articles have been heavily revised to reflect the newly gained knowledge from the documents.

But the released documents also included information on very recent events surrounding these pumps...


In the last post, I noted that two pumps at 17th Street (W8 and W9) had been pulled out in the last couple of months. Those two pumps were already pulled out in 2009 for repairs, yet here they are coming out again. Why is that happening?

Because the parts that weren't overhauled in the 2009 repairs - which make up most of the pumps - gave out.

Just as a reminder, the repairs to pumps in 2009 (covered in earlier posts here and here) only concerned themselves with replacing carbon steel piping on the inside of the pumps with stainless steel:



The piping on the outside of the pumps was replaced, but with the same carbon steel material that had rusted in the first place. Then it was repainted with the same paint as before as well:



Had there been new stainless steel piping on the outside of the pumps, it would appear the same as the silvery stainless steel piping shown in the picture of the inside of the pump above.

In addition, all the moving parts like the bearings and the Rineer motors were also put back.

So essentially, about a third of what needed to be fixed was fixed, and the pumps were put back in the brackish Lake Pontchartrain waters to start corroding again. Six 60" pumps - five at 17th Street and one at London Avenue - as well as an unknown number of 42" pumps at 17th Street received this suite of repairs. It is no surprise that these pumps would be subject to the same corrosion damage as any other, mostly-carbon steel piece of equipment sitting in salty water.

We saw the first evidence of this earlier this spring, when pump E1 at the London Avenue site was removed as part of a planned pump pull. I only recently figured out that this pump was repaired in 2009 in addition to the five 60" pumps at 17th Street.

The London E1 stainless steel interior pipes, installed in 2009 and seen in the photo above, did not appear severely rusted in spring, 2011...


...while the carbon steel external piping and coolers looked like they took some serious corrosion damage since June, 2009:


[More details on the 2011 repairs to this pump are in my earlier post, "The 2011 pump rebuild scramble."]

That this pump was taken out as part of a planned complete overhaul in the spring of 2011 is evidence the Corps recognized the 2009 repairs were inadequate. But with many other pumps completely untouched since 2007, they triaged and focused on the more damaged ones rather than the other five pumps repaired in 2009.

However, it was only a matter of time before the pumps partially fixed in 2009 also started breaking. That's what happened with 17th Street pumps W8 and W9 in the last two months.

The first failure appears to have occurred in May or June of this year. We learned this from $89,489.55 task order #8 on the second Healtheon contract, issued on June 7, 2011. Prior to this, the second Healtheon contract had lain fallow since the issuance of task order #7 and its immediate follow-up modification on December 9th and 10th, 2010. After that, the 60" hydraulic pump repair work had been moved over to the third Healtheon contract, which was inked in January, 2011. Issuance of a task order on a contract previously thought ended was a little odd, but not unprecedented. The first Healtheon contract has continued to get task orders issued on it as well.

Anyhow, the body of task order #8 reads,
"Remove pump # 8W at 17th St. Canal to inspect, recommend what need to be repaired and reinstall it.

Completion of this work will be six weeks after notice of award.

Note:
1. Pump # 8W was overhauled in Aug 2009 and is currently leaked [sic] outside of the pump.
2. After the contractor inspects and finds out what the pump problems are, we will provide Contracting Division a separate scope of work for repair."

So the Corps had discovered an oil leak on pump W8. Did they report it to the National Response Center, as required under federal law? Apparently not, because searches on the NRC website show no reports for such an incident prior to June 7, 2011. That's really no surprise - the Corps has little regard for the law on reporting oil spills.

Pump W8 was pulled out of the 17th Street site and sent to Conhagen's shop. A $30,178.32 modification to task order #8 was issued June 20, 2011:
"The following scope of work was found necessary upon removal and inspection of Pump #8W at the 17th St. ICS:
- Remove hydraulic motor and motor adapter. Replace motor with gov-furnished new one.
- Build up weld eroded and corroded areas on motor adapter and other pump surfaces.
- Replace oil coolers and connected piping. Spot prep minor corrosion on other exterior piping.
- Replace small bypass hose for motor case drain. Replace exterior pipe u-bolts.
- Provide a full coat of epoxy paint on pump assembly in addition to touch up of original scope.
- Have mechanical seal reconditioned at subcontractor’s shop (to be returned to gov). Install new gov-furnished seal."

I would have expected the scope to simply be a restatement of the normal scope of work, with perhaps the exception of replacement of the interior piping. Instead, we find this oddly worded scope, which is rather lacking in detail. Nevertheless, it appears to cover much of the work which would be expected.

Fortunately, Conhagen had a better grasp on what needed to be done. The Conhangen repair report for pump W8's second time in their shop gives a detailed scope which covers everything needed to bring this pump up to same status as other completely rebuilt units.

That report shows the areas which one would expect to be the most damaged - those not upgraded in the 2009 repairs - were indeed the most damaged. The carbon steel oil coolers installed in 2009 had unsurprisingly accumulated a great deal of marine growth and had rusted:


Admittedly, the damage wasn't as great as we have seen on other pumps, but it is evident. Here's a similar area on one of the pumps after it was repaired in 2009:


The Rineer motor and its mount were also rusty:


Here's the "before" picture of that area in 2009:


Conhagen fixed up the pump, leak tested it, and put it back in the water on June 6, 2011, according to the acceptance testing record included in Conhagen's report.

Wait, what?

The original task order for the removal of this pump was dated June 7, 2011, and the follow-up modification with the scope of repairs was dated June 20, 2011. So how did the pump go back in the water before there was any contract for repairs?

It appears this is once again the Corps' Contracting Division issuing after-the-fact task orders and modifications for work already completed. In fact, the cover page of the Conhagen repair report indicates their work on the pump actually started three weeks before they received their first contractual documents:


One has to wonder how exactly contracts are run at the New Orleans District, when we find stuff like this.

So apart from the contracting slight-of-hand, I think we can take the Conhagen photos of W8's repairs in May and June of 2011 as proof that it has been fully rebuilt, meaning the 17th Street pump status can be updated again. However, the fact that W8 after previously being partially repaired still sprung a leak means that the remaining 60" pumps fixed in 2009 are now not just questionable, but effectively unrepaired. So the revised 17th Street status is:


This now means there are sixteen 60" hydraulic pumps across all three sites which cannot be trusted to be free from corrosion-related failures in the future:

17th Street: E5, E7, W2, W3, W9, W10 (W2 and W3 never previously repaired)
Orleans Avenue: E1, E2, E3, E4, E5, W1, W2, W3, W5 (all never previously repaired)
London Avenue: W2 (not previously repaired)

Indeed, this assertion was confirmed just last week, as 17th Street pump W9, another of the pumps only partially repaired in 2009, also sprung a leak.

This time, there was an oil spill report to the National Response Center, on July 5, 2011:
"Official Material Name: HYDRAULIC OIL
Qty Released: 8 GALLON(S)
Qty in Water: 8 GALLON(S)
CALLER IS REPORTING A DISCHARGE OF HYDRAULIC OIL FROM A HYDRAULIC PUMP DUE TO UNKNOWN CAUSES.
REMEDIAL ACTIONS: CLEANING UP SPILL WITH ABSORBENT PADS WITHIN THE BOOM SYSTEM."

Two days later, on July 7, 2011, the Corps issued $89,489.55 task order #9 on the second Healtheon contract. That dollar amount is exactly the same as the initial task order (#8) on 17th Street pump W8 in June. This time, the task order reads:
"Remove pump # 9W at 17th St. Canal to inspect, recommend what need to be repaired and reinstall it.

Note:
1. Pump # 9W was overhauled in Aug 2009 and is currently leaked [sic] outside of the pump.
2. After the contractor inspects and finds out what the pump problems are, we will provide Contracting Division a separate scope of work for repair.
3. Estimated date to start is 7/8/ 2011 and overtime for the contractors to work on Saturday 7/9/11 and Sunday 7/10/11 should be authorized."

Like the dollar amount, this is a copy and paste from the initial task order for pump W8, down to the typo on the work "leaked."

Since my receipt of documents via FOIA only covered stuff through last week, I don't have the task order modification which covers the actual repairs of 17th Street pump W9, but I am certain it looks nearly identical to the corresponding modification for pump W8. Right now, pump W9 is sitting on Conhagen's shop floor, and it will probably go back in the water before July 31.

Update, 7/6/12

The modification that covered the additional scope of repairs on 17th Street W9 was issued July 19, 2011 for $29,803.42. Again, this appears to be an after-the-fact contract action, since documents attached to the Conhagen repair report indicate the pump went back in the water on July 15, 2011.

That report includes the usual sorts of pictures of excessive marine growth and corrosion on the pump:


Original post resumes

The pattern of pump repairs over the past two years has been for scheduled rebuild work to only be undertaken outside hurricane season, meaning from December 1 to May 31st. If individual pumps fail during or close to hurricane season, as 17th Street pumps W8 and W9 have in the past couple of months - or as three other pumps did during the 2010 hurricane season - they are yanked out and rebuilt. Everything else - including pumps that have been rusting for nearly five years now - is left in the canals. This is a remnant of a foolish Corps decision made after Katrina - and carried out for two years of engineering and construction in 2006 and 2007 - not to supply any spare pumping capacity at their closure structures.

At the pace demonstrated by the repair contractor over the past two years, it seems impossible all 16 remaining 60" pumps will be completely rebuilt by June 1, 2012. Only ten pumps each were rebuilt in the 2009-10 and 2010-11 hurricane off seasons. So it seems likely - barring further (admittedly very likely) oil spills from unrebuilt pumps - there will still be rusty pumps in the canals all the way through the 2012 hurricane season.

Labels:

Wednesday, July 13, 2011

Finally

On July 8, 2011, the Corps finally started turning over the records on the rebuilding of the hydraulic pumps at all three outfall canal sites. I had requested these documents through the Freedom of Information Act over a year before. In the interim, there were a number of additional repairs, and many documents associated with those actions were also given to me.

As a result, I am currently heavily editing my previous entries about the pump fixes, adding in pictures and descriptions of the repairs taken from Corps contract actions and reports written by the pump repair shop, Conhagen, Inc, located in Kenner, LA.

Previously, I had only received a single Conhagen report which covered the repair of two pumps. I now have an additional nine reports documenting the repairs of 15 other pumps.

Here's links to the reports:

17th Street pump W10 and bridge pumps #1 and #6 (May-July, 2009) - no Conhagen report generated because the Corps did the work themselves

17th Street pumps E5, E7, W8 and W9 (July-September, 2009) (report received previously)

17th Street pumps W5 and W6 (February-March, 2010)

17th Street pumps E1 and E2 (March-April, 2010)

London Avenue pumps E3 and E4 (March-April, 2010)

17th Street pumps E6 and E8 (April-June, 2010)

London Avenue pumps W3 and W4 (April-June, 2010)

17th Street pump W1 (June-July, 2010)

London Avenue pump W1 (July-August, 2010)

Orleans Avenue pump W4 (October-November, 2010)

London Avenue pumps E2 and E6 (December, 2010 - January, 2011)

London Avenue pumps E1 and E5 (March-April, 2011)

London Avenue pumps W5 and W6 (April-May, 2011)

17th Street pumps W4 and W7 (March-May, 2011)

17th Street pumps E3 and E4 (May-June, 2011)

17th Street pump W8 (May-June, 2011)

17th Street pump W9 (July, 2011 - ?) - pump currently under repair.

Sharp eyed readers will notice the last two pumps, W8 and W9 at 17th Street, had supposedly already been rebuilt in 2009. However, those rebuilds only replaced a few of the carbon steel pipes, not all of them. They also didn't touch the bearings, Rineer motors or any moving parts. Unsurprisingly, these pumps have failed again, this time from the unrepaired parts.

The entries I'm updating or have already updated include:

Imminent, originally posted May 13, 2010

This year's scramble, originally posted June 3, 2010

Worse than previously known, originally posted June 11, 2010

The latest on lakefront pump repairs, originally posted March 23, 2011

The 2011 pump rebuild scramble, originally posted May 9, 2011

Brief update on London Avenue pumps, originally posted June 24, 2011

Other posts might also get updated or revised. And there will be coming posts on the current repairs, or "re-repairs" as it were.

In the meantime, here's some photos from the newly received reports...

A hole rusted straight through one of the oil coolers on pump W5 or W6 at 17th Street (report here):


Rust inside the supposedly sealed bearing housing on pump W5 or W6 at 17th Street (report here):


Extremely severe rust on London Avenue pump E4 oil cooler and attached piping (report here):


Identical corrosion on 17th Street pump W1 (report here):


Chunks missing from inlet suction bell on Orleans Avenue pump W4, due to corrosion (report here):


Other chunks missing from inlet suction bell on 17th Street pump E6 (report here):



More chunks missing from vanes inside suction bell on 17th Street pump E2 (report here):



Rusty Rineer hydraulic motor and attached piping on London Avenue pump E2 (report here):


Similarly rusty Rineer motor and piping on Orleans Avenue pump W4 (report here):



There's much more where that came from. But the important thing to remember is this: there are currently at least 16 pumps in the water which have had inadequate or no repairs to address these problems, which were evident after just two years in the canals. That includes 9 of the 10 pumps at the Orleans Avenue gates. Those 16 pumps have been in the canals for over four years now, and the only reason the Corps pulls them out is if they spring an oil leak.

Labels:

Friday, July 01, 2011

Permanent pumps slip to Katrina 10th anniversary

[Updated July 12, 2011 - see bottom of post]

[Corps confirms 2015 permanent pump completion. See July 16, 2011 update at bottom of post.]

In my last post, I mentioned the outfall canal updates the Corps gives the Southeast Louisiana Flood Protection Authority - East. In the most recent update, from early June, there was also a slide about the permanent pump station project:


It indicates that the completion date for the permanent pumps will likely slip to 2015, due in part to the bid protests filed by losing bidders (but really due to the Corps' lassitude and intransigence in getting the project underway from 2006 through 2010). These stations were supposed to have been in service in 2010.

The permanent pump stations project was awarded April 13, 2011 to a coalition of companies calling themselves "CBY Builders." It includes CDM, Brasfield & Gorrie, and Yates Construction. It immediately garnered bid protests from two of the losing bidders. That's not a surprise, since the job value is $674,998,555.36. Notably, that's over $129 million less than the $804 million congressional appropriation for the job.

Who are the protestors? Who were the other bidders?

The Corps used a two stage bidding process. In the first stage, they accepted bids from anyone. They also made public the technical specification. Then, in the second and much more secret stage, they narrowed the list to the five top players (i.e. "the list that counts"). In this second stage the Corps didn't let anyone know who was bidding, they didn't release any specifications or drawings, and they didn't put out any press releases. It was remarkably hush-hush for a project worth over half a billion dollars.

We already know the winner. Here's the other four:

Bechtel team (protestor)

Bechtel partnered with New Orleans-area engineering firms Waldemar S. Nelson; Brown, Cunningham & Gannuch (aka "BCG"); N-Y Associates; and Eustis Engineering. During the bidding process, Bechtel had a website up at http://www.nops-pccp.com, but it went away after the contract was awarded.

PCCP Constructors (protestor)

PCCP Constructors is made up of Kiewit, Traylor Brothers, and the New Orleans-area based M.R. Pittman group. This is the same set of companies that make up the joint venture "Gulf Intracoastal Constructors," who has the prime contract for the West Closure Complex.

Louisiana Canal Constructors (no protest)

"Louisiana Canal Constructors" appears to be Alberici and Archer-Western (a subsidiary of Chicago-based Walsh Group), the same team that worked levee project LPV-111 along the Gulf Intracoastal Waterway in New Orleans East. Parsons was also involved. We know this because all three firms are listed on the Louisiana Secretary of State entry on LCC. Incidentally, that entry shows that LCC is actually not in good standing due to failure to file an annual report this year. That's not surprising, since their lack of bid protest activity says the LCC team has decided to call it a day on this project.

Weston group (no protest)

The Weston group also includes engineering heavyweight HNTB. Weston was so interested in this project last year that they actually lobbied congress on it. A lobbying disclosure form for Weston's chief lobbyist, Sean McGraw, notes the New Orleans Permanent Pump Stations as a target for their lobbying during debates over the FY2010 Energy & Water bill

Other companies floating around the bidding process included Odebrecht, Arcadis, Shaw (who had advertisements on job boards for work on this project two years ago) and Volkert, though I don't know who they were partnered with. All these names are rather heavy hitters in the engineering industry, which is no surprise considering the mammoth size of this prize.

Corps helps out finalists

Part of the reason we know who's who among the bidders is because of solicitations for small business subcontractors each team issued last fall. These sorts of subcontracting gatherings are common for bigger projects; they're useful for signing up small businesses to meet the Corps' small business goals. Sometimes the Corps puts them on, but in this case, each firm - each privately-owned firm - staged them. PCCP Constructors' event was on October 19th, 2010 at the University of New Orleans (UNO):


Bechtel's was on October 6th at the Best Western on Severn in Metairie:



Louisiana Canal Constructors' event was on September 23rd at Southern University:



Weston's team had an event on October 20th at UNO:



I can't find an announcement for an event hosted by the winning bidder, the CDM team.

Here's the twist: all these announcements for events put on by private companies were - and are - residing on the Corps' computers, specifically the subdirectory associated with the Corps' Small Business office. PCCP Constructors' is here, Bechtel's is here, LCC's is here, and Weston's is here.

At one point, the Corps provided links to the Weston and PCCP Constructors flyers on their Small Business Programs page:



[Note to Corps: I've already got Google screenshots showing each of these flyers on your servers, so don't bother taking them down.]

If you think having a government agency advertise for private companies looking to do business with that agency is unseemly, you're right. But it's par for the course for the New Orleans District, which enjoys cozy ties to many large firms (see the details on Kiewit's protest below). One of the flyers notes that two officials from the Small Business Administration (Jo Lawrence and actually-Corps-guy Randy Marchiafava) were confirmed attendees, and two Corps officials (Ned Foley and Michael H Bloom, both Corps small business guys) were invited, so the government was out there appearing at events held by private firms to help them improve their bids.

Details on the protests

It is noteworthy that there's not just one protest for each of the two protesting bidders. In fact, each of them have filed three protests since the contract award. According to the General Accountability Office's bid protest docket, Bechtel filed protests on April 26th, May 2nd, and June 6th. Kiewit's group filed protests on April 26th, June 6th, and June 24th.

I'm sure the April 26th protests by each firm were probably already prepared before the contract was even awarded. The June 6th protests appear to be protestors' responses to the Corps' initial report to the GAO, if the schedule above is to be believed.

[Update July 12, 2011

The July outfall canal update indicates the June protests by the bidders were "supplementals," whatever that means. See complete update at the bottom of this post.]

Back before all these protests started rolling in, the Corps' schedule (found on the last page of the April, 2011 Orleans Levee District status update) naively called for absolutely no impact by them, and a project completion date of October, 2014:


With Kiewit's protest filed on June 24th, the GAO's latest response date is now October 3, 2011 (the GAO has 100 days to decide bid protests). Note this is later than what early June schedule above shows, since the slide was produced before the latest protests were filed. Comparing the two schedules and the GAO protest dates, it is easy to conclude about six months of schedule slippage, pushing the project completion date to approximately March, 2015. And that assumes the project doesn't get rebid as a result of the protests, which is what one of the protestors is calling for.

[Update July 12, 2011

It is possible the protests are all part of one big file, with an August deadline still in effect. The Corps is playing the project delay card close to their vest. See the complete update at the bottom of this post.]

Kiewit goes public
Alleges Corps revolving door killed their bid

Kiewit went to the local media about their protest soon after filing in April. On May 4, 2011, WVUE-TV ran a story titled "Former employee accused of giving inside info to competing company."

The story reports:
"PCCP Constructors believes a senior level Corps employee provided inside information to the company that was ultimately chosen for the project, CBY Builders.

'It’s something that needs to be looked at and examined so that we know exactly what was said when and what kind of inside information may or may not have been disclosed that would benefit one party over another,' PCCP spokesman Kent Grisham explained. [note: Grisham actually works for Kiewit]

Grisham is talking about Richmond Kendrick. Kendrick was the chief of program execution for the Corps’ pump stations project. But in August 2010, Kendrick left the Corps and went to work for a company named CDM.

CDM is a partner of CBY Builders."

That's a pretty interesting charge. Of course, the revolving door between the Corps and contractors is well known within construction circles, but less so to the general public. Many of the most prominent, and some of the less prominent, figures involved in the Corps' post-Katrina work have left for contracting jobs which brought them right back into the Corps' offices working on that same post-Katrina work. In fact, here's a 2009 article in an industry trade organ featuring two of them: Dan Hitchings (former Task Force Hope leader and now working for Arcadis on Corps hurricane protection contracts) and Mike Rossi (who wasn't directly involved in the post-K work, but who now works for Vali Cooper International, the New Orleans-area company started by former Hurricane Protection Office leader Jeff Bedey and which claims to have Corps hurricane protection contracts, though they're more likely subcontracts). Ironically, the article also features an interview of Richmond Kendrick in his now-former capacity as deputy of program execution for the Corps’ Hurricane Protection Office.

Anyhow, it's unusual for the revolving door to get publicized in this fashion, but when there's hundreds of millions of dollars on the table...

The WVUE-TV report continues with an actual quote from one of Kiewit's protests. Given the May 4th date of the story, it could only be their first one, filed on April 26th:
"The protest filed by PCCP to the Government Accountability Office states, 'The most critical events in selecting the winning offeror for the PCCP (Permanent Canal Closure and Pumping) contract occurred after Mr. Kendrick took his inside knowledge concerning this procurement to CDM.'

Grisham says the company’s goal is to have the Army Corps re-open the bid process, now that Kendrick has left the government agency."

That would certainly be remarkable, though given the size of the contract, not unprecedented.

I'll stick close to this $674,998,555.36 story.

Update, July 12, 2011

The latest outfall canal remediation project update - issued in the first week of July - also contains a small update on the permanent pumps:


This update seemingly still calls for an August 4, 2011 response to all the protests. It calls the latest round of protests from June "supplemental protests." It also notably removes any reference to the start or completion date of the project.

I'm not sure what to make of the Corps insisting the GAO will come back August 4th with a decision. That date is based on the date of the original protests in April. The GAO's own docket indicates that each protest restarts the 100-day clock for them, so that while the earliest protest could be resolved in August, the later June ones might not get a decision until October.

Of course, we could just be seeing a limitation of the GAO's website, in that all three protests for each firm might indeed be part of a whole, but the website doesn't have a way to show that. Indeed, ther eis some support for that. The protest numbers given to PCCP Constructors' (nee Kiewit's) protest are B-405036.1, B-405036.4, and B-405036.6, while those assigned to Bechtel's protest are B-405036.2, B-405036.3, and B-405036.5. That would seem to indicate that they are simply part of an overall file numbered B-405036.

Two other shreds of information come out of this latest stab at a schedule by the Corps:

1) There was apparently some kind of hearing the last week of June in Washington.

2) That hearing was followed up by a briefing (presumably by GAO to the Corps) on July 7th, last Thursday.

Again, I'll try to stay on top of this.

Update, July 16, 2011

The Corps has confirmed the 2015 completion date for the permanent pumps.

In a July 12, 2011 presentation given to FEMA about the hurricane protection system by the Corps' Mississippi Valley Division (one step above the New Orleans District), the 2015 date was made explicit on this slide of completion dates for incomplete projects:


Incidentally, that is slide 1 of a pair of slides listing 30 incomplete 100 year protection projects:


The 16 projects with blue rows are those only at the 100 year level of protection if the Corps brings in Hesco baskets and sandbags immediately before a storm. The 13 projects with yellow rows actually have something built (like sheetpile or the like) to give interim 100 year protection. Remarkably, there's a still a project in St Charles Parish which hasn't even been awarded. So much for breaking their backs to meet the June 1st, 2011 deadline.

And finally, it turns out the Corps actually confirmed the 2015 date the same day this post first appeared. In a July 1, 2011 press release trumpeting their piecemeal, inadequate remediation work along the outfall canals, they included this:
"The 17th Street, Orleans Avenue and London Avenue outfall canals extend from interior pump stations to Lake Pontchartrain in metro New Orleans. Interim Closure Structures currently block 100-year storm surge from entering the canals; they will be replaced with permanent canal closures and pump stations that are scheduled to be complete in 2015. Because of the canal closures, the outfall canals’ primary purpose is to remove rainwater from the city. The floodwalls and levees lining the canals are no longer the primary storm surge defenses for New Orleans."

Thursday, June 30, 2011

Making it up as they go along

The Corps gives updates on the outfall canal remediation projects (those are the projects to raise the Safe Water Elevation, or SWE, along all three outfall canals to 8 feet) every month to the Southeast Louisiana Flood Protection Authority - East (SLFPA-E). SLFPA-E has been good enough to upload the slides from those presentations to their website. Here's links for the ones I've seen thus far:

late March, 2011 (at end of report)
late April, 2011
early June, 2011

The June update includes this information on the 17th Street project:
"Reach 16 Surveys being conducted to determine if improvements are needed"

and

"Reach 16 may need 6” to 18” of embankment at the toe of the levee, determination will be made after the survey is analyzed."

When I first read that, I thought it was good, because I had remembered there was a sub-8 foot SWE reach along the 17th Street canal that wasn't included in the original remediation plans. Then I went back and looked and realized it was actually Reach 30:


So where is Reach 16?

It's across the canal from Reach 30, down south of the I-10 bridges:


The Corps included pictures of it in the presentation:




Reach 16 was not included in the 17th Street canal remediation project when the contract was awarded in December, 2010. But now the Corps is saying extra dirt might be required at the toe of the levee (the part of the levee at its bottom where it goes from sloped to flat), and is even including pictures of the Reach 16 toe.

The changing Safe Water Elevation of Reach 16

So what's going on with Reach 16? The main chart of Safe Water Elevations on page 12 of the October, 2010 17th Street SWE report (report and all appendices here) shows the Safe Water Elevation for Reach 16 was the bare minimum of 8 feet, meaning no remediation was required:



So if the SWE was 8 feet, based on seepage considerations, what's the problem?

A clue comes from further inside the report. These types of reports are written from the inside out, with the individual results sections compiled first, and then the introductory and conclusory sections written around them. Those "outer" sections are the ones written last and which have the heaviest attention paid to them during editing, because they are the sections most likely to be read (who except crazies like me reads the whole report?).

So when one reads the introductory and conclusory sections, one finds this unambiguous statement reinforcing the main chart:
"Seepage was not found to be a concern on any reach of the 17th Street canal for an SWE below +8.0 feet."

But when we look at the individual section on the seepage results, we find this chart on page 124:



The actual calculations originally resulted in a seepage SWE of 7.5 feet, 6 inches below the 8 foot target. A SWE of 7.5 feet may have meant more remediation along the canal (theoretically, though we've seen the Corps ignore other sub-8-foot calculations). Reach 16 is 1615 feet long, so that's a lot of remediation.

So how did the SWE jump from an unacceptable value of 7.5 feet to the barely acceptable value of 8 feet? The report itself provides the rather bizarre answer, but first there's a brief detour to explain (in my rudimentary understanding) how seepage works.

The (very) basics of seepage

To help, here's a diagram of the layers of various soil along Reach 16:


This diagram is taken from the calculations for "slope stability," not seepage. The diagram for seepage doesn't include such clear delineations of the different soil types, so I grabbed this one.

As you can see, there's a thick layer of sand - shown in green - underlying the whole thing. I've noted that the bottom of the canal is at -9.4 feet, while the sand starts at -9 feet.

On the "protected" side - i.e. the non-canal side in Jefferson Parish - there's stronger, less permeable soil overlying this sand, both within the levee and next to it. That's the stuff shown in yellow and orange, labeled either "Marsh," "Embankment Fill," or simply "Fill." Below the sand is a layer of pretty impermeable clay, shown in red and brown. And at the top of the diagram is the I-wall, which penetrates down to -1.5 feet, or 7.5 feet short of the top of the sand layer and 44.5 feet short of the clay layer.

For seepage to occur, water from the canal must contact the sand surface directly. It must then travel through the sand to the protected side. For it to truly be a problem, though, it must exert an upward force against the protected-side soils which is greater than the force exerted downward by the weight of those soils. If it does this, then a connection like a pipe is made between the canal and the land side, and the foundation underneath the levee and floodwall can be quickly undermined.

In Reach 16's case, it would seem the first part of a potential seepage problem - a canal bottom lower than the top of the sand - is in place. The bottom of the canal is 0.4 feet into the sand layer. This condition is referred to by the Corps' consultants as an "open bottom," and Reach 16 is not the only reach along the 17th Street canal to exhibit it. Indeed, Black & Veatch analyzed five other reaches with survey data indicating an open bottom: 15 (later split into 15A and 15B), 31, 32, and 33. All these are reaches near the I-10 or Veterans bridges:


These calculations, while quite complicated in their guts, are simple in concept. The canal is modeled from known survey data. Water is placed in the canal up to the target SWE of +8 feet and to a maximum of +10 feet (the Corps limited SWE calculations to 10 feet of water, even though the walls are much higher). If the resultant calculated safety factor is above the prescribed safety factor for the particular type of failure under study (global stability, slope stability, gap, seepage, etc, each of which has its own safety factor), then the reach under consideration passes.

In the case of all those other "open bottom" reaches, Black & Veatch was able to load the canal up with water to an elevation of +10 feet and not fall below the Corps' seepage safety factor of 1.6.

But for Reach 16, Black & Veatch could only load the canal up with water to an elevation of +7.5 feet before they fell below the 1.6 safety factor. Just for reference's sake, the canal was able to be loaded up to +10 feet on Reach 16 for all the other types of failures Black & Veatch was looking at. So seepage was what is called the "controlling" criterion for Reach 16.

Sediment

Now, there's one other thing that can mitigate against seepage. That is stopping the canal water from getting to the sand in the first place. All three outfall canals have sediment along their bottoms, carried there from the city's streets and drainage tunnels and pipes. It is deposited there constantly, and especially when there's a rainstorm and water is pumped along the canals out to the lake.

I wrote about this sediment in 2009, when I based an entry on the 2007 iteration of the 17th Street SWE report. That report indicated sediment was quite thick along the entire canal. However, those thickness readings were based on soundings, not direct borings into the canal bottom. In addition, the 2007 report did not characterize the sediment at all, including its permeability or how much of it was mixed in with sand. The October, 2010 report notes (and this is quite important) that the Corps has still not measured the sediment, referred to as "fine grained materials:"
"[T]here are no borings in the canal to confirm the thickness of the fine grain materials over the beach deposits."

This is a yawning gap in the data required to completely analyze seepage.

As noted above, a number of reaches were analyzed as "open bottom" reaches. With regard to sediment, that likely means that Black & Veatch and the Corps made a judgement that the sediment blanket along those stretches was too thin to stop the water flowing through the direct connection exposed by the top of the sand layer being above the bottom of the canal. In addition, Black & Veatch also noted six other reaches - 14, 17, 18, 29, 30, and 34 - which had thin sediment layers, but whose canal bottoms were apparently not below the top of the sand layer. These weren't analyzed as open bottom reaches.

So they were making judgements about the thickness of the sediment layer throughout the preparation of the report. But how did they do so when they had no detailed canal borings to confirm those thicknesses? Where was the hard data?

Detailed borings of the sediment along the London Avenue canal bottom in 2010 led to a radical recasting of SWE calculations there, essentially restarting the engineering effort and eventually leading to the inclusion of many reaches in the remediation project along that canal which had previously been thought safe. Yet, no similar effort was undertaken along the 17th Street canal (or the Orleans Avenue canal for that matter).

So with no detailed information on the sediment overlying the beach sand at the bottom of the canal along Reach 16, and with a calculation showing a sub-8-foot SWE along that reach from seepage due to an open bottom, how then did the SWE jump 6 inches to 8 feet? What was that judgement based upon?

1983

This is where it gets bizarre.

Instead of either:

a) using the lack of detailed sediment data to take the conservative course - assuming an open bottom along Reach 16 and sticking with with the 7.5 foot SWE within the seepage section of the report, or

b) revealing the source of their heretofore undisclosed sediment thickness data,

Black & Veatch and the Corps chose a third course. They startlingly reached back almost 30 years to rebut their own seepage calculation. The report starts describing a December, 1983 test along the 17th Street canal's Reach 16 to determine whether a direct sand connection existed at the canal bottom. This test took place ahead of 1984-85 dredging of the canal by the Sewerage & Water Board, before the later walls were designed and built. The test was used to help get a permit from the Corps for that dredging. Presumably, reassuring the Corps that the canal would not be undermined by seepage during the dredging would go a long way toward getting the dredging permit. Since the dredging happened, it clearly did the trick.

In fact, seven years later, as part of the now-discredited 1990 17th Street canal design document which led to the placement of the I-walls destined to fail in 2005, the Corps themselves used the report of this test in the same way: to rebut concerns about seepage. Thus, it is beyond surprising to see it resurrected here to do the same thing a third time - rebut concerns about seepage - as if nothing had happened in the intervening decades.

Nonetheless, two whole pages of the 2010 report are devoted to blithely describing the 1983 test, recommended by Eustis Engineering to the then-consulting engineers on the dredging, Modjeski & Masters (Eustis still does millions of dollars of geotechnical work for the New Orleans District). The complete report on the 1983 test can be found within volume 2 of the 1990 Design Memorandum 20 (or DM-20), the base design document for the 17th Street canal walls. Scroll to page 78.

In 1983, at Eustis' recommendation, a 100' by 100' square area in the middle of the canal was excavated to expose the beach sand. Also, at Eustis' recommendation, piezometers were installed on the land side of the Reach 16 levee, just south of I-10. These piezometers sensed the water elevation in the soil below. If there were a seepage connection between the water in the canal and the piezometers outside the canal, changes in the canal water elevation would be reflected in the piezometer readings. Similar arrangements were a part of the 2007 London Avenue canal load test.

According to the 2010 report, during the 1983 test the water in the canal went up 0.41 feet while the sand was exposed on December 16, 1983, but the piezometers outside the canal either went down or remained unchanged. The 1983 test authors also noted that the test location was partially covered in sediment on the 16th, and that the entire location was covered in sediment three days later. To Eustis, this demonstrated that there was no seepage connection, either from the sediment cover or because there just wasn't one. Frankly, their reasoning is quite muddled.

However, Black & Veatch accepted the "no seepage" finding of the 1983 test, while including a token suggestion that before the SWE for Reach 16 is finalized, "a sampling program be performed to determine whether a [sediment] blanket exists at this location." But nowhere in the 2010 report does it indicate such a sampling program took place before the SWE for Reach 16 was raised to 8 feet in that same report. Indeed, there was no such sampling program during the 1983 test either. It appears that the 8 foot number came about directly - and only - as a result of Black & Veatch and the Corps accepting the findings of the 1983 test as gospel. It's like they just willed it into being. This is supported by the footnote on the master SWE chart on page 12 of the report:


...as well as the plain text of the 8 foot SWE in the main chart itself. This was stunning for me to read - a consultant in 2010, using the Corps' incomplete, muddled pre-Katrina design data to support a conclusion not supported by the post-Katrina data record, a data record also lacking key pieces of information. It's almost like they were looking for any shred of evidence that might support their pre-made conclusion that seepage could not possibly be a problem, their own calculations be damned.

I was not the only one to notice this.

Behind the scenes

On a consultant-written report such as this one, after the report is drafted it is sent to lots of people outside and inside the Corps for comments. They use a system called "Dr Checks" to log those comments, the responses, and the responses to the responses. Thus, the Dr Checks printout is the sort of document the public almost never sees - the official record of the internal engineering debates that go into the formation of these reports.

When I got the October, 2010 17th Street and Orleans Avenue reports, I was stunned to find the Dr Checks printouts were included as appendices (the London Avenue report didn't have one). The Orleans Avenue one runs 159 pages (complete Orleans Avenue report and appendices here). The 17th Street one is significantly shorter - just 25 pages long.

That doesn't mean the 17th Street Dr Checks printout is any less valuable. In fact it contains a brief but vigorous debate over seepage along Reach 16.

Keep in mind the Dr Checks comments are based upon draft versions of the report which we don't have. Thus, sometimes the comments mention a criticism which - when one reads the final report - it is clear eventually got addressed. Nonetheless, this particular chain of comments is extremely enlightening.

The comment chain starts with Noah Vroman, a Corps seepage expert working at the Corps' Engineer Research & Development Center (ERDC) in Vicksburg, Mississippi. Vroman's name has popped up often in the past few years in the many reports generated on seepage concerns along the London Avenue canal.

On July 23, 2010, Vroman says of Reach 16,
"I calculated a FOS [Factor of Safety] of 1.2 at a WL [Water Level] of 10.0"

He is noting this because the acceptable seepage safety factor is 1.6.

Black & Veatch geotechnical engineer John Koontz is the first to reply to Vroman, on August 7th:
"This reach was reviewed. A full scale canal seepage test was performed by excavating the beach deposits. No response was observed at the toe. The report is in DM-20 Vol. II. Discussion of this test has been added to the report."

As noted earlier, the "toe" is the section of the levee at the bottom, where it goes from sloped to flat. It is generally the most vulnerable section, because there's less soil there than at the top.

Also as noted above, "DM-20" is the 17th Street Design Memorandum, or the official design document underpinning the floodwalls and levees which failed in 2005.

As to the substance of Koontz's comment, we can surmise that the description of the 1983 test had not been included in the draft version Vroman was reviewing in July.

Vroman responded on September 23rd with one of the longest Dr Checks comments I've seen in these documents. From the date and the context of the comment, I surmise this September 23rd comment is based upon a subsequent draft, which we don't have. Note that I've added paragraph breaks to Vroman's comment to make it more readable:
"Do not agree with response.

I believe key elements of the results of the test section are not adequately addressed in the revised SWE report. The test section that is reference[d] was conducted in 1983 due [to] concerns with dredging operations in the canal that could potentially expose beach deposits in the canal. Eustis Engineering Company provided expertise and guidance for the test. They concluded from the test that "the possibility of blow out during high-water conditions in the canal is probably slight" and additional monitoring during dredging operations should be conducted. It is obvious that Eustis Engineering was concerned after the test that a "blow out" condition could still occur."

Let me pause here and point out that additional monitoring was conducted during that dredging. Piezometers were installed outside the canal as the dredging advanced along the canal between December, 1984 and April, 1985. They didn't really show any evidence of seepage. However, there was no characterization of the canal bottom as part of that monitoring.

Returning to Vroman's comment:
"The key elements that should be included in the SWE report concerning seepage and the canal test are:

- This was not a full canal test but merely monitoring an area where the possibly [sic] of exposing the beach sand was likely.

- after excavation of the test section, a boring was taken just three days later and there was 2.7 ft to 4 ft of sedimentation above the sand. The authors of the test section report indicated that this sediment was the likely reason the piezometers did not respond to changes in canal water elevations.

In addition, the authors of the test section report also indicated that the beach sand at the bottom of the canal was possibly contaminated or mixed with fines and "sealed" the bottom of the canal."

Based on the final text of the 2010 report, it appears this information was not in the September draft, but was included in the final report as a result of this comment.

Vroman continues:
"Thus it is important for the SWE evaluation analysis to consider whether this sedimentation layer exists within this reach. The same diligence used on the London Avenue SWE report for seepage concerns and sedimentation in the canal should be used in this reach.

Regarding this SWE report and the condition of the canal in 2010 and beyond, does a reliable sedimentation layer exist above the beach sand that will prevent adverse seepage conditions?"

And there it is - the critical question. Essentially, "what hard data do we have on the sediment?" And he's not asking for what was known in 1983 about a 100' by 100' section, but what is known along the entire canal - today. As was demonstrated on the London Avenue SWE effort, complete characterization of the sediment is unbelievably critical to understanding seepage. Since there has not been any comprehensive effort to do so along the 17th Street canal, it is only conservative to stick with the results from the "open bottom" calculations, which gave the 7.5 foot SWE for Reach 16. And, more importantly, the lack of that current, comprehensive sediment data brings into question how correct the seepage calculations are for the entire canal.

Vroman didn't just bring this up on the 17th Street report. We find him asking the same questions about sediment (also called "silt") on the Orleans Avenue report, because - like along the 17th Street canal - there was no comprehensive characterization of canal sediment at Orleans either. From a comment on the Orleans Avenue Dr Checks printout marked "this item flagged as a critical issue:"
"Different approach was used in Orleans canal as opposed to London.

For the London analysis, silt layer in the canal was critical feature for seepage and was deemed nonexistent if silt is less than 2 feet thick. For Orleans no canal side investigation was performed to determine the thickness of any impervious layers above the sand. This aspect is very critical to the analyses.

For Reach 1, there is only about 1.6 feet of marsh shown above the sand. This assumption was made purely on survey data and assuming the [base of the] marsh is constant between borings 200 feet apart (east bank and west bank). I think using this approach would warrant more conservatism than used for London. Anything less than 3 feet thick should be assumed to be nonexistent and direct connection to the bottom of the canal could be possible."


Vroman's comments on the Orleans Avenue report appear to have won the day, as his concerns about seepage led to the seepage calculations getting redone to include consideration of an "open bottom" on Reach 1 and other places. And on Reach 1, seepage was found to be to be the controlling criterion. In fact, sheet pile is being driven there to completely cut off the seepage path as part of the Orleans Avenue remediation project, as shown in this photo from the June update:


Of course, without a comprehensive characterization of the sediment along the Orleans Avenue canal, doubt remains about many of the other seepage calculations.

The outcome on the 17th Street canal report was quite different. The Black & Veatch SWE project manager, Larry Almaleh, responded to Vroman's 17th Street comment on October 1, 2010, just before - or perhaps the day of - the report's final issuance:
"The SWE for a FOS of 1.6 for an open bottom condition at this location as identified in the report is elevation +7.5 [feet]. Based on the identification of a significant sedimentation at the base of the canal and the lack of response in the piezometers raising of the SWE to +8 is not unreasonable. Note in multiple meetings and corresondence B&V has been told there is no seepage concerns in the 17th street canal by the HPO and the external review group."

Nowhere in the report is a precise identification of "significant sedimentation" outside of the 1983 test actually named or described.

But most telling is that the Corps ("HPO" is the Corps' Hurricane Protection Office, co-located with the New Orleans District) had apparently told Black & Veatch the result of the seepage calculations before they were done: "B&V has been told there is no seepage concerns in the 17th street canal by the HPO..."

I thought Black & Veatch was supposed to be reporting to the Corps on seepage concerns, not the other way around?

Handwaiving

So stepping back from all this, it looks to me like there was a ton of handwaiving going on around the seepage concerns at the 17th Street canal's Reach 16 specifically, and along the entire 17th Street canal generally, possibly on orders from the Corps Hurricane Protection Office in New Orleans. And while Vroman's comments apparently got an expanded description of the 1983 test added to the final report, it didn't change the underlying fact that Black & Veatch and the Corps were solely relying on a nearly 30-year-old test from a discredited document to support their pre-reached conclusion that seepage was nothing to worry about along the 17th Street canal.

So not surprisingly, despite calculations and the Corps' own non-New Orleans-based experts pointing to the absolute need to fully characterize the depth and composition of the sediment along the canal bottom, not to mention possibly remediate the 1615 feet of Reach 16 because of seepage, Black & Veatch issued their report in October, 2010 with the conclusive statement,
"Seepage was not found to be a concern on any reach of the 17th Street canal for an SWE below +8.0 feet."

That conclusion then flowed directly into the scope of the remediation project along the 17th Street canal. Until this most recent project update nine months after the SWE report was issued and seven months after the remediation contract was signed, there has been no effort to address seepage along the canal, including at Reach 16. All the remediation efforts are geared to other concerns like overall stability.

The way the Corps and Black & Veatch got to that outcome - even if they are possibly correcting it now - should make people very queasy. How could anyone in good conscience use nearly 30-year-old incomplete 17th Street design data and ignore post-Katrina data collection and calculations to support conclusions about the canal's safety in the present day? After all that has transpired, and after all the Corps' pronouncements that they've changed, it looks like more of the same over five years later.

Reach 16 to be fixed anyway?

That brings us full circle to the June remediation project update, which mentions that the Corps is now looking at putting 6 to 18 inches of dirt at the toe of the levee along Reach 16, based on surveys of the reach. That toe is undoubtedly the point found vulnerable in the Black & Veatch seepage calculation that found the 7.5 foot SWE. Thus, if more soil is piled over it, presumably water carried through the sand by seepage would not be able to push up and cause a failure.

Of course, the only definitive solution to stopping seepage is not piling more dirt on the landside, but driving of 60 foot steel sheet piling into the clay layer along the entire reach. In fact, that has been how seepage has been controlled everywhere else, including on both the Orleans Avenue and London Avenue remediation projects. I assume the Corps thinks that's too expensive or time-consuming or embarassing to do so along the 17th Street canal, so they are proposing a band-aid of a slightly thicker levee toe. It's a half-measure that doesn't fully address the problem.

The June update does not make clear whether the survey is of the canal bottom or the levee. We can hope it is of the canal bottom. Perhaps this is the "sampling program" given a token mention in the 2010 report? Indeed, we can hope there is a comprehensive effort underway to characterize the sedimentation along the canal bottom, and that effort will lead to a recasting of the engineering similar to what happened along the London Avenue canal last year. Since this update is about a month old, we should know whether those hopes are just that within a couple of weeks.

Go to older posts Go to newer posts