Twice in the past four years, a top Justice Department lawyer warned the presiding judge of a secret surveillance court that information overheard in President Bush's eavesdropping program may have been improperly used to obtain wiretap warrants in the court, according to two sources with knowledge of those events.

The revelations infuriated U.S. District Judge Colleen Kollar-Kotelly -- who, like her predecessor, Royce C. Lamberth, had expressed serious doubts about whether the warrantless monitoring of phone calls and e-mails ordered by Bush was legal. Both judges had insisted that no information obtained this way be used to gain warrants from their court, according to government sources, and both had been assured by administration officials it would never happen.

[snip]

The two judges' discomfort with the NSA spying program was previously known. But this new account reveals the depth of their doubts about its legality and their behind-the-scenes efforts to protect the court from what they considered potentially tainted evidence. The new accounts also show the degree to which Baker, a top intelligence expert at Justice, shared their reservations and aided the judges.

Both judges expressed concern to senior officials that the president's program, if ever made public and challenged in court, ran a significant risk of being declared unconstitutional, according to sources familiar with their actions. Yet the judges believed they did not have the authority to rule on the president's power to order the eavesdropping, government sources said, and focused instead on protecting the integrity of the FISA process.

The legal controversy over the NSA surveillance program has obscured an intelligence issue that is at least as important to the nation’s future: sheer competence. Do we have any idea what we’re doing? One reason the NSA is listening in on so many domestic conversations fruitlessly—few of the thousands of tips panned out, according to The Washington Post—is that the agency barely has a clue as to who, or what, it is supposed to be monitoring.

While soaking up the lion’s share of the $40 billion annual intel budget, the NSA continues to preside over an antiquated cold-war apparatus, one designed to listen in on official communications pipelines in nation-states. Today it is overwhelmed by cell-phone and Internet traffic. While terror groups multiply, the NSA is still waiting for the next Soviet Union to arise (which many in the Pentagon see as China, say, 50 to 100 years from now). As a December 2002 report by the Senate Select Intelligence Committee noted, "Only a tiny fraction" of the NSA’s 650 million daily intercepts worldwide "are actually ever reviewed by humans, and much of what is collected gets lost in the deluge of data."

What’s needed is a fundamental rethinking that would put some of those billions of dollars that go into NSA’s global surveillance into more human intelligence and Internet surveillance instead. But that’s not happening. "There’s no question that technology changes have created a tidal-wave type of problem," says one former senior NSA official. "NSA’s been talking about it for 10 years at least. Will they ever get in front of it? No."

As our esteemed senators fret over whether the NSA has violated their outdated 1978 law, the Foreign Intelligence Surveillance Act, they are not paying enough attention to the competence issue. And no one seems to recall that the same Senate intelligence committee report from 2002 also criticized the "NSA's cautious approach to any collection of intelligence relating to activities in the United States," and its "failure to address modern communications technology aggressively." In recent years the agency tried to do so, but failed. To little notice, a giant $1 billion-plus program called Trailblazer that was to have brought the NSA up to date in data mining and pattern analysis—transforming the NSA's blizzard of signals intelligence into an easily searchable database—has turned into such a boondoggle that, one intelligence official says, "nothing can be salvaged out of it." "It’s a complete and abject failure," says Robert D. Steele, a CIA veteran who is familiar with the program.


What went wrong? The NSA, using traditional defense contractors like Science Applications International Corp. (SAIC), sought to do too much at once, applying a clunky top-down solution to what was a Silicon Valley problem, says Ed Giorgio, who was the chief codebreaker at NSA for 30 years. "The biggest problem with Trailblazer was there was a grand theory of unification that was going to solve the problem, as if the ‘central committee’ could really do what’s best done by a distributed network of people," he says. Adds Fred Cohen, a former computer scientist at Sandia Labs: "The scope and magnitude of this problem is enormous. What they have failed at historically and are failing to do today is to put out enough small money to enough different creative thinkers to explore a lot more possibilities."

By most accounts, no one at senior levels has a good idea of how to replace the failed Trailblazer. Now, time’s awasting. Former NSA senior director Philip Bobbitt, writing recently in The New York Times, provided a vivid example of the importance of data mining and pattern analysis. On Sept. 10, 2001, he wrote, the NSA intercepted two messages: ''The match begins tomorrow'' and ''Tomorrow is zero hour.'' They were picked up from random monitoring of pay phones in areas of Afghanistan where Al Qaeda was active. No one knew what to make of them, and in any case they were not translated or disseminated until Sept. 12. But "had we at the time cross-referenced credit card accounts, frequent-flyer programs and a cellphone number shared by those two men, data mining might easily have picked up on the 17 other men linked to them and flying on the same day at the same time on four flights," Bobbitt wrote. Today the NSA seems hardly more capable of piecing together the next "tomorrow is zero hour" intercept.

The US government is developing a massive computer system that can collect huge amounts of data and, by linking far-flung information from blogs and e-mail to government records and intelligence reports, search for patterns of terrorist activity.
The system - parts of which are operational, parts of which are still under development - is already credited with helping to foil some plots. It is the federal government's latest attempt to use broad data-collection and powerful analysis in the fight against terrorism. But by delving deeply into the digital minutiae of American life, the program is also raising concerns that the government is intruding too deeply into citizens' privacy.

The core of this effort is a little-known system called Analysis, Dissemination, Visualization, Insight, and Semantic Enhancement (ADVISE). Only a few public documents mention it. ADVISE is a research and development program within the Department of Homeland Security (DHS), part of its three-year-old "Threat and Vulnerability, Testing and Assessment" portfolio. The TVTA received nearly $50 million in federal funding this year.

DHS officials are circumspect when talking about ADVISE. "I've heard of it," says Peter Sand, director of privacy technology. "I don't know the actual status right now. But if it's a system that's been discussed, then it's something we're involved in at some level."

A major part of ADVISE involves data-mining - or "dataveillance," as some call it. It means sifting through data to look for patterns. If a supermarket finds that customers who buy cider also tend to buy fresh-baked bread, it might group the two together. To prevent fraud, credit-card issuers use data-mining to look for patterns of suspicious activity.

What sets ADVISE apart is its scope. It would collect a vast array of corporate and public online information - from financial records to CNN news stories - and cross-reference it against US intelligence and law-enforcement records. The system would then store it as "entities" - linked data about people, places, things, organizations, and events, according to a report summarizing a 2004 DHS conference in Alexandria, Va. The storage requirements alone are huge - enough to retain information about 1 quadrillion entities, the report estimated. If each entity were a penny, they would collectively form a cube a half-mile high - roughly double the height of the Empire State Building.

But ADVISE and related DHS technologies aim to do much more, according to Joseph Kielman, manager of the TVTA portfolio. The key is not merely to identify terrorists, or sift for key words, but to identify critical patterns in data that illumine their motives and intentions, he wrote in a presentation at a November conference in Richland, Wash.

For example: Is a burst of Internet traffic between a few people the plotting of terrorists, or just bloggers arguing? ADVISE algorithms would try to determine that before flagging the data pattern for a human analyst's review.

At least a few pieces of ADVISE are already operational. Consider Starlight, which along with other "visualization" software tools can give human analysts a graphical view of data. Viewing data in this way could reveal patterns not obvious in text or number form. Understanding the relationships among people, organizations, places, and things - using social-behavior analysis and other techniques - is essential to going beyond mere data-mining to comprehensive "knowledge discovery in databases," Dr. Kielman wrote in his November report. He declined to be interviewed for this article.

Starlight has already helped foil some terror plots, says Jim Thomas, one of its developers and director of the government's new National Visualization Analytics Center in Richland, Wash. He can't elaborate because the cases are classified, he adds. But "there's no question that the technology we've invented here at the lab has been used to protect our freedoms - and that's pretty cool."

As envisioned, ADVISE and its analytical tools would be used by other agencies to look for terrorists. "All federal, state, local and private-sector security entities will be able to share and collaborate in real time with distributed data warehouses that will provide full support for analysis and action" for the ADVISE system, says the 2004 workshop report.

[snip]

Yet the scope of ADVISE - its stage of development, cost, and most other details - is so obscure that critics say it poses a major privacy challenge.

"We just don't know enough about this technology, how it works, or what it is used for," says Marcia Hofmann of the Electronic Privacy Information Center in Washington. "It matters to a lot of people that these programs and software exist. We don't really know to what extent the government is mining personal data."

Even congressmen with direct oversight of DHS, who favor data mining, say they don't know enough about the program.

"I am not fully briefed on ADVISE," wrote Rep. Curt Weldon (R) of Pennsylvania, vice chairman of the House Homeland Security Committee, in an e-mail. "I'll get briefed this week."

Privacy concerns have torpedoed federal data-mining efforts in the past. In 2002, news reports revealed that the Defense Department was working on Total Information Awareness, a project aimed at collecting and sifting vast amounts of personal and government data for clues to terrorism. An uproar caused Congress to cancel the TIA program a year later.

Echoes of a past controversial plan

ADVISE "looks very much like TIA," Mr. Tien of the Electronic Frontier Foundation writes in an e-mail. "There's the same emphasis on broad collection and pattern analysis."

But Mr. Sand, the DHS official, emphasizes that privacy protection would be built-in. "Before a system leaves the department there's been a privacy review.... That's our focus."