Spy Agency Mined Vast Data Trove, Officials Report - New York Times

Here's some more on wiretapping issue. This NY Times article gives an overview of the depth of the program. The program is about what they call "data mining". In case you don't know, "data mining" is a common technique in IT organizations. Essentially data mining about processing one set of data to create a second set of data.

Spy Agency Mined Vast Data Trove, Officials Report (By ERIC LICHTBLAU and JAMES RISEN, NYTimes.com, Published: December 24, 2005)

This might help understand what this is

Officials in the government and the telecommunications industry who have knowledge of parts of the program say the N.S.A. has sought to analyze communications patterns to glean clues from details like who is calling whom, how long a phone call lasts and what time of day it is made, and the origins and destinations of phone calls and e-mail messages. Calls to and from Afghanistan, for instance, are known to have been of particular interest to the N.S.A. since the Sept. 11 attacks, the officials said.

This is what they call the "envelope" information. This would be like the post office writing down the addresses on the outside of envelopes they handle. By tracking just the sources and destinations of physical mail you learn a lot. And it's the telecommunications equivalent to this, such as the source and destination phone numbers of all phone calls, that the article talks about.

It's not just this envelope information, but they're looking for "patterns" of contacts. This kind of analysis is not about listening to the actual phone calls, but who is calling whom, and how often.

The article discusses how the government has made special arrangements with the telecommunication companies. The NSA has a secret "backdoor" letting them directly tap into the core of the telecommunication infrastructure.

This says that what's happened has had widespread agreement from a broad spectrum of government and business entities. This took a long time and had to involve a lot of people to implement.

The volume of information harvested from telecommunication data and voice networks, without court-approved warrants, is much larger than the White House has acknowledged, the officials said. It was collected by tapping directly into some of the American telecommunication system's main arteries, they said.

As part of the program approved by President Bush for domestic surveillance without warrants, the N.S.A. has gained the cooperation of American telecommunications companies to obtain backdoor access to streams of domestic and international communications, the officials said.

One final interesting piece is whether U.S. laws govern privacy of telephone calls that just pass through equipment which happens to be located in the U.S. It may seem counter-intuitive, but the way telecommunications is installed in the world it is sometimes cheaper or easier for transmissions from one country to another country to pass through equipment in the U.S.

The government's collection and analysis of phone and Internet traffic have raised questions among some law enforcement and judicial officials familiar with the program. One issue of concern to the Foreign Intelligence Surveillance Court, which has reviewed some separate warrant applications growing out of the N.S.A.'s surveillance program, is whether the court has legal authority over calls outside the United States that happen to pass through American-based telephonic "switches," according to officials familiar with the matter.

"There was a lot of discussion about the switches" in conversations with the court, a Justice Department official said, referring to the gateways through which much of the communications traffic flows. "You're talking about access to such a vast amount of communications, and the question was, How do you minimize something that's on a switch that's carrying such large volumes of traffic? The court was very, very concerned about that."

... The switches are some of the main arteries for moving voice and some Internet traffic into and out of the United States, and, with the globalization of the telecommunications industry in recent years, many international-to-international calls are also routed through such American switches.

One outside expert on communications privacy who previously worked at the N.S.A. said that to exploit its technological capabilities, the American government had in the last few years been quietly encouraging the telecommunications industry to increase the amount of international traffic that is routed through American-based switches.