Re: Software Being Developed to Monitor Opinions of U.S.

Four and a half years ago I wrote about: DARPA's Information Awareness Office, The Total Information Awareness System; Or, Big Brother in-carnate. The Total Information Awareness (TIA) project had just been revealed, and this was before the harumphing that later drove the program underground. The goal of the TIA was to bring together a variety of technologies such as datamining, language translation, sparse data analysis, and use those technologies to drastically increase the ability of U.S. Government spooks to know what's going on. In the wake of the September 11, 2001 attacks "they" claimed to want better ability to predict and prevent the next terrorist attack. But at the same time the whole program just seemed like the reincarnation of Big Brother.

In Software Being Developed to Monitor Opinions of U.S. the NY Times is publicising a research program that smacks of the TIA. If you follow the above link you'll see the list of programs that was publicised on the TIA web site in early 2002. This NY Times article discusses a research effort to track news articles, these are said to be foreign news articles, and automatically detect statements which might help the spooks figure out terror threats. Looking at the list of TIA projects you can easily identify several that apply to this project.

Translingual Information Detection, Extraction and Summarization (TIDES): covers automatic translation of crucial information from foreign languages to English.

Wargaming the Asymmetric Environment (WAE): Involves detecting patterns indicating terroristical actions.

Genisys: Covers a different effort to track terroristical actions, involving identification of people and groups.

There's several considerations at play with this. First it is clearly a priority of government officials to protect their citizens. That means the U.S. Intelligence agencies clearly have a role to detect and prevent threats to the U.S. Clearly there are groups who would use terror or geurilla tactics to attack the U.S. Clearly modern technology gives the spooks a lot of capabilities along the lines of tracking information and making sense of widely scattered information.

An example of that technology when placed in the hands of average people is the cooperative research website. That website's primary purpose is to construct detailed timelines and connections between people and the actions they're taking in the world. It's well understood that a lot of data is published in the "public record" such as newspapers, but the snippets of information are widely spread from one story to the next. Only by drawing together dozens of threads spread over many news reports can you begin to get the whole picture.

That's what the U.S. Government research described in the NY Times article is meaning to do. But presumably the goal is to do this at a larger scale, and use automatic language translation and natural language processing to more rapidly tie together the pieces.

Where one begins to call this Big Brother is the fact that the technology doesn't know anything about morals. You can just as easily program those computers to track U.S. news media as it can foreign news media. And it is against U.S. law for the U.S. government agencies to spy on U.S. citizens or organizations.

And of course the software doesn't have to stop with news stories printed in the media. The software can just as easily be spidering the whole of the Internet just as Google and the other search engines are doing so. If the search engines can let us "search the Internet" then why couldn't the government set up their own server farm(s) to scoope up the Internet and do a similar analysis of the content?

And of course the software doesn't have to stop with national borders. The software can just as easily scoop up the whole of the Internet as it can to limit itself to foreign web sites.

Therefore when we have a U.S. Government who has been brazenly violating U.S. laws against spying on U.S. citizens, can we trust their assurances that this program will only be trained on foreign news sources?