The U.S. government had a problem: Spying in the digital age required access to the fiber-optic cables traversing the world's oceans, carrying torrents of data at the speed of light. And one of the biggest operators of those cables was being sold to an Asian firm, which might complicate American surveillance efforts.
Enter "Team Telecom."
In months of private talks, the team of lawyers from the FBI and the departments of Defense, Justice and Homeland Security demanded that the company maintain what amounted to an internal corporate cell of American citizens with government clearances. Among their jobs, documents show, was ensuring that surveillance requests got fulfilled quickly and confidentially.
This "Network Security Agreement," signed in September 2003 by Global Crossing, became a model for other deals over the past decade as foreign investors increasingly acquired pieces of the world's telecommunications infrastructure.
The publicly available agreements offer a window into efforts by U.S. officials to safeguard their ability to conduct surveillance through the fiber-optic networks that carry a huge majority of the world's voice and Internet traffic.
The agreements, whose main purpose is to secure the U.S. telecommunications networks against foreign spying and other actions that could harm national security, do not authorize surveillance. But they ensure that when U.S. government agencies seek access to the massive amounts of data flowing through their networks, the companies have systems in place to provide it securely, say people familiar with the deals.
Negotiating leverage has come from a seemingly mundane government power: the authority of the Federal Communications Commission to approve cable licenses. In deals involving a foreign company, say people familiar with the process, the FCC has held up approval for many months while the squadron of lawyers dubbed Team Telecom developed security agreements that went beyond what's required by the laws governing electronic eavesdropping.
The security agreement for Global Crossing, whose fiber-optic network connected 27 nations and four continents, required the company to have a "Network Operations Center" on U.S. soil that could be visited by government officials with 30 minutes of warning. Surveillance requests, meanwhile, had to be handled by U.S. citizens screened by the government and sworn to secrecy — in many cases prohibiting information from being shared even with the company's executives and directors.
"Our telecommunications companies have no real independence in standing up to the requests of government or in revealing data," said Susan Crawford, a Yeshiva University law professor and former Obama White House official. "This is yet another example where that's the case."
The full extent of the National Security Agency's access to fiber-optic cables remains classified. The Office of the Director of National Intelligence issued a statement saying that legally authorized data collection "has been one of our most important tools for the protection of the nation's — and our allies' — security. Our use of these authorities has been properly classified to maximize the potential for effective collection against foreign terrorists and other adversaries."
It added, "As always, the Intelligence and law enforcement communities will continue to work with all members of Congress to ensure the proper balance of privacy and protection for American citizens."
Documents obtained by The Washington Post and Britain's Guardian newspaper in recent weeks make clear how the revolution in information technology sparked a revolution in surveillance, allowing the U.S. government and its allies to monitor potential threats with a reach impossible only a few years earlier.
Yet any access to fiber-optic cables allows for possible privacy intrusions into Americans' personal communications, civil libertarians say.
As people worldwide chat, browse and post images through online services, much of the information flows within the technological reach of U.S. surveillance. Though laws, procedural rules and internal policies limit how that information can be collected and used, the data from billions of devices worldwide flow through Internet choke points that the United States and its allies are capable of monitoring.
This broad-based surveillance of fiber-optic networks runs parallel to the NSA's PRISM program, which allows analysts access to data from nine major Internet companies, including Google, Facebook, Microsoft, Yahoo, AOL and Apple, according to classified NSA PowerPoint slides. (The companies have said the collection is legal and limited.)
One NSA slide titled, "Two Types of Collection," shows both PRISM and a separate effort labeled "Upstream" and lists four code names: Fairview, Stormbrew, Blarney and Oakstar. A diagram superimposed on a crude map of undersea cable networks describes the Upstream program as collecting "communications on fiber cables and infrastructure as data flows past."
The slide has yellow arrows pointing to both Upstream and PRISM and says, "You Should Use Both." It also has a header saying "FAA 702 Operations," a reference to a section of the amended Foreign Intelligence Surveillance Act that governs surveillance efforts of foreign targets related to suspected terrorism and other foreign intelligence.
Under that provision, the government may serve a court order on a company compelling it to reach into its networks for data on multiple targets who are foreigners reasonably believed to be overseas. At an Internet gateway, the government may specify a number of e-mail addresses of foreigners to be targeted without the court signing off on each one.
When the NSA is collecting the communications of a foreign, overseas target who is speaking or e-mailing with an American, that American's e-mail or phone call is considered to be "incidentally" collected. It is considered "inadvertently" collected if the target actually turns out to be an American, according to program rules and people familiar with them. The extent of incidental and inadvertent collection has not been disclosed, leading some lawmakers to demand disclosure of estimates of how many Americans' communications have been gathered. No senior intelligence officials have answered that question publicly.
Using software that scans traffic and "sniffs out" the targeted e-mail address, the company can pull out e-mail traffic automatically to turn over to the government, according to several former government officials and industry experts.
It is unclear how effective that approach is compared with collecting from a "downstream" tech company such as Google or Facebook, but the existence of separate programs collecting data from both technology companies and telecommunications systems underscores the reach of government intelligence agencies.
"People need to realize that there are many ways for the government to get vast amounts of e-mail," said Chris Soghoian, a technology expert with the American Civil Liberties Union.
The drive for new intelligence sources after the Sept. 11, 2001, attacks relied on a key insight: American companies controlled most of the Internet's essential pipes, giving ample opportunities to tap the torrents of data flowing by. Even terrorists bent on destruction of the United States, it turned out, talked to each other on Web-based programs such as Microsoft's Hotmail.
Yet even data not handled by U.S.-based companies generally flowed across parts of the American telecommunications infrastructure. Most important were the fiber-optic cables that largely have replaced the copper telephone wires and the satellite and microwave transmissions that, in an earlier era, were the most important targets for government surveillance.
Fiber-optic cables, many of which lie along the ocean floor, provide higher-quality transmission and greater capacity than earlier technology, with the latest able to carry thousands of gigabits per second.
The world's hundreds of undersea cables now carry 99 percent of all intercontinental data, a category that includes most international phone calls as well, says TeleGeography, a global research firm.
The fiber-optic networks have become a rich source of data for intelligence agencies.The Guardian newspaper reported last month that the Government Communications Headquarters, the British equivalent of the NSA, taps and stores data flowing through the fiber-optic cables touching that nation, a major transit point for data between Europe and the Americas. That program, code-named Tempora, shares data with the NSA, the newspaper said.
Tapping undersea transmission cables had been a key U.S. surveillance tactic for decades, dating back to the era when copper lines carrying sensitive telephone communications could be accessed by listening devices divers could place on the outside of a cable's housing, said naval historian Norman Polmar, author of "Spy Book: The Encyclopedia of Espionage."
"The U.S. has had four submarines that have been outfitted for these special missions," he said.
But the fiber-optic lines — each no thicker than a quarter — were far more difficult to tap successfully than earlier generations of undersea technology, and interception operations ran the risk of alerting cable operators that their network had been breached.
It's much easier to collect information from any of dozens of cable landing stations around the world — where data transmissions are sorted into separate streams — or in some cases from network operations centers that oversee the entire system, say those familiar with the technology, who spoke on the condition of anonymity to discuss sensitive intelligence matters.
In the aftermath of the Sept. 11 attacks, the NSA said its collection of communications inside the United States was constrained by statute, according to a draft report by the agency's inspector general in 2009, which was obtained by The Washington Post and The Guardian. The NSA had legal authority to conduct electronic surveillance on foreigners overseas, but the agency was barred from collecting such information on cables as it flowed into and through the United States without individual warrants for each target.
"By 2001, Internet communications were used worldwide, underseas cables carried huge volumes of communications, and a large amount of the world's communications passed through the United States," the report said. "Because of language used in the [Foreign Intelligence Surveillance] Act in 1978, NSA was required to obtain court orders to target e-mail accounts used by non-U.S. persons outside the United States if it intended to intercept the communications at a webmail service within the United States. Large numbers of terrorists were using such accounts in 2001."
As a result, after White House and CIA officials consulted with the NSA director, then-President George W. Bush expanded the NSA's legal authority to collect communications inside the United States through a presidential order. The President's Surveillance Program, the report said, "significantly increased [NSA's] access to transiting foreign communications."
Gen. Michael Hayden, then the NSA director, described that information as "the real gold of the program" that led to the identification of threats within the United States, according to the inspector general's report.
Elements of the President's Surveillance Program became public in 2005, when The New York Times reported the government's ability to intercept e-mail and phone call content inside the United States without court warrants, sparking controversy. The FISA court began oversight of those program elements in 2007.
As these debates were playing out within the government, Team Telecom was making certain that surveillance capacity was not undermined by rising foreign ownership of the fiber-optic cables that the NSA was using.
The Global Crossing deal created particular concerns. The company had laid an extensive network of undersea cables in the world, but it went bankrupt in 2002 after struggling to handle more than $12 billion in debt.
Two companies, one from Singapore and a second from Hong Kong, struck a deal to buy a majority stake in Global Crossing, but U.S. government lawyers immediately objected as part of routine review of foreign investment into critical U.S. infrastructure.
President Gerald Ford in 1975 had created an interagency group — the Committee on Foreign Investment in the United States, or CFIUS — to review deals that might harm U.S. national security. Team Telecom grew out of that review process. Those executive branch powers were expanded several times over the decades and became even more urgent after the Sept. 11 attacks, when the Defense Department became an important player in discussions with telecommunications companies.
The Hong Kong company soon withdrew from the Global Crossing deal, under pressure from Team Telecom, which was worried that the Chinese government might gain access to U.S. surveillance requests and infrastructure, according to people familiar with the negotiations.
Singapore Technologies Telemedia eventually agreed to a slate of concessions, including allowing half of the board of directors of a new subsidiary managing the undersea cable network to consist of American citizens with security clearances. They would oversee a head of network operations, a head of global security, a general counsel and a human resources officer — all of whom also would be U.S. citizens with security clearances. The FBI and the departments of Defense, Justice or Homeland Security had the power to object to any appointments to those jobs or to the directors who had to be U.S. citizens.
U.S. law already required that telecommunications companies doing business in the United States comply with surveillance requests, both domestic and international. But the security agreement established the systems to ensure that compliance and to make sure foreign governments would not gain visibility into the working of American telecommunications systems — or surveillance systems, said Andrew D. Lipman, a telecommunications lawyer who has represented Global Crossing and other firms in negotiating such deals.
"These Network Security Agreements flesh out the details," he said.
Lipman, a partner with Bingham McCutchen, based in Washington, said the talks with Team Telecom typically involve little give and take. "It's like negotiating with the Motor Vehicle Department."
Singapore Technologies Telemedia sold Global Crossing in 2011 to Level 3 Communications, a company based in Colorado. But the Singaporean company maintained a minority ownership stake, helping trigger a new round of review by Team Telecom and a new Network Security Agreement that added several new conditions.
A spokesman for Level 3 Communications declined to comment for this story.