The Puzzle Palace

The papers of the late William Friedman, who had served as the NSA's chief cryptologist, were a valuable source for Bamford. The NSA later confiscated the documents from the library at which they were held.

The NSA describes Bamford's research process in a partially declassified history of postwar American cryptography.[5] The history begins by describing how Bamford approached Houghton-Mifflin with a proposal to write a book on the NSA. The publisher accepted and paid Bamford a $7,500 advance.

According to the NSA's historical account, "Public Law 86-36 had served as a useful barrier against this type of research, but Bamford proved to be cleverer than others. He began with a barrage of requests for information under the Freedom of Information Act (FOIA)." Bamford eventually discovered "the Mother Lode," a collection of documents deposited at the George C. Marshall Foundation Library by former NSA chief cryptologist William Friedman. These documents included copies of the NSA Newsletter, addressed to "NSA Employees and their families." The account continues, "Bamford then submitted a FOIA request for the entire collection, using as his rationale the offending phrase indicating that the information had been intended for dissemination to uncleared people." The FOIA requests gave Bamford access to an only lightly redacted set of documents, and a former NSA employee later gave Bamford access to an almost complete collection.

During the 1975 Church Committee hearings, the United States Department of Justice (DOJ) had investigated the legal culpability of various intelligence agencies. Bamford submitted a FOIA request for resulting documents, and received most of their report on the NSA. The DOJ did not inform the NSA of the release because the investigation was ongoing, and the NSA was a possible target. The released papers were later to become the subject of threatened litigation, and their release prompted eventual rule changes that allowed for document reclassification. According to the NSA's history, "the document, with some Justice redactions, contained a good deal of information about the NSA-GCHQ [Government Communications Headquarters, a British intelligence agency] relationship, and served as the basis for Bamford's information on Second Party issues [i.e., issues regarding outside agencies with which the NSA had partnered]." The NSA history dryly notes that "GCHQ was not amused" about the disclosure.

Bamford also interviewed retired NSA senior officials, including former director Marshall Carter, with whom he talked for a day and a half. None of the discussed subject matter was classified, but, according to the NSA, "it helped Bamford to complete his mosaic." The NSA account also describes how Bamford drove through the NSA parking lot, recording diplomatic license plates and cross-checking them against known lists to determine which countries maintained representatives at Fort Meade.

The NSA's historical account summarizes Bamford's work thusly: "James Bamford broke new ground in intelligence agency research, and his techniques were adopted by others seeking to investigate reclusive federal agencies. He did it all within the limits of the law – through attributable interviews, FOIA'ed documents, and meticulous research in public libraries and newspapers. ... He 'wrote the book' on how to put together a comprehensive picture of an organization that wanted no such comprehensive picture."

Before the book's publication, the Reagan administration claimed that unclassified source documents were released to Bamford in error, and threatened him with prosecution if he did not return 250 pages of documents he had obtained through Freedom of Information Act (FOIA) requests. The documents in question related to a 1975-76 DOJ investigation, and described the NSA's widespread illegal monitoring of domestic communication, warrantless surveillance of Americans, and monitoring of commercial cable and telex traffic.[1] Bamford's attorney, believing that they would prevail in court, invited the DOJ to prosecute, but no case was ever brought.[5] In the wake of the dispute, classification rules were revised to allow document reclassification, and the contested documents were reclassified. NSA agents then visited libraries to remove other source documents from circulation.[6][7][8][9][10][11]

The new rules allowing document reclassification were outlined in Executive Order 12356. Previously, Executive Order 12065, issued by Jimmy Carter in 1978, had prohibited document reclassification.[12] Executive Order 12356, issued by Ronald Reagan in 1982, eliminated the reclassification prohibition and described situations in which documents could be reclassified. Released documents could be reclassified as long as they could be reasonably recovered (meaning that documents available to the public at large would not meet this criterion). It also allowed documents requested under the FOIA or Privacy Act to be classified or reclassified provided they met specified requirements (e.g., certain matters related to national security).[13][14] According to Bamford, he could not be prosecuted under the new rules because of the principle of ex post facto.[15]

Reclassified and re-sequestered source documents that Bamford had used when writing The Puzzle Palace were the subject of subsequent litigation. The NSA's historical account states that documents removed from the Marshall Library were "sequestered portions of the Friedman collection," i.e., the collection that included the copies of the NSA Newsletter that spurred one of Bamford's FOIA requests. The materials removed from circulation included three government publications and 31 pieces of Friedman's private correspondence. The American Library Association (ALA) challenged the document removal in court, and in 1987 the U.S. Court of Appeals for the District of Columbia dismissed the case. Ruth Bader Ginsburg, who was at that time a Court of Appeals judge, ruled that the ALA lacked standing in the case. A lower court ruling had already affirmed that the NSA had authority to remove the reclassified documents, but criticized the NSA's "cavalier attitude" toward the classification determination of those documents.[5][16][17]

The New York Times reviewed the book favorably, writing that "Until now no one has published a comprehensive and detailed report on the agency. The quality and depth of Mr. Bamford's research are remarkable." The review concluded, "By revealing the scope and opening up the operations of the N.S.A. without giving away its most sensitive secrets, Mr. Bamford has performed an important public service in this impressive book."[1]

In a review of Shane Harris' book The Watchers: The Rise of America's Surveillance State, New York Times reporter Eric Lichtblau described The Puzzle Palace as "the benchmark study of the N.S.A. [that] first pulled back the curtain to provide a glint of unwanted sunlight on the place".[18] Michael Duffy, reviewing Bamford's 2004 book Pretext for War, wrote in Time magazine that The Puzzle Palace "is still considered the classic account of the mysterious National Security Agency."[19]

The Puzzle Palace has been generally well received by experts, and has been used as a textbook at the Defense Intelligence Agency's National Intelligence University.[20] The NSA itself says the book "brought a new focus to the efforts of journalists and independent writers to break down the Agency's vaunted anonymity," and describes the book as "the most significant breach in NSA's anonymity since David Kahn's The Codebreakers in 1967."[5] In the years between the book's publication and 9/11, the relationship between Bamford and the NSA grew less adversarial. The NSA cooperated with Bamford on a later book titled Body of Secrets, and in April 2001 the NSA hosted a book-signing event for Bamford at their Maryland headquarters.[20] Bamford was critical of NSA actions in the years following 9/11, and the relationship between the two once again cooled.

• Bamford, James (1982). The Puzzle Palace: A Report on America's Most Secret Agency. Houghton Mifflin. ISBN 0-14-006748-5.
• Bamford, James (2001). The Puzzle Palace: Inside the National Security Agency, America's Most Secret Intelligence Organization. Viking Pr. ISBN 0-14-023116-1.