September 22, 2016

Secret report reveals: German BND also uses XKEYSCORE for data collection

(Updated: September 23, 2016)

Over the past few years we learned a lot about Germany's foreign intelligence service BND, although not from leaks, but from the public hearings of the parliamentary commission that investigates NSA spying operations and its cooperation with German agencies.

Recently however a secret government report was leaked to German media, which not only identifies violations of the data protection act but also reveals the codenames for several BND systems and the fact that BND uses the American XKEYSCORE system not only for analysis, but also for collection purposes.

Here, the new information from the secret report is combined with things we know from earlier sources and reportings.

- A secret report
- The SUSLAG liaison office
- Selectors provided by NSA
- Operations SMARAGD and ZABBO
- Metadata analysis: VERAS
- Analysis and collection: XKEYSCORE
- Integrated analysis: MIRA 4
- Legal defects

The BND satellite intercept station at Bad Aibling, Germany
(Photo: AFP/Getty Images)

A secret report

The report that now has been published goes back to September 2013, when the then federal data protection commissioner Peter Schaar ordered a thorough inspection of the BND satellite intercept station in Bad Aibling, which took place on December 3 and 4 of that year.

In October 2014, Schaar's successor Andrea Voßhoff conducted a second visit to Bad Aibling, which in July 2015 resulted in an extensive and detailed report (German: Sachstandsbericht) about all the systems used at this BND station. This report was (and still is) classified as Top Secret.

Additionally, Voßhoff made a legal assessment based upon the Sachstandsbericht. This was finished in March 2016 and sent to then BND president Schindler and the federal chancellery. It was classified as Secret, but was leaked to regional broadcasters NDR and WDR and a transcription of the full document was published by the digital rights platform on September 1.

Both reports are about the cooperation between BND and NSA, which goes back to 2004, when the Americans turned their satellite intercept station Bad Aibling (codenamed GARLICK) over to German intelligence. In return, BND had to share the results from its satellite collection with the NSA, for which the latter provided selectors, like e-mail addresses, phone numbers, etc. of the targets they were interested in.

Google Maps view of the Mangfall Barracks in Bad Aibling, Germany.
The building at the very top seems to be the BND facility,
the one nearby with the white roof NSA's "Tin Can".

The SUSLAG liaison office

After taking over the Bad Aibling satellite station, BND seems to have moved the control facility to the nearby Mangfall Barracks, which were taken over from the German armed forces (Bundeswehr) in 2002. For the Special US Liaison Activity Germany (SUSLAG), which is the liaison office of NSA for Germany, a new highly secure container building was built on the Mangfall Barracks premises in 2003 (nicknamed "tin can" or Blechdose).

According to the commissioner's report, the SUSLAG building and the building with BND servers and equipment are connected through a 100 MBit/s fiber optic cable. SUSLAG also has a technical data link to the NSA's primary communications hub in Europe, the European Technical Center (ETC) in the Mainz-Kastel district of the city of Wiesbaden.

Cooperation between the US and Germany in the Joint SIGINT Activity (JSA, 2004-2012) took place inside the BND building, for which NSA personnel had access permissions. After the JSA was terminated, SUSLAG personnel kept their entrance rights for the BND building, but it has separate rooms for highly sensitive information to which none of the Americans have access.

A letter from BND from October 15, 2015 says that at that moment, 10 people from NSA worked at SUSLAG, with following access rights:
- 2 have access to building 7 (SUSLAG) only
- 4 have access to building 7 and building 4 (Administration)
- 4 have access to building 7 and building 8 (BND)

The SUSLAG building is only used by NSA personnel and BND claims that the data protection commissioner has no jurisdiction over the SUSLAG, but she disputes that and says the SUSLAG building is simply part of the BND complex. She also regrets that SUSLAG doesn't recognize her oversight authority.

Selectors provided by NSA

For the satellite interception in Bad Aibling, some 4 out of 5 selectors come from NSA, the rest from BND. According to Süddeutsche Zeitung, NSA provided BND with roughly 690.000 phone numbers and 7,8 million internet identifiers between 2002 and 2013. That is an average of something like 60.000 phone numbers and 700.000 internet identifiers a year, or 164 phone numbers and over 1900 internet identifiers each day.

From the parliamentary hearings we already knew that BND personnel pulls the American selectors from an NSA server, and the commissioner's report now reveals that this server is in NSA's ETC in Wiesbaden. On this server BND puts back any results for these selectors. These data transfers from and to ETC go through the SUSLAG facility, but BND is able to get direct access to the NSA server in Wiesbaden through an FTP-gateway (a "BACOM system").

Selector databases

From an earlier parliamentary hearing we know that BND stores the selectors from NSA in two databases: one for IP selectors (from NSA only), and one for telephone selectors (from both NSA and BND). Each agency had access to its own IP database; the phone database was managed jointly, but BND could only approve or disapprove NSA selectors, and NSA could only do so with those from BND.

The names of these databases were not known until now, but the commissioner's report mentions them, along with some additional details:
- Target Number Database (TND), which exists since 2008 and holds the telephone selectors from both NSA and BND. The latter either come from BND's own tasking database PBDB or are provided by domestic security services.

- SCRABBLE, which only holds selectors for packet-switched (internet) communications provided by NSA, after their format has been converted. These selectors initially had no description (Deutung, like a justification for the target). Because of this, BND temporarily stopped using them as of May 2015, and for the commissioner any results from them are unlawful because BND was not able to determine whether they are necessary for its mission.

Their names indicate that these database systems were provided by NSA, and together with the fact that they also contain NSA-provided selectors, this is likely the reason why these names were never mentioned during the parliamentary hearings - unlike those of BND's own systems.
Update: it was noticed that TND and SCRABBLE were actually mentioned once during the parliamentary hearings, when former BND president Schindler said that "the US has [its own] databases TND and SCRABBLE".


Before being stored in the SCRABBLE and TND databases, both the telephone and internet selectors have to pass the DAFIS filtering system, which checks whether they belong to German citizens or companies or may otherwise contradict German interests. Accordingly, the selectors are marked as "allowed" or "protected".

Those marked "allowed" are subsequently being activated ("tasked") on the actual data collection systems. The report says that for this, hard selectors like phone numbers and e-mail addresses can be freely combined with content search terms (Inhaltssuchbegriffe) like key words, which could refer to the GENESIS language used for more complex XKEYSCORE searches.

According to the report, selectors marked as "protected" are send back to NSA and are also deactivated in the TND and SCRABBLE databases - to make sure that they won't get activated when NSA provides them a second time (this confirms that there's no separate database (Ablehnungsdatei) with rejected selectors as was suggested during the earlier parliamentary commission hearings).

BND refused the data protection commissioner access to TND and SCRABBLE, so she wasn't able to check the individual selectors. She regarded that as a massive restriction of her supervision authority.

Operations SMARAGD and ZABBO

Selectors that have been approved are send to the systems that filter out communications that match those selectors. Some of these systems are in Germany, others are abroad. The report of commissioner Voßhoff for the first time discloses two specific data collection operations and their codewords:

- SMARAGD, a cable tapping operation somewhere outside Europe and in cooperation with another foreign intelligence agency.

- ZABBO, collection in Bad Aibling of satellite communications from Afghanistan.

There's no explanation for why the commissioner only mentions these two operations. The satellite antennas in Bad Aibling undoubtedly collect from many more countries, but maybe these are the only operations from which, during the investigation period, data were shared with NSA.


The way SMARAGD is described perfectly fits a certain type of operations in which a 3rd Party partner of NSA like in this case BND, cooperates with yet another country that secretly provides access to data traffic, which is then also shared with NSA. According to the book Der NSA Komplex, BND and NSA conducted about half a dozen of such operations in recent years.

In its english version of the news report about this issue, the website points to an NSA document that was published earlier by Der Spiegel. In it, we see EMERALD mentioned as an alternate codename for the NSA operation WHARPDRIVE, which is exactly such a trilateral program in which a third secret service participates.

WHARPDRIVE was still active in 2013, but in the Spring of that year, employees of the private company that operated the communication cables, accidently discovered the clandestine BND/NSA equipment, but the operation was rescued by providing a plausible cover story.*

The NSA report from April 2013 however said that "WHARPDRIVE has been identified for possible termination due to fiscal constraints", but this may have been coincided with the exposure of the program in the book Der NSA Komplex in March 2014.

It should also be noted that came up with this identification by translating the German codename SMARAGD into its English equivalent EMERALD. It is possible that the Americans also translated the German codeword SMARAGD into EMERALD, but just as likely is that it's a different program (maybe as a successor with the same set-up).

Operation Eikonal

But there's another codeword connection: from 2004 till 2008, NSA cooperated with BND in operation EIKONAL in order to get access to fiber optic cables from Deutsche Telekom in Frankfurt.

From the parliamentary hearings we know that operation EIKONAL had GRANAT as its internal BND codename. And with GRANAT being German for garnet, and SMARAGD for emerald, we see that both operations are actually named after a gemstone, which often indicates some kind of similarity.

In October 2014, the Danish paper Information reported that the WHARPDRIVE access was opened in February 2013 and had the same size as EIKANOL. This operation EIKANOL or EIKONAL was a typical example of the way NSA cooperates with 3rd Party partner agencies under its RAMPART-A program, but unlike the SMARAGD/WHARPDRIVE operations with the cable access point being inside Germany:

Left: bilateral cable access operation (RAMPART-A) - Right: trilateral cable access operation
In the cases discussed here, Germany would be "Country X"
(click to enlarge)

It is tempting to identify SMARAGD and ZABBO as the two collection programs (SIGADs US-987LA and US-987LB) from the BOUNDLESSINFORMANT chart for Germany that was published in July 2013. For both facilities together, more than 552 million metadata records were counted between December 10, 2012 and January 8, 2013.

Provided that this chart shows the only data shared by BND, it's very well possible that the satellite collection program ZABBO is one of them. For the cable access SMARAGD this is less certain and depends on when this program started and whether it is identical with WHARPDRIVE (which started in February 2013).

BOUNDLESSINFORMANT screenshot showing metadata provided by BND
(click to enlarge)

Data transfer

The report of the data protection commissioner also provides an impression of the BND networks through which collected data are brought back to headquarters.

Data collected abroad are send back to Germany over the operational network ISNoVPN (apparently something that goes "over VPN" for secure tunneling) and then arrives at a dedicated demilitarized zone (DMZ) network for data collection (Datenabholungs-DMZ).

In this DMZ network there's a virtual machine (VM) that acts as a host for data that come in from each collection facility (Erfassungsansatz). The report mentions the virtual machines "Import VM SMARAGD" and "Import VM ZABBO" for the operations SMARAGD and ZABBO respectively.

In these virtual machines, the metadata go through an Application Level Gateway (ALG), which is a security components combined with a firewall. Such an ALG is able to detect, filter and when necessary, delete data from an incoming data stream. Again, there's an ALG for each collection facility: for example SMARAGD-ALG for data from the SMARAGD collection effort.

Finally, the collected data arrive at a network called NG-Netz, which is the back-end in Bad Aibling of the transfer system that pulls in data collected at a front-end access point (Erfassungskopf) somewhere abroad.

(click to enlarge)

Metadata analysis: VERAS

The system that BND uses for analysing bulk metadata from circuit-switched communications is called VERAS, which stands for Verkehrs-Analyse-System or Traffic Analysis System. VERAS stores metadata only for up to 90 days and according to the commissioner's report they are derived from two sources:

- Metadata that come with communications collected after matching with specific selectors (the related content goes to the INBE database)

- All the metadata from selected communication links (satellite frequencies and fiber optic channels) that are regarded useful for intelligence purposes, but only after passing the DAFIS filter.

According to the manual for VERAS version 4.3.x from 2010, the system has a topology mode, in which connections can be created level after level, similar to the "hops" we know from the NSA's contact chaining method. There's no limitation to the number of levels that can be added and analysts can also focus on specific targets to create patterns-of-life (Bewegungsprofile) for them.

This kind of contact-chaining and metadata analysis inevitably involves metadata from innocent people. BND distinguished between directly and indirectly relevant. Directly relevant are metadata related to people who are already known or suspected for being relevant for intelligence purposes.

Indirectly relevant are metadata related to people who have some kind of connection to directly relevant people, or when such metadata are being stored from a "geographical point of view", which apparently refers to metadata of people being somewhere near a target without having been in direct contact.

The report says that metadata connected on such a geographical basis results in much more people being involved than when using call or connection chaining. Data related to indirectly relevant people are also used by BND, for example as new selectors.

VERAS was introduced in 2002 and recently, VERAS 4 has been replaced by VERAS version 6, which was developed by the German armed forces (Bundeswehr) as part of the VERBA (VERkehrs-Beziehungs-Analyse) project.

For VERAS 6 there's not yet a database establishing order (see below), but in February 2015 BND sent the commissioner a draft version, which she already considers illegal because BND admits that it is technically impossible to prevent that data of innocent people are being used in the VERAS system.

Analysis and collection: XKEYSCORE

Already in July 2013, Der Spiegel reported that BND president Schindler had informed the parliamentary intelligence oversight commission (PKGr) that his agency was using NSA's XKEYSCORE system since 2007, but only for analysis, not for data collection. This was confirmed by W. K., a sub-division manager in the BND's Signals Intelligence division, during a parliamentary hearing.

But now, the report of the data protection commissioner says that BND uses XKEYSCORE not just for analysis, but also for the collection of both metadata and content.

The report explains that in its data collection, or front-end function, XKEYSCORE uses selectors, single ones or freely combined ones in the form of fingerprints, to search for matches in IP traffic of both public and privat networks, and stores anything that matches these selectors.

Remarkably enough, the commissioner writes that XKEYSCORE searches all internet traffic worldwide ("weltweit den gesamten Internetverkehr"), which seems to be a copy/paste from sensationalistic press reports, as XKEYSCORE can only search data which are collected at some physical access points and not even NSA has access to all the world's communications traffic, let alone BND.

Slide from an NSA presentation about the XKEYSCORE system

Besides picking out and storing communications that match specific selectors, XKEYSCORE is also able to store a so-called "full take", a temporary rolling buffer of all data from a particular link. This in order to find files which aren't directly associated with specific selectors - which was heralded as its unique capability.

The commissioner's report only mentions this buffer function when it cites a BND response calling XKEYSCORE "a local and temporary buffering of data" which in their opinion doesn't make it a database. The commissioner disagrees and says it's a database, because even when it's just for a short time, the data are available for usage. This means a there should have been a database establishing order for XKEYSCORE (see below).

Front-end and back-end

The report doesn't explain what XKEYSCORE actually does in its function as a back-end analysis tool. But maybe instead of distinguishing between collection and analysis, we should look at the difference between the front-end and the back-end functions of the system, which is explained in a manual for its so-called Deepdive version.

This learns us that the back-end performs high-speed filtering and selection using both strong selectors (like e-mail addresses) and soft selectors (like key words), and also uses various plug-ins to extract and index the metadata, which are also used for the rolling buffer-functionality of XKEYSCORE:

Diagram showing the dataflow for the DeepDive version of XKEYSCORE

The front-end is where the intercepted data streams come in, which are first reassembled by the METTLESOME and xFip components. Then, only the most useful streams are forwarded based upon rules using country codes, keywords and such. Finally, the Defrag component conducts full sessionizing, which means that the separate IP packets that travel over the internet are reassembled into their original readable form again.

The commissioner's report says that initially the sessionizing of data from a particular communications link was conducted by another NSA system codenamed WEALTHYCLUSTER (WC, which is for lower data rates), but that this kind of processing was more and more taken over by XKEYSCORE (XKS).

So, if the distinction between collection and analysing corresponds to that between front-end and back-end, that means that the new thing we learned from the commissioner's report is that BND apparently also uses XKEYSCORE for sessionizing the data they collect, and not only for filtering and analysing them.

This sessionizing might seem rather obvious, but real-time filtering and sessionizing at data rates as high as 10 Mbit/s requires very fast, specialized and expensive equipment. Well-known manufacturers are Narus and Verint, and it seems likely that their equipment is used for XKEYSCORE too.

As XKEYSCORE is only used for internet communications, the NSA selectors are derived from the SCRABBLE database. The results of the collection are transferred to NSA, after having been filtered by DAFIS to get rid of data related to Germans.

Integrated analysis: MIRA 4

Besides all the systems mentioned before, BND also uses MIRA 4, which stands for Modulare Integrierte Ressourcen Architektur or Modular Integrated Ressource Architecture, version 4. According to a letter from BND from Februar 2015, this system is used to store all the content, whether from e-mail, voice, fax or teletype messages, within a certain BND station and apparently also enables software to process and select raw data in order to create intelligence reports (Meldungen).

This was however contradicted by a letter from BND from December 2015 which said that MIRA 4 is only used to store just those Meldungen. The commissioner replied that she would be thankful when BND could clarify this discrepancy.

Apparently not noticed by the commissioner is an NSA report from 2006, which was published by earlier Der Spiegel, and which says that German analytic tool suites like MIRA 4:
"integrate multiple database analytic functions (such as viewing voice and listening to fax [sic]), much like NSA Headquarters has UIS (User Integrated Services). In some ways, these tools have features that surpass US SIGINT capabilities. Among a series of interesting items, NSA analysts noted that BND analysts could seamlessly move from VERAS (call-chaining software) to the associated voice cuts."

Later on, the 2006 NSA report says: "The BND responded positively to NSA's request for a copy of MIRA4 and VERAS software, and made several requests from NSA concerning target and tool development and data".

During a parliamentary hearing in October 2014, BND's own data protection officer Ms. H. F. said that in 2010, MIRA 4 was replaced by INBE as a system that apparently not only stores the content of communications, but also makes it available for analysis.

The 2016 commissioner's report says that data stored in MIRA 4 were not migrated to INBE, when the latter system became operational in 2011. Data in MIRA 4 seem to have been automatically "aged off" after 90 days and the last backup of the system was destroyed in the Summer of 2014.

Legal defects

The purpose of the secret report by federal data protection commissioner Andrea Voßhoff was to determine the legality of the data collection, processing, storing and analysing systems at the BND field station in Bad Aibling. The two main problems she identified are about necessity and the lack of database establishing orders.


According to the German data protection law, BND is only allowed to receive, store, process and analyse personal data after checking that they are necessary and relevant for its foreign intelligence mission as authorized by German law. In various cases, especially when it comes to bulk collection of metadata and receiving the selectors from NSA, the agency doesn't or cannot check the necessity for each piece of data. This makes it unlawful for BND to posess and use those data.

The problem behind this is that when such laws were made, there was no awareness of secret services using large sets of metadata, which also includes those of innocent people. Also in this particular case, almost all data collected in Bad Aibling and shared with NSA will be collected from crisis zones like Afghanistan, which makes them more relevant for BND's mission and less likely of containing German communications.

Database establishing orders

Another major legal defect the commissioner found was that for the BND databases VERAS 4, VERAS 6, XKEYSCORE, TND, SCRABBLE, INBE, and DAFIS there was no database establishing order (Dateianordnung) and that they were also set up without prior approval by the commissioner. This makes the existance of these databases unlawful, which means the data they contain should be deleted immediatly until an establishing order is provided.

BND argued that the absence of a database establishing order is just a formal defect and doesn't affect the legal status of a database and its content. The commissioner doesn't agree with that and says that one of the functions of an establishing order is to determine the purpose of a database, which limits and restricts the use of the personal data in it. The lack of such an order also means that there are no rules for when approvals by oversight bodies are required, thus making the use of these databases both unlawful and uncontrolled.

In response

Meanwhile, on September 7, the German interior ministry released a draft for a new data protection act, in which it is proposed that in the future, the data protection commissioner will not have the authority anymore to impose sanctions or fines on the secret services - so restricting the commissioner's authority rather than strenghten it.

Finally, on September 15, Edward Snowden also mentioned the commissioner's report on Twitter, saying that it "confirms mass surveillance". Apparently he didn't read the report, as it is actually about the lack of specific legal restrictions, not about the scope of BND's collection efforts.

Links and Sources
- Rolf Weber: Der geleakte BND-Bericht der BfDI Voßhoff -- wie gewohnt bei näherem Hinsehen wenig skandalträchtig
- Netzpolitik: Secret Report: German Federal Intelligence Service BND Violates Laws And Constitution By The Dozen
- Der Spiegel: NSA-Standorte in Deutschland: Wiesbaden

August 20, 2016

Is the Shadow Brokers leak the latest in a series?

(Updated: September 24, 2016)

Earlier this week, a group or an individual called the Shadow Brokers published a large set of files containing the computer code for hacking tools. They were said to be from the Equation Group, which is considered part of the NSA's hacking division TAO.

The leak got quite some media attention, but so far it was not related to some earlier leaks of highly sensitive NSA documents. These show interesting similarities with the Shadow Brokers files, which were also not attributed to Edward Snowden, but seem to come from an unknown second source.

Screenshot of some computer code with instructions
from the Shadow Brokers archive
(click to enlarge)

The Shadow Brokers files

Since August 13, Shadow Brokers posted a manifesto and two large encrypted files on Pastebin, on GitHub, on Tumblr and on DropBox (all of them closed or deleted meanwhile).

One of the encrypted files could be decrypted into a 301 MB archive containing a large number of computer codes for server side utility scripts and exploits for a variety of targets like firewalls from Cisco, Juniper, Fortinet and TOPSEC. The files also include different versions of several implants and instructions on how to use them, so they're not just the malware that could have been found on the internet, but also files that were only used internally.

A full and detailed list of the exploits in this archive can be found here.

Security experts as well as former NSA employees considered the files to be authentic, and earlier today the website The Intercept came with some unpublished Snowden documents that confirm the Shadow Brokers files are real.

Besides the accessible archive, Shadow Brokers also posted a file that is still encrypted, and for which the key would only be provided to the highest bidder in an auction. Would the auction raise 1 million bitcoins (more than 500 million US dollars), then Shadow Brokers said they would release more files to the public. This auction however is likely just meant to attract attention.

Screenshot of a file tree from the Shadow Brokers archive
(click to enlarge)

From the Snowden documents?

According to security experts Bruce Schneier and Nicholas Weaver the new files aren't from the Snowden trove. Like most people, they apparently assume that Snowden took mostly powerpoint presentations and internal reports and newsletters, but that's not the whole picture. The Snowden documents also include various kinds of operational data, but this rarely became public.

Most notable was a large set of raw communications content collected by NSA under FISA and FAA authority, which also included incidentally collected data from Americans, as was reported by The Washington Post on July 5, 2014. The Snowden documents also include technical reports, which are often very difficult to understand and rarely provide a newsworthy story on their own.

Someone reminded me as well that in January 2015, the German magazine Der Spiegel published the full computer code of a keylogger implant codenamed QWERTY, which was a component of the NSA's WARRIORPRIDE malware framework. So with the Snowden trove containing this one piece of computer code, there's no reason why it should not contain more.

Contradicting the option that the Shadow Brokers files could come from Snowden is the fact that some of the files have timestamps as late as October 18, 2013, which is five months after Snowden left NSA. Timestamps are easy to modify, but if they are authentic, then these files have to be from another source.

A second source?

This brings us to a number of leaks that occured in recent years and which were also not attributed to Snowden. These leaks involved highly sensitive NSA files and were often more embarrassing than stuff from the Snowden documents - for example the catalog of hacking tools and techniques, the fact that chancellor Merkel was targeted and intelligence reports proving that NSA was actually successful at that.

It is assumed that these and some other documents came from at least one other leaker, a "second source" besides Snowden, which is something that still not many people are aware of. The files that can be attributed to this second source have some interesting similarities with the Shadow Brokers leak. Like the ANT catalog published in December 2013, they are about hacking tools and like the XKEYSCORE rules published in 2014 and 2015 they are internal NSA computer code.

This alone doesn't say much, but it's the choice of the kind of files that makes these leaks look very similar: no fancy presentations, but plain technical data sets that make it possible to identify specific operations and individual targets - the kind of documents many people are most eager to see, but which were rarely provided through the Snowden reporting.

As mainstream media became more cautious in publishing such files, it is possible that someone who also had access to the Snowden cache went rogue and started leaking documents just for harming NSA and the US - without attributing these leaks to Snowden because he would probably not approve them, and also to suggest that more people followed Snowden's example.

Of course the Shadow Brokers leak can still be unrelated to the earlier ones. In that case it could have been that an NSA hacker mistakenly uploaded his whole toolkit to a server outside the NSA's secure networks (also called a "staging server" or "redirector" to mask his true location) and that someone was able to grab the files from there - an option favored by for example Edward Snowden and security researcher the grugq.

Diagram showing the various stages and networks involved
in botnet hacking operations by NSA's TAO division
(source - click to enlarge)

An insider?

Meanwhile, several former NSA employees have said that the current Shadow Brokers leak might not be the result of a hack from the outside, but that it's more likely that the files come from an insider, who stole them like Snowden did earlier.

Of course it's easier for an insider to grab these files than for a foreign intelligence agency, let alone an ordinary hacker, to steal them from the outside. But if that's the case, it would mean that this insider would still be able to exfiltrate files from NSA premises (something that shouldn't be possible anymore after Snowden), and that this insider has the intent to embarrass and harm the NSA (Snowden at least said he just wanted to expose serious wrongdoings).

Here we should keep in mind that such an insider is not necessarily just a frustrated individual, but can also be a mole from a hostile foreign intelligence agency.

On August 21, NSA expert James Bamford also confirmed that TAO's ANT catalog wasn't included in the Snowden documents (Snowden didn't want to talk about it publicly though). Bamford favors the option of a second insider, who may have leaked the documents through Jacob Appelbaum and Julian Assange.

Russian intelligence?

On Twitter, Edward Snowden said that "Circumstantial evidence and conventional wisdom indicates Russian responsibility", but it's not clear what that evidence should be. It seems he sees this leak as a kind of warning from the Russians not to take revenge for the hack of the Democratic National Committee (DNC) e-mails, which was attributed to Russian intelligence.

This was also what led Bruce Schneier to think it might be the Russians, because who other than a state actor would steal so much data and wait three years before publishing? Not mentioned by Schneier is that this also applies to the documents that can be attributed to the second source: they also pre-date June 2013.

A related point of speculation is the text that accompanied the Shadow Brokers files, which is in bad English, as if it was written by a Russian or some other non-western individual. This is probably distraction, as it looks much more like a fluent American/English speaker who tried to imitate unexperienced English.

The text also holds accusations against "Elites", in a style which very much resembles the language used by anarchist hacker groups, but that can also be faked to distract from the real source (it was also noticed that the e-mail address used by Shadow Brokers ( seems to refer to the manga Code Geass in which an exiled prince takes revenge against the "Britannian Empire").

Screenshot of some file folders from the Shadow Brokers archive
(click to enlarge)


With the authenticity of the Shadow Brokers files being confirmed, the biggest question is: who leaked them? There's a small chance that it was a stupid accident in which an NSA hacker uploaded his whole toolkit to a non-secure server and someone (Russians?) found it there.

Somewhat more likely seems the option that they came from an insider, and in that case, this leak doesn't stand alone, but fits into a series of leaks in which, since October 2013, highly sensitive NSA data sets were published.

So almost unnoticed by the mainstream media and the general public, someone was piggybacking on the Snowden-revelations with leaks that were often more embarrassing for NSA than many reportings based upon the documents from Snowden.

Again, obtaining such documents through hacking into highly secured NSA servers seems less likely than the chance that someone from inside the agency took them. If that person was Edward Snowden, then probably someone with access to his documents could have started his own crusade against NSA.

If that person wasn't Snowden, then it's either another NSA employee who was disgruntled and frustrated, or a mole for a hostile foreign intelligence agency. But for an individual without the protection of the public opinion like Snowden, it must be much harder and riskier to conduct these leaks than for a foreign state actor.

Former NSA counterintelligence officer John Schindler also thinks there could have been a (Russian) mole, as the agency has a rather bad track record in finding such spies. If this scenario is true, then it would be almost an even bigger scandal than that of the Snowden-leaks.

During an FBI-led investigation of the ShadowBrokers leak, NSA officials reportedly said that a former agency operative carelessly left the hacking tool files available on a remote computer, where Russian hackers found them. After this was discovered, NSA tuned its sensors to detect use of any of the tools by other parties, like China and Russia. But as that wasn't the case, NSA did not feel obligated to warn the US manufacturers.

Links and Sources
- How the NSA got hacked
- Where Are NSA’s Overseers on the Shadow Brokers Release?
- NSA ‘Shadow Brokers’ Hack Shows SpyWar With Kremlin Is Turning Hot
- Everything you need to know about the NSA hack (but were afraid to Google)
- Powerful NSA hacking tools have been revealed online
- ‘Shadow Brokers’ Leak Raises Alarming Question: Was the N.S.A. Hacked?
- NSA and the No Good, Very Bad Monday

June 24, 2016

E-mails from inside the NSA bureaucracy

Earlier this month, the NSA declassified a huge set of internal e-mails, following FOIA-requests about the issue of whether Edward Snowden had raised concerns about the NSA's surveillance programs through proper channels inside the agency.

> Download the declassified e-mails (very large pdf)

Here, we will take a look at the administrative details these internal NSA e-mails provide. Next time we will see what their content says about the concerns that Snowden claimed to have raised.

Internal e-mail from NSA director Michael Rogers. In the signature block we see his
NSANet and SIPRNet e-mail addresses and his non-secure phone number (all redacted)
(Click to enlarge - See also: NSA director Alexander's phones)

E-mail addresses

Except from the classification markings, the NSA's internal e-mails aren't very different from those exchanged by most other people around the world. But they do show for example some details about the internal communications networks of the agency.

From the signature blocks underneath the e-mails we learn that, depending on their function and tasks, NSA employees have e-mail addresses for one or more of the following four computer networks:

- NSANet for messages classified up to Top Secret/SCI (Five Eyes signals intelligence). On this network the address format for e-mail is jjdoe@nsa

- JWICS for messages classified up to Top Secret/SCI (US intelligence). The address format is

- SIPRNET for messages classified up to Secret (mainly US military). The address format is

- UNCL for unclassified messages, likely through NIPRNet. The address format is

For e-mail, all NSA employees have display names in a standardized format: first comes their family name, given name and middle initial, sometimes followed by "Jr" or a high military rank. Then follows "NSA" and the proper organizational designator, then "USA" for their nationality and finally "CIV" for civilian employees, "CTR" for contractors, "USN" for Navy, "USA" for Army or "USAF" for Air Force members.

Thus, the display name of the current NSA director is "Rogers Michael S ADM NSA-D USA USN", while that of the previous director was "Alexander Keith B GEN NSA-D USA USA". In 2012, Snowden had the display name "Snowden Edward J NSA-FHX4 USA CTR":

E-mail from Snowden as systems administrator in Hawaii, August 2012
The redacted part of the classification marking
seems to hide a dissemination marking *
(Click to enlarge)

The organizational designator FHX4 is interesting. FH stands for Field station Hawaii, but X4, being unit 4 of division X, is still a mystery. The field station divisions have the same designators as those at NSA headquarters, where there's also a division X, but so far no document gave an indication what it does.

The signature block shows that Snowden worked as a systems administrator for Dell's Advanced Solutions Group and that he was deployed at the Technology Department of NSA's Cryptologic Center in Hawaii, more specifically at the Office of Information Sharing. The latter has the organizational designator (F)HT322 and is therefore different from that in Snowden's display name.

In the declassified messages we only see display names, not the actual e-mail addresses behind them. Therefore, only the classification markings on the messages provide an indication on which network they were exchanged.

From an e-mail that was declassified earlier we know that in April 2013 Snowden used the address "", which is the format for the JWICS network, but was apparently used on NSANet.*

From one of the declassified e-mails about NSA's internal investigation it seems that Snowden had just two mail accounts: "we have his TS [Top Secret] NSANet email and his UNCLASSIFIED email", but this is followed by some redacted lines.*

Finally, the signature blocks of some NSA employees also provide a link to their dropbox for sending them files that may be too large for e-mail. Such dropboxes have addresses like "[...]".

Example of an NSA message, with in the signature block e-mail addresses for JWICS and an
unclassified network, and phone numbers for the NSTS and the non-secure phone networks
OPS 2B is the wider and lower one of the two black NSA headquarters buildings
(Click to enlarge)

Telephone numbers

Besides e-mail addresses, many messages also have phone numbers in the signature blocks. They show numbers for one or more of the telephone systems used at NSA:

- NSTS, which stands for National Secure Telephone System and is NSA's internal telephone network for secure calls. Numbers for this network have the format 969-8765 and are often marked with "(s)" for "secure"

- STE, which stands for Secure Terminal Equipment, being a telephone device capable of encrypting phone calls on its own. Telephone numbers can be written in the format (301) 234-5678 or as STE 9876.

- BLACK, CMCL or Commercial, which are numbers for non-secure telephones that may also access the public telephone network. They have the regular format (301) 234-5678 and are often marked with "(b)" for "black" (as opposed to "red") or with "(u)" for unclassified.

The NSA/CSS Threat Operations Center (NTOC) at NSA headquarters, with from left to right:
an STE secure phone, a probably non-secure telephone and a phone for the NSTS
(Photo: NSA, 2012 - Click to enlarge)


Finally, releasing such a huge set of documents in which many parts had to be redacted always bears the risk that something is overlooked. That also happened this time, as in one e-mail from an investigator from NSA's Counterintelligence Investigations unit Q311 they forgot to redact the codeword TIKICUBE:

TIKICUBE appears to be a unit of the Investigations Division Q3. Whether this might be a special unit investigating the Snowden leak isn't clear though.

The abbreviations behind the investigators name are: CFE for Certified Fraud Examiner and CISSP for Certified Information Systems Security Professional.

We also see that this investigation division is not located at the NSA headquarters complex at Fort Meade, but at FANX. This stands for Friendship Annex, a complex of NSA office buildings in Linthicum, near Baltimore, some 12 km. or 7.5 miles north-east of Fort Meade.

The famous blue-black glass headquarters buildings are OPS 2A and OPS 2B, while the SIGINT division is apparently in the flat 3-story building from the late 1950s, designated OPS 1.

May 19, 2016

German journalists about working with the Snowden documents

Last Monday, the website The Intercept started publishing larger batches of documents from the Snowden trove, so they can now also be examined by the public. It's a new phase after previously documents were generally disclosed as part of journalistic reports, but the number of such publications steadily declined over the last two years.

For how it was to work with the Snowden documents can be learned from an interesting interview with two journalists from the German Magazine Der Spiegel. They not only published a whole range of articles based upon the Top Secret NSA documents, but also a book which is much more informative than that of Glenn Greenwald.

The interview with Marcel Rosenbach and Holger Stark from Der Spiegel, as well as with Svea Eckert from the German broadcaster NDR, was part of the Network Research (Netzwerk Recherche) annual conference, which was held on July 3 and 4, 2015:

Interview with Marcel Rosenbach, Holger Stark
and Svea Eckert, July 2015 (in German)

Because the interview is fully in German, here's an extensive summary in English, which also looks more closely at a few specific revelations:
- The Snowden documents
- The National Intelligence Priority Framework (NIPF)
- Eavesdropping on chancellor Merkel
- Some other issues

The Snowden documents

Journalists from Der Spiegel were provided with several ten thousand digital documents through the documentary film maker Laura Poitras, who had been in direct contact with Edward Snowden.

According to Holger Stark, it was clear that Snowden had sorted the documents, not very fine-grained, but he had put them in a few folders, according to topics that had his special interest, like operations of the NSA divisions TAO (hacking) and SSO (cable tapping). Rosenberg said that it looked like Snowden selected the documents based upon his concerns regarding civil liberties and that het didn't some "collect it all" scraping.

(although in the film CitizenFour, Snowden himself said: "I cast such a wide net" that it would be difficult for NSA to determine how many documents he actually took)*

The journalists tried to search and filter the documents automatically, but a huge number of them had to be read and analysed manually, and read over and over again, in order to understand what was in them and what their importance could be. For that, they also consulted experts for cryptography and network architecture as well as former NSA employees like Binney and Drake (independent intelligence experts were not mentioned).

It was possible to ask Snowden, but not in a regular or easy way, also because he wanted to stay at a distance of the journalistic work. The journalists couldn't tell or estimate how many documents Snowden actually took. Der Spiegel got the documents unredacted but in the documents that were published, editors redacted most of the names.

Der Spiegel frequently asked NSA to review the documents they wanted to publish, in order to prevent that lives could become in danger. Sometimes NSA asked to remove things, but when it was obvious that that was for political reasons, the request was ignored. But in a few other cases Der Spiegel didn't publish or partly redacted the documents.


Despite all their efforts, there were still many gaps and questions. This resulted in for example a wrong interpretation of NSA's data visualisation tool BOUNDLESSINFORMANT. In August 2013, Der Spiegel published charts from this tool that were initially interpreted as showing how many data NSA collected from several European countries. Soon, BND and NSA denied this and explained that the charts show data that European agencies provided to the Americans.

Holger Stark admitted that their initial interpretation was apparently not correct, but that there are still many questions about this issue. One of the difficulties was that NSA and US government were not willing to respond to questions about this program, so they decided to publish their best guess. Rosenbach added that major foreign papers also shared their initial interpretation (maybe because the wrong interpretation came from Greenwald?).

BOUNDLESSINFORMANT screenshot showing metadata provided by BND
(click to enlarge)


The National Intelligence Priority Framework (NIPF)

One document that wasn't published, but only reported about is the National Intelligence Priority Framework (NIPF), which contains the priorities for the US intelligence community as set by the White House. During the interview a part of the original NIPF document was shown for the first time:

The NIPF consists of a large matrix with each cell indicating the intersection between a state or non-state actor and an intelligence topic. A readable reconstruction of the NIPF based upon this new piece and earlier sources, can be found here (pdf).

Over time, Rosenbach and Stark learned to interpret the Snowden documents by combining information from multiple documents. A separate document, an internal NSA newsletter from December 2009, for example provided additional information about the priorities of the NIPF chart:

This newsletter says that updated versions of the NIPF are released about twice a year, and that these are run against the National SIGINT Requirements Process (NSRP), which sets the priorities for acquiring Signals Intelligence (SIGINT). The 5 levels of NIPF priorities are then translated (by the SIGINT Committee or SIGCOM) to the 9 levels of SIGINT priorities, based upon the importance of the SIGINT contribution.

The first NIPF was issued in 2003 and at that time the matrix contained over 2300 cells! There were hundreds of issues with priority 1 and 2, way too many to be managable. So over the years the number of priorities, particularly the numbers of priority 1s and 2s had been reduced.

According to the journalists, the newsletter also explains that topics with priority 1 and 2 are meant for the president and the White House, while priority 3 is for cabinet ministers, the Chiefs of Staff and the Pentagon. For these highest priorities, covert intelligence methods are used. For priorities 4 and 5 open sources may be sufficient and their results are mainly used for political analysis.

For the Spiegel journalists this bureaucratic process illustrates that NSA isn't an agency that went rogue, but that they are directed by the political information needs from the White House (something that was usually conveniently ignored).



Svea Eckert, a documentary maker for the regional German broadcasster NDR, was also present at the interview, and she had brought with her the laptop they had used for working with the Snowden documents. The computer was newly bought for this purpose and was never connected to the internet.

At NDR, Eckert was doing research for a documentary about the internet as a battle space, when a colleague of her in the US was provided with a thumb drive containing Snowden documents that had been selected on their relevance for the topic of the documentary. It wasn't told who the middlemen for these documents were, and apparently different German news media got documents from different sources.

The source had said that for these documents only the external TAILS operating system should be used. The same system was used by other people who worked with Snowden documents, like Laura Poitras, Glenn Greenwald, and Barton Gellman. On the dedicated laptop, Eckert showed an example of what these documents look like:

In the window we see for example an internal NSA newsletter with an interview with a hacker from NSA's TAO division, a Cyber Warfare Lexicon and a powerpoint presentation. The latter has the filename "MONSTERMIND_presentation (copy).pptx", but when it was opened, it actually had the cover term CYBERCOP on the front slide and it was prepared by the "CyberCOP Product Manager".

Eckert explained that although most of these documents were very interesting, not everything was newsworthy enough or in the public interest to publish. Also the opinions of various experts had to be asked, because journalists were not always able to judge what the context or the importance of particular pieces of information was.


The CYBERCOP presentation is from April 11, 2013 and contains several screenshots of a graphical user interface in which NSA analysts can see where cyber attacks occur. The map part seems very similar to a well-known flashy visualisation on the website of the Norwegian cyber security company Norse:

It was decided not to publish the full MONSTERMIND/CYBERCOP presentation, but the documentary Schlachtfeld Internet ("Battlefield Internet") did contain several slides, which showed that NSA is apparently powerful enough to trace such attacks and that therefore the agency must be present at numerous points on the internet. This was considered newsworthy enough to report about.

In the documentary itself it was explained that an analysis tool called CYBERCOP makes it possible for NSA to monitor "cyber war" in real time. The presentation described at least one specific attack: on April 10, 2012, the US federal banking system in New York was succesfully attacked by Iran, not directly, but through thousands of computers around the world, controlled through internet servers in Germany.

Broadcaster NDR published three slides of the CYBERCOP presentation here (pdf). Two of them show the CYBERCOP interface in a high resolution:

(click to enlarge)


The MONSTERMIND system was first disclosed in a very long interview that James Bamford had with Edward Snowden in August 2014. There, Snowden said that MONSTERMIND is a frightening program that automated "the process of hunting for the beginnings of a foreign cyberattack".

It could also automatically prevent attacks from entering the country, but its unique capability is that "instead of simply detecting and killing the malware at the point of entry, MonsterMind would automatically fire back, with no human involvement" - with the risk of hitting the wrong one, as Snowden warned.

The "killing" capability was also described in Eckert's documentary, but without mentioning the codename MONSTERMIND. It didn't became clear whether this just came from Snowden's recollection or that it's mentioned in the CYBERCOP presentation (or other documents).


Eavesdropping on chancellor Merkel

The journalists from Der Spiegel also found interesting things purely by accident. The cache of documents for example contained an NSA presentation from the Center for Content Extraction (CCE, unit designator T1221) about a system to automatically sort out interesting and useful parts of intercepted phone calls.

One slide of this presentation shows an example list of some chiefs of state (cos), among which German chancellor Angela Merkel was listed. The presentation was not about actual interception operations, but did provide an indication that Merkel had been a target:

Der Spiegel published this slide on March 29, 2014 and the full presentation (pdf) was released online in June 2014. That chancellor Merkel had been a target of NSA had already been revealed in October 2013, based upon a database entry that allegedly did not came from the Snowden documents, but from another and yet unidentified second source.

So far, it seems that this example from the chiefs-of-state list is the only confirmation of NSA's targeting of chancellor Merkel that came from the Snowden documents. The intercepted content published by Wikileaks is also supposed to be from the second source.


Some other issues

During and after the interview, Stark, Rosenbach and Eckert were also asked about various aspects of working with Snowden Documents:

- Contrary to some claims made by the US government, there seemed to be little danger that these documents could endanger the lives of operatives or other people. The work that NSA does is highly technical and therefore the documents hardly contain any names. Most of the names they do contain are of authors, not of operative field agents.

- Eckert found it disappointing that the documents had almost no code or malware signatures in them, which could have been useful to identify hacking operations conducted by the NSA (Eckert said the XKEYSCORE rules were not included in the set she received). Again this was because the documents were often for management and training purposes and contained information on a meta level instead of actual operational details.

- The journalists were aware of the fact that these presentations had to be judged according to their intended purpose and audience and that the audio of these presentations was of course absent, although some presentations came with speaker's notes, which proved to be useful. Important was also to that presentations will often have presented things in a positive way.

Finally, when asked about the future of the Snowden documents, the journalists thought that it could be good to make them available for scientific research, but that it's not up to them to decide. They were not in favor of making all the documents publicly available, like in the way Wikileaks used to do.