The existence of the websites emerged in various stages, some of which may refer to this network or to other closely related communications failure since the published information is sometimes not clear enough.
May 21, 2011: various Iranian news outlets reported that:
30 individual suspected of spying for the US were arrested and 42 CIA operatives were identified in connection with the network.
The network, which was set up by a considerable number of seasoned CIA operatives in several countries, attempted to trick citizens into spying for them under the guise of issuing visa, helping with permanent residency, and making job and study offers.
Iranian sources include:The news were picked up and repeated by Western outlets on the same day e.g.:At this point there were still no clear indications that the recruitment had been made with websites, however later revelations would later imply that.
December 2014: McClathy DC reported on “Intelligence, defense whistleblowers remain mired in broken system” that CIA contractor John A. Reidy had started raising concerns about the security of a communication systems used by the CIA and other sources mention that he started this in 2008[ref] The focus of the article is how he was then ignored and silenced for raising these concerns, which later turned out to be correct and leading to an intelligence catastrophe that started in 2010.[ref][ref][ref]web.archive.org/web/20150101173203/ This appears to have come out after a heavily redacted appeal by Reidy against the CIA from October 2014 came into McClathy’s possession.[ref] While Reidy’s disclosures were responsible and don’t give much away, given the little that they disclose it feels extremely likely that they were related to the same system we are interested in. Even heavily redacted, the few unredacted snippets of the appeal are pure gold and give a little bit of insight into the internal workings of the CIA. Some selections:
As our efforts increased, we started to notice anomalies in our operations and conflicting intelligence reporting that indicated that several of our operations had been compromised. The indications ranged from [ redacted ] to sources abruptly and without reason ceasing all communications with us.
These warning signs were alarming due to the fact that our officers were approaching sources using [operational technique] (ledger item 16)
When our efforts began, ultimate operational authority rested with us. The other component provided the finances for the operation while we gave the operational guidance and the country specific knowledge.
knew we had a massive intelligence failure on our hands. All of our assets [ redacted ] were in jeopardy.
All of this information was collected under the project cryptonym [cryptonym] (ledger item 52)
Meanwhile throughout 2010, I started to hear about catastrophic intelligence failures in the government office I formally worked for. More than one government employee reached out to me and notified me that the “nightmare scenario” I had described and tried to prevent had transpired. I was told that in upwards of 70% of our operations had been compromised.
August 2018: Foreign Policy reported at “Botched CIA Communications System Helped Blow Cover of Chinese Agents” that:
It was considered one of the CIA’s worst failures in decades: Over a two-year period starting in late 2010, Chinese authorities systematically dismantled the agency’s network of agents across the country, executing dozens of suspected U.S. spies. But since then, a question has loomed over the entire debacle. How were the Chinese able to roll up the network?
and:
U.S. intelligence officers were also able to identify digital links between the covert communications system and the U.S. government itself, according to one former official—links the Chinese agencies almost certainly found as well. These digital links would have made it relatively easy for China to deduce that the covert communications system was being used by the CIA. In fact, some of these links pointed back to parts of the CIA’s own website, according to the former official.
Although no clear mention of websites is made in that article, the fact that there were “links” back to the CIA website strongly suggests that the communication was done through websites.
The report also reveals that there was a temporary “interim system” that new sources would use while they were being vetted, but that it used the same style of system as the main system. It would be cool if we managed to identify which sites are interim or not somehow:
When CIA officers begin working with a new source, they often use an interim covert communications system—in case the person turns out to be a double agent.
This interim, or “throwaway,” system, an encrypted digital program, allows for remote communication between an intelligence officer and a source, but it is also separated from the main communications system used with vetted sources, reducing the risk if an asset goes bad.
Although they used some of the same coding, the interim system and the main covert communication platform used in China at this time were supposed to be clearly separated.
The usage of of Google dorking is then mentioned:
In fact, the Iranians used Google to identify the website the CIA was using to communicate with agents.
It seems to us that this would have been very difficult on the generically themed websites that we have found so far. This suggests the existence of a separate recruitment website network, perhaps the one reported in 2011 by Iran offering VISAs. It would be plausible that such network could link back to the CIA and other government websites. Recruited agents would only then later use the comms network to send information back. The target countries may have first found the recruitment network, and then injected double agents into it, who later came to know about the comms network. TODO: it would be awesome to find some of those recruitment websites!
Another very interesting mention is the platform had been over extended beyond its original domain application, which is in part why things went so catastrophically bad:
Former U.S. officials said the internet-based platform, which was first used in war zones in the Middle East, was not built to withstand the sophisticated counterintelligence efforts of a state actor like China or Iran. “It was never meant to be used long term for people to talk to sources,” said one former official. “The issue was that it was working well for too long, with too many people. But it was an elementary system.”
December 2018: a followup Yahoo News article “At the CIA, a fix to communications system that left trail of dead agents remains elusive” gives an interesting internal organizational overview of the failed operation:
As a result, many who are directly responsible for working with sources on the ground within the CIA’s Directorate of Operations are furious
The fiascos in Iran and China continue to be sticking points between the Directorate of Operations and the CIA’s Directorate of Science and Technology (DS&T) — the technical scientists. “There is a disconnect between the two directorates,” said one former CIA official. “I’m not sure that will be fixed anytime soon.”
Entire careers in the CIA’s Office of Technical Service — the part of DS&T directly responsible for developing covert communications systems — were built on these internet-based systems, said a former senior official. Raising concerns about them was “like calling someone’s baby ugly,” said this person.
Much as in the case of Reidy, it is partly because of such internal dissatisfaction that so much has come out to the press, as agents feel that they have nowhere else to turn to.
The most important thing that this article gave were screenshots of nine websites, including the domain names of two of them: iraniangoals.com and iraniangoalkicks.com:
In addition, some sites bore strikingly similar names. For example, while Hosseini was communicating with the CIA through Iraniangoals.com, a site named Iraniangoalkicks.com was built for another informant. At least two dozen of the 350-plus sites produced by the CIA appeared to be messaging platforms for Iranian operatives, the analysts found.
The “350-plus” number is a bit random, given that their own analysts stated a much higher 885 in their report.
The article also reveals the critical flaw of the system; the usage of sequential IPs:
Online records they analyzed reveal the hosting space for these front websites was often purchased in bulk by the dozen, often from the same internet providers, on the same server space. The result was that numerical identifiers, or IP addresses, for many of these websites were sequential, much like houses on the same street.
It also mentions that other countries besides Iran and Chine were also likely targeted:
One of the most important information given in that report is the large number of sites found, 885, and the fact that they are available on Wayback Machine:The million dollar question is “which website did they use” and “how much does it cost if anything” since our investigation has so far had to piece together a few different hacky sources but didn’t spend any money. And a lot of money could be poured into this, e.g. DomainTools which might contain one of the largest historical databases seems to start at 15k USD / 1000 queries. One way to try and deduce which website they used is to look through their other research, e.g.:
They describe the most common subject matters and language of the websites:
They also give the dates range in which the system was active, which is very helpful for better targeting our searches:
The bulk of the websites that we discovered were active at various periods between 2004 and 2013.
And then a bomb, they claim to have found information regarding specific officers:This basically implies that they must have either
found some communication layer level identifier, e.g. domain name registration HTTPS certificate certificate because it is impossible to believe that real agent names would have been present on the website content itself!
or they may be instead talking about a separate recruitment network which offered the VISAs which we conjecture might have existed but currently have no examples of, and which might conceivably contain real embassy contacts
We have so for not yet found any such clear references to real individuals.
Chris, get fucked.
Thankfully however, either by carelessness or intentionally, this was easy to do by inspecting the address of the screenshots provided. For example, one of the URLs was:
The next step was to use our knowledge of the sequential IP flaw to look for more neighbor websites to the nine we knew of.
This was not so easy to do because the websites are down and so it requires historical data. But for our luck we found viewdns.info which allowed for 200 free historical queries (and they seem to have since removed this hard limit and moved to only throttling), leading to the discovery or some or our own new domains!
This gave us a larger website sample size in the order of the tens, which allowed us to better grasp more of the possible different styles of website and have a much better idea of what a good fingerprint would look like.
The next major and difficult step would be to find new IP ranges.
This was and still is a hacky heuristic process for us, but we’ve had the most success with the following methods:
Finally, at the very end of our pipeline, we were left with a a few hundred domains, and we just manually inspected them one by one as far as patience would allow it to confirm or discard the.
This section contains some of the most interesting and a few representative screenshots of the websites found.
Here are the websites likely targeting democracies based on their language and content found so far, defining a democracy as a country with score 7.0 or more in the Democracy index 2010:
France (6: affairesdumonde.com, guide-daventure.com, lesummumdelafinance.com, football-de-luxe.com, romulusactualites.com, suparakuvi.com)
In English, so more deniable:”Almost democracies”:Ciro couldn’t help but feel as if looking through the Eyes of Sauron himself!
Snowden’s 2013 revelations particularly shocked USA “allies” with the fact that they were being spied upon, and as of the 2020’s, everybody knows this and has “stopped caring”, and or moved to end-to-end encryption by default. This is beautifully illustrated in the Snowden when Snowden talks about his time in Japan working for Dell as an undercover NSA operative:
NSA wanted to impress the Japanese. Show them our reach. They loved the live video from drones. This is Pakistan right now . They were not as excited about that we wanted their help to spy on the Japanese population. They said it was against their laws.
And we did not stop there. Once we had their communications we continued with the physical infrastructure. We sneaked into small programs in their power grids, dams, hospitals. The idea was that if Japan one day was not our allies we could turn off the lights.
Another noteworthy scene from that movie is Video 1. “Aptitude test on communication networks scene from the 2016 Snowden film”, where a bunch of new CIA recruits are told that:
Each of you is going to build a covert communications network in your home city [i.e. their fictitious foreign target location written on each person’s desk such as Berlin, Istanbul and Bangkok, not necessarily where they were actually born], you’re going to deploy it, backup your site, destroy it, and restore it again.
This section contains a list of all the websites that we consider belong to the network beyond reasonable doubt.
This section is about possible real-world information leaks found in the HTML of the pages. Domain DNS metadata may of course expose more, and is more likely to do so, this section is only about in-page findings, notably in the HTML.
We haven found much so far, but the ones we have are curious.
A similar thing happened to alljohnny.com
A few separate websites have an archive with the same pid parameter:
It is unclear what it means. All of them contain something like:
Error. Page cannot be displayed. Please contact your service provider for more details. (11)
so looks like an archival artifact only.
This section tries to explain how the discoveries were made in more detail.
Some of the subsections are quite readable, while others are mostly data dumps and work logs, so bear with us.
Oleg Shakirov later discovered that the Carson one had its domain written right on the screenshot, as part of a watermark present on the original website itself. Therefore the URLs of all the websites were in one way or another essentially given on the article.
The full list of domains from screenshots is:
From The Reuters websites and others we’ve found, we can establish see some clear stylistic trends across the websites which would allow us to find other likely candidates upon inspection:
natural sounding, sometimes long-ish, domain names generally with 2 or 3 full words. Most in English language, but a few in Spanish, and very few in other languages like French.
shallow websites with a few tabs, many external links, sometimes many images, and few internal pages
common themes include:
.com and .net top-level domains, plus a few other very rare non .com .net TLDs, notably .info and .org
each one has one “communication mechanism file”: communication mechanisms
narrow page width like in the days of old, lots of images
split header images
some common pattern they follow in their news lists:
The most notable dissonance from the rest of the web is that there are no commercial looking website of companies, presumably because it was felt that it would be possible to verify the existence of such companies.
Most domains are the only domain for its IP, i.e. the websites are mostly private hosted. However we have later found many exceptions to this general indicator, so it should not be used as a strong exclusion rule.
It would be fun to actually reverse search into one of their stock image provider’s original images. Ones we’ve found:
Some possibly interesting searches include:
list all HTML comments, maybe something spicy was left over:
Varios of the non-English websites seem to have comments translating the content e.g.:
./noticiasmusica.net/20101230165001/index.html:
Alguns dos Melhores Sites Nacionais
This feels like it could be the translation helping the technical webdev team know what is what.
Some URLs existed both in HTML and .php extension, or were converted at some point:
It is impossible to tell if these were oversights, or intentional to simulate common web development quircks. But they are cute in any case.
One promising way to find more of those would be with IP searches, since it was stated in the Reuters article that the CIA made the terrible mistake of using several contiguous IP blocks for those website. What a phenomenal OPSEC failure!!!
Our current results indicate that the typical IP range is about 30 IPs wide.
E.g. searching: viewdns.info/iphistory and considering only hits from 2011 or earlier we obtain:
capture-nature.com
activegaminginfo.com
iraniangoals.com
rastadirect.net
iraniangoalkicks.com
headlines2day.com
118.139.174.1 – Singapore – Web Hosting Service – 2013-06-30. Source: viewdns.info
184.168.221.91 2013-08-12T06:17:39. Source: 2013 DNS Census grep
Ciro then tried some of the other IPs, and soon hit gold.
Summaries of the IP range exploration done so far follows, combined data from all databases above.
Here we list of suspected domains for which the correct IP was apparently not found since there are no neighbouring hits.
These are suspicious, and suggest either that we didn’t obtain the correct reverse IP, or a change in CIA methodology from an older time at which they were not yet using the obscene IP ranges.
For example, in the case of inews-today.com, 2013 DNS Census gave one IP 193.203.49.212, but then viewdns.info gave another one 66.175.106.146 which fit into an existing IP range, and which assumed to be the correct IP of interest.
It is also possible that some of them are simply false positives so they should be taken with a grain of salt. Further reverse engineering e.g. of comms or HTML analysis might be able to exclude some of them.
It is interesting to note that Reuters seems to have featured disproportionately many hits from that range, one wonders why that happened. It is possible that they chose these because they actually didn’t have any nearby hits to give away less obvious information, though they did pick some from the ranges as wel.
In what follows we list the domains with possible reverse IPs and what was explored so far for each. We consider IPs not in a range to be uncertain, and that instead their domains might have been previously in a range which we
dailynewsandsports.com. Found with: 2013 DNS Census virtual host cleanup heuristic keyword searches
216.119.129.94. rdns source: viewdns.info “location”: “United States”, “owner”: “A2 Hosting, Inc.”, “lastseen”: “2012-04-13”. Tested viewdns.info range: 216.119.129.85 – 216.119.129.86, 216.119.129.89 – 216.119.129.99, ran out of queries for 87 and 88
216.119.129.90: eastdairies.com 2011-04-04. Promising name and date, but no archives alas.
216.119.129.97: miideaco.com 2016-02-01
216.119.129.114 Found with: 2013 DNS Census virtual host cleanup heuristic keyword searches, also present on viewdns.info but at a later date from previous “location”: “United States”, “owner”: “A2 Hosting, Inc.”, “lastseen”: “2013-11-29”. Tested viewdns.info range: 216.119.129.109 – 216.119.129.119
216.119.129.110: dommoejmechty.com.ua. Legit.
216.119.129.111: dailybeatz.com: Legit
216.119.129.113:
audreygeneve.com
reyzheng.com
jacintorey.com
216.119.129.114: dailynewsandsports.com. hit.
216.119.129.115: afxchange.com legit/broken
216.119.129.116: danafunkfinancial.com: legit
208.73.33.194 on securitytrails.com
football-enthusiast.com:
212.4.18.14: Tested viewdns.info range: 212.4.18.1 – 212.4.18.29. This is a curious case, rather close to 212.4.18.129 sightseeingnews.com, but not quite in the same range apparently. Viewdns.info also agrees on its history with only “212.4.18.14”, “location” : “Milan – Italy”, “owner” : “MCI Worldcom Italy Spa”, “lastseen” : “2013-06-30” of interest.
farsi-newsandweather.com:
Likely hits possible but whose archives is too broken to be easily certain. If:were to ever be found, these would be considered hits.
todaysolar.com. This might just be legit, but keeping it around just in case.
62.22.60.49: telecom-headlines.com. UUNET in Spain. Found with: visual inspection of full 2013 DNS Census virtual host cleanup list just before worldnewsnetworking.com. Tested viewdns.info range: 62.22.60.34 – 62.22.60.66
62.22.60.33: newsperk.com. Almost certainly a hit. Stylistically perfect, rss-item. But no comms not found. Ennerving! 2011. English. Egypt. news. Later legitimately reused.
62.22.60.34: freeslideshow.net. Legit? Attempting to open any HTML archives leads to an infinite page load loop, e.g. 2010. A subpage however exists: web.archive.org/web/20101230001640/ and appears legit.
65.61.127.165: travelconnectionsonline.com. Ciro initially though this might be a hit. But upon Googling it, there’s now a mirror at: travelconn.tripod.com/. Combined with the lack of a standard communications mechanism and the 2001 copyright, maybe it isn’t a hit after all
65.61.127.222: asianwhitecoffee.com 2012-07-16T09:21:05 web.archive.org/web/20110903080036/ Could be legit.
66.45.179.205 noticiasporjanua.com. ADHOST in Edmonds – United States. Found with: 2013 DNS Census virtual host cleanup. Tested viewdns.info range: 66.45.179.187 – 66.45.179.223
66.45.179.187: mail03.gatesfoundation.org. Legit.
66.45.179.192: thegraceofislam.com. Hit.
66.45.179.193: arabicnewsunfiltered.com. Hit.
66.45.179.194: raulsonsglobalnews.com. Hit.
66.45.179.195: aryannews.net. Hit.
66.45.179.199: attivitaestremi.com. Hit.
66.45.179.200: foodwineandsuch.com. Hit.
66.45.179.201: hitthepavementnow.com. Hit.
66.45.179.203: noticiascontinental.com. Hit.
66.45.179.205: noticiasporjanua.com. Hit.
66.45.179.206: podisticamondiale.com. Hit.
66.45.179.207: reflectordenoticias.com. Hit.
66.45.179.208: havenofgamerz.com. Hit.
66.45.179.209: vejaaeuropa.com. Hit.
66.45.179.210: sa-michigan.com. Hit.
66.45.179.211: absolutebearing.net. Hit.
66.45.179.212: grandretirement.net. No archives. cqcounter.com/whois/www/grandretirement.net.html blank image.
66.45.179.213: myportaltonews.com. Hit.
66.45.179.214: investmentintellect.com. Hit.
66.45.179.215: nigeriastar.net 2012-03-12. Hit.
66.104.173.186 myworldlymusic.com. XO-AS15 in United States. Found with: 2013 DNS Census virtual host cleanup heuristic keyword searches. Tested viewdns.info range: 66.104.173.158 – 66.104.173.194
66.104.173.161: fanatic-pc-gamers.com. domainsbyproxy.com. 2013: Welcome to the US Petabox. cqcounter.com/whois/www/fanatic-pc-gamers.com.html somewhat in-style with large “Login to our Members Forum” message and copyright 2005.
66.104.173.163: runakonews.com. Hit.
66.104.173.164: shoppingadventure.net. Hit.
66.104.173.165: entertaining-ly.com. Hit. Network Solutions, LLC for Matthew Sorrell. tools.whoisxmlapi.com/reverse-whois-search hits:
66.175.106.140: aroundthemiddleeast.com. No Wayback Machine hits. Last resolved: 2012-06-29. cqcounter.com/whois/www/aroundthemiddleeast.com.html not found.
66.175.106.146: inews-today.com. Initially found with 2013 DNS Census virtual host cleanup heuristic keyword searches which gave IP address 193.203.49.212. But that has no nearby hits. 66.175.106.146 was later found on viewdns.info, and slotted into this other existing IP range.
193.203.49.211 datingso.com: legit? Russian dating website
193.203.49.212 inews-today.com. Hit.
193.203.49.223 zatysi.net: legit
193.203.49.226 kinotopik.com: legit? Russian
193.203.49.229 rotor-volgograd.com. Legit.
193.203.49.233 ordercytotec.com. Broken. cqcounter.com/whois/www/ordercytotec.com.html not found.
66.175.106.147: starwarsweb.net. Hit.
66.175.106.148: activegaminginfo.com. Hit. Network Solutions, LLC for Elizabeth Corral. tools.whoisxmlapi.com/reverse-whois-search reverse search “Corral, Elizabeth” only has that hit
66.175.106.149: feedsdemexicoyelmundo.com. Hit.
66.175.106.150: noticiasmusica.net. Hit. Network Solutions, LLC for Megan See. tools.whoisxmlapi.com/reverse-whois-search only this hit.
tools.whoisxmlapi.com/reverse-whois-search search for “Alger, Jennifer” has four domain:but more interestingly this address is the same as other hits: activegameinfo.com and noticiasmusica.net! “PO Box 459” anywhere search has 10k+ domains and so does Drums so not helping.
72.34.53.174 technologytodayandtomorrow.com. IHNET in United States. This IP is special. This IP is somehow closely linked to the “Mass Deface III” pastebin as it seems to have been hosted by Condor hosting. They also have many old sites, and links to Russia which is apparently where this was hosted.
68.178.232.100 United States AS-26496-GO-DADDY-COM-LLC 2011-11-13 virtual
72.34.53.174 United States IHNET 2011-09-08. Tested viewdns.info range: 72.34.53.164 72.34.53.184 viewdns.info/reverseip/?t=1&host=72.34.53.174 went through all of them;
possible hits * intloil.org 2012-04-27. 2011, Possible hit, a bit off style, but possibly because too broken. rss-item. Copyright 2005. Present at pastebin.com/CTXnhjeSp (now lost without archives I’m an idiot). cqcounter.com/whois/www/intloil.org.html from 2011 somewhat in style but interestingly also similarly broken. The “Login” button leads to another domain: “condorsecure.com”: web.archive.org/web/20110721052801/ which is megaweird and is what is mentioned in the “Mass Deface III” pastebin. domainsbyproxy.com. A similar thing happens in europeantravelcafe.com but to another domain. * islamicnewsonline.com 2013-03-23. No archives in date range. cqcounter.com/whois/www/islamicnewsonline.com.html not found, sad
not hits
securitytrails.com/domain/technologytodayandtomorrow.com/history/a same
173.208.81.2 LEASEWEB-USA-CHI in Lombard – United States:
weblognewsinfo.com:
newsincirculation.com
199.85.212.118 just-kidding-news.com. ATT-INTERNET4 in United States.
199.85.212.118 rdns source: 2013 DNS Census virtual host cleanup heuristic keyword searches, dnshistory.org (2009-09-23 -> 2011-01-25) and viewdns.info: “location”: “United States”, “owner”: “VIMRO, LLC”, “lastseen”: “2012-01-11”. Tested viewdns.info range: 199.85.212.95 – 199.85.212.128. Not sure worth it given the many 2013 DNS Census misses surrounding.
209.85.45.160: bieltvedt.net. No archives of time.
209.85.45.160: golfstats.dk. No archives.
209.85.45.225: infokus.ca
209.85.45.225: mail.tomlatham.net
209.85.45.225: mail.tomlatham.org
209.85.45.239: flavacationcenter.com
204.176.38.143 noticiassofisticadas.com. UUNET in United States. Found with: 2013 DNS Census virtual host cleanup. Tested viewdns.info range: 204.176.38.125 – 204.176.38.154
204.176.38.130: i-pressnews.com. Hit.
204.176.38.132: turkishnewslinks.com. Hit.
204.176.38.134: photographyarecord.com. Hit.
204.176.38.135: breakingthewicket.com. Hit.
204.176.38.136: politicalworldtoday.com. Hit.
204.176.38.137: hi-tech-today.com. Hit.
204.176.38.138: continental-business-news.com. TODO. rss-item, split images. 2011. Cannot find comms. Also header and footer are not limited width which is unusual. Further HTML similarity reversing would be needed.
204.176.38.139: bigscreenbattles.com. Hit.
204.176.38.141: rakotafootball.com. Hit.
204.176.38.142: senderosdemontana.com. Hit.
204.176.38.143: noticiassofisticadas.com. Hit.
204.176.38.144: techno-today.com. Hit.
204.176.38.145: tickettonews.com. Hit.
204.176.38.146: dps-digitalphotosharing.com. Hit.
204.176.38.147: theputtingreen.com. Hit.
204.176.38.149: sportsnewstodayar.com. Hit.
204.176.38.150: kairuafricanews.com. Hit.
204.176.39.115 globalprovincesnews.com. UUNET in United States. Tested viewdns.info range: 204.176.39.93 – 204.176.39.124
207.150.191.68 technologypresstoday.com. Saudi Telecom Company JSC in Saudi Arabia.
208.93.112.105 fastnews-online.com. TULIP-SYSTEMS in United States. Checked viewdns.info range: 208.93.112.90 – 208.93.112.155
208.254.38.39 todaysengineering.com. COLO-PREM-VZB in United States.
Tested viewdns.info range: 208.254.38.9 – 208.254.38.86. Weirdly empty, doesn’t even show the domain iteslf!
208.254.40.117 worldnewsandent.com. COLO-PREM-VZB in United States. whois.arin.net/rest/net/NET-208-192-0-0-1/pft?s=208.254.40.117: Net Range 208.192.0.0 – 208.255.255.255. Tested viewdns.info range: 208.254.40.92 – 208.254.40.135
208.254.40.96: sixty2media.com. Hit.
208.254.40.99: newspoliticssource.com. Hit.
208.254.40.110 musical-fortune.net. Hit.
208.254.40.113: ashoka-gemstones.com. Hit.
208.254.40.117: worldnewsandent.com. Hit.
208.254.40.124: riskandrewardnews.com. Hit.
208.254.40.129: mailb.casella.com. Legit.
208.254.42.205 driversinternationalgolf.com. COLO-PREM-VZB in United States. Tested viewdns.info range: 208.254.42.178 – 208.254.42.233.
210.80.75.35: aroundtheworldnews.net. No archives. ipinf.ru/domains/210.80.75.33/ disagrees and places it at .33.
210.80.75.36: e-commodities.net. Hit.
210.80.75.37: trekkingtoday.com. Hit.
210.80.75.41: multinews-33.com. Hit.
210.80.75.42: movimientodenticias.com. No archives. cqcounter.com/whois/www/movimientodenticias.com.html blank.
210.80.75.43: gulfandmiddleeastnews.com. Hit.
210.80.75.44: whirlybirdinflight.com. Hit.
210.80.75.45: kings-game.net. Hit.
210.80.75.46: topglobalnewsdaily.com. Hit.
210.80.75.49: recipe-dujour.com. Hit.
210.80.75.53: sportsman-elite.com. Hit.
210.80.75.55: philippinenewsonline.net. Hit.
210.80.75.56: technewsforme.com. Hit.
210.80.75.59: goldeportesnoticias.com. Hit.
210.80.75.68: gigabyte-usa.com. Legit.
212.4.17.38 fightwithoutrules.com. UUNET in Cassano d’Adda – Italy. whois.arin.net/rest/net/NET-208-192-0-0-1/pft?s=208.254.40.117. Net Range: 208.192.0.0 – 208.255.255.255. Organization: Name: Verizon Business. Tested viewdns.info range: see 212.4.16.* above
212.4.17.38: fightwithoutrules.com. Hit.
212.4.17.41: newtechfrontier.com. Hit.
212.4.17.43: smart-travel-consultant.com. Hit.
212.4.17.46: atentlaloc.com. Hit.
212.4.17.53: newsresolution.net. Hit.
212.4.17.56: lesummumdelafinance.com. Hit.
212.4.17.56: thepinnacleoffinance.com. No Wayback machine archives. cqcounter.com/whois/www/thepinnacleoffinance.com.html blank.
212.4.17.61: tech-stop.org. Archive: 2011. Feels likely. No commons found. .org hit? Has subdomain “gear.tech-stop.org” according to 2013 DNS Census, which suggests CGI comms, but no links to it
212.4.17.98: topbillingsite.com. Hit.
212.4.17.122: b2bworldglobal.com. Hit.
212.4.17.125: worldaroundyunnan.com. Hit.
212.4.17.160: localtoglobalnews.com. Hit.
There were also some other reverse IP hits for fightwithoutrules.com, but no CIA websites there:
204.11.56.25 – British Virgin Islands – Confluence Networks Inc – 2013-09-26. Many domains.
208.91.197.19 – British Virgin Islands – Confluence Networks Inc – 2013-05-20. Many domains.
Other hits:
208.91.197.132. rdns source: viewdns.info: “location” : “British Virgin Islands”, “owner” : “Confluence Networks Inc”, “lastseen” : “2013-09-26”. So this is after the previous one, unlikely to be correct.
205.178.189.131. source: securitytrails.com
212.209.79.40 hydradraco.com. UUNET in Sweden. Found with: visual inspection of full 2013 DNS Census virtual host cleanup list just after globalbaseballnews.com. Tested viewdns.info range: 212.209.79.35 – 212.209.79.63
212.209.79.34: fgnl.net. Hit. securitytrails.com provides IP history:
both under MCI Communications Services, Inc. d/b/a Verizon Business.
212.209.79.37: fitness-sources.com. Hit.
212.209.79.40: hydradraco.com. Hit.
212.209.79.41: noticiasdelmundolatino.com. Hit.
212.209.79.42: suparakuvi.com. Hit.
212.209.79.44: myigadgets.net. Unclear. 2010. tech. Contains some helpers to: iGoogle. This page is very interesting. and quite different from the others, as it contains highly specialized functionality. No known comms found. The choice of homepage languages is also very suspicious: Arabic, Farsi, French, Chinese and Spanish.
212.209.79.52: newel-adserver.com. Redirects to newel.com which is legit. cqcounter.com/whois/www/newel-adserver.com.html blank.
212.209.79.53: ourscubaworld.com. Hit.
212.209.79.58: tech-love-home.com. Hit.
212.209.79.60: first-solo-aviation.com. Hit.
212.209.79.61: china-destinations.org. Hit.
216.93.248.194 esmundonoticias.com. TWDX in Chelmsford – United States.
216.105.98.152: modernarabicnews.com. SAVVY-NET in United States. Found with: 2013 DNS Census virtual host cleanup heuristic keyword searches. Tested viewdns.info range: 216.105.98.125 – 216.105.98.167
216.105.98.118:
216.105.98.132: europeantravelcafe.com. Hit.
216.105.98.134: fuenteneta.com. Hit.
216.105.98.135: ilat-news.com. Hit.
216.105.98.136: etherealinspirations.net. Hit.
216.105.98.137: the-news-zone.com. Hit.
216.105.98.138: photozoomnews.com. No archives. cqcounter.com/whois/www/photozoomnews.com.html empty
216.105.98.139: cultura-digital.net. Hit.
216.105.98.140: uaeshoppingspree.com. Hit.
216.105.98.141: jabarifootball.com. No archives. “Jabari” is a Swahili/Arabic name[ref]. cqcounter.com/whois/www/jabarifootball.com.html not found.
216.105.98.142: globalreview-ar.com. No archives. Shame, could have been our first Argentinian site. cqcounter.com/whois/www/globalreview-ar.com.html empty.
216.105.98.144: garanziadellasicurezza.com. Hit.
216.105.98.145: montanismoaventura.com. Hit.
216.105.98.146: large-format-news.com. Hit.
216.105.98.147: nepalnewsbrief.com. Hit. dnshistory.org marks it as having IP 2010-03-10 -> 2010-08-15 216.169.148.94 [ref]. This range does feel a bit different from the others, too many broken archives, and relatively early ones too. Explored viewdns.info range: 216.169.148.84 – 216.169.148.104, empty for period. domainsbyproxy.com.
216.105.98.148: teclafinance.com. Hit.
216.105.98.149: entreman.com. Hit.
216.105.98.152: modernarabicnews.com. Hit.
216.105.98.153: global-headlines.com. Hit.
216.105.98.154: everythingcricket.org. Hit.
216.105.98.156: familyhealthonline.net. Hit.
216.105.98.157: delacorne.com. Hit.
216.105.98.158: econfutures.com. Hit.
216.105.98.161: kstcloud.com. No archives. cqcounter.com/whois/www/kstcloud.com.html not found
All IP ranges have some holes in them for which we don’t have a domain name.
It is because there was nothing there, or just because we don’t have a good enough reverse IP database?
Censys is another option that would be good to try.
Putting 140 USD into WhoisXMLAPI to get all whois histories of interest for possible reverse searches would also be of interest.
It can’t be HTML crawl because presumably there wouldn’t have been links to those websites? Presumably this is why Common Crawl doesn’t seem to have any hits.
So they must have had some kind of DNS A record database?
Or would IPv4 sweep have worked, without the Host header with the CIA’s setup?
The same question also applies to the 2013 DNS Census. It has less hits, but still has many.
Whatever they did, we are so so glad that they did!
Others that had been previously found in IP ranges but without clear comms:
65.61.127.177: material-science.org
212.4.17.61: tech-stop.org
74.116.72.244 arborstribune.org
.org is very rare, and has been excluded from some of our search heuristics. That was a shame, but likely not much was missed.
This is a dark art, and many of the sources are shady as fuck! We often have no idea of their methodology. Also no source is fully complete. We just piece up as best we can.
This is our primary data source, the first article that pointed out a few specific CIA websites which then served as the basis for all of our research.
We take the truth of this article as an axiom. And then all we claim is that all other websites found were made by the same people due to strong shared design principles of the such websites.
The Common Crawl project attempts in part to address this lack of querriability, but we haven’t managed to extract any hits from it.
This allows to filter down 10 thousands of possible domains in a few hours. But 100s of thousands would be too much. This is because you have to query exactly one URL at a time, and they possibly rate limit IPs. But no IP blacklisting so far after several hours, so it’s not that bad.
From then on, you can just manually inspect for hist on your browser.
Since archive is so abysmal in its data access, e.g. a Google BigQuery would solve our issues in seconds, we have to come up with creative ways of getting around their IP throttling.
The CIA doesn’t play fair. They’re actually the exact opposite of fair. So neither shall we.
This should allow a full sweep of the 4.5M records in 2013 DNS Census virtual host cleanup in a reasonable amount of time. After JAR/SWF/CGI filtering we obtained 5.8k domains, so a reduction factor of about 1 million with likely very few losses. Not bad.
5.8k is still a bit annoying to fully go over however, so we can also try to count CDX hits to the domains and remove anything with too many hits, since the CIA websites basically have very few archives:
cd 2013-dns-census-a-novirt-domains.txt.cdx
./cdx-tor.sh -d out.post domain-list.txt
cd out.post.cdx
cut -d' ' -f1 out | uniq -c | sort -k1 -n | awk 'match($2, /([^,]+),([^)]+)/, a) {printf("%s.%s %d\n", a[2], a[1], $1)}' > out.count
This ignores the obviously atypical JavaScript with SHAs from iranfootballsource, and the particularly small old menu.js from cutabovenews.com, which we embed into cia-2010-covert-communication-websites/cdx-post-js.sh.
The size helps a bit, but it’s not insanely good unfortunately, only about 3x, these are some common JS sizes right there!
Accounts used so far: 6 (1500 reverse IP checks).
Their historic DNS and reverse DNS info was very valuable, and served as Ciro’s the initial entry point to finding hits in the IP ranges given by Reuters.
Since this source is so scarce and valuable, we have been quite careful to note down all the domain and IP ranges that have been explored.
For domain to IP queries from the API you should use “iphistory” viewdns.info/api/docs/ip-history.php:
TODO can they do historical reverse IP or not? I.e. determine which domains were hosted on a given IP at a given date in the past?
Hit overlap:
jq -r '.[].host' ../media/cia-2010-covert-communication-websites/hits.json ) | xargs -I{} sqlite3 aiddcu.sqlite "select * from t where d = '{}'"
Domain hit count when we were at 279 hits: 142 hits, so about half of the hits were present.
The timing of the database is perfect for this project, it is as if the CIA had planted it themselves!
We’ve noticed that often when there is a hit range:
there is only one IP for each domain
there is a range of about 20-30 of those
and that this does not seem to be that common. Let’s see if that is a reasonable fingerprint or not.
Note that although this is the most common case, we have found multiple hits that viewdns.info maps to the same IP.
First we create a table u (unique) that only have domains which are the only domain for an IP, let’s see by how much that lowers the 191 M total unique domains:
time sqlite3 u.sqlite 'create table t (d text, i text)'
time sqlite3 av.sqlite -cmd "attach 'u.sqlite' as u" "insert into u.t select min(d) as d, min(i) as i from t where d not like '%.%.%' group by i having count(distinct d) = 1"
The not like '%.%.%' removes subdomains from the counts so that CGI comms are still included, and distinct in count(distinct is because we have multiple entries at different timestamps for some of the hits.
Let’s start with the 208 subset to see how it goes:
time sqlite3 av.sqlite -cmd "attach 'u.sqlite' as u" "insert into u.t select min(d) as d, min(i) as i from t where i glob '208.*' and d not like '%.%.%' and (d like '%.com' or d like '%.net') group by i having count(distinct d) = 1"
OK, after we fixed bugs with the above we are down to 4 million lines with unique domain/IP pairs and which contains all of the original hits! Almost certainly more are to be found!
This data is so valuable that we’ve decided to upload it to: archive.org/details/2013-dns-census-a-novirt.csv Format:
Which gives the following useless noise, there is basically no pattern:
grep -e news -e noticias -e nouvelles -e world -e global
iran + football:
iranfootballsource.com: the third hit for this area after the two given by Reuters! Epic.
3 easy hits with “noticias” (news in Portuguese or Spanish”), uncovering two brand new ip ranges:
66.45.179.205 noticiasporjanua.com
66.237.236.247 comunidaddenoticias.com
204.176.38.143 noticiassofisticadas.com
Let’s see some French “nouvelles/actualites” for those tumultuous Maghrebis:
216.97.231.56 nouvelles-d-aujourdhuis.com
news + world:
210.80.75.55 philippinenewsonline.net
news + global:
204.176.39.115 globalprovincesnews.com
212.209.74.105 globalbaseballnews.com
212.209.79.40: hydradraco.com
OK, I’ve decided to do a complete Wayback Machine CDX scanning of news… Searching for .JAR or https.*cgi-bin.*\.cgi are killers, particularly the .jar hits, here’s what came out:
62.22.60.49 telecom-headlines.com
62.22.61.206 worldnewsnetworking.com
64.16.204.55 holein1news.com
66.104.169.184 bcenews.com
69.84.156.90 stickshiftnews.com
74.116.72.236 techtopnews.com
74.254.12.168 non-stop-news.net
193.203.49.212 inews-today.com
199.85.212.118 just-kidding-news.com
207.210.250.132 aeronet-news.com
212.4.18.129 sightseeingnews.com
212.209.90.84 thenewseditor.com
216.105.98.152 modernarabicnews.com
“headline”: only 140 matches in 2013-dns-census-a-novirt.csv and 3 hits out of 269 hits. Full inspection without CDX led to no new hits.
“today”: only 3.5k matches in 2013-dns-census-a-novirt.csv and 12 hits out of 269 hits, TODO how many on those on 2013-dns-census-a-novirt? No new hits.
“world”, “global”, “international”, and spanish/portuguese/French versions like “mondo”, “mundo”, “mondi”: 15k matches in 2013-dns-census-a-novirt.csv. No new hits.
Let’ see if there’s anything in records/mx.xz.
mx.csv is 21GB.
They do have " in the files to escape commas so:
then:
# uniq not amazing as there are often two or three slightly different records repeated on multiple timestamps, but down to 11 GB
python3 mx.py | uniq > mx-uniq.csv
sqlite3 mx.sqlite 'create table t(d text, m text)'
# 13 GB
time sqlite3 mx.sqlite ".import --csv --skip 1 'mx-uniq.csv' t"
# 41 GB
time sqlite3 mx.sqlite 'create index td on t(d)'
time sqlite3 mx.sqlite 'create index tm on t(m)'
time sqlite3 mx.sqlite 'create index tdm on t(d, m)'
# Remove dupes.
# Rows: 150m
time sqlite3 mx.sqlite <
Let’s see what the hits use:
awk -F, 'NR>1{ print $2 }' ../media/cia-2010-covert-communication-websites/hits.csv | xargs -I{} sqlite3 mx.sqlite "select distinct * from t where d = '{}'"
At around 267 total hits, only 84 have MX records, and from those that do, almost all of them have exactly:
sqlite3 mx.sqlite "select count(*) from t where m = 'mailstore1.secureserver.net'"
which gives, ~18M, so nope, it is too much by itself…
Let’s try to use that to reduce av.sqlite from 2013 DNS Census virtual host cleanup a bit further:
time sqlite3 mx.sqlite '.mode csv' "attach 'aiddcu.sqlite' as 'av'" '.load ./ip' "select ipi2s(av.t.i), av.t.d from av.t inner join t as mx on av.t.d = mx.d and mx.m = 'mailstore1.secureserver.net' order by av.t.i asc" > avm.csv
where avm stands for av with mx pruning. This leaves us with only ~500k entries left. With one more figerprint we could do a Wayback Machine CDX scanning scan.
Let’s check that we still have most our hits in there:
so yeah, most of those are likely going to be humongous just by looking at the names.
The smallest ones by far from the total are: frienddns.ru with only 487 hits, all others quite large or fake hits due to CSV. Did a quick Wayback Machine CDX scanning there but no luck alas.
Let’s check the smaller ones:
inews-today.com,2013-08-12T03:14:01,ns1.frienddns.ru
source-commodities.net,2012-12-13T20:58:28,ns1.namecity.com -> fake hit due to grep e-commodities.net
dailynewsandsports.com,2013-08-13T08:36:28,ns3.a2hosting.com
just-kidding-news.com,2012-02-04T07:40:50,jns3.dailyrazor.com
fightwithoutrules.com,2012-11-09T01:17:40,sk.s2.ns1.ns92.kolmic.com
fightwithoutrules.com,2013-07-01T22:46:23,ns1625.ztomy.com
half-court.net,2012-09-10T09:49:15,sk.s2.ns1.ns92.kolmic.com
half-court.net,2013-07-07T00:31:12,ns1621.ztomy.com
We have not managed to extract much from this source, they don’t have as much data on the range of interest.
But they do have some unique data at least, perhaps we should try them a bit more often, e.g. they were the only source we’ve seen so far that made the association: headlines2day.com -> 212.209.74.126 which places it in the more plausible globalbaseballnews.com IP range.
TODO can it do IP to domain? Or just domain to IP? Asked on their Discord: discord.com/channels/698151879166918727/968586102493552731/1124254204257632377. Their banner suggests that yes:
With our new look website you can now find other domains hosted on the same IP address, your website neighbours and more even quicker than before.
Owner replied, you can’t:
At the moment you can only do this for current not historical records
This is a shame, reverse IP here could be quite valuable.
In principle, we could obtain this data from search engines, but Google doesn’t track that entire website well, e.g. no hits for site:dnshistory.org "62.22.60.48" presumably due to heavy IP throttling.
Homepage dnshistory.org/ gives date starting in 2009:and it is true that they do have some hits from that useful era.
They appear to piece together data from various sources. This is the most complete historical domain -> IP database we have so far. They don’t have hugely more data than viewdns.info, but many times do offer something new. It feels like the key difference is that their data goes further back in the critical time period a bit.
TODO do they have historical reverse IP? The fact that they don’t seem to have it suggests that they are just making historical reverse IP requests to a third party via some API?
But searching the IP 62.22.60.55 is empty and there’s no historical data option?
Account creation blacklists common email providers such as gmail to force users to use a “corporate” email address. But using random domains like ciro@cirosantilli.com works fine.
Their data seems to date back to 2008 for our searches.
So url_host_3rd_last_part might be a winner for CGI comms fingerprinting!
Naive one for one index:
select * from "ccindex"."ccindex" where url_host_registered_domain = 'conquermstoday.com' limit 100;
have no results… data scanned: 5.73 GB
Let’s see if they have any of the domain hits. Let’s also restrict by date to try and reduce the data scanned:
select * from "ccindex"."ccindex" where
fetch_time < TIMESTAMP '2014-01-01 00:00:00' AND
url_host_registered_domain IN (
'activegaminginfo.com',
'altworldnews.com',
...
'topbillingsite.com',
'worldwildlifeadventure.com'
)
Humm, data scanned: 60.59 GB and no hits… weird.
Sanity check:
select * from "ccindex"."ccindex" WHERE
crawl="CC-MAIN-2013-20" AND
subset="warc" AND
url_host_registered_domain IN (
'google.com',
'amazon.com'
)
has a bunch of hits of course. Data scanned: 212.88 MB, WHEREcrawl and subset are a must! Should have read the article first.
Let’s widen a bit more:
select * from "ccindex"."ccindex" WHERE
crawl IN (
'CC-MAIN-2013-20',
'CC-MAIN-2013-48',
'CC-MAIN-2014-10'
) AND
subset="warc" AND
url_host_registered_domain IN (
'activegaminginfo.com',
'altworldnews.com',
...
'worldnewsandent.com',
'worldwildlifeadventure.com'
)
Still nothing found… they don’t seem to have any of the URLs of interest?
We could not find anything useful in it so far, but there is great potential to use this tool to find new IP ranges based on properties of existing IP ranges. Part of the problem is that the dataset is huge, and is split by top 256 bytes. But it would be reasonable to at least explore ranges with pre-existing known hits…
We have started looking for patterns on 66.* and 208.*, both selected as two relatively far away ranges that have a number of pre-existing hits. 208 should likely have been 212 considering later finds that put several ranges in 212.
... similar down
208.254.40.95 1334668500 down no-response
208.254.40.95 1338270300 down no-response
208.254.40.95 1338839100 down no-response
208.254.40.95 1339361100 down no-response
208.254.40.95 1346391900 down no-response
208.254.40.96 1335806100 up unknown
208.254.40.96 1336979700 up unknown
208.254.40.96 1338840900 up unknown
208.254.40.96 1339454700 up unknown
208.254.40.96 1346778900 up echo-reply (0.34s latency).
208.254.40.96 1346838300 up echo-reply (0.30s latency).
208.254.40.97 1335840300 up unknown
208.254.40.97 1338446700 up unknown
208.254.40.97 1339334100 up unknown
208.254.40.97 1346658300 up echo-reply (0.26s latency).
... similar up
208.254.40.126 1335708900 up unknown
208.254.40.126 1338446700 up unknown
208.254.40.126 1339330500 up unknown
208.254.40.126 1346494500 up echo-reply (0.24s latency).
208.254.40.127 1335840300 up unknown
208.254.40.127 1337793300 up unknown
208.254.40.127 1338853500 up unknown
208.254.40.127 1346454900 up echo-reply (0.23s latency).
208.254.40.128 1335856500 up unknown
208.254.40.128 1338200100 down no-response
208.254.40.128 1338749100 down no-response
208.254.40.128 1339334100 down no-response
208.254.40.128 1346607900 down net-unreach
208.254.40.129 1335699900 up unknown
... similar down
Suggests exactly 127 – 96 + 1 = 31 IPs.
208.254.42:
... similar down
208.254.42.191 1334522700 down no-response
208.254.42.191 1335276900 down no-response
208.254.42.191 1335784500 down no-response
208.254.42.191 1337845500 down no-response
208.254.42.191 1338752700 down no-response
208.254.42.191 1339332300 down no-response
208.254.42.191 1346499900 down net-unreach
208.254.42.192 1334668500 up unknown
208.254.42.192 1336808700 up unknown
208.254.42.192 1339334100 up unknown
208.254.42.192 1346766300 up echo-reply (0.40s latency).
208.254.42.193 1335770100 up unknown
208.254.42.193 1338444900 up unknown
208.254.42.193 1339334100 up unknown
... similar up
208.254.42.221 1346517900 up echo-reply (0.19s latency).
208.254.42.222 1335708900 up unknown
208.254.42.222 1335708900 up unknown
208.254.42.222 1338066900 up unknown
208.254.42.222 1338747300 up unknown
208.254.42.222 1346872500 up echo-reply (0.27s latency).
208.254.42.223 1335773700 up unknown
208.254.42.223 1336949100 up unknown
208.254.42.223 1338750900 up unknown
208.254.42.223 1339334100 up unknown
208.254.42.223 1346854500 up echo-reply (0.13s latency).
208.254.42.224 1335665700 down no-response
208.254.42.224 1336567500 down no-response
208.254.42.224 1338840900 down no-response
208.254.42.224 1339425900 down no-response
208.254.42.224 1346494500 down time-exceeded
... similar down
Suggests exactly 223 – 192 + 1 = 31 IPs.
Let’s have a look at the file 68: outcome: no clear hits like on 208. One wonders why.
It does appears that long sequences of ranges are a sort of fingerprint. The question is how unique it would be.
First:
n=208
time awk '$3=="up"{ print $1 }' $n | uniq -c | sed -r 's/^ +//;s/ /,/' | tee $n-up-uniq
t=$n-up-uniq.sqlite
rm -f $t
time sqlite3 $t 'create table tmp(cnt text, i text)'
time sqlite3 $t ".import --csv $n-up-uniq tmp"
time sqlite3 $t 'create table t (i integer)'
time sqlite3 $t '.load ./ip' 'insert into t select str2ipv4(i) from tmp'
time sqlite3 $t 'drop table tmp'
time sqlite3 $t 'create index ti on t(i)'
This reduces us to 2 million IP rows from the total possible 16 million IPs.
OK now just counting hits on fixed windows has way too many results:
sqlite3 208-up-uniq.sqlite "\
SELECT * FROM (
SELECT min(i), COUNT(*) OVER (
ORDER BY i RANGE BETWEEN 15 PRECEDING AND 15 FOLLOWING
) as c FROM t
) WHERE c > 20 and c < 30
"
n=208
time awk '$3=="up"{ print $1 }' $n | uniq -c | sed -r 's/^ +//;s/ /,/' | tee $n-up-uniq-cnt
t=$n-up-uniq-cnt.sqlite
rm -f $t
time sqlite3 $t 'create table tmp(cnt text, i text)'
time sqlite3 $t ".import --csv $n-up-uniq-cnt tmp"
time sqlite3 $t 'create table t (cnt integer, i integer)'
time sqlite3 $t '.load ./ip' 'insert into t select cnt as integer, str2ipv4(i) from tmp'
time sqlite3 $t 'drop table tmp'
time sqlite3 $t 'create index ti on t(i)'
Let’s see how many consecutives with counts:
sqlite3 208-up-uniq-cnt.sqlite <= 3)
GROUP BY grp
ORDER BY i
) where c > 28 and c < 32
EOF
Let’s check on 66:
grep -e '66.45.179' -e '66.45.179' 66
not representative at all… e.g. several convfirmed hits are down:
66.45.179.215 1335305700 down no-response
66.45.179.215 1337579100 down no-response
66.45.179.215 1338765300 down no-response
66.45.179.215 1340271900 down no-response
66.45.179.215 1346813100 down no-response
Let’s check relevancy of known hits:
grep -e '208.254.40' -e '208.254.42' 208 | tee 208hits
Domain list only, no IPs and no dates. We haven’t been able to extract anything of interest from this source so far.
Domain hit count when we were at 69 hits: only 9, some of which had been since reused. Likely their data collection did not cover the dates of interest.
When you Google most of the hit domains, many of them show up on “expired domain trackers”, and above all Chinese expired domain trackers for some reason, notably e.g.:
First known working day: 2011-07-29.
Scraping script: cia-2010-covert-communication-websites/hupo.sh. Scraping does about 1 day every 5 minutes relatively reliably, so about 36 hours / year. Not bad.
The hits are very well distributed amongst days and months, at least they did a good job hiding these potential timing fingerprints. This feels very deliberately designed.
There are lots of hits. The data set is very inclusive. Also we understand that it must have been obtains through means other than Web crawling, since it contains so many of the hits.
Nice output format for scraping as the HTML is very minimal
They randomly changed their URL format to remove the space before the .com after 2012-02-03:
Some of their files are simply missing however unfortunately, e.g. neither of the following exist:webmasterhome.cn did contain that one however: domain.webmasterhome.cn/com/2012-07-01.asp. Hmm. we might have better luck over there then?
2018-11-19 is corrupt in a new and wonderful way, with a bunch of trailing zeros:
More generally, several files contain invalid domain names with non-ASCII characters, e.g. 2013-01-02 contains 365л.com. Domain names can only contain ASCII charters: stackoverflow.com/questions/1133424/what-are-the-valid-characters-that-can-show-up-in-a-url-host Maybe we should get rid of any such lines as noise.
Some files around 2011-09-06 start with an empty line. 2014-01-15 starts with about twenty empty lines. Oh and that last one also has some trash bytes the end . Beauty.
First known working day: 2011-08-18.
Also heavily IP throttled, and a bit more than hupo apparently.
Also has some randomly missing dates like hupo.com, though different missing ones from hupo, so they complement each other nicely.
Some of the URLs are broken and don’t inform that with HTTP status code, they just replace the results with some Chinese text 无法找到该页 (The requested page could not be found):
Several URLs just return length 0 content, e.g.:
curl -vvv
* Trying 125.90.93.11:80...
* Connected to domain.webmasterhome.cn (125.90.93.11) port 80 (#0)
> GET /com/2015-10-31.asp HTTP/1.1
> Host: domain.webmasterhome.cn
> User-Agent: curl/7.88.1
> Accept: */*
>
< HTTP/1.1 200 OK
< Date: Sat, 21 Oct 2023 15:12:23 GMT
< Server: Microsoft-IIS/6.0
< X-Powered-By: ASP.NET
< Content-Length: 0
< Content-Type: text/html
< Set-Cookie: ASPSESSIONIDCSTTTBAD=BGGPAONBOFKMMFIPMOGGHLMJ; path=/
< Cache-control: private
<
* Connection #0 to host domain.webmasterhome.cn left intact
It is not fully clear if this is a throttling mechanism, or if the data is just missing entirely.
Starting around 2018, the IP limiting became very intense, 30 mins / 1 hour per URL, so we just gave up. Therefore, data from 2018 onwards does not contain webmasterhome.cn data.
Starting from 2013-05-10 the format changes randomly. This also shows us that they just have all the HTML pages as static files on their server. E.g. with:we see:
2013-05-09:
20130509յڹ 0-3y.com
2013-05-10:
20130510յڹ
justdropped.com: e.g. www.justdropped.com/drops/010112com.html. First known working day: 2006-01-01. Unthrottled.
yoid.com: e.g.: yoid.com/bydate.php?d=2016-06-03&a=a. First known workding day: 2016-06-01.
This suggests that scraping these lists might be a good starting point to obtaining “all expired domains ever”.
Data comparison:
2012-01-01Looking only at the .com:
webmastercn has just about ten extra ones than justdropped, the rest is exactly the same
justdropped has some extra and some missing from hupo
The lists are quite similar however.
Considering toplevels:
hupo has several toplevels that webmastercn does not have, e.g. .org and many others
justdropped only covers exactly 6 tlds: .us, .org, .net, .info, .com and .biz. The .com lists are very similar to hupo + webmastercn. But it has a lot more non-.com domains apparently.
We’ve made the following pipelines for hupo.com + webmasterhome.cn merging:
./hupo.sh &
./webmastercn.sh &
./justdropped.sh &
wait
./justdropped-post.sh
./hupo-merge.sh
# Export as small Google indexable files in a Git repository.
./hupo-repo.sh
# Export as per year zips for Internet Archive.
./hupo-zip.sh
# Obtain count statistics:
./hupo-wc.sh
Count unique domains in the repos:
( echo */*/*/* | xargs cat ) | sort -u | wc
The extracted data is present at:Soon after uploading, these repos started getting some interesting traffic, presumably started by security trackers going “bling bling” on certain malicious domain names in their databases:
Next, we can start searching by keyword with Wayback Machine CDX scanning with Tor parallelization with out helper cia-2010-covert-communication-websites/hupo-cdx-tor.sh, e.g. to check domains that contain the term “news”:
./hupo-cdx-tor.sh mydir 'news|global' 2011 2019
produces per-year results for the regex term news|global between the years under:
./hupo-cdx-tor.sh out 'news|headline|internationali|mondo|mundo|mondi|iran|today'
Other searches that are not dense enough for our patience:
OMG news search might be producing some golden, golden new hits!!! Going full into this. Hits:
thepyramidnews.com
echessnews.com
tickettonews.com
airuafricanews.com
vuvuzelanews.com
dayenews.com
newsupdatesite.com
arabicnewsonline.com
arabicnewsunfiltered.com
newsandsportscentral.com
networkofnews.com
trekkingtoday.com
financial-crisis-news.com
and a few more. It’s amazing.
TODO what does this Chinese forum track? New registrations? Their focus seems to be domain name speculation
Some of the threads contain domain dumps. We haven’t yet seen a scrapable URL pattern, but their data goes way back and did have various hits. The forum seems to have started in 2006: club.domain.cn/forum.php?mod=forumdisplay&fid=41&page=10127
club.domain.cn/forum.php?mod=viewthread&tid=241704 “【国际域名拟删除列表】2007年06月16日” is the earliest list we could find. It is an expired domain list.
This pastebin contained a few new hits, in addition to some pre-existing ones. Most of the hits them seem to be linked to the IP 72.34.53.174, which presumably is a major part of the fingerprint found by CYBERTAZIEX, though unsurprisingly methodology is unclear. As documented, the domains appear to be linked to a “Condor hosting” provider, but it is hard to find any information about it online.
From the title, it would seem that someone hacked into Condor and defaced all of its sites, including unknowingly some CIA ones which is LOL.
Ciro Santilli checked every single non-subdomain domain in the list.
www.zone-h.com lists some of the domains. They also seem to have intended to have snapshots of the defaces but we can’t see them which is sad:
But they do reverse IP, and they show which nearby reverse IPs have hits on the same page, for free, which is great!
Shame their ordering is purely alphabetical, doesn’t properly order the IPs so it is a bit of a pain, but we can handle it.
The data here had a little bit of non-overlap from other sources. 4 new confirmed hits were found, plus 4 possible others that were left as candidates.
Unfortunately I can’t find a reverse IP search method.
And perhaps due to having lots of CAPTCHAs, Google doesn’t seem to index that website very well… it even has a tiny screenshot! And it also shows some more metadata beyond IP, e.g. HTTP response headers, which notably contain stuff like Server: Apache-Coyote/1.1.
They seem to have an exceptionally complete database.
They also have some random localized versions:These can be useful if your IP gets blacklisted on the main site because you were checking too many sites.
There are four main types of communication mechanisms found:
JAR is the most common comms, and one of the most distinctive, making it a great fingerprint.
Several of the JAR files are named something like either:as if to pose as Internet speed testing tools? The wonderful subtleties of the late 2000s Internet are a bit over our heads.
All JARs are directly under root, not in subdirectories, and the basename usually consist of one word, though sometimes two camel cased.
JavaScript file. There are two subtypes:
JavaScript with SHAs. Rare. Likely older. Way more fingerprintable.
JavaScript without SHAs. They have all been obfuscated slightly different and compressed. But the file sizes are all very similar from 8kB to 10kB, and they all look similar, so visually it is very easy to detect a match with good likelyhood.
Adobe Flash swf file. In all instances found so far, the name of the SWF matches the name of the second level domain exactly, e.g.:
http://tee-shot.net/tee-shot.swf
While this is somewhat of a fingerprint, it is worth noting that is was a relatively commonly used pattern. But it is also the rarest of the mechanisms. This is a at a dissonance with the rest of the web, which circa 2010 already had way more SWF than JAR apparently.
Some of the SWF websites have archives for empty /servlet pages:
which makes us think that it is a part of the SWF system.
CGI comms
These have short single word names with some meaning linked to their website.
Because the communication mechanisms are so crucial, they tend to be less varied, and serve as very good fingerprints. It is not ludicrous, e.g. identical files, but one look at a few and you will know the others.
We’ve come across a few shallow and stylistically similar websites on suspicious ranges with this pattern.
No JS/JAR/SWF comms, but rather a subdomain, and an HTTPS page with .cgi extension that leads to a login page. Some names seen for this subdomain:
secure.: most common
ssl.: also common
various other more creative ones linked to the website theme itself, e.g.:
musical-fortune.net has a backstage.musical-fortune.net
The question is, is this part of some legitimate tooling that created such patterns? And if so which? Or are they actual hits with a new comms mechanism not previously seen?
The fact that:
hits of this type are so dense in the suspicious ranges
they are so stylistically similar between on another
citizenlabs specifically mentioned a “CGI” comms method
suggests to Ciro that they are an actual hit.
Later on, we’ve also come across some stylistic hits in IP ranges with apparent slight variations of the CGI comms pattern:
no .cgi, but also http on subdomain:
no subdomain, no https, no .cgi
Since these are so rare, it is still a bit hard to classify them for sure, but they are of great interest no doubt, as as we start to notice these patterns more tend to come if it is a thing.
crt.sh appears to be a good way to look into this:
backstage.musical-fortune.net:
clients.smart-travel-consultant.com
members.it-proonline.com
members.metanewsdaily.com
miembros.todosperuahora.com
secure.altworldnews.com
secure.driversinternationalgolf.com
secure.freshtechonline.com
secure.globalnewsbulletin.com
secure.negativeaperture.com
secure.riskandrewardnews.com
secure.theworld-news.net
secure.topbillingsite.com
secure.worldnewsandent.com
ssl.beyondnetworknews.com
ssl.newtechfrontier.com
www.businessexchangetoday.com
heal.conquermstoday.com
They all appear to use either of:
TODO it would be cool to have a look at the JARs and see if they have anything in common that makes for a good fringerprint. Would not help find new ones, but would help to confirm possible hits.
There are two types of JavaScript found so far. The ones with SHA and the ones without. There are only 2 examples of JS with SHA:Both files start with precisely the same string:
throw new Error("B64 D.1");};if(at[1]==-1){throw new Error("B64 D.2");};if(at[2]==-1){if(f
Googling most domains gives only very few results, and most of them are just useless lists of expired domains. Skipping those for now.
Googling "dedrickonline.com" has a git at www.webwiki.de/dedrickonline.com# Furthermore, it also contains the IP address “65.61.127.174” under the “Technik” tab!
Unfortunately that website appears to be split by language? E.g. the English version does not contain it: www.webwiki.com/dedrickonline.com, which would make searching a bit harder, but still doable.
But if we can Google search those IPs there, we might just hit gold.
IP search did work! www.webwiki.de/65.61.127.174
But doesn’t often/ever work unfortunately for others.
Searching on github.com: github.com/DrWhax/cia-website-comms by Jurre van Bergen from September 2022 contains some of the links to some of the ones reported by Reuters including some of their JARs, presumably for reversing purposees. Pinged him at: github.com/DrWhax/cia-website-comms/issues/1
Some less-trivial breakthroughs:
Grepping the 2013 DNS Census first by overused CGI comms subdomains secure. and ssl. leaves 200k lines. Grepping for the overused “news” led to hits:
After 2013 DNS Census virtual host cleanup heuristic keyword searches we later understood why there were so few hits here: the 2013 DNS Census didn’t capture the secure. subdomains of many domains it had for some reason. Shame, because if it had, this method would have yielded many more results.
He then proceeded to give Carson and 5 other domains in private communication. His name is given here with his consent. His advances besides not being blind were Yandexing for some of the known hits which led to pages that contained other hits:
Unfortunately, these methods are not very generalizable, and didn’t lead to a large number of other hits. But every domain counts!
Figure 1.
2004 Wayback Machine archive of alljohnny.com
.
What follows is the previous
Some text visible on the Reuters screenshot:It is unclear however if this text is plaintext or part of a an image.
Some failed attempts, either dry guesses or from DNS grepping dataset searches:
Scrapped justdropped data, patched:
+++ b/cia-2010-covert-communication-websites/cdx-post.sh
@@ -1,7 +1,7 @@
#!/usr/bin/env bash
# Post process the output of cdx.sh to enrich IDs even further, and reconstruct easier to Web Archive inspect domain names.
-grep -P -e '([^,)]+)\)\/\1\.swf|\)/[^/]+.jar|([^,)]+),([^,)]+),([^,)]+)\)/cgi-bin/[^/]+\.cgi' "$1" |
- sed -r 's/\).*//' | awk -F, '{ printf("%s.%s\n", $2, $1) }' | uniq -c | awk '$1 == 1{ print $2 }' | tee $1.post
+grep -P -e '([^,)]+)\)\/\1\.swf|\)/[^/]+.jar|([^,)]+),([^,)]+),([^,)]+)\)/cgi-bin/[^/]+\.cgi' "$1"|
+ sed -r 's/\).*//' | awk -F, '{ printf("%s.%s\n", $2, $1) }' | uniq -c | awk '{ print $2 }' | tee $1.post
and then:
./hupo-cdx-tor.sh out 'news|headline|internationali|mondo|mundo|mondi|iran|today' 2006 2022
2010. Suspicious. But no clear fingrenprint. Also not as shallow as others. Also Joomla based which would be novel.
Summary: this is just a red herring. Wakatime owner likely registered the domains just after this article was published as a publicity stunt. Fair play though.
Running e.g.
curl -vvv dedrickonline.com
gives:
* Trying 162.255.119.197:80...
* Connected to dedrickonline.com (162.255.119.197) port 80 (#0)
> GET / HTTP/1.1
> Host: dedrickonline.com
> User-Agent: curl/7.88.1
> Accept: */*
>
< HTTP/1.1 301 Moved Permanently
< Date: Mon, 12 Jun 2023 20:30:19 GMT
< Content-Type: text/html; charset=utf-8
< Content-Length: 55
< Connection: keep-alive
< Location:
< X-Served-By: Namecheap URL Forward
< Server: namecheap-nginx
<
Moved Permanently.
* Connection #0 to host dedrickonline.com left intact
so we see that he must have setup redirection with Namecheap as mentioned at: www.namecheap.com/support/knowledgebase/article.aspx/385/2237/how-to-redirect-a-url-for-a-domain/
