CSEC Surveillance analysis of IP and user data

The most recent story from the Snowden documents is from Canada: it claims the CSEC (Communications Security Establishment Canada) used airport Wi-Fi information to track travelers. That's not really true. What the top-secret presentation shows is a proof-of-concept project to identify different IP networks, using a database of user IDs found on those networks over time, and then potentially using that data to identify individual users. This is actually far more interesting than simply eavesdropping on airport Wi-Fi sessions. Between Boingo and the cell phone carriers, that's pretty easy.
The researcher, with the cool-sounding job-title of "tradecraft developer," started with two weeks' worth of ID data from a redacted "Canadian Special Source." (The presentation doesn't say if they compelled some Internet company to give them the data, or if they eavesdropped on some Internet service and got it surreptitiously.) This was a list of userids seen on those networks at particular times, presumably things like Facebook logins. (Facebook, Google, Yahoo and many others are finally using SSL by default, so this data is now harder to come by.) They also had a database of geographic locations for IP addresses from Quova (now Neustar). The basic question is whether they could determine what sorts of wireless hotspots the IP addresses were.
You'd expect airports to look different from hotels, and those to look different from offices. And, in fact, that's what the data showed. At an airport network, individual IDs are seen once, and briefly. At hotels, individual IDs are seen over a few days. At an office, IDs are generally seen from 9:00 AM to 5:00 PM, Monday through Friday. And so on.
Pretty basic so far. Where it gets interesting his how this kind of dataset can be used. The presentation suggests two applications. The first is the obvious one. If you know the ID of some surveillance target, you can set an alarm when that target visits an airport or a hotel. The presentation points out that "targets/enemies still target air travel and hotels"; but more realistically, this can be used to know when a target is traveling.
The second application suggested is to identify a particular person whom you know visited a particular geographical area on a series of dates/times. The example in the presentation is a kidnapper. He is based in a rural area, so he can't risk making his ransom calls from that area. Instead, he drives to an urban area to make those calls. He either uses a burner phone or a pay phone, so he can't be identified that way. But if you assume that he has some sort of smart phone in his pocket that identifies itself over the Internet, you might be able to find him in that dataset. That is, he might be the only ID that appears in that geographical location around the same time as the ransom calls and at no other times.
The results from testing that second application were successful, but or slow. The presentation sounds encouraging, stating that something called Collaborative Analysis Research Environment (CARE) is being trialed "with NSA launch assist": presumably technology, money, or both. CARE reduces the run-time "from 2+ hours to several seconds." This was in May 2012, so it's probably all up and running by now. We don't know if this particular research project was ever turned into an operational program, but the CSEC, the NSA, and the rest of the Five Eyes intelligence agencies have a lot of interesting uses for this kind of data.
Since the Snowden documents have been reported on last June, the primary focus of the stories has been the collection of data. There has been very little reporting about how this data is analyzed and used. The exception is the story on the cell phone location database, which has some pretty fascinating analytical programs attached to it. I think the types of analysis done on this data are at least as important as its collection, and likely more disturbing to the average person. These sorts of analysis are being done with all of the data collected. Different databases are being correlated for all sorts of purposes. When I get back to the source documents, these are exactly the sorts of things I will be looking for. And when we think of the harms to society of ubiquitous surveillance, this is what we should be thinking about.