Expectation of privacy was nice while it lastedFollow

Kind of an impossible request when people panic at the slightest change and insist the worst case scenario is the only scenario possible.

Possibly, but you're still not catching everything on everyone, which is what gbaji was arguing.

Let's just grab some numbers, and say the NSA datacenter can hold about 10 exabytes of information. Let's say that each year the world produces maybe 10 zettabytes of information a year Spoiler(yes I'm using round numbers here for simplicity, and feel free to give me better numbers if you have them, I just grabbed what was readily accessible.). That would mean that the world is producing 1,000x more information every year than can be stored on the servers. One would also assume that's a best case scenario, as they'll have to generate some of their own data to properly label, categorize, and annotate the giant amount of information there.

That goes beyond the absurdity that you have to somehow summarize the information to make it useful for people. Most people look at a raw database query and have their eyes glaze over. You'd need to get that information into a usable format before someone looking up information on a potential terrorist lead could make good use of it (which, of course, means more data to store).

In addition, you're duplicating information that's already stored elsewhere by people with a motivation to retain that information. While it makes perfect sense to have your own secure copy of relevant information just in case it gets deleted or whatever from the 3rd party site, there's no reason to have your very own copy of every cat video on youtube just because. We know these external companies get requests to view this information (which suggests that the NSA people aren't retaining everything), and again, making someone have to go submit a form to a judge before they can access it cuts down on the potential for abuse.

Edit: I don't believe that 10 zettabytes includes phone calls, TV broadcast information, information on personal computers, or other things not linked to the internet that they might want to retain? Huh, this is a surprisingly hard number to track down...

Edit2: Hmm, this page seems to suggest 2.5 exabytes a day, which works out to over 900 zettabytes a year. Wonder if that adds in the other non-internet sources to get to that much higher number? That would maybe make sense? Bleh...

Edited, Jul 20th 2016 10:42am by someproteinguy

Oh! that one specific program is defunct, well then nevermind then. My fears have been assuaged and suspicions allayed.

I guess prohibitions on genocide or hate crimes are unfounded as well because the final solution is not in effect.

It is so nice that once consigned to the dustbin of history, ideas stay dead and gone forevermore.

I said "can be a felony", not a misdemeanor which is true, and the law specifically mentions that as well as the conditions, which would be applicable in many cases, thus it my statement was perfectly reasonable. I did not lie to you and the law was pretty explicit.

In case you still want to hem and haw, a direct cite:

Sigh. You're not getting it. At some point, "check every single boat for what we're looking for" will actually be faster than "figure out which boat has what we're looking for and check just that one". You really don't seem to be able to grasp this. It's not choosing the mystery box. That suggests that we're still picking *one* thing at a time. The paradigm shift here is that when you have sufficient data scanning and analysis capability, you no longer have to "pick one", or even "pick some". You don't have to start out by catching and following leads and then directing your surveillance in the directions indicated. You can actually simply look at *everything* for what you're looking for and work the other way.

You keep wanting to frame this as a choice between directing our efforts at places we already suspect might bear fruit versus picking the same number of "random" places to look. But that's not at all what I'm talking about here. I honestly don't know how many different ways I can explain the same concept to you to get you to understand this.



If we allow it to. My point is that we should not just blindly trust that no one would ever abuse power they are given. We must start with the assumption that they will, and the only reason they wont is if we keep an eye on them to make sure they don't.



Ok. But do you see how saying "we don't need to worry about this, because people wont allow this to happen" acts to reduce the likelihood that people will actually not allow that thing to happen? If everyone is just sitting back, trusting that "the people" wont allow something to happen, then none of "the people" actually are. Which is a potential problem.



Yes. More specifically, I'm making the point that by over focusing our privacy concerns at a program like the one Congress knew about and approved, and which only uses publicly accessible data, we run the risk of creating a kind of "boy who cried wolf" scenario, where we exhaust ourselves fighting against the wrong things, and continually fail to succeed (because there isn't actually a 4th amendment violation), leading to people adopting the idea that there is no such thing as privacy anymore, which may result in apathy when/if the government actually does attempt to use data collection methods that violate the 4th amendment. Because after the 50th time some privacy advocates insist that something the government does is "a horrible violation of our privacy and must be stopped", and then nothing comes of it, then the one time they are trying to point out an actual real violation of our privacy, no one will listen. We're programming the population to ignore cries about government domestic spying by doing this.


I'm reasonably sure I've already explained this several times. i didn't think it was that difficult a concept to grasp. Apparently, I was wrong.

Edited, Jul 20th 2016 3:39pm by gbaji

Spoiler

Gbaji wrote:

Sigh. You're not getting it. At some point, "check every single boat for what we're looking for" will actually be faster than "figure out which boat has what we're looking for and check just that one". You really don't seem to be able to grasp this. It's not choosing the mystery box. That suggests that we're still picking *one* thing at a time. The paradigm shift here is that when you have sufficient data scanning and analysis capability, you no longer have to "pick one", or even "pick some". You don't have to start out by catching and following leads and then directing your surveillance in the directions indicated. You can actually simply look at *everything* for what you're looking for and work the other way.

You keep wanting to frame this as a choice between directing our efforts at places we already suspect might bear fruit versus picking the same number of "random" places to look. But that's not at all what I'm talking about here. I honestly don't know how many different ways I can explain the same concept to you to get you to understand this.

When you do research for a paper, do you look for specific documents that might assist your paper, or do you download a random large number of documents, to sort through later? You seem to think that the sorting only happens in one scenario.

Spoiler

Gbaji wrote:

If we allow it to. My point is that we should not just blindly trust that no one would ever abuse power they are given. We must start with the assumption that they will, and the only reason they wont is if we keep an eye on them to make sure they don't.

Gbaji wrote:

Ok. But do you see how saying "we don't need to worry about this, because people wont allow this to happen" acts to reduce the likelihood that people will actually not allow that thing to happen? If everyone is just sitting back, trusting that "the people" wont allow something to happen, then none of "the people" actually are. Which is a potential problem.

Gbaji wrote:

Yes. More specifically, I'm making the point that by over focusing our privacy concerns at a program like the one Congress knew about and approved, and which only uses publicly accessible data, we run the risk of creating a kind of "boy who cried wolf" scenario, where we exhaust ourselves fighting against the wrong things, and continually fail to succeed (because there isn't actually a 4th amendment violation), leading to people adopting the idea that there is no such thing as privacy anymore, which may result in apathy when/if the government actually does attempt to use data collection methods that violate the 4th amendment. Because after the 50th time some privacy advocates insist that something the government does is "a horrible violation of our privacy and must be stopped", and then nothing comes of it, then the one time they are trying to point out an actual real violation of our privacy, no one will listen. We're programming the population to ignore cries about government domestic spying by doing this.


I'm reasonably sure I've already explained this several times. i didn't think it was that difficult a concept to grasp. Apparently, I was wrong.

Once again, your "concern" of misuse of power is true for everyone who has power. We can manage this scenario the same way we manage other powerful people/organizations with power, i.e. the people who control your money. Are you more concerned about someone reading your diary than someone spending your life savings?

Sure. But how do you trim your database to just those people who are "relevant to your query"? You'd have to do the same sorting either way.



Why not start with a massive amount of data on a massive amount of things, and then use a set of search parameters to trim your search down? It's the same thing, except doing it that way reduces the chance that you miss something because you decided not to look at people and information that you didn't start out thinking might be relevant. I think we're really describing the same thing here. Obviously, if data storage is an issue you must do some of your sorting before doing data collection so as to trim the resulting database to a manageable size. But if storage isn't an issue, then there's no reason at all not to collect everything you can and then search on that. Keep trimming your hits on those searches until it reaches a manageable number with a higher probability of actually matching what you are looking for.

If you restrict your data collection first, you may very well eliminate data that you might not see a pattern in unless it's included with a whole bunch of other seemingly unrelated data, and you might just miss something.



Not really. It's a purely technical limitation. Again, you're doing the same sorting either way. I'd actually argue that it would waste more man hours if you first start out trying to figure out criteria to determine what to collect and then search on it, rather than just grabbing everything and running searches on the resulting (much larger) dataset. The only reason not to do it the way I'm suggesting is if you have storage restrictions.




How do you know which are the relevant parts of the data if you don't first collect it and search through it? If there's X amount of total data out there, you can either trim X down to .01X and search on the much smaller set, or just search on the whole thing. But to trim it down to .01X you have to have some kind of parameters to decide what is relevant and what isn't? How do you decide that? You can't. You can't possibly know if the contents of any of a million people's social media is going to provide a pattern match indicating terrorist behavior before looking at all of the million people's social media. So you either decide to just not use anyone's social media, or you decide to use everyone's social media, or you have some other initial search through social media to detect patterns that you think might later match a pattern indicating terrorist behavior and collect those people's social media. Um... But you had to do that search, right?

Which means you're doing the same amount of work. More really. You want to leverage the technological capability of the tools you have to the maximum degree possible. Why insist on doing a step that avoids using those tools? Just dump everything into a big pot and look through it. A computer can do that far far more efficiently than any other method. I'm not sure how pre-sorting helps here.



That's a limitation of the search tools, not the data set. Again, you'd have to somehow use the same exact sorting methods to eliminate data from the set in the first place. That seems like an unnecessary thing to do that will only decrease the accuracy of your search.

Your argument is like saying that if we just eliminated 90% of all the metadata google uses for searching for stuff, we could get more accurate and faster searches. Well, they'd be faster, but less accurate. Because the very things we're looking for (perhaps the obscure and hard to correlate things) would be the most likely to have been eliminated in our first round of data cuts. So instead of just having a difficult search to find what we're looking for, we have an impossible one because what we're looking for will never be found because it's no longer in the searchable dataset.



Sure. But it becomes a more likely possibility if too many people adopt the mindset that we don't have privacy today. You're more likely to fight to keep something you believe you actually have than to continue to fight for a cause you believe is already lost. That's my point. I get that it's hypothetical and a bit esoteric, but that's what it is. It's just that I've seen more and more people making statements like "privacy is dead", and becoming increasingly apathetic towards the notion of privacy. That's what I find dangerous and a bit scary. If people don't think that privacy exists, or don't think it's important, then that's when we may actually lose it.

Spoiler

Gbaji wrote:

Ok. How do you "look for specific documents that might assist your paper"? You search though a dataset that someone has collected and indexed for you to search. If Google hasn't created a reference to a given paper, you will never find it. Similarly, if our hypothetical government database doesn't contain information about a given person, place, action, whatever, they will never be able to find it in a search. They can't "just search for people who might be of interest", without first... you know... doing some kind of search. I think you are getting stuck on the whole idea of "collecting a dataset". This doesn't require physically copying every single thing in existence, but in most cases, just a reference to it (just as google just holds metadata used for their searches). There's no need, for example, for said government database to duplicate things that can already be located via a google search (although they might use their own more capable tools on the same data). Other data, that is perhaps more transitory in nature would need to be actively collected in its entirely and stored. How long would depend on a number of factors.

Life is a database, so answer the question.

Spoiler

Gbaji wrote:

I'm concerned about both. I'm not sure what your point is here. I don't think anyone is arguing that the expectation that your bank not embezzle all your money was "nice while it lasted". But some people are arguing that with regard to our expectation of privacy (again, it's in the thread title), specifically with regard to unwarranted searches of private computers in someone's home. So yeah, I'm going to talk about that issue, because... wait for it... it's the topic we're talking about.

If someone creates a thread about the safety of my savings account, I'll talk about that instead. Cause then that would be the topic. See how that works?

So, let me get this straight. You are equally concerned about your bank embezzling your money as the government creating this automated program to gain all information from everyone all at once?

And? Did I ever say that it couldn't (although... good luck with that)? I'd also like to point out that there's a difference between understanding how something works, and understanding how to use it. You only know the latter. All that stuff I was talking about earlier that you dismissed? That's what nmap is doing on the back end. The side that you don't know about, and appears to be "magic". Newsflash: It's not.



You don't understand what I'm saying. As I mentioned earlier in the thread, we're speaking different languages. You're discounting what I'm saying out of ignorance.

But here, let me try to educate you on the right way to do it:

If you want to hack "everything", you don't sit on one computer and run a single scan on "everything". You write a hack that targets local systems, embeds itself in them, replicates itself and spreads to other systems via a variety of methods, jumping across subnet boundaries and compromising more and more systems. Why? Because if you understand how network communication actually works, you'll understand that any attempt to do something like you are describing will be easily be detected. You *can* scan ports on a system across a subnet boundary (anywhere on the internet if you want), but you have to use active methods to discover which systems are on a remote subnet, which will be very very detectable. If you restrict your direct hacks to systems on your local subnet (like say folks connected to your local coffee shops wi-fi network maybe, or (less recommended) the computers connected to the same loop your home cable modem is connected to), you can identify targets quietly. If you are located on the same switch (and not just the same router), you can direct scans at them that can't easily be detected (since they are directed from mac to mac and never touch a router or even cross a switch boundary, so there's no broadcast being done to discover the system you are targeting). This, btw, is why I said that understanding how networks actually work is important.

Once you infect a number of systems with this kind of hack, they'll continue to infect others. The aforementioned coffee shop example works well since most systems connected to them are mobile and will walk around infecting other systems at other hotspots, which will do the same elsewhere, and so on, and so on. All you have to do is wait.

Now what your hack "does" is up to you. One of the more common hacks is to just drop some code in place that targets ping floods at a given address or address range on a given day and time. So a month later, several thousand systems of some unsuspecting dupes take part in a DDoS attack on the servers of a game company that banned you for using cheats. Those bastards probably had it coming, right? Of course, you could also use those system both as infection spreaders and as storehouses for future hacks (even the more direct attacks since they're not coming from you now). Maybe they periodically "check in", for updates. Maybe they upload certain files on their system to another hacked remote system (which you know about cause you hacked it and set it up). Now you can trigger any of a number of different actions that you want, all being performed by your zombie horde of hacked computers around the internet.

Now, imagine that times like 1000 for what a sophisticated government hack could do. Replace "infect computers that happen to connect to an open hotspot" with "set up our own trapped hotspots around the country" (heck, we could even create a funded government program to provide "free wi-fi", as cover for this), or even "drop embedded custom binary code into backbone routers that insert infected code into appropriate packets crossing the router", or "plant code writers at Apple and Microsoft and just put our own hacks into their tcp stack code", or "embed our hack into the login for Obamacare". Remember, all you really need to have is a hack that allows you to direct the hacked computers to do something you want at a later date. That's actually a lot easier than you might think.

You're thinking in terms of brute force hacks directly across a wan. That's a great way to get yourself on the FBI's radar, but not much else. And that's not remotely scalable to the level I'm talking about. The fact that you can't imagine how broader and more subtle hacks could be done, doesn't mean that there aren't dozens of ways to do it.



Just one subnet at a time. What I think you fail to grasp is that this isn't time consuming on a local level. It is, in fact, exactly how the network already works. Systems "scan" their local subnet all the time, constantly. You'd be shocked at the amount of network traffic is merely involved in "mapping" the local subnet, which in turn finds a route to a remote subnet, which in turn is "mapped" to find the exact computer you are connecting to. This happens on every subnet, thousands of times a second. All the time. All traffic is "local" in this regard. The very design of the internet is to work this way. You don't need to know or care about anything "far away". Each portion knows about it's own portion and knows how to talk to adjacent portions. Add enough of those bits together and you have the massive thing we call the internet. But each piece is the same. And it works the same.

The assumed reason for using a database system is to not have to rely on known leads.



Sure. But there's no reason to believe that adding more data to the set will increase false-positives. It will, however, increase true-positives that you would have missed in a smaller set.

Here's an easy way to look at it. Let's say my search criteria is "left handed males". One dataset includes 5% of the US population, the other includes 100%. There's no reason to expect that I'm going to accidentally match people who are not males or not left handed in my search simply because I increased the total size of the dataset. I'm still searching for the same criteria. I'm just not missing 95% of the hits.



The same can be said of whatever method you'd use to trim your dataset down in the first place though. And in your example, it's the "unknown terrorists" we're presumably looking for, right?

Again, I'm not seeing how adding a ton of extra data to the set that doesn't match the search criteria in any way affects the accuracy of the search. If I add a million bits of information about people who are clearly not terrorists, they're still clearly not terrorists and will not match whatever search parameters I'm using to search for terrorists. However, it's quite possible that a few of those million bits of information that I would have excluded at first blush might work in concern with some other bits of information in the dataset that would show me someone I thought was clearly not a terrorist, but who actually is.

And yes, you're right that this could be a false positive, but assuming that these sorts of searches are used to initiate additional human surveillance actions, then the harm from a false positive in this search is relatively low, while the harm from failing to detect a true positive is very high. Let me point out that I'm not making any kind of moral judgement here. I'm just pointing out how using the largest dataset possible is a better way to go here. The restriction is really just about how much data you can either store yourself, or index against other available sources.



Which is a totally valid point, and I'm not disputing it. I was mainly responding to the idea that this would not be a good way to do searches at all. I heartily disagree. The more data, the better (well, from a "we want to find as many bad guys as possible" point of view and not necessarily from a "we want to protect people's privacy" point of view).



I'm not sure how any of that is relevant to pattern searches for behavior that may indicate some kind of terrorist or organized criminal activity. I think that once you even begin talking about searching a database we're past the point of "things we know directly". If we knew who all the terrorists were, we wouldn't need a database in the first place, right? Seems axiomatic that the very discussion assumes that we don't know what we don't know, and are trying to search through large amounts of data for patterns that a computer search can spot, but that humans will miss.



Again though, we're specifically not talking about those things. We're talking about things that can only be found by searching a giant database.



If said database existed solely to assist us in tracking the activities and behavior of already known "bad guys", you'd have a point. But that's not the extent of what these things are constructed for today, and certainly not the limit of what they could do tomorrow. The government wants to use these sorts of things to search through large amounts of seemingly unrelated data for patterns at a speed and accuracy that humans could not come remotely close to. By definition, we're looking for things that we can't already figure out, or else we wouldn't need the system in the first place.



Again though, you're going to miss the unknown terrorists that way. Also, as I've stated a couple times already, if that's all you're doing, you don't need a searchable database at all. You're just collecting evidence at this point, which, as you stated, we could do by hand before and can do by hand now. I just can't get past the point that the assumption behind this whole thing is to add an additional tool that could be used to spot terrorist plots that we can't detect just by following the trail of known terrorists. And to do that, you want to cast your net as wide as possible.



Because the search for "known terrorists" didn't stop at some date in the past. It's ongoing. So it's not "already been done". It's being done today. It will be done tomorrow. It'll still be being done a year from now. So given that you have to spend the time and effort generating that list in the first place, why not use the best tools to do so?




Exactly. But by limiting the dataset, you'd be effectively limiting the results to just the top 1000 "known terrorists and obvious associates". You basically just made my point for me. We want a tool that does allow us to look at the 100,000th guy who matches our search criteria, and then narrow our search to see if that guy still matches those parameters. Then narrow it some more. And if he's still there after we've eliminated number 1001 to 999,999, we might realize we need to take a closer look at this guy. This same guy we would have completely missed doing it the old way.



Exactly. Because we're only looking at the obvious signs that the dumb guys (or the guys who don't care) give us. But the idea behind the kinds of mass data systems I'm talking about (again, I'm not making a moral judgement here) is that if you could include a lot more data in your set, you could find patterns of behavior that could accurately flag those "unknown subjects" before they commit their acts of terrorism or whatever. Patterns that you would otherwise miss because you're simply not collecting the data that you need to make the correlation.

Edited, Jul 20th 2016 8:08pm by gbaji

That doesn't seem like a prudent idea. I mean we're certainly not great at catching terrorists, but are we really so bad we need to start over from scratch? Remember you likely will have to pull people from active investigations into known terrorist networks to chase these new leads, so you're losing out on gathering further intelligence in those areas.

Sure we would, that's the whole point of the database, to expand on existing knowledge and track known individuals. Even in a perfect world where we know who all the terrorists are we still don't know where they are, where they'll strike, if they're recruiting more people, etc. etc. These are things that could be gleamed from information without exhaustive database searching.

I can certainly use my outside knowledge to limit what I'm searching against. If there's subsets of the data I know aren't relevant, say maybe all those cute cat videos on youtube, there's no reason to sift through them. If I know there are parts of the database with useless information why would I use them? Why would I even put them into the database in the first place if I have limited storage capacity?

That's what I'm assuming the primary purpose of it is, yes.

This is where we start sliding down the slippery slope isn't it? Hypothesizing about future capabilities and uses of technology isn't something we've been especially good at as a whole in the past. I mean, we're still waiting for our flying cars and vacations on the moon and whatnot right? The Russians never did shoot nukes at us. We could probably continue down the road of things we were worried about in the past that never happened. It's very likely the things we really should be afraid of in the future aren't even on our radar now. Hey, even Al Gore didn't see the internet coming.

In the end I imagine all of this you're talking about is technically possible, but I'm still dubious it's useful in generating actionable information. At least enough more-so than traditional means to make it a worthwhile endeavor.

Edited, Jul 20th 2016 8:15pm by someproteinguy

Saudis did 9/11.