"...the field has real standards of scholarly validity."

[Dan Fromm]

Everything I've seen points to the simple fact that their system is completely useless unless they also have the content of phone calls (using speech-to-text software) and messages for analysis. And even with that, they don't have nearly the manpower they would need to follow up on each potential positive. Kinda makes me wonder if the system was built for some other purpose.

Edit: if this is getting too political for this forum I will cease and desist.

Jody, what they have seems pretty useless for predicting who will cause an outrage. We're very lucky that so few people -- poor deranged mass murderers except, and I'm not being sarcastic, I feel as sorry for them as for their victims -- commit outrages in the US. A case can be made that when there's more than a lone madman or a couple of brothers involved in an outrage telco message records can be useful for finding the associates. A case can also be made that having all of the message records in one giant database makes retrieval faster and easier. Given the state of the communications system in the US -- there are many carriers, each with its own network and stand-alone billing system -- having a master database make make sense from law enforcement's point of view. Letting the secret police play with it at will is another very separate issue.

At this point, I think we're discussing what's possible, not which way to vote. Brad's defending his tribe. But we have indeed wandered away from the original topic.

[Jac@stafford.net]

Dan Fromm: " It is indeed possible that the secret police have other data no one's talking about and that they're not boasting about having that will improve their ability to find bad guys before they misbehave, but given bad guys rarity its hard to believe that a targeting model will do the job."

Don't the outliers usually self identify?

[Dan Fromm]

Don't the outliers usually self identify?

Jac, one of the premises behind the great data gathering effort is that it will enable our protectors to stop outrages before they happen. I wish they could, strongly doubt they can.

[cikaziva]

this reminds me of a fellow that i meet once, a mathematician, who had a life long project of calculating a beauty of a poetry. basically he would convert words in numbers and then he would look for proportions and rhythm in the models (i dont know anything about mathematics and i cant remember what was exact method ). He spent years and decades calculating in order to conclude that poetry of Frederico Garcia Lorca is beautiful. but then i remember guys laughing at me because i was taking photographs of mud

[Jody_S]

this reminds me of a fellow that i meet once, a mathematician, who had a life long project of calculating a beauty of a poetry. basically he would convert words in numbers and then he would look for proportions and rhythm in the models (i dont know anything about mathematics and i cant remember what was exact method ). He spent years and decades calculating in order to conclude that poetry of Frederico Garcia Lorca is beautiful. but then i remember guys laughing at me because i was taking photographs of mud

It's probably a lot easier to compute the beauty of a photograph.

[BradS]

Brad, are you a statistician?

Oh, I do what I can...but don't hold it against me. I dropped out of grad school after ten years reading for a phd in applied math/statistics. My thesis topic was rejected, I had a full time engineering job, a two year old child, a mortgage and...and one day, I just walked away. Despite this, I worked as an engineeering statistician for a couple of large tech companies early in my career. I designed a bunch of microcomputer hardware stuff before that too. However, I long ago realized that it is far easier and more lucrative to do software and so that is mainly what I have done lately. Every once in a while I still do some statistics but very few companies are willing to pay for real statistical analysis. Most who claim to want a statistician really seem to just want some flunky to punch data into MS Excel, whip out pretty charts and graphs or make up fairy tales shouded in mystical technical jargon and supported by numbers that "look impressive" .


(EDITED out a long winded opinionated tirade wherein I even managed to confuse myself.)

As for honesty, there are econometricians who are dishonest or incompetent. I've run up against enough of 'em. The same seems to be true of statisticians. We've had pharmaceutical company statisticians in for interviews, many have complained about extreme pressure to be, um, creative. Some bent under it, others didn't.

This is why I left statistics as a profession. Some clients/investigators wanted to discover useful information that was not inconsitent with the true but unknown state of nature. Too many wanted (me) to abuse statistics to support (their) lies.

[Darin Boville]

Jac, one of the premises behind the great data gathering effort is that it will enable our protectors to stop outrages before they happen. I wish they could, strongly doubt they can.

And what if they *could*? Thats the real question. This sort of data gathering is in it's embryonic stages. Maybe not now but soon they will have the ability to stop bad things from happening. Will that justify the surveillance? If the answer is an automatic "yes" then there is no point in bothering with the controversy now--if it is just a technical argument.

Switching gear to economist and "scholarly validity"--though I love economics and have had my share of classes in the topic--even worked as an 'economist" once upon a time (I wasn't one, a complicated story)--I think it is obvious that economics is probably the most politicized of all academic disciplines. (I put aside the politics of the English department because here I am only talking about politics that matter. )

--Darin

P.S. My take on the take on the meta records claim is that while yes, it is useful to have a sort of "friend map" of who called who, etc what they really have is an index to the actual phone calls. Voice recognition isn't good enough yet to catalog/transcribe all the actual phone recordings so they have an index which allows them to map all those calls, allowing court orders, etc to gather data, get the actual recording, after the fact. Actual phone calls, as low fidelity as they are nowadays, can't be that hard to store.

[Jody_S]

(EDITED out a long winded opinionated tirade wherein I even managed to confuse myself.)

Re. your now deleted comments on scientists and their disdain for, and misuse of, statistics: in the field in which I studied, now a small sub-field of biology, no one that I ever met did their own statistics. For that matter, they had little involvement in the actual construction of experiments. All of this was left to their slaves, er, 'research assistants'. Once in a while, they would bring a mess of data to the biology dep't resident statistician (a lovely woman named Rhonda), and ask her to 'fix' it. Important errors are obviously made, every day. The idea of it all is that the system is supposed to be self-correcting. And given the current rate of progress in the field of biology, I tend to agree that it is.

[Jody_S]

Voice recognition isn't good enough yet to catalog/transcribe all the actual phone recordings ...

Maybe not the commercial software we have access to.

My comments are based on the simple observation that the computing power they have at their disposal, and what they are building, is of far greater capacity than what might be required to do what they have told us they're doing. And, obviously, if they have the technical ability to store the content of phone calls (in whatever electronic form), they will do so.

Regarding the question: should they be doing this if it allowed them to prosecute 'pre-crime'; my answer is obviously 'no', not unless they can prove their system is infallible, and that it cannot possibly be used for any other purpose.

[Dan Fromm]

Darin, the problem isn't whether they can predict behavior. The question is whether they can predict it well enough. We can't afford to sanction a person because we fear it might do something bad and be wrong.

More data isn't always the answer. I recall telling a client that the heap of data he was so proud of was complete garbage and that there was no way around the fact. He had a real problem but no statistical approach would solve it. We'd have to get creative and solve it without statistics. His response was to scream at me that we should be able to do something useful with all that data. No, not always.

Telcos collect message records to be able to bill customers and, when a call is handled by more than one company, each other. They're not allowed to listen to calls, don't, and don't record them. And they're not equipped to record them.

If you want to be paranoid, many entities including the secret police use deep packet sniffers to see and exploit what's sent across the 'net.

Brad, late in my career I used the stream of message records I received to identify some kinds of fraud that involved cell phones. We fired customers, sometimes, who used their cell phones to defraud us. I didn't use statistics to find bad customers, I kept a database of phone numbers that were used for the kinds of fraud I was concerned about and looked at what customers who called them did. This involved a lot of tedious unpleasant soul-deadening leg work, but in forensics one can't afford a mistake. I passed my finds to our corporate fraud group, who verified that the customers should be fired -- more leg work, but we couldn't afford to cut anyone off mistakenly -- and then cut off service for the cell phones they were allowed to cut off and passed the rest to our wireless group for treatment. In effect, police work informed by data, not data mining.

Brad, in the end machine learning models produce a probability of something conditioned on observable factors. How they get there is often very mysterious, but they produce exactly what a fitted targeting model does. Actually, I gather, not as much, since I'm not sure they produce confidence intervals. The key operational difference is the machine learning approach involves nearly continuous refitting. The conventional statistical approach fits once in a while. In my experience how often targeting models are refitted is driven largely by IT groups' reluctance to rescore records in data bases.