TSA Secure Flight use of commercial data
Bruce Schneier always has interesting insights in his Crypto-Gram newsletter. He wrote about the Transportation Security Administration (TSA) and the development of their Secure Flight system. His story is available here. I urge you to subscribe to his newsletter.
I had a few thoughts about "testing the use of commercial data".
I worked in a traditional IT environment for 35 years, and "testing" usually meant testing an application, or a prototype, or an algorithm. In any case, before you could test, you need to have something pretty well defined. If this is what the TSA is doing, they must have some rules or algorithm already defined for what they expect to accomplish. Have they explained that at all? Is there anyone they would need to explain that to - a working group similar to the one Bruce Schneier is on, or Congress? In any case, it brings up the question of how they will test - more on that below.
The other thing they might be doing besides testing an application/prototype/algorithm is analyzing the data - trying to discover ways to use the data to find threats. This is a reasonable approach from a purely theoretical view - altho it raises all sorts of privacy issues. I am not a statistician, but I think some such discovery tests look for data correlations that may be totally unexpected, and unexplainable. For example, they may "discover" that anyone who, say, reads Bruce Schneier's columns, is more likely to be a terrorist that someone who doesn't.
So now we get to the issue of determining the success or failure of their tests. It seems to me that the only thing they can do is match their results, from either program or data testing, against their list of known terrorists or threats. Of course, that means they will only find the TYPE of terrorist they already know about - it probably would NOT find terrorists like the ones who did the recent subway and bus bombings in London. If they think they have a way of finding new terrorists in the general population, how do they determine if it is accurate? Knock on your door and ask if you are a terrorist? Do a background check? Just add the names to the watch list? And what is the likelihood of getting accurate positive hits? Let's say there are 10,000 terrorists in this country (just to pick a number). That represents something like .003% of the population. So they are looking for a very small number of positive results.
I am sure these questions are not new, but I sure wish more answers were available.
I had a few thoughts about "testing the use of commercial data".
I worked in a traditional IT environment for 35 years, and "testing" usually meant testing an application, or a prototype, or an algorithm. In any case, before you could test, you need to have something pretty well defined. If this is what the TSA is doing, they must have some rules or algorithm already defined for what they expect to accomplish. Have they explained that at all? Is there anyone they would need to explain that to - a working group similar to the one Bruce Schneier is on, or Congress? In any case, it brings up the question of how they will test - more on that below.
The other thing they might be doing besides testing an application/prototype/algorithm is analyzing the data - trying to discover ways to use the data to find threats. This is a reasonable approach from a purely theoretical view - altho it raises all sorts of privacy issues. I am not a statistician, but I think some such discovery tests look for data correlations that may be totally unexpected, and unexplainable. For example, they may "discover" that anyone who, say, reads Bruce Schneier's columns, is more likely to be a terrorist that someone who doesn't.
So now we get to the issue of determining the success or failure of their tests. It seems to me that the only thing they can do is match their results, from either program or data testing, against their list of known terrorists or threats. Of course, that means they will only find the TYPE of terrorist they already know about - it probably would NOT find terrorists like the ones who did the recent subway and bus bombings in London. If they think they have a way of finding new terrorists in the general population, how do they determine if it is accurate? Knock on your door and ask if you are a terrorist? Do a background check? Just add the names to the watch list? And what is the likelihood of getting accurate positive hits? Let's say there are 10,000 terrorists in this country (just to pick a number). That represents something like .003% of the population. So they are looking for a very small number of positive results.
I am sure these questions are not new, but I sure wish more answers were available.
posted by Dan at 11:05 AM
link for this entry
0 Comments:
Post a Comment
<< Home