On big data, the Boston Marathon and civil liberties

[Commentary] For all the concerns over mobile phone logs, video footage and other data collection that could potentially be used to survail American citizens, it’s times like this that I think we see their real value. According to a Los Angeles Times article about the bomb attack at the Boston Marathon, the FBI has collected 10 terabytes that it’s sifting through in order to seek out clues about what exactly happened and who did it. Maybe I’m just a techno-optimist, but I find this very reassuring. It’s reassuring because I’ve spoken with so many smart people over the years who can do amazing things with data. Ten terabytes isn’t a huge data set by any stretch of the imagination, but it’s plenty to work with if it’s of high quality. It’s very possible there are some needles in that haystack of call logs, and I’m optimistic the analysts within the FBI — possibly with some outside help — will be able to find them.