Crushing the Cost of Predicting the Future

Large and diverse sets of data have become abundant as more information is posted on the Internet, from stock prices and news pages to sensor readings. Now the cost of analyzing and visualizing that data is dropping fast, too, with implications for all kinds of decision-making.

A company called Recorded Future looks at 100,000 Web pages an hour, scanning across 50,000 sources that include everything from Securities and Exchange Commission filings to Twitter comments. The idea is to look for statements about the future, like notice of an annual meeting or predictions about when a product might be released, look at past developments and then create a “temporal index” that suggests trends. “The Web has come to reflect the world,” says Christopher Ahlberg, the co-founder and chief executive of Recorded Future. “We can use that to predict things.” Recorded Future is financed with $8 million from the likes of Google’s venture arm and In-Q-Tel, which makes investments to benefit the United States intelligence community, and its clients have included government agencies and banks. Its products include a $9,000-a-month service for hedge funds that plugs Recorded Future’s insights into their trading networks. There are both direct and indirect insights. Expected news, like a meeting or filing, can create volatility in stock prices. Recorded Future predicts five days ahead of time.