Introducing LINQ to HPC: Processing Big Data on Windows
Monday, May 23, 2011
Big data is a big deal. Effectively processing large amounts of unstructured information can have real value, but doing it requires a technology focused on this problem.
If you're interested in the business value of big data, take a look at this recent McKinsey report
, which walks through possibilities in several different areas. If you're interested in the technology, you might read the Microsoft-sponsored white paper
I've written on LINQ to HPC, a new option for working with big data on Windows.
Part of Windows HPC Server, LINQ to HPC targets problems similar to those addressed by the open source Hadoop. And although it's just been released as a commercial product, LINQ to HPC is in fact a commercialization of Dryad, a technology created by Microsoft Research that's been used within Microsoft for several years. In a world full of big data, it's good to have options for how we make sense out of this sea of bits.