Up to Discussions in Open Management Consortium
2 Replies Last post: Feb 20, 2008 4:00 PM by Paco NATHAN  
Click to view John Willis's profile Beginner
John Willis
16 posts since
Dec 29, 2007

Feb 8, 2008 8:42 AM

Is anyone using Hadoop/Mapreduce?

I recently read an interesting blog about how Rackspace is using Hadoop to process email logs. They are able to provide answers to complex questions that would otherwise be impossible to answer. I was wondering if anyone on the OMC know if any good Hadoop stories.

 

TIA

johnmwillis.com

Tags: hadoop, mapreduce
Click to view John Butler's profile Novice
John Butler
1 posts since
Jan 24, 2008
Feb 9, 2008 2:33 AM in response to: John Willis
Re: Is anyone using Hadoop/Mapreduce?

Our team at my last place was using it for building large amounts of very large BDB indexes simultaneously. Log processing still seems to be the best candidate, though one of the neat pieces of Hadoop is DFS. DFS is fantastic for storing a large file across several nodes and making distributing parts and even copies of parts to other systems.

Click to view Paco NATHAN's profile Novice
Paco NATHAN
4 posts since
Dec 28, 2007
Feb 20, 2008 4:00 PM in response to: John Willis
Re: Is anyone using Hadoop/Mapreduce?

 

Here's a series of Hadoop articles on Ceteri.org intended as an intro to using it. Those blog articles are based on an open source project which I started as a place to collect Hadoop and MapReduce coding examples. I found that Hadoop application examples were few and far between. The first example uses data from the jyte.com cred graph to calculate a simplified PageRank metric.

 

 

 

 

We were using Hadoop at my previous firm to handle parts of our natural language processing and text analytics. I have a hunch that we'll be using Hadoop or something much like it at my next firm, wherever that might be

 

 

 

 

Paco NATHAN

 

 

More Like This

  • Retrieving data ...