How to control file assignation in different slave in hadoop distributed system?

Tag: hadoop Author: wushihaowoaini Date: 2011-09-05
  1. How to control file assignation in different slave in hadoop distributed system?
  2. Is it possible to write 2 or more file in hadoop as map reduce task Simultaneously?

I am new to hadoop.It will be really helpful to me. If you know please answer.

Best Answer

This is my answer for your #1:

You can't directly control where map tasks go in your cluster or where files get sent in your cluster. The JobTracker and the NameNode handle these, respectively. The JobTracker will try to send the map tasks to be data local to improve performance. (I had to guess what you meant for your question , if I didn't get it right, please elaborate)

This is my answer for your #2:

MultipleOutputs is what you are looking for when you want to write multiple files out from a single reducer.

comments:

Is it not possible to write output file in running node without or outside the hdfs.