Please explain what's dfs.include file purpose and how to define it. I've added a new node to the Hadoop cluster but it's not identified by the namenode. In one of the posts I found that dfs.include can resolve this issue. Thank you in advance, Vladi
Hadoop dfs.include file
|
Other Answer1
Just including the node name in the dfs.include and mapred.include is not sufficient. The slave file has to be updated on the namenode/jobtracker. The tasktracker and the datanode have to be started on the new node and the refreshNodes command has to be run on the NameNode and the JobTracker to make them aware of the new node. Here are the instructions on how to do this. According to the 'Hadoop : The Definitive Guide'
|
Relate
- Compression in Hadoop Sequence File
- File split/partition in hadoop
- Hadoop file size Clarification
- Hadoop HDFS and Sequence File
- Hadoop - File to block mapping
- SFTP file system in hadoop
- BZip2 file read in Hadoop
- Hadoop Distribution File System
- hadoop in a shared file system
- File Transfer in datanodes - hadoop
- Hadoop Distributed File system
- Explanation of the hadoop file system
- hadoop- determine if a file is being written to
- Hadoop HDFS maximum file size
- Fastest access of a file using Hadoop
comments: