Building hadoop using ant

Tag: hadoop Author: sgy1111 Date: 2011-09-17

I tried to build hadoop-mapreduce-project using ant.I tried with maven it suceeded but i need to build it with ant. OR is their any alternative of "ant compile-mapred-test" in maven build? The error occurs is:

io problem while parsing ivy file: https://repository.apache.org/content/repositories/snapshots/org/apache/hadoop/hadoop-yarn-server-common/0.24.0-SNAPSHOT/hadoop-yarn-server-common-0.24.0-20110914.035045-1.pom: Impossible to load parent for file:/home/mohyt/.ivy2/cache/org.apache.hadoop/hadoop-yarn-server-common/ivy-0.24.0-SNAPSHOT.xml.original. Parent=org.apache.hadoop#hadoop-yarn-server;${yarn.version}
[ivy:resolve]       module not found: org.apache.hadoop#hadoop-yarn-server-common;0.24.0-SNAPSHOT
[ivy:resolve]   ==== apache-snapshot: tried
[ivy:resolve]     https://repository.apache.org/content/repositories/snapshots/org/apache/hadoop/hadoop-yarn- server/${yarn.version}/hadoop-yarn-server-${yarn.version}.pom
[ivy:resolve]     -- artifact org.apache.hadoop#hadoop-yarn-server;${yarn.version}!hadoop-yarn-server.jar:
[ivy:resolve]     https://repository.apache.org/content/repositories/snapshots/org/apache/hadoop/hadoop-yarn-server/${yarn.version}/hadoop-yarn-server-${yarn.version}.jar
[ivy:resolve]   ==== maven2: tried
[ivy:resolve]     http://repo1.maven.org/maven2/org/apache/hadoop/hadoop-yarn-server-common/0.24.0-SNAPSHOT/hadoop-yarn-server-common-0.24.0-SNAPSHOT.pom
[ivy:resolve]     -- artifact org.apache.hadoop#hadoop-yarn-server-common;0.24.0-SNAPSHOT!hadoop-yarn-server-common.jar:
[ivy:resolve]     http://repo1.maven.org/maven2/org/apache/hadoop/hadoop-yarn-server-common/0.24.0-SNAPSHOT/hadoop-yarn-server-common-0.24.0-SNAPSHOT.jar
[ivy:resolve]   io problem while parsing ivy file: https://repository.apache.org/content/repositories/snapshots/org/apache/hadoop/hadoop-mapreduce-client-core/0.24.0-SNAPSHOT/hadoop-mapreduce-client-core-0.24.0-20110914.035126-1.pom: Impossible to load parent for file:/home/mohyt/.ivy2/cache/org.apache.hadoop/hadoop-mapreduce-client-core/ivy-0.24.0-SNAPSHOT.xml.original. Parent=org.apache.hadoop#hadoop-mapreduce-client;${hadoop-mapreduce.version}
[ivy:resolve]       module not found: org.apache.hadoop#hadoop-mapreduce-client-core;0.24.0-SNAPSHOT
[ivy:resolve]   ==== apache-snapshot: tried
[ivy:resolve]     https://repository.apache.org/content/repositories/snapshots/org/apache/hadoop/hadoop-mapreduce-client/${hadoop-mapreduce.version}/hadoop-mapreduce-client-${hadoop-mapreduce.version}.pom
[ivy:resolve]     -- artifact org.apache.hadoop#hadoop-mapreduce-client;${hadoop-mapreduce.version}!hadoop-mapreduce-client.jar:
[ivy:resolve]     https://repository.apache.org/content/repositories/snapshots/org/apache/hadoop/hadoop-mapreduce-client/${hadoop-mapreduce.version}/hadoop-mapreduce-client-${hadoop-mapreduce.version}.jar
[ivy:resolve]   ==== maven2: tried
[ivy:resolve]     http://repo1.maven.org/maven2/org/apache/hadoop/hadoop-mapreduce-client-core/0.24.0-SNAPSHOT/hadoop-mapreduce-client-core-0.24.0-SNAPSHOT.pom
[ivy:resolve]     -- artifact org.apache.hadoop#hadoop-mapreduce-client-core;0.24.0-SNAPSHOT!hadoop-mapreduce-client-core.jar:
[ivy:resolve]     http://repo1.maven.org/maven2/org/apache/hadoop/hadoop-mapreduce-client-core/0.24.0-SNAPSHOT/hadoop-mapreduce-client-core-0.24.0-SNAPSHOT.jar
[ivy:resolve]   io problem while parsing ivy file: https://repository.apache.org/content/repositories/snapshots/org/apache/hadoop/hadoop-yarn-common/0.24.0-SNAPSHOT/hadoop-yarn-common-0.24.0-20110914.034958-1.pom: Impossible to load parent for file:/home/mohyt/.ivy2/cache/org.apache.hadoop/hadoop-yarn-common/ivy-0.24.0-SNAPSHOT.xml.original. Parent=org.apache.hadoop#hadoop-yarn;${yarn.version}
[ivy:resolve]       module not found: org.apache.hadoop#hadoop-yarn-common;0.24.0-SNAPSHOT
[ivy:resolve]   ==== apache-snapshot: tried
[ivy:resolve]     https://repository.apache.org/content/repositories/snapshots/org/apache/hadoop/hadoop-yarn/${yarn.version}/hadoop-yarn-${yarn.version}.pom
[ivy:resolve]     -- artifact org.apache.hadoop#hadoop-yarn;${yarn.version}!hadoop-yarn.jar:
[ivy:resolve]     https://repository.apache.org/content/repositories/snapshots/org/apache/hadoop/hadoop-yarn/${yarn.version}/hadoop-yarn-${yarn.version}.jar
[ivy:resolve]   ==== maven2: tried
[ivy:resolve]     http://repo1.maven.org/maven2/org/apache/hadoop/hadoop-yarn-common/0.24.0-SNAPSHOT/hadoop-yarn-common-0.24.0-SNAPSHOT.pom
[ivy:resolve]     -- artifact org.apache.hadoop#hadoop-yarn-common;0.24.0-SNAPSHOT!hadoop-yarn-common.jar:
[ivy:resolve]     http://repo1.maven.org/maven2/org/apache/hadoop/hadoop-yarn-common/0.24.0-SNAPSHOT/hadoop-yarn-common-0.24.0-SNAPSHOT.jar
[ivy:resolve]       ::::::::::::::::::::::::::::::::::::::::::::::
[ivy:resolve]       ::          UNRESOLVED DEPENDENCIES         ::
[ivy:resolve]       ::::::::::::::::::::::::::::::::::::::::::::::
[ivy:resolve]       :: org.apache.hadoop#hadoop-yarn-server-common;0.24.0-SNAPSHOT: not found
[ivy:resolve]       :: org.apache.hadoop#hadoop-mapreduce-client-core;0.24.0-SNAPSHOT: not found
[ivy:resolve]       :: org.apache.hadoop#hadoop-yarn-common;0.24.0-SNAPSHOT: not found
[ivy:resolve]       ::::::::::::::::::::::::::::::::::::::::::::::
[ivy:resolve] 
[ivy:resolve] :: USE VERBOSE OR DEBUG MESSAGE LEVEL FOR MORE DETAILS

Other Answer1

Hadoop is going towards maven from ant in branch-0.23 and trunk. As of now, some of the modules like MRv1 have to be mavenized though. Ant builds won't be supported in branch-0.23 and trunk. The artifacts produced by the ant and the maven build should be more or less the same. There is no point in using ant, unless there is a specific reason for using it.

Here are the different maven options for building Hadoop.