[HADOOP] Hadoop 2.2.0 MapReduce가 작동하지 않음

Hadoop 2.2.0 MapReduce가 작동하지 않음

Hadoop 2.2.0을 pseudo-distributed 모드로 설치 및 구성한 후 jps에서 볼 수있는 것처럼 모든 것이 실행 중입니다.

$ jps
2287 JobHistoryServer
1926 ResourceManager
2162 NodeManager
1834 DataNode
1756 NameNode
3013 Jps

그런 다음 단어 수를

hadoop jar $HADOOP_HOME/share/hadoop/mapreduce/hadoop-mapreduce-examples-2.2.0.jar wordcount /user/hdfs/file /output

그리고 실행은 (?) 다음과 같이 고정시킵니다.

$ hadoop jar $HADOOP_HOME/share/hadoop/mapreduce/hadoop-mapreduce-examples-2.2.0.jar wordcount /user/hdfs/file /output                                          
OpenJDK 64-Bit Server VM warning: You have loaded library /home/hduser/hadoop-src/hadoop-2.2.0/lib/native/libhadoop.so.1.0.0 which might have disabled stack guard. The VM will try to fix the stack guard now.
It's highly recommended that you fix the library with 'execstack -c <libfile>', or link it with '-z noexecstack'.
14/04/22 22:17:24 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
14/04/22 22:17:25 INFO client.RMProxy: Connecting to ResourceManager at /192.168.33.10:8032
14/04/22 22:17:25 INFO input.FileInputFormat: Total input paths to process : 1
14/04/22 22:17:25 INFO mapreduce.JobSubmitter: number of splits:1
14/04/22 22:17:25 INFO Configuration.deprecation: user.name is deprecated. Instead, use mapreduce.job.user.name
14/04/22 22:17:25 INFO Configuration.deprecation: mapred.jar is deprecated. Instead, use mapreduce.job.jar
14/04/22 22:17:25 INFO Configuration.deprecation: mapred.output.value.class is deprecated. Instead, use mapreduce.job.output.value.class
14/04/22 22:17:25 INFO Configuration.deprecation: mapreduce.combine.class is deprecated. Instead, use mapreduce.job.combine.class
14/04/22 22:17:25 INFO Configuration.deprecation: mapreduce.map.class is deprecated. Instead, use mapreduce.job.map.class
14/04/22 22:17:25 INFO Configuration.deprecation: mapred.job.name is deprecated. Instead, use mapreduce.job.name
14/04/22 22:17:25 INFO Configuration.deprecation: mapreduce.reduce.class is deprecated. Instead, use mapreduce.job.reduce.class
14/04/22 22:17:25 INFO Configuration.deprecation: mapred.input.dir is deprecated. Instead, use mapreduce.input.fileinputformat.inputdir
14/04/22 22:17:25 INFO Configuration.deprecation: mapred.output.dir is deprecated. Instead, use mapreduce.output.fileoutputformat.outputdir
14/04/22 22:17:25 INFO Configuration.deprecation: mapred.map.tasks is deprecated. Instead, use mapreduce.job.maps
14/04/22 22:17:25 INFO Configuration.deprecation: mapred.output.key.class is deprecated. Instead, use mapreduce.job.output.key.class
14/04/22 22:17:25 INFO Configuration.deprecation: mapred.working.dir is deprecated. Instead, use mapreduce.job.working.dir
14/04/22 22:17:26 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_1398204897594_0002
14/04/22 22:17:26 INFO impl.YarnClientImpl: Submitted application application_1398204897594_0002 to ResourceManager at /192.168.33.10:8032
14/04/22 22:17:26 INFO mapreduce.Job: The url to track the job: http://vagrant-ubuntu-saucy-64:8088/proxy/application_1398204897594_0002/
14/04/22 22:17:26 INFO mapreduce.Job: Running job: job_1398204897594_0002
14/04/22 22:17:36 INFO mapreduce.Job: Job job_1398204897594_0002 running in uber mode : false
14/04/22 22:17:36 INFO mapreduce.Job:  map 0% reduce 0%

어떤 아이디어?

해결법

1. ==============================

   1.문제는 yarn-site.xml 파일에 있습니다. 속성은 3072 Mb보다 커야하며 1024 Mb로 구성 했으므로 올바른 방법은 다음과 같습니다.

   문제는 yarn-site.xml 파일에 있습니다. 속성은 3072 Mb보다 커야하며 1024 Mb로 구성 했으므로 올바른 방법은 다음과 같습니다.

   <property>
       <name>yarn.nodemanager.resource.memory-mb</name>
       <value>4096</value>
       <description>Physical memory, in MB, to be made available to running containers</description>
   </property>
   

2. ==============================

   2.나는 로그가 충분히 상세하지 않다고 생각한다. 먼저 디버그 모드를 열 수있다 :

   나는 로그가 충분히 상세하지 않다고 생각한다. 먼저 디버그 모드를 열 수있다 :

   export HADOOP_ROOT_LOGGER=DEBUG,console
   

   더 많은 로그 및 붙여 넣기를 보려면 wordcount 작업을 실행하십시오.

from https://stackoverflow.com/questions/23231403/mapreduce-in-hadoop-2-2-0-not-working by cc-by-sa and MIT license