[HADOOP] 실행시 hadoop 페이지 랭크 오류

실행시 hadoop 페이지 랭크 오류

내 VM웨어에 hadoop을 설치하고 jar 파일 pagerank 파일을 디자인했습니다. 다음 명령 실행 :

hadoop jar PageRank-1.0.0.jar PageRankDriver init 입력 출력 2, 다음과 같은 오류가 발생합니다.

Failing this attempt.Diagnostics: [2017-12-01 12:55:58.278]Exception from container-launch.
Container id: container_1512069161738_0011_02_000001
Exit code: 1
Stack trace: ExitCodeException exitCode=1: 
        at org.apache.hadoop.util.Shell.runCommand(Shell.java:994)
        at org.apache.hadoop.util.Shell.run(Shell.java:887)
        at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:1212)
        at org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerExecutor.java:295)
        at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.launchContainer(ContainerLaunch.java:457)
        at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:277)
        at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:90)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)

[2017-12-01 12:55:58.278]
[2017-12-01 12:55:58.279]Container exited with a non-zero exit code 1. Last 4096 bytes of stderr :
Error: Could not find or load main class org.apache.hadoop.mapreduce.v2.app.MRAppMaster

Please check whether your etc/hadoop/mapred-site.xml contains the below configuration:
<property>
  <name>yarn.app.mapreduce.am.env</name>
  <value>HADOOP_MAPRED_HOME=${full path of your hadoop distribution directory}</value>
</property>
<property>
  <name>mapreduce.map.env</name>
  <value>HADOOP_MAPRED_HOME=${full path of your hadoop distribution directory}</value>
</property>
<property>
  <name>mapreduce.reduce.e nv</name>
  <value>HADOOP_MAPRED_HOME=${full path of your hadoop distribution directory}</value>
</property>
[2017-12-01 12:55:58.279]
[2017-12-01 12:55:58.279]Container exited with a non-zero exit code 1. Last 4096 bytes of stderr :
Error: Could not find or load main class org.apache.hadoop.mapreduce.v2.app.MRAppMaster

Please check whether your etc/hadoop/mapred-site.xml contains the below configuration:
<property>
  <name>yarn.app.mapreduce.am.env</name>
  <value>HADOOP_MAPRED_HOME=${full path of your hadoop distribution directory}</value>
</property>
<property>
  <name>mapreduce.map.env</name>
  <value>HADOOP_MAPRED_HOME=${full path of your hadoop distribution directory}</value>
</property>
<property>
  <name>mapreduce.reduce.e nv</name>
  <value>HADOOP_MAPRED_HOME=${full path of your hadoop distribution directory}</value>
</property>
[2017-12-01 12:55:58.279]
For more detailed output, check the application tracking page: http://number9.cs.stevens.edu:8088/cluster/app/application_1512069161738_0011 Then click on links to logs of each attempt.
. Failing the application.
2017-12-01 12:55:59,219 INFO mapreduce.Job: Counters: 0
Init Job Error

어떻게하면이 문제를 해결할 수 있을지 아무도 모른다.

해결법

1. ==============================

   1.아래와 같이 mapred-site.xml에서 구성을 변경하십시오.

   아래와 같이 mapred-site.xml에서 구성을 변경하십시오.

   <configuration>
       <property>
           <name>mapreduce.framework.name</name>
           <value>yarn</value>
       </property>
       <property>
           <name>yarn.app.mapreduce.am.env</name>
           <value>HADOOP_MAPRED_HOME=${HADOOP_HOME}</value>
       </property>
       <property>
           <name>mapreduce.map.env</name>
           <value>HADOOP_MAPRED_HOME=${HADOOP_HOME}</value>
       </property>
       <property>
           <name>mapreduce.reduce.env</name>
           <value>HADOOP_MAPRED_HOME=${HADOOP_HOME}</value>
       </property>
   </configuration>
   

2. ==============================

   2.위의 하나는 모든 서비스를 다시 시작해야하는 필요성을 편집 한 후에 저를 위해 일하고 있습니다.

   위의 하나는 모든 서비스를 다시 시작해야하는 필요성을 편집 한 후에 저를 위해 일하고 있습니다.

   사용 :

   stop-all.sh
   start-all.sh
   

from https://stackoverflow.com/questions/47599789/hadoop-pagerank-error-when-running by cc-by-sa and MIT license