맵리 듀스 작업 상태는 실행 중 상태에 갇혀있다

나는 Oozie (4.1.0)에서 맵리 듀스 프로그램을 실행하기 위해 노력하고있어.

그러나 그것의 상태는 RUNNING 상태에와 같은 상태에 붙어.

workflow.xml

<workflow-app xmlns="uri:oozie:workflow:0.4" name="simple-Workflow">
   <start to="RunMapreduceJob" />
   <action name="RunMapreduceJob">
      <map-reduce>
         <job-tracker>localhost:8088</job-tracker>
         <name-node>hdfs://localhost:9000</name-node>
         <prepare>
            <delete path="hdfs://localhost:9000/dataoutput"/>
         </prepare>
         <configuration>
            <property>
               <name>mapred.job.queue.name</name>
               <value>default</value>
            </property>
            <property>
               <name>mapred.mapper.class</name>
               <value>DataDividerByUser.DataDividerMapper</value>
            </property>
            <property>
               <name>mapred.reducer.class</name>
               <value>DataDividerByUser.DataDividerReducer</value>
            </property>
            <property>
               <name>mapred.output.key.class</name>
               <value>org.apache.hadoop.io.IntWritable</value>
            </property>
            <property>
               <name>mapred.output.value.class</name>
               <value>org.apache.hadoop.io.Text</value>
            </property>
            <property>
               <name>mapred.input.dir</name>
               <value>/data</value>
            </property>
            <property>
               <name>mapred.output.dir</name>
               <value>/dataoutput</value>
            </property>
         </configuration>
      </map-reduce>
      <ok to="end" />
      <error to="fail" />
   </action>
   <kill name="fail">
      <message>Mapreduce program Failed</message>
   </kill>
   <end name="end" />
</workflow-app>

job.properties

nameNode=hdfs://localhost:9000
jobTracker=localhost:8088
queueName=default
oozie.use.system.libpath=true
oozie.wf.application.path=${nameNode}/Config

작업 추적기는 여기에, 실행하는 스크린 샷입니다 https://prnt.sc/pbvb5i

OOzie URL에 JobInfo 점점 오류

JA009: Failed on local exception: com.google.protobuf.InvalidProtocolBufferException: Protocol message end-group tag did not match expected tag.; Host Details : local host is: "ec2-18-222-170-204.us-east-2.compute.amazonaws.com/18.222.170.204"; destination host is: "localhost":8088;

나는 무슨 일이 일어나고 무슨하시기 바랍니다 알고 있습니다 ...?

최신 정보:

이제 모든 노드 잘 https://prnt.sc/pc4a7n 노력하고 있습니다

Oozie 로그

hdfs://localhost:9000/user/hduser/share/lib/lib_20190928171545/sqoop/oozie-sharelib-sqoop-4.1.0.jar, hdfs://localhost:9000/user/hduser/share/l
ib/lib_20190928171545/sqoop/sqoop-1.4.3-hadoop100.jar]
2019-09-28 17:34:29,232  INFO Services:541 - SERVER[localhost] Initialized
2019-09-28 17:34:29,234  INFO Services:541 - SERVER[localhost] Running with JARs for Hadoop version [2.3.0]
2019-09-28 17:34:29,234  INFO Services:541 - SERVER[localhost] Oozie System ID [oozie-hdus] started!
2019-09-28 17:34:29,526  WARN AuthenticationFilter:341 - SERVER[localhost] AuthenticationToken ignored: AuthenticationToken expired
2019-09-28 17:34:29,526  WARN AuthenticationFilter:341 - SERVER[localhost] AuthenticationToken ignored: AuthenticationToken expired
2019-09-28 17:34:29,536  WARN AuthenticationFilter:341 - SERVER[localhost] AuthenticationToken ignored: AuthenticationToken expired
2019-09-28 17:34:29,536  WARN AuthenticationFilter:341 - SERVER[localhost] AuthenticationToken ignored: AuthenticationToken expired
2019-09-28 17:34:29,560  WARN AuthenticationFilter:341 - SERVER[localhost] AuthenticationToken ignored: AuthenticationToken expired
2019-09-28 17:34:29,560  WARN AuthenticationFilter:341 - SERVER[localhost] AuthenticationToken ignored: AuthenticationToken expired
2019-09-28 17:34:29,562  WARN AuthenticationFilter:341 - SERVER[localhost] AuthenticationToken ignored: AuthenticationToken expired
2019-09-28 17:34:29,562  WARN AuthenticationFilter:341 - SERVER[localhost] AuthenticationToken ignored: AuthenticationToken expired
2019-09-28 17:34:39,222  INFO StatusTransitService$StatusTransitRunnable:541 - SERVER[localhost] USER[-] GROUP[-] Acquired lock for [org.apache
.oozie.service.StatusTransitService]
2019-09-28 17:34:39,224  INFO StatusTransitService$StatusTransitRunnable:541 - SERVER[localhost] USER[-] GROUP[-] Running coordinator status se
rvice first instance
2019-09-28 17:34:39,222  INFO PauseTransitService:541 - SERVER[localhost] USER[-] GROUP[-] Acquired lock for [org.apache.oozie.service.PauseTra
nsitService]
2019-09-28 17:34:39,519  INFO StatusTransitService$StatusTransitRunnable:541 - SERVER[localhost] USER[-] GROUP[-] Running bundle status service
 first instance
2019-09-28 17:34:39,521  INFO CoordMaterializeTriggerService$CoordMaterializeTriggerRunnable:541 - SERVER[localhost] USER[-] GROUP[-] TOKEN[-] 
APP[-] JOB[-] ACTION[-] CoordMaterializeTriggerService - Curr Date= 2019-09-28T12:09Z, Num jobs to materialize = 0
2019-09-28 17:34:39,521  INFO CoordMaterializeTriggerService$CoordMaterializeTriggerRunnable:541 - SERVER[localhost] USER[-] GROUP[-] TOKEN[-] 
APP[-] JOB[-] ACTION[-] Released lock for [org.apache.oozie.service.CoordMaterializeTriggerService]
2019-09-28 17:34:39,570  INFO StatusTransitService$StatusTransitRunnable:541 - SERVER[localhost] USER[-] GROUP[-] Released lock for [org.apache
.oozie.service.StatusTransitService]
2019-09-28 17:34:39,571  INFO PurgeXCommand:541 - SERVER[localhost] USER[-] GROUP[-] TOKEN[-] APP[-] JOB[-] ACTION[-] STARTED Purge to purge Wo
rkflow Jobs older than [30] days, Coordinator Jobs older than [7] days, and Bundlejobs older than [7] days.
2019-09-28 17:34:39,571  INFO PurgeXCommand:541 - SERVER[localhost] USER[-] GROUP[-] TOKEN[-] APP[-] JOB[-] ACTION[-] ENDED Purge deleted [0] w
orkflows, [0] coordinatorActions, [0] coordinators, [0] bundles
2019-09-28 17:34:39,639  INFO PauseTransitService:541 - SERVER[localhost] USER[-] GROUP[-] Released lock for [org.apache.oozie.service.PauseTra
nsitService]
2019-09-28 17:35:39,571  INFO StatusTransitService$StatusTransitRunnable:541 - SERVER[localhost] USER[-] GROUP[-] Acquired lock for [org.apache
.oozie.service.StatusTransitService]
2019-09-28 17:35:39,572  INFO StatusTransitService$StatusTransitRunnable:541 - SERVER[localhost] USER[-] GROUP[-] Running coordinator status se
rvice from last instance time =  2019-09-28T12:04Z
2019-09-28 17:35:39,616  INFO StatusTransitService$StatusTransitRunnable:541 - SERVER[localhost] USER[-] GROUP[-] Running bundle status service
 from last instance time =  2019-09-28T12:04Z
2019-09-28 17:35:39,630  INFO StatusTransitService$StatusTransitRunnable:541 - SERVER[localhost] USER[-] GROUP[-] Released lock for [org.apache
.oozie.service.StatusTransitService]
2019-09-28 17:35:39,647  INFO PauseTransitService:541 - SERVER[localhost] USER[-] GROUP[-] Acquired lock for [org.apache.oozie.service.PauseTra
nsitService]
2019-09-28 17:35:39,662  INFO PauseTransitService:541 - SERVER[localhost] USER[-] GROUP[-] Released lock for [org.apache.oozie.service.PauseTra
nsitService]
2019-09-28 17:35:39,814  INFO ActionStartXCommand:541 - SERVER[localhost] USER[hduser] GROUP[-] TOKEN[] APP[simple-Workflow] JOB[0000000-190928
171702728-oozie-hdus-W] ACTION[0000000-190928171702728-oozie-hdus-W@RunMapreduceJob] Start action [0000000-190928171702728-oozie-hdus-W@RunMapr
educeJob] with user-retry state : userRetryCount [0], userRetryMax [0], userRetryInterval [10]
2019-09-28 17:36:39,631  INFO StatusTransitService$StatusTransitRunnable:541 - SERVER[localhost] USER[-] GROUP[-] Acquired lock for [org.apache
.oozie.service.StatusTransitService]
2019-09-28 17:36:39,632  INFO StatusTransitService$StatusTransitRunnable:541 - SERVER[localhost] USER[-] GROUP[-] Running coordinator status se
rvice from last instance time =  2019-09-28T12:05Z
2019-09-28 17:36:39,639  INFO StatusTransitService$StatusTransitRunnable:541 - SERVER[localhost] USER[-] GROUP[-] Running bundle status service
 from last instance time =  2019-09-28T12:05Z
2019-09-28 17:36:39,643  INFO StatusTransitService$StatusTransitRunnable:541 - SERVER[localhost] USER[-] GROUP[-] Released lock for [org.apache
.oozie.service.StatusTransitService]
2019-09-28 17:36:39,663  INFO PauseTransitService:541 - SERVER[localhost] USER[-] GROUP[-] Acquired lock for [org.apache.oozie.service.PauseTra
nsitService]
2019-09-28 17:36:39,685  INFO PauseTransitService:541 - SERVER[localhost] USER[-] GROUP[-] Released lock for [org.apache.oozie.service.PauseTra
nsitService]
2019-09-28 17:37:39,644  INFO StatusTransitService$StatusTransitRunnable:541 - SERVER[localhost] USER[-] GROUP[-] Acquired lock for [org.apache
.oozie.service.StatusTransitService]
2019-09-28 17:37:39,645  INFO StatusTransitService$StatusTransitRunnable:541 - SERVER[localhost] USER[-] GROUP[-] Running coordinator status se
rvice from last instance time =  2019-09-28T12:06Z
2019-09-28 17:37:39,656  INFO StatusTransitService$StatusTransitRunnable:541 - SERVER[localhost] USER[-] GROUP[-] Running bundle status service
 from last instance time =  2019-09-28T12:06Z
2019-09-28 17:37:39,661  INFO StatusTransitService$StatusTransitRunnable:541 - SERVER[localhost] USER[-] GROUP[-] Released lock for [org.apache
.oozie.service.StatusTransitService]
2019-09-28 17:37:39,686  INFO PauseTransitService:541 - SERVER[localhost] USER[-] GROUP[-] Acquired lock for [org.apache.oozie.service.PauseTra
nsitService]
2019-09-28 17:37:39,705  INFO PauseTransitService:541 - SERVER[localhost] USER[-] GROUP[-] Released lock for [org.apache.oozie.service.PauseTra
nsitService]
2019-09-28 17:37:53,297  WARN AuthenticationFilter:341 - SERVER[localhost] AuthenticationToken ignored: AuthenticationToken expired
2019-09-28 17:37:53,297  WARN AuthenticationFilter:341 - SERVER[localhost] AuthenticationToken ignored: AuthenticationToken expired
2019-09-28 17:37:53,299  WARN AuthenticationFilter:341 - SERVER[localhost] AuthenticationToken ignored: AuthenticationToken expired
2019-09-28 17:37:53,299  WARN AuthenticationFilter:341 - SERVER[localhost] AuthenticationToken ignored: AuthenticationToken expired
2019-09-28 17:37:53,312  WARN AuthenticationFilter:341 - SERVER[localhost] AuthenticationToken ignored: AuthenticationToken expired
2019-09-28 17:37:53,312  WARN AuthenticationFilter:341 - SERVER[localhost] AuthenticationToken ignored: AuthenticationToken expired
2019-09-28 17:37:53,478  WARN AuthenticationFilter:341 - SERVER[localhost] AuthenticationToken ignored: AuthenticationToken expired
2019-09-28 17:37:53,478  WARN AuthenticationFilter:341 - SERVER[localhost] AuthenticationToken ignored: AuthenticationToken expired
2019-09-28 17:37:53,631  WARN ParameterVerifier:544 - SERVER[localhost] USER[-] GROUP[-] TOKEN[-] APP[-] JOB[-] ACTION[-] The application does 
not define formal parameters in its XML definition
2019-09-28 17:37:53,893  INFO ActionStartXCommand:541 - SERVER[localhost] USER[hduser] GROUP[-] TOKEN[] APP[simple-Workflow] JOB[0000000-190928
173423962-oozie-hdus-W] ACTION[0000000-190928173423962-oozie-hdus-W@:start:] Start action [0000000-190928173423962-oozie-hdus-W@:start:] with u
ser-retry state : userRetryCount [0], userRetryMax [0], userRetryInterval [10]
2019-09-28 17:37:53,895  INFO ActionStartXCommand:541 - SERVER[localhost] USER[hduser] GROUP[-] TOKEN[] APP[simple-Workflow] JOB[0000000-190928
173423962-oozie-hdus-W] ACTION[0000000-190928173423962-oozie-hdus-W@:start:] [***0000000-190928173423962-oozie-hdus-W@:start:***]Action status=
DONE
2019-09-28 17:37:53,895  INFO ActionStartXCommand:541 - SERVER[localhost] USER[hduser] GROUP[-] TOKEN[] APP[simple-Workflow] JOB[0000000-190928
173423962-oozie-hdus-W] ACTION[0000000-190928173423962-oozie-hdus-W@:start:] [***0000000-190928173423962-oozie-hdus-W@:start:***]Action updated
 in DB!
2019-09-28 17:37:54,128  INFO ActionStartXCommand:541 - SERVER[localhost] USER[hduser] GROUP[-] TOKEN[] APP[simple-Workflow] JOB[0000000-190928
173423962-oozie-hdus-W] ACTION[0000000-190928173423962-oozie-hdus-W@RunMapreduceJob] Start action [0000000-190928173423962-oozie-hdus-W@RunMapr
educeJob] with user-retry state : userRetryCount [0], userRetryMax [0], userRetryInterval [10]
2019-09-28 17:38:39,662  INFO StatusTransitService$StatusTransitRunnable:541 - SERVER[localhost] USER[-] GROUP[-] Acquired lock for [org.apache
.oozie.service.StatusTransitService]
2019-09-28 17:38:39,663  INFO StatusTransitService$StatusTransitRunnable:541 - SERVER[localhost] USER[-] GROUP[-] Running coordinator status se
rvice from last instance time =  2019-09-28T12:07Z
2019-09-28 17:38:39,671  INFO StatusTransitService$StatusTransitRunnable:541 - SERVER[localhost] USER[-] GROUP[-] Running bundle status service
 from last instance time =  2019-09-28T12:07Z
2019-09-28 17:38:39,677  INFO StatusTransitService$StatusTransitRunnable:541 - SERVER[localhost] USER[-] GROUP[-] Released lock for [org.apache
.oozie.service.StatusTransitService]
2019-09-28 17:38:39,706  INFO PauseTransitService:541 - SERVER[localhost] USER[-] GROUP[-] Acquired lock for [org.apache.oozie.service.PauseTra
nsitService]
2019-09-28 17:38:39,722  INFO PauseTransitService:541 - SERVER[localhost] USER[-] GROUP[-] Released lock for [org.apache.oozie.service.PauseTra
nsitService]
2019-09-28 17:39:39,527  INFO CoordMaterializeTriggerService$CoordMaterializeTriggerRunnable:541 - SERVER[localhost] USER[-] GROUP[-] TOKEN[-] 
APP[-] JOB[-] ACTION[-] CoordMaterializeTriggerService - Curr Date= 2019-09-28T12:14Z, Num jobs to materialize = 0
2019-09-28 17:39:39,528  INFO CoordMaterializeTriggerService$CoordMaterializeTriggerRunnable:541 - SERVER[localhost] USER[-] GROUP[-] TOKEN[-] 
APP[-] JOB[-] ACTION[-] Released lock for [org.apache.oozie.service.CoordMaterializeTriggerService]
2019-09-28 17:39:39,679  INFO StatusTransitService$StatusTransitRunnable:541 - SERVER[localhost] USER[-] GROUP[-] Acquired lock for [org.apache
.oozie.service.StatusTransitService]
2019-09-28 17:39:39,680  INFO StatusTransitService$StatusTransitRunnable:541 - SERVER[localhost] USER[-] GROUP[-] Running coordinator status se
rvice from last instance time =  2019-09-28T12:08Z
2019-09-28 17:39:39,687  INFO StatusTransitService$StatusTransitRunnable:541 - SERVER[localhost] USER[-] GROUP[-] Running bundle status service
 from last instance time =  2019-09-28T12:08Z
2019-09-28 17:39:39,691  INFO StatusTransitService$StatusTransitRunnable:541 - SERVER[localhost] USER[-] GROUP[-] Released lock for [org.apache
.oozie.service.StatusTransitService]
2019-09-28 17:39:39,723  INFO PauseTransitService:541 - SERVER[localhost] USER[-] GROUP[-] Acquired lock for [org.apache.oozie.service.PauseTra
nsitService]
2019-09-28 17:39:39,743  INFO PauseTransitService:541 - SERVER[localhost] USER[-] GROUP[-] Released lock for [org.apache.oozie.service.PauseTra
nsitService]

하지만 여전히 맵리 듀스는 PREP의 실행 상태에있다. 오류는 다음과 같이 말에서 보여줍니다

JA009: Failed on local exception: com.google.protobuf.InvalidProtocolBufferException: Protocol message end-group tag did not match expected tag.; Host Details : local host is: "localhost/127.0.0.1"; destination host is: "localhost":8088;

업데이트 -1 하둡 FS -ls HDFS : // localhost를 : 9000 출력

drwxr-xr-x   - hduser supergroup          0 2019-09-28 19:44 hdfs://localhost:9000/user/hduser/oozie-hdus
drwxr-xr-x   - hduser supergroup          0 2019-09-28 17:15 hdfs://localhost:9000/user/hduser/share

해결법

from https://stackoverflow.com/questions/58139368/mapreduce-job-status-is-stuck-in-running-state by cc-by-sa and MIT license

'HADOOP' 카테고리의 다른 글

[HADOOP] , 데이터 삭제 후 어떤 방법을 하이브 외부 테이블을 드롭하지? (0)	2019.10.01
[HADOOP] 예외 JDBC를 통해 연결하는 동안 하이브 (0)	2019.10.01
[HADOOP] HDP 2.4, 어떻게 하나 개의 파일에 수로를 이용하여 하둡 맵리 듀스 로그를 수집하고 무엇이 가장 좋은 방법입니다 (0)	2019.10.01
[HADOOP] 어떻게 매퍼 하둡에서 시스템 환경 변수를 설정하는 방법? (0)	2019.10.01
[HADOOP] Giraph의 추정 클러스터 힙 xxxxMBs 물어 0메가바이트의 현재 가용성 클러스터 힙보다 더 크다. 작업 중단 (0)	2019.10.01

복붙노트

[HADOOP] 맵리 듀스 작업 상태는 실행 중 상태에 갇혀있다

맵리 듀스 작업 상태는 실행 중 상태에 갇혀있다

해결법

'HADOOP' 카테고리의 다른 글

티스토리툴바