[SPRING] 버전 스프링 배치와 cloudera hadoop 사이의 비 호환성

버전 스프링 배치와 cloudera hadoop 사이의 비 호환성

나는 봄 배치 단어 카운트 프로그램을 시도하고 있었고 다음과 같은 버전 문제에 직면했다 :

ERROR [org.springframework.batch.core.step.AbstractStep] - <Encountered an error executing the step>
java.lang.IncompatibleClassChangeError: Found interface org.apache.hadoop.mapreduce.Counter, but class was expected

나는 Cloudera Hadoop2 cdh4.5.0과 Spring hadoop 버전 1.0.1.RELEASE를 사용한다. Spring 배치가 hadoop cdh4와 호환되므로 정확한 문제를 식별 할 수 없습니다.

내 의존성 트리는 아래와 같습니다.

[INFO] org.springframework.data:batch-wordcount:jar:0.0.1
[INFO] +- org.springframework:spring-context:jar:3.0.5.RELEASE:compile
[INFO] |  +- org.springframework:spring-aop:jar:3.0.5.RELEASE:compile
[INFO] |  +- org.springframework:spring-beans:jar:3.0.5.RELEASE:compile
[INFO] |  +- org.springframework:spring-expression:jar:3.0.5.RELEASE:compile
[INFO] |  \- org.springframework:spring-asm:jar:3.0.5.RELEASE:compile
[INFO] +- org.springframework:spring-core:jar:3.0.5.RELEASE:compile
[INFO] |  \- commons-logging:commons-logging:jar:1.1.1:compile
[INFO] +- org.springframework:spring-tx:jar:3.0.5.RELEASE:compile
[INFO] |  +- aopalliance:aopalliance:jar:1.0:compile
[INFO] +- org.springframework.batch:spring-batch-core:jar:2.2.0.RELEASE:compile
[INFO] |  +- com.thoughtworks.xstream:xstream:jar:1.3:compile
[INFO] |  |  \- xpp3:xpp3_min:jar:1.1.4c:compile
[INFO] |  +- org.codehaus.jettison:jettison:jar:1.1:compile
[INFO] +- org.springframework.batch:spring-batch-infrastructure:jar:2.2.0.RELEASE:compile
[INFO] |  \- org.springframework.retry:spring-retry:jar:1.0.2.RELEASE:compile
[INFO] +- org.springframework.batch:spring-batch-test:jar:2.2.0.RELEASE:compile
[INFO] |  +- (junit:junit:jar:4.10:compile - omitted for conflict with 4.11)
[INFO] |  +- (org.springframework:spring-test:jar:3.2.0.RELEASE:compile - omitted for conflict with 3.0.5.RELEASE)
[INFO] |  +- commons-io:commons-io:jar:1.4:compile
[INFO] |  +- commons-dbcp:commons-dbcp:jar:1.2.2:compile
[INFO] |  |  \- commons-pool:commons-pool:jar:1.3:compile
[INFO] |  \- commons-collections:commons-collections:jar:3.2:compile
[INFO] +- junit:junit:jar:4.11:test (scope not updated to compile)
[INFO] |  \- org.hamcrest:hamcrest-core:jar:1.3:test
[INFO] +- org.springframework.data:spring-data-hadoop:jar:1.0.2.RELEASE:compile
[INFO] +- org.apache.hadoop:hadoop-common:jar:2.0.0-cdh4.5.0:compile
[INFO] |  +- org.apache.hadoop:hadoop-annotations:jar:2.0.0-cdh4.5.0:compile
[INFO] |  |  \- jdk.tools:jdk.tools:jar:1.6:system
[INFO] |  +- com.google.guava:guava:jar:11.0.2:compile
[INFO] |  +- commons-net:commons-net:jar:3.1:compile
[INFO] |  +- javax.servlet:servlet-api:jar:2.5:compile
[INFO] |  +- org.mortbay.jetty:jetty:jar:6.1.26.cloudera.2:compile
[INFO] |  +- org.mortbay.jetty:jetty-util:jar:6.1.26.cloudera.2:compile
[INFO] |  +- com.sun.jersey:jersey-core:jar:1.8:compile
[INFO] |  +- com.sun.jersey:jersey-json:jar:1.8:compile
[INFO] |  |  +- com.sun.xml.bind:jaxb-impl:jar:2.2.3-1:compile
[INFO] |  |  |  \- javax.xml.bind:jaxb-api:jar:2.2.2:compile
[INFO] |  |  |     \- javax.activation:activation:jar:1.1:compile
[INFO] |  |  \- (com.sun.jersey:jersey-core:jar:1.8:compile - omitted for duplicate)
[INFO] |  +- log4j:log4j:jar:1.2.17:compile
[INFO] |  +- org.apache.hadoop:cloudera-jets3t:jar:2.0.0-cdh4.5.0:compile
[INFO] |  |  \- (net.java.dev.jets3t:jets3t:jar:0.9.0:compile - omitted for conflict with 0.6.1)
[INFO] |  +- net.java.dev.jets3t:jets3t:jar:0.6.1:compile
[INFO] |  |  +- (commons-codec:commons-codec:jar:1.3:compile - omitted for conflict with 1.4)
[INFO] |  |  +- (commons-logging:commons-logging:jar:1.1.1:compile - omitted for duplicate)
[INFO] |  |  \- (commons-httpclient:commons-httpclient:jar:3.1:compile - omitted for duplicate)
[INFO] |  +- (junit:junit:jar:4.8.2:compile - omitted for conflict with 4.11)
[INFO] |  +- commons-lang:commons-lang:jar:2.5:compile
[INFO] |  +- commons-configuration:commons-configuration:jar:1.6:compile
[INFO] |  |  +- commons-digester:commons-digester:jar:1.8:compile
[INFO] |  |  |  +- commons-beanutils:commons-beanutils:jar:1.7.0:compile
[INFO] |  |  \- commons-beanutils:commons-beanutils-core:jar:1.8.0:compile
[INFO] |  +- org.slf4j:slf4j-api:jar:1.6.1:compile
[INFO] |  +- org.slf4j:slf4j-log4j12:jar:1.6.1:runtime
[INFO] |  +- org.codehaus.jackson:jackson-core-asl:jar:1.8.8:compile
[INFO] |  +- org.codehaus.jackson:jackson-mapper-asl:jar:1.8.8:compile
[INFO] |  +- net.sf.kosmosfs:kfs:jar:0.3:compile
[INFO] |  +- com.google.protobuf:protobuf-java:jar:2.4.0a:compile
[INFO] |  +- org.apache.hadoop:hadoop-auth:jar:2.0.0-cdh4.5.0:compile
[INFO] |  +- com.jcraft:jsch:jar:0.1.42:compile
[INFO] |  \- org.apache.zookeeper:zookeeper:jar:3.4.5-cdh4.5.0:compile
[INFO] |     \- jline:jline:jar:0.9.94:compile
[INFO] +- org.apache.hadoop:hadoop-hdfs:jar:2.0.0-cdh4.5.0:compile
[INFO] |  +- tomcat:jasper-runtime:jar:5.5.23:compile
[INFO] |  |  \- commons-el:commons-el:jar:1.0:compile
[INFO] +- org.apache.hadoop:hadoop-tools:jar:2.0.0-mr1-cdh4.5.0:compile
[INFO] |  +- com.cloudera.cdh:hadoop-ant:pom:2.0.0-mr1-cdh4.5.0:compile
[INFO] |  \- org.apache.hadoop:hadoop-core:jar:2.0.0-mr1-cdh4.5.0:compile
[INFO] |     +- hsqldb:hsqldb:jar:1.8.0.10:compile
[INFO] |     +- org.eclipse.jdt:core:jar:3.1.1:compile
[INFO] +- org.hamcrest:hamcrest-all:jar:1.1:compile
[INFO] +- org.apache.hadoop:hadoop-client:jar:2.0.0-mr1-cdh4.5.0:compile
[INFO] +- org.apache.hadoop:hadoop-streaming:jar:2.0.0-mr1-cdh4.5.0:compile
[INFO] \- org.springframework:spring-test:jar:3.0.5.RELEASE:compile

또한, 중복 종속성에 대해 생략하면 오류가 발생합니까?

해결법

1. ==============================

   1.Cloudera CDH4와 함께 1.0.2.RELEASE-cdh4 버전을 사용해야합니다 -이 배포판의 특정 릴리즈는 Spring 저장소에서 사용할 수 있습니다 :

   Cloudera CDH4와 함께 1.0.2.RELEASE-cdh4 버전을 사용해야합니다 -이 배포판의 특정 릴리즈는 Spring 저장소에서 사용할 수 있습니다 :

   <repository>
     <id>spring-releases</id>
     <name>Spring Release Repository</name>
     <url>http://repo.spring.io/libs-release</url>
   </repository>
   

   자세한 내용은 여기 : https://github.com/spring-projects/spring-hadoop/wiki/Build-with-Cloudera-CDH4

from https://stackoverflow.com/questions/21133953/version-incompatibility-between-spring-batch-and-cloudera-hadoop by cc-by-sa and MIT license