[HADOOP] 오류 하이브에서 HBase를에 삽입하는 동안

오류 하이브에서 HBase를에 삽입하는 동안

나는 CDH 4.7.1 클러스터를 사용하고 있습니다. 지도는 100 %를 완료하고이 부분을 줄일 실패 보인다. 나는 하이브-site.xml을 아래 부분을 추가했습니다. 실제 오류 메시지가이 글의 마지막 부분에 붙여 넣습니다. 감사. 어떤 도움에 감사드립니다.

<property>
    <name>hive.aux.jars.path</name>
    <value> file:///opt/cloudera/parcels/CDH/lib/hbase/hbase.jar, 
        file:///opt/cloudera/parcels/CDH-4.7.1-1.cdh4.7.1.p0.47/lib/hive/lib/hive-hbase-handler-0.10.0-cdh4.7.1.jar,
        file:///opt/cloudera/parcels/CDH-4.7.1-1.cdh4.7.1.p0.47/lib/hive/lib/zookeeper.jar,
        file:///opt/cloudera/parcels/CDH-4.7.1-1.cdh4.7.1.p0.47/lib/hive/lib/guava-11.0.2.jar
    </value>
</property>

오류:

java.lang.ClassNotFoundException: org.apache.hadoop.hive.hbase.HBaseSerDe
Continuing ...
java.lang.ClassNotFoundException: org.apache.hadoop.hive.hbase.HiveHBaseTableInputFormat
Continuing ...
java.lang.ClassNotFoundException: org.apache.hadoop.hive.hbase.HiveHBaseTableOutputFormat
Continuing ...
java.lang.NullPointerException
Continuing ...
java.lang.NullPointerException
at org.apache.hadoop.hive.ql.exec.FileSinkOperator.initializeOp(FileSinkOperator.java:315)
at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:360)
at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:436)
at org.apache.hadoop.hive.ql.exec.Operator.initializeChildren(Operator.java:392)
at org.apache.hadoop.hive.ql.exec.Operator.initializeOp(Operator.java:377)
at org.apache.hadoop.hive.ql.exec.LimitOperator.initializeOp(LimitOperator.java:41)
at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:360)
at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:436)
at org.apache.hadoop.hive.ql.exec.Operator.initializeChildren(Operator.java:392)
at org.apache.hadoop.hive.ql.exec.ExtractOperator.initializeOp(ExtractOperator.java:40)
at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:360)
at org.apache.hadoop.hive.ql.exec.ExecReducer.configure(ExecReducer.java:150)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:106)
at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:75)
at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:133)
at org.apache.hadoop.mapred.ReduceTask.runOldReducer(ReduceTask.java:469)
at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:447)
at org.apache.hadoop.mapred.Child$4.run(Child.java:268)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:396)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1438)
at org.apache.hadoop.mapred.Child.main(Child.java:262)


MORE ERROR LOG: FROM REDUCE TASK
2015-02-19 16:20:09,624 WARN org.apache.hadoop.mapred.Child: Error running child
java.lang.RuntimeException: Error in configuring object
    at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:109)
    at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:75)
    at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:133)
    at org.apache.hadoop.mapred.ReduceTask.runOldReducer(ReduceTask.java:469)
    at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:447)
    at org.apache.hadoop.mapred.Child$4.run(Child.java:268)
    at java.security.AccessController.doPrivileged(Native Method)
    at javax.security.auth.Subject.doAs(Subject.java:396)
    at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1438)
    at org.apache.hadoop.mapred.Child.main(Child.java:262)
Caused by: java.lang.reflect.InvocationTargetException
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
    at java.lang.reflect.Method.invoke(Method.java:597)
    at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:106)
    ... 9 more
Caused by: java.lang.RuntimeException: Reduce operator initialization failed
    at org.apache.hadoop.hive.ql.exec.ExecReducer.configure(ExecReducer.java:157)
    ... 14 more
Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: java.lang.NullPointerException
    at org.apache.hadoop.hive.ql.exec.FileSinkOperator.initializeOp(FileSinkOperator.java:373)
    at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:360)
    at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:436)
    at org.apache.hadoop.hive.ql.exec.Operator.initializeChildren(Operator.java:392)
    at org.apache.hadoop.hive.ql.exec.Operator.initializeOp(Operator.java:377)
    at org.apache.hadoop.hive.ql.exec.LimitOperator.initializeOp(LimitOperator.java:41)
    at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:360)
    at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:436)
    at org.apache.hadoop.hive.ql.exec.Operator.initializeChildren(Operator.java:392)
    at org.apache.hadoop.hive.ql.exec.ExtractOperator.initializeOp(ExtractOperator.java:40)
    at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:360)
    at org.apache.hadoop.hive.ql.exec.ExecReducer.configure(ExecReducer.java:150)
    ... 14 more
Caused by: java.lang.NullPointerException
    at org.apache.hadoop.hive.ql.exec.FileSinkOperator.initializeOp(FileSinkOperator.java:315)
    ... 25 more

해결법

1. ==============================

   1.당신의 hive-env.sh에서 HIVE_AUX_JARS_PATH의 값을 설정합니다. 당신은 하이브에 액세스 할 수있는 경우, 쉘에서 다음을 수행합니다

   당신의 hive-env.sh에서 HIVE_AUX_JARS_PATH의 값을 설정합니다. 당신은 하이브에 액세스 할 수있는 경우, 쉘에서 다음을 수행합니다

   hive> add jar /opt/cloudera/parcels/CDH/lib/hbase/hbase.jar; 
   hive> add jar /opt/cloudera/parcels/CDH-4.7.1-1.cdh4.7.1.p0.47/lib/hive/lib/hive-hbase-handler-0.10.0-cdh4.7.1.jar;
   hive> add jar /opt/cloudera/parcels/CDH-4.7.1 1.cdh4.7.1.p0.47/lib/hive/lib/zookeeper.jar;
   hive> add jar /opt/cloudera/parcels/CDH-4.7.1-1.cdh4.7.1.p0.47/lib/hive/lib/guava-11.0.2.jar;
   

   이 당신을 도움이되기를 바랍니다

from https://stackoverflow.com/questions/28593227/error-while-inserting-from-hive-to-hbase by cc-by-sa and MIT license