[HADOOP] 클라우 데라 하이브 : 어디 JSON-serde-1.3.7 jar 파일을 추가합니다

클라우 데라 하이브 : 어디 JSON-serde-1.3.7 jar 파일을 추가합니다

나는 클라우 데라 5.8.0를 사용하고 있습니다

우선이 명령을 실행합니다 :

hive> ADD JAR /usr/lib/hive/lib/hive-serdes-1.0-SNAPSHOT.jar;
Added [/usr/lib/hive/lib/hive-serdes-1.0-SNAPSHOT.jar] to class path
Added resources: [/usr/lib/hive/lib/hive-serdes-1.0-SNAPSHOT.jar]

그리고 나는 JSON-serde-1.3.7 jar 파일을 추가하는 것보다

hive> ADD JAR /usr/lib/hive/lib/json-serde-1.3.7-jar-with- dependencies.jar;
Added [/usr/lib/hive/lib/json-serde-1.3.7-jar-with-dependencies.jar] to class path
Added resources: [/usr/lib/hive/lib/json-serde-1.3.7-jar-with-dependencies.jar]

하지만 하이브에이 테이블을 만들려고 할 때 :

CREATE EXTERNAL TABLE tweets (    
    id BIGINT,
    created_at STRING,
    source STRING,
    favorited BOOLEAN,
    retweet_count INT,
    retweeted_status STRUCT<
    text:STRING,
    `user`:STRUCT<screen_name:STRING,name:STRING>>,
    entities STRUCT<
    urls:ARRAY<STRUCT<expanded_url:STRING>>,
    user_mentions:ARRAY<STRUCT<screen_name:STRING,name:STRING>>,
    hashtags:ARRAY<STRUCT<text:STRING>>>,
    text STRING,
    `user` STRUCT<
    screen_name:STRING,
    name:STRING,
    friends_count:INT,
    followers_count:INT,
    statuses_count:INT,
    verified:BOOLEAN,
    utc_offset:INT,
    time_zone:STRING>,
    in_reply_to_screen_name STRING
)
ROW FORMAT SERDE 'com.cloudera.hive.SERDE.JSONSerDe'
LOCATION '/user/cloudera/flume/tweets'

나는이 오류가 발생합니다 :

오류 처리하는 동안 문 : 실패 : 실행 오류, 리턴 코드 (1) org.apache.hadoop.hive.ql.exec.DDLTask에서. serde를 확인할 수 없습니다 : com.cloudera.hive.SERDE.JSONSerDe

해결법

1. ==============================

   1.이 문제를 해결

   이 문제를 해결

   ADD JAR /usr/lib/hive/lib/hive-serdes-1.0-SNAPSHOT.jar;
   ADD JAR /usr/lib/hive/lib/json-serde-1.3.7-jar-with-dependencies.jar;
   
   CREATE EXTERNAL TABLE tweets (
      id BIGINT,
      created_at STRING,
      source STRING,
      favorited BOOLEAN,
      retweet_count INT,
      retweeted_status STRUCT<
         text:STRING,
         user:STRUCT<screen_name:STRING,name:STRING>>,
      entities STRUCT<
         urls:ARRAY<STRUCT<expanded_url:STRING>>,
         user_mentions:ARRAY<STRUCT<screen_name:STRING,name:STRING>>,
         hashtags:ARRAY<STRUCT<text:STRING>>>,
      text STRING,
      user STRUCT<
         screen_name:STRING,
         name:STRING,
         friends_count:INT,
         followers_count:INT,
         statuses_count:INT,
         verified:BOOLEAN,
         utc_offset:INT,
         time_zone:STRING>,
      in_reply_to_screen_name STRING
   ) 
       ROW FORMAT SERDE 'com.cloudera.hive.serde.JSONSerDe'
       LOCATION '/user/cloudera/flume/tweets';
   

from https://stackoverflow.com/questions/40854177/cloudera-hive-where-to-add-json-serde-1-3-7-jar-file by cc-by-sa and MIT license