[HADOOP] 파이썬 모듈 impyla를 사용하여 Kerberos를의 하둡 클러스터에 연결

파이썬 모듈 impyla를 사용하여 Kerberos를의 하둡 클러스터에 연결

나는 하둡 클러스터를 베로스에 연결할 수 impyla 모듈을 사용하고 있습니다. 나는 액세스 할 hiveserver2 / 하이브하지만 난 다음 오류가 발생했습니다 :

from impala.dbapi import connect
import os
connection_string = 'hdp296m1.XXX.XXX.com'
conn = connect(host=connection_string, port=21050,auth_mechanism="GSSAPI",kerberos_service_name='testuser@Myrealm.COM',password='testuser')
cursor = conn.cursor()
cursor.execute('select count(*) form t_all_types_simple_t')
print cursor.description
results = cursor.fetchall()

스택 트레이스 :

[vagrant@localhost vagrant]$ python test_conn.py
Traceback (most recent call last):
  File "test_conn.py", line 4, in <module>
    conn = connect(host=connection_string, port=21050, auth_mechanism="GSSAPI",kerberos_service_name='testuser@Myrealm.COM',password='testuser')
  File "/usr/lib/python2.7/site-packages/impala/dbapi.py", line 147, in connect
    auth_mechanism=auth_mechanism)
  File "/usr/lib/python2.7/site-packages/impala/hiveserver2.py", line 758, in connect
    transport.open()
  File "/usr/lib/python2.7/site-packages/thrift_sasl/__init__.py", line 61, in open
    self._trans.open()
  File "/usr/lib64/python2.7/site-packages/thrift/transport/TSocket.py", line 101, in open
    message=message)
thrift.transport.TTransport.TTransportException: Could not connect to hdp296m1.XXX.XXX.com:21050

testuser를 내가 kinit 명령을 수행하기 위해 사용하는 것입니다 내 Kerberos 주체이다.

해결법

1. ==============================

   1.연결이 잘못된 것 같습니다 .. 시도

   연결이 잘못된 것 같습니다 .. 시도

   from impala.dbapi import *
   import sys, os
   # set your parms
   host=os.environ.get("CDH_HIVE",'x.x.x.x')
   port=os.environ.get("CDH_HIVE_port",'10000')
   auth_mechanism=os.environ.get("CDH_auth",'GSSAPI')
   user='hive' 
   db='mydb' 
   # No password use kinit 
   password=''
   # hive is principal with krb
   kbservice='hive'  
   
   class Hive:
   
       def __init__(self,db):
           self.database=db
           self.__conn = connect(host=host,
                               port=port,
                               auth_mechanism=auth_mechanism,
                               user=user,
                               password=password,
                               database=db,
                               kerberos_service_name=kbservice
                               )
   
   
           self.__cursor = self.__conn.cursor()
   
   
   h = Hive(db)
   

from https://stackoverflow.com/questions/41738526/connecting-to-kerberized-hadoop-cluster-using-python-module-impyla by cc-by-sa and MIT license