[HADOOP] 설정 EC2 클러스터에 씽하는 소리를 사용하여

설정 EC2 클러스터에 씽하는 소리를 사용하여

나는 씽하는 소리를 사용하여 아마존 EC2에서 10 개 노드의 클러스터를 시작할 수 있었다. 지금은 R 및 패키지를 설치해야합니다.

이것은 명령입니다 :

whirr run-script --script /home/cloudera/TutorialBreen/config/whirr-ec2/install-r+packages.sh --config /home/cloudera/TutorialBreen/config/whirr-ec2/hadoop-ec2.properties

의 .sh 파일의 RMR 패키지에 대한 링크가 더 이상 살고 있지 않기 때문에 불행하게도, 난 오류가 발생합니다. 이것은 원래 설치-R + packeges.sh 파일입니다 :

sudo yum -y --enablerepo=epel install R R-devel

sudo R --no-save << EOF
install.packages(c('RJSONIO', 'itertools', 'digest', 'Rcpp', 'plyr'), repos="http://cran.revolutionanalytics.com", INSTALL_opts=c('--byte-compile') )
EOF

# if you always like to be up-to-date, you can install the latest version
# of rmr directly from RHadoop's github repository:
#
# branch=master
#
# wget --no-check-certificate https://github.com/RevolutionAnalytics/RHadoop/tarball/$branch -O - | tar zx
# mv RevolutionAnalytics-RHadoop* RHadoop
# sudo R CMD INSTALL --byte-compile RHadoop/rmr/pkg/

# but I'm usually not that adventurous:

wget --no-check-certificate https://github.com/downloads/RevolutionAnalytics/RHadoop/rmr_1.3.1.tar.gz
sudo R CMD INSTALL rmr_1.3.1.tar.gz

sudo su << EOF1 
cat >> /etc/profile <<EOF

export HADOOP_HOME=/usr/lib/hadoop

EOF
EOF1

나는 그것을 수정 rmr1.3.1에 새 링크를 삽입 : https://github.com/RevolutionAnalytics/RHadoop/tarball/rmr-1.3.1/RevolutionAnalytics-RHadoop-rmr-1.3.1-0-gfff2743.tar.gz

이 새로운 .sh로-파일입니다 :

sudo yum -y --enablerepo=epel install R R-devel

sudo R --no-save << EOF
install.packages(c('RJSONIO', 'itertools', 'digest', 'Rcpp', 'plyr'), repos="http://cran.revolutionanalytics.com", INSTALL_opts=c('--byte-compile') )
EOF

# if you always like to be up-to-date, you can install the latest version
# of rmr directly from RHadoop's github repository:
#
# branch=master
#
# wget --no-check-certificate https://github.com/RevolutionAnalytics/RHadoop/tarball/$branch -O - | tar zx
# mv RevolutionAnalytics-RHadoop* RHadoop
# sudo R CMD INSTALL --byte-compile RHadoop/rmr/pkg/

# but I'm usually not that adventurous:

wget --no-check-certificate https://github.com/RevolutionAnalytics/RHadoop/tarball/rmr-1.3.1/RevolutionAnalytics-RHadoop-rmr-1.3.1-0-gfff2743.tar.gz
sudo R CMD INSTALL RevolutionAnalytics-RHadoop-rmr-1.3.1-0-gfff2743.tar.gz

sudo su << EOF1 
cat >> /etc/profile <<EOF

export HADOOP_HOME=/usr/lib/hadoop

EOF
EOF1

불행하게도 그 중 하나가 작동하지 않습니다. 나는 (출력 페이지 후) 다음과 같은 오류가 발생합니다 :

    --2012-11-05 10:28:02--  https://github.com/RevolutionAnalytics/RHadoop/tarball/rmr-1.3.1/RevolutionAnalytics-RHadoop-rmr-1.3.1-0-gfff2743.tar.gz
    Resolving github.com... 207.97.227.239
    Connecting to github.com|207.97.227.239|:443... connected.
    WARNING: cannot verify github.com's certificate, issued by `/C=US/O=DigiCert Inc/OU=www.digicert.com/CN=DigiCert High Assurance EV CA-1':
      Unable to locally verify the issuer's authority.
    HTTP request sent, awaiting response... 302 Found
    Location: https://nodeload.github.com/RevolutionAnalytics/RHadoop/legacy.tar.gz/rmr-1.3.1 [following]
    --2012-11-05 10:28:02--  https://nodeload.github.com/RevolutionAnalytics/RHadoop/legacy.tar.gz/rmr-1.3.1
    Resolving nodeload.github.com... 207.97.227.252
    Connecting to nodeload.github.com|207.97.227.252|:443... connected.
    WARNING: cannot verify nodeload.github.com's certificate, issued by `/C=US/O=DigiCert Inc/OU=www.digicert.com/CN=DigiCert High Assurance CA-3':
      Unable to locally verify the issuer's authority.
    HTTP request sent, awaiting response... 200 OK
    Length: 15699365 (15M) [application/x-gzip]
    Saving to: `rmr-1.3.1.4'

100%[======================================>] 15,699,365  14.6M/s   in 1.0s    

2012-11-05 11:31:49 (14.6 MB/s) - `rmr-1.3.1.4' saved [15699365/15699365]

Warning: invalid package 'RevolutionAnalytics-RHadoop-rmr-1.3.1-0-gfff2743.tar.gz'
Error: ERROR: no packages specified

누군가가 내가 (아마도 설치-R + packages.sh 파일)을 변경해야 할 무엇을 알고 있나요?

해결법

from https://stackoverflow.com/questions/13235767/using-whirr-to-setup-ec2-cluster by cc-by-sa and MIT license