복붙노트

[HADOOP] 컴파일 오류 WordCount.java를 컴파일하는 동안

HADOOP

컴파일 오류 WordCount.java를 컴파일하는 동안

나는 WordCount.java를 컴파일하는 동안 '기호를 찾을 수 없습니다'라는 오류가 계속. 나는 다른 라이브러리를 가져 오는 시도했지만 그것은 단지 오류 목록까지 추가합니다. 나는 추가해야합니다 몇 가지 구성 요소를 실종?

  javac -classpath /Documents/hadoop/hadoop-1.2.1/hadoop-core-1.2.1.jar:/Documents/hadoop/hadoop-1.2.1/lib/commons-cli-1.2.jar -d wordcount_classes WordCount.java


    WordCount.java:4: error: package org.apache.hadoop.fs does not exist
    import org.apache.hadoop.fs.Path;
                               ^
    WordCount.java:5: error: package org.apache.hadoop.io does not exist
    import org.apache.hadoop.io.*;
    ^
    WordCount.java:6: error: package org.apache.hadoop.mapred does not exist
    import org.apache.hadoop.mapred.*;
    ^
    WordCount.java:8: error: package org.apache.hadoop.mapred.lib.input does not exist
    import org.apache.hadoop.mapred.lib.input.FileInputFormat;
                                             ^
    WordCount.java:9: error: package org.apache.hadoop.mapred.lib.output does not exist
    import org.apache.hadoop.mapred.lib.output.FileOutputFormat;
                                          ^
    WordCount.java:10: error: package org.apache.hadoop.mapred.lib.input does not exist
    import org.apache.hadoop.mapred.lib.input.TextInputFormat;
                                         ^
    WordCount.java:15: error: cannot find symbol
        public static class Map extends MapReduceBase implements Mapper<LongWritable,    Text, Text, IntWritable> {
                                        ^
    symbol:   class MapReduceBase
    location: class WordCount
    WordCount.java:15: error: cannot find symbol
        public static class Map extends MapReduceBase implements Mapper<LongWritable,   Text, Text, IntWritable> {
                                                                 ^
    symbol:   class Mapper
    location: class WordCount
    WordCount.java:15: error: cannot find symbol
        public static class Map extends MapReduceBase implements Mapper<LongWritable,     Text, Text, IntWritable> {
                                                                        ^
    symbol:   class LongWritable
    location: class WordCount
    WordCount.java:15: error: cannot find symbol
        public static class Map extends MapReduceBase implements Mapper<LongWritable, Text, Text, IntWritable> {
                                                                                      ^
    symbol:   class Text
    location: class WordCount 
    WordCount.java:15: error: cannot find symbol
        public static class Map extends MapReduceBase implements Mapper<LongWritable,     Text, Text, IntWritable> {
                                                                                             ^
    symbol:   class Text
    location: class WordCount
WordCount.java:15: error: cannot find symbol
        public static class Map extends MapReduceBase implements Mapper<LongWritable, Text, Text, IntWritable> {
                                                                                                  ^
    symbol:   class IntWritable
    location: class WordCount
    WordCount.java:16: error: cannot find symbol
          private final static IntWritable one = new IntWritable(1);
                               ^
    symbol:   class IntWritable
    location: class Map
    WordCount.java:17: error: cannot find symbol
          private Text word = new Text();
                  ^
    symbol:   class Text
    location: class Map
    WordCount.java:19: error: cannot find symbol
          public void map(LongWritable key, Text value, OutputCollector<Text, IntWritable> output, Reporter reporter) throws IOException {
                          ^
    symbol:   class LongWritable
    location: class Map
    WordCount.java:19: error: cannot find symbol
          public void map(LongWritable key, Text value, OutputCollector<Text, IntWritable>   output, Reporter reporter) throws IOException {
                                            ^
    symbol:   class Text
    location: class Map
    WordCount.java:19: error: cannot find symbol
          public void map(LongWritable key, Text value, OutputCollector<Text, IntWritable>     output, Reporter reporter) throws IOException {
                                                        ^
    symbol:   class OutputCollector
    location: class Map
    WordCount.java:19: error: cannot find symbol
          public void map(LongWritable key, Text value, OutputCollector<Text, IntWritable> output, Reporter reporter) throws IOException {
                                                                        ^
  symbol:   class Text
  location: class Map
WordCount.java:19: error: cannot find symbol
          public void map(LongWritable key, Text value, OutputCollector<Text, IntWritable> output, Reporter reporter) throws IOException {
                                                                              ^
  symbol:   class IntWritable
  location: class Map
WordCount.java:19: error: cannot find symbol
          public void map(LongWritable key, Text value, OutputCollector<Text, IntWritable> output, Reporter reporter) throws IOException {
                                                                                                   ^
  symbol:   class Reporter
  location: class Map
WordCount.java:28: error: cannot find symbol
        public static class Reduce extends MapReduceBase implements Reducer<Text, IntWritable, Text, IntWritable> {
                                           ^
  symbol:   class MapReduceBase
  location: class WordCount
WordCount.java:28: error: cannot find symbol
        public static class Reduce extends MapReduceBase implements Reducer<Text, IntWritable, Text, IntWritable> {
                                                                    ^
  symbol:   class Reducer
  location: class WordCount
WordCount.java:28: error: cannot find symbol
        public static class Reduce extends MapReduceBase implements Reducer<Text, IntWritable, Text, IntWritable> {
                                                                            ^
  symbol:   class Text
  location: class WordCount
WordCount.java:28: error: cannot find symbol
        public static class Reduce extends MapReduceBase implements Reducer<Text, IntWritable, Text, IntWritable> {
                                                                                  ^
  symbol:   class IntWritable
  location: class WordCount
WordCount.java:28: error: cannot find symbol
        public static class Reduce extends MapReduceBase implements Reducer<Text, IntWritable, Text, IntWritable> {
                                                                                               ^
  symbol:   class Text
  location: class WordCount
WordCount.java:28: error: cannot find symbol
        public static class Reduce extends MapReduceBase implements Reducer<Text, IntWritable, Text, IntWritable> {
                                                                                                     ^
  symbol:   class IntWritable
  location: class WordCount
WordCount.java:29: error: cannot find symbol
          public void reduce(Text key, Iterator<IntWritable> values, OutputCollector<Text, IntWritable> output, Reporter reporter) throws IOException {
                             ^
  symbol:   class Text
  location: class Reduce
WordCount.java:29: error: cannot find symbol
          public void reduce(Text key, Iterator<IntWritable> values, OutputCollector<Text, IntWritable> output, Reporter reporter) throws IOException {
                                                ^
  symbol:   class IntWritable
  location: class Reduce
WordCount.java:29: error: cannot find symbol
          public void reduce(Text key, Iterator<IntWritable> values, OutputCollector<Text, IntWritable> output, Reporter reporter) throws IOException {
                                                                     ^
  symbol:   class OutputCollector
  location: class Reduce
WordCount.java:29: error: cannot find symbol
          public void reduce(Text key, Iterator<IntWritable> values, OutputCollector<Text, IntWritable> output, Reporter reporter) throws IOException {
                                                                                     ^
  symbol:   class Text
  location: class Reduce
WordCount.java:29: error: cannot find symbol
          public void reduce(Text key, Iterator<IntWritable> values, OutputCollector<Text, IntWritable> output, Reporter reporter) throws IOException {
                                                                                           ^
  symbol:   class IntWritable
  location: class Reduce
WordCount.java:29: error: cannot find symbol
          public void reduce(Text key, Iterator<IntWritable> values, OutputCollector<Text, IntWritable> output, Reporter reporter) throws IOException {
                                                                                                                ^
  symbol:   class Reporter
  location: class Reduce
WordCount.java:16: error: cannot find symbol
          private final static IntWritable one = new IntWritable(1);
                                                     ^
  symbol:   class IntWritable
  location: class Map
WordCount.java:17: error: cannot find symbol
          private Text word = new Text();
                                  ^
  symbol:   class Text
  location: class Map
WordCount.java:34: error: cannot find symbol
            output.collect(key, new IntWritable(sum));
                                    ^
  symbol:   class IntWritable
  location: class Reduce
WordCount.java:39: error: cannot find symbol
          JobConf conf = new JobConf(WordCount.class);
          ^
  symbol:   class JobConf
  location: class WordCount
WordCount.java:39: error: cannot find symbol
          JobConf conf = new JobConf(WordCount.class);
                             ^
  symbol:   class JobConf
  location: class WordCount
WordCount.java:42: error: cannot find symbol
          conf.setOutputKeyClass(Text.class);
                                 ^
  symbol:   class Text
  location: class WordCount
WordCount.java:43: error: cannot find symbol
          conf.setOutputValueClass(IntWritable.class);
                                   ^
  symbol:   class IntWritable
  location: class WordCount
WordCount.java:49: error: cannot find symbol
          conf.setInputFormat(TextInputFormat.class);
                              ^
  symbol:   class TextInputFormat
  location: class WordCount
WordCount.java:50: error: cannot find symbol
          conf.setOutputFormat(TextOutputFormat.class);
                               ^
  symbol:   class TextOutputFormat
  location: class WordCount
WordCount.java:52: error: cannot find symbol
          FileInputFormat.setInputPaths(conf, new Path(args[0]));
                                                  ^
  symbol:   class Path
  location: class WordCount
WordCount.java:52: error: cannot find symbol
          FileInputFormat.setInputPaths(conf, new Path(args[0]));
          ^
  symbol:   variable FileInputFormat
  location: class WordCount
WordCount.java:53: error: cannot find symbol
          FileOutputFormat.setOutputPath(conf, new Path(args[1]));
                                                   ^
  symbol:   class Path
  location: class WordCount
WordCount.java:53: error: cannot find symbol
          FileOutputFormat.setOutputPath(conf, new Path(args[1]));
          ^
  symbol:   variable FileOutputFormat
  location: class WordCount
WordCount.java:55: error: cannot find symbol
      JobClient.runJob(conf);
      ^
  symbol:   variable JobClient
  location: class WordCount
46 errors

해결법

  1. ==============================

    1.이 하나의 시도 할 수 있습니다 :

    이 하나의 시도 할 수 있습니다 :

    javac -classpath hadoop-core-1.2.1.jar:lib/commons-cli-1.2.jar -d [path where u want to put the result] [path]/WordCount.java
    
  2. ==============================

    2.당신의 클래스 패스에 필요한 모든 libs와 항아리를 포함하지 않는 것 같다.

    당신의 클래스 패스에 필요한 모든 libs와 항아리를 포함하지 않는 것 같다.

    하둡 2.7의 경우 쉽게 실행할 수 있습니다 :

    $hadoop classpath 
    

    이것은 당신이 당신의 클래스 패스에 포함 할 필요가 libs와 항아리의 전체 목록을 반환합니다.

    당신이 당신의 코드를 컴파일 할 때 또는 간단한 당신은 위의 명령을 사용할 수 있습니다

    $  javac -classpath $(hadoop classpath) -d wordcount_classes WordCount.java
    
  3. ==============================

    3.필요한 패키지가 javac의 클래스 경로에 포함되지 않은 때문이다.

    필요한 패키지가 javac의 클래스 경로에 포함되지 않은 때문이다.

    하둡 2.7.3의 경우, 3 개 패키지를 추가하는 것은 나를 위해 작동합니다 :

    javac -classpath $HADOOP_HOME$/share/hadoop/common/hadoop-common-2.7.3.jar:$HADOOP_HOME$/share/hadoop/mapreduce/hadoop-mapreduce-client-core-2.7.3.jar:$HADOOP_HOME$/share/hadoop/common/lib/commons-cli-1.2.jar
    -d wordcount_classes WordCount.java
    
  4. ==============================

    4.다른 답변 중 일부는 본론 및 실용, 빠른 솔루션을 제공 - 일 것이다 - 그러나 공식 하둡 단어 수의 튜토리얼은 빈 / 하둡에서 javac의 전화를하는 것이 좋습니다. 이 방법은 하둡은 자신의 라이브러리에 대한 클래스 경로 충돌 처리됩니다.

    다른 답변 중 일부는 본론 및 실용, 빠른 솔루션을 제공 - 일 것이다 - 그러나 공식 하둡 단어 수의 튜토리얼은 빈 / 하둡에서 javac의 전화를하는 것이 좋습니다. 이 방법은 하둡은 자신의 라이브러리에 대한 클래스 경로 충돌 처리됩니다.

    상기의 튜토리얼은 그러나 당신이 이렇게 실종 tools.jar를합니다 (JDK와 JRE에 대해서는 다른 사람이있다) 잘못된 lib 디렉토리에 HADOOP_CLASSPATH을 가리 키도록 오도 할 수있다. *

    내 솔루션은 다음과 같이 진행하는 것입니다 :

    $ export HADOOP_PREFIX="${absolute/path/to/your/hadoop/binaries}"
    $ export JDK_HOME="${absolute/path/to/your/jdk}"  
    
    $ export JAVA_HOME="${JDK_HOME}/jre"  
    $ export PATH="${JAVA_HOME}/bin:${PATH}"  
    $ export HADOOP_CLASSPATH="${JDK_HOME}/lib/tools.jar"  
    
    $ cd $HADOOP_PREFIX
    $ bin/hadoop com.sun.tools.javac.Main ./path/to/WordCount.java  
    

    실제 값은 JAVA_HOME에 할당하고 HADOOP_PREFIX 그 도구가 실제로 컴퓨터에 설치되어있는 위치를 지정해야합니다.

    *이 오류가 발생합니다 : 찾거나 컴파일시 메인 클래스의 com.sun.tools.javac.Main를로드 할 수 없습니다

  5. from https://stackoverflow.com/questions/22600927/compilation-errors-while-compiling-wordcount-java by cc-by-sa and MIT license