계획에 대한 파일 시스템 없습니다 : HDFS

내가 HBase를 볼트를 포함 내 스톰 토폴로지를 실행하고있을 때 나는 다음과 같은 오류가 있습니다.

java.io.IOException: No FileSystem for scheme: hdfs
at org.apache.hadoop.fs.FileSystem.getFileSystemClass(FileSystem.java:2298) ~[hadoop-common-2.0.0-cdh4.7.0.jar:na]
at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:2305) ~[hadoop-common-2.0.0-cdh4.7.0.jar:na]
at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:89) ~[hadoop-common-2.0.0-cdh4.7.0.jar:na]
at org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:2344) ~[hadoop-common-2.0.0-cdh4.7.0.jar:na]
at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:2326) ~[hadoop-common-2.0.0-cdh4.7.0.jar:na]
at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:353) ~[hadoop-common-2.0.0-cdh4.7.0.jar:na]
at org.apache.hadoop.fs.Path.getFileSystem(Path.java:194) ~[hadoop-common-2.0.0-cdh4.7.0.jar:na]
at org.apache.hadoop.hbase.util.DynamicClassLoader.<init>(DynamicClassLoader.java:104) ~[hbase-common-0.98.1-hadoop2.jar:0.98.1-hadoop2]
at org.apache.hadoop.hbase.protobuf.ProtobufUtil.<clinit>(ProtobufUtil.java:201) [hbase-client-0.98.1-hadoop2.jar:0.98.1-hadoop2]
at org.apache.hadoop.hbase.ClusterId.parseFrom(ClusterId.java:64) [hbase-client-0.98.1-hadoop2.jar:0.98.1-hadoop2]
at org.apache.hadoop.hbase.zookeeper.ZKClusterId.readClusterIdZNode(ZKClusterId.java:69) [hbase-client-0.98.1-hadoop2.jar:0.98.1-hadoop2]
at org.apache.hadoop.hbase.client.ZooKeeperRegistry.getClusterId(ZooKeeperRegistry.java:83) [hbase-client-0.98.1-hadoop2.jar:0.98.1-hadoop2]
at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.retrieveClusterId(HConnectionManager.java:857) [hbase-client-0.98.1-hadoop2.jar:0.98.1-hadoop2]
at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.<init>(HConnectionManager.java:662) [hbase-client-0.98.1-hadoop2.jar:0.98.1-hadoop2]
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) [na:1.7.0_72]
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57) [na:1.7.0_72]
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) [na:1.7.0_72]
at java.lang.reflect.Constructor.newInstance(Constructor.java:526) [na:1.7.0_72]
at org.apache.hadoop.hbase.client.HConnectionManager.createConnection(HConnectionManager.java:414) [hbase-client-0.98.1-hadoop2.jar:0.98.1-hadoop2]
at org.apache.hadoop.hbase.client.HConnectionManager.createConnection(HConnectionManager.java:393) [hbase-client-0.98.1-hadoop2.jar:0.98.1-hadoop2]
at org.apache.hadoop.hbase.client.HConnectionManager.getConnection(HConnectionManager.java:274) [hbase-client-0.98.1-hadoop2.jar:0.98.1-hadoop2]
at org.apache.hadoop.hbase.client.HTable.<init>(HTable.java:194) [hbase-client-0.98.1-hadoop2.jar:0.98.1-hadoop2]
at org.apache.hadoop.hbase.client.HTable.<init>(HTable.java:156) [hbase-client-0.98.1-hadoop2.jar:0.98.1-hadoop2]
at org.apache.storm.hbase.bolt.HBaseBolt$1.run(HBaseBolt.java:97) [storm-hbase-0.1.2.jar:na]
at org.apache.storm.hbase.bolt.HBaseBolt$1.run(HBaseBolt.java:94) [storm-hbase-0.1.2.jar:na]
at java.security.AccessController.doPrivileged(Native Method) [na:1.7.0_72]
at javax.security.auth.Subject.doAs(Subject.java:415) [na:1.7.0_72]
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1438) [hadoop-common-2.0.0-cdh4.7.0.jar:na]
at org.apache.storm.hbase.bolt.HBaseBolt.prepare(HBaseBolt.java:94) [storm-hbase-0.1.2.jar:na]
at backtype.storm.daemon.executor$fn__3352$fn__3364.invoke(executor.clj:690) [storm-core-0.9.2-incubating.jar:0.9.2-incubating]
at backtype.storm.util$async_loop$fn__452.invoke(util.clj:429) [storm-core-0.9.2-incubating.jar:0.9.2-incubating]
at clojure.lang.AFn.run(AFn.java:24) [clojure-1.5.1.jar:na]
at java.lang.Thread.run(Thread.java:745) [na:1.7.0_72]

16 : 44 : 32.839 [스레드 31 HbasePersistorBolt] 정보 backtype.storm.daemon.executor - 준비 HbasePersistorBolt 볼트 (5)

여기 내 pom.xml 파일입니다

<?xml version="1.0" encoding="UTF-8"?>

http://maven.apache.org/xsd/maven-4.0.0.xsd "> 원조-CIM fr.aid.cim 0.9-SNAPSHOT 4.0.0 이벤트 구조체 토폴로지

<dependencies>
    <!-- Hadoop Hbase Storm Kafka dependencies-->
    <dependency>
        <groupId>org.apache.hadoop</groupId>
        <artifactId>hadoop-client</artifactId>
        <version>${org.apache.hadoop.version}</version>
        <exclusions>
            <exclusion>
                <artifactId>com.google.protobuf</artifactId>
                <groupId>protobuf-java</groupId>
            </exclusion>
        </exclusions>
    </dependency>

    <dependency>
        <groupId>com.google.protobuf</groupId>
        <artifactId>protobuf-java</artifactId>
        <version>2.5.0</version>
    </dependency>

    <dependency>
        <groupId>com.github.ptgoetz</groupId>
        <artifactId>storm-hbase</artifactId>
        <version>${storm-hbase.version}</version>
    </dependency>

    <dependency>
        <groupId>org.apache.storm</groupId>
        <artifactId>storm-kafka</artifactId>
        <version>0.9.2-incubating</version>
    </dependency>

    <dependency>
        <groupId>org.apache.kafka</groupId>
        <artifactId>kafka_2.10</artifactId>
        <version>0.8.1.1</version>
        <exclusions>
            <exclusion>
                <groupId>org.apache.zookeeper</groupId>
                <artifactId>zookeeper</artifactId>
            </exclusion>
            <exclusion>
                <groupId>log4j</groupId>
                <artifactId>log4j</artifactId>
            </exclusion>
        </exclusions>
    </dependency>
    <!-- END Hadoop Hbase Storm Kafka dependencies-->

    <!-- Project Dependencies -->
    <dependency>
        <groupId>fr.aid.cim</groupId>
        <artifactId>commons</artifactId>
        <version>${project.version}</version>
    </dependency>

    <dependency>
        <groupId>fr.aid.cim</groupId>
        <artifactId>storm-hazelcast</artifactId>
        <version>${project.version}</version>
    </dependency>
    <!-- END Project Dependencies -->

    <!-- Integration TEST Dependencies -->
    <dependency>
        <groupId>org.apache.hadoop</groupId>
        <artifactId>hadoop-core</artifactId>
        <scope>test</scope>
    </dependency>

    <dependency>
        <groupId>org.apache.hadoop</groupId>
        <artifactId>hadoop-common</artifactId>
        <type>test-jar</type>
        <scope>test</scope>
    </dependency>

    <dependency>
        <groupId>org.apache.hbase</groupId>
        <artifactId>hbase</artifactId>
        <version>${org.apache.hbase.version}</version>
        <type>test-jar</type>
        <scope>test</scope>
    </dependency>

    <dependency>
        <groupId>org.apache.hadoop</groupId>
        <artifactId>hadoop-hdfs</artifactId>
        <version>${org.apache.hadoop.version}</version>
        <type>test-jar</type>
        <scope>test</scope>
    </dependency>

    <dependency>
        <groupId>org.apache.hadoop</groupId>
        <artifactId>hadoop-hdfs</artifactId>
        <version>${org.apache.hadoop.version}</version>
        <scope>test</scope>
    </dependency>

    <dependency>
        <groupId>org.apache.zookeeper</groupId>
        <artifactId>zookeeper</artifactId>
        <scope>test</scope>
    </dependency>
    <!-- END TEST Dependencies -->

    <!-- Other Dependencies -->
    <dependency>
        <groupId>org.json</groupId>
        <artifactId>json</artifactId>
        <version>20140107</version>
    </dependency>

    <dependency>
        <groupId>com.google.guava</groupId>
        <artifactId>guava</artifactId>
        <version>11.0</version>
    </dependency>

    <dependency>
        <groupId>com.fasterxml.jackson.core</groupId>
        <artifactId>jackson-databind</artifactId>
    </dependency>
    <!-- END Other Dependencies -->

</dependencies>

<build>
    <plugins>
        <plugin>
            <artifactId>maven-assembly-plugin</artifactId>
            <configuration>
                <descriptorRefs>
                    <descriptorRef>jar-with-dependencies</descriptorRef>
                </descriptorRefs>
            </configuration>
            <executions>
                <execution>
                    <id>make-assembly</id>
                    <phase>package</phase>
                    <goals>
                        <goal>single</goal>
                    </goals>
                </execution>
            </executions>
        </plugin>
        <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-dependency-plugin</artifactId>
        </plugin>
    </plugins>
</build>

<profiles>
    <profile>
        <id>local</id>
        <activation>
            <activeByDefault>true</activeByDefault>
        </activation>
        <dependencies>
            <dependency>
                <groupId>org.apache.storm</groupId>
                <artifactId>storm-core</artifactId>
            </dependency>
            <dependency>
                <groupId>org.apache.zookeeper</groupId>
                <artifactId>zookeeper</artifactId>
            </dependency>
        </dependencies>
    </profile>
    <profile>
        <id>cluster</id>
        <dependencies>
            <dependency>
                <groupId>org.apache.storm</groupId>
                <artifactId>storm-core</artifactId>
                <scope>provided</scope>
            </dependency>
            <dependency>
                <groupId>org.apache.zookeeper</groupId>
                <artifactId>zookeeper</artifactId>
                <scope>provided</scope>
            </dependency>
        </dependencies>
    </profile>

</profiles>

어떤 아이디어? 감사

해결법

==============================
1.컴파일 범위 종속성으로 하둡 HDFS를 추가하려고 :

컴파일 범위 종속성으로 하둡 HDFS를 추가하려고 :

<의존성> <의 groupId> org.apache.hadoop 하둡 HDFS <버전> $ {org.apache.hadoop.version}
==============================
2.다음 코드를 추가하려고

다음 코드를 추가하려고
```
 Configuration conf = new Configuration();
 conf.set("fs.hdfs.impl", "org.apache.hadoop.hdfs.DistributedFileSystem");
```
answer.https을 얻을 것입니다 아마 다음 링크 : //www.cnblogs.com/justinzhang/p/4983673.html

from https://stackoverflow.com/questions/26958865/no-filesystem-for-scheme-hdfs by cc-by-sa and MIT license

'HADOOP' 카테고리의 다른 글

[HADOOP] MRUnit와 브로가있는 InstantiationException을 제공합니다 (0)	2019.10.11
[HADOOP] 읽기 / Windows 서버에서 HDFS에 파일을 작성 (0)	2019.10.11
[HADOOP] 어떻게 하둡에서 자바에 대한 combineFileInputFormat를 사용 하는가? (0)	2019.10.11
[HADOOP] 웹 응용 프로그램에서 맵리 듀스 작업을 실행 (0)	2019.10.11
[HADOOP] 하둡 스트리밍 : 보류 상태에서 작업을 줄일 수는 "여지 작업을 줄일 수 없기 때문."라고 (0)	2019.10.11

복붙노트

[HADOOP] 계획에 대한 파일 시스템 없습니다 : HDFS

계획에 대한 파일 시스템 없습니다 : HDFS

해결법

1.컴파일 범위 종속성으로 하둡 HDFS를 추가하려고 :

2.다음 코드를 추가하려고

'HADOOP' 카테고리의 다른 글

티스토리툴바