Tuesday, November 12, 2013

Learned a bit - ACCUMULO - set classpath in accumulo-site.xml

Thank You for comment on my post -  Learned a bit - ACCUMULO. So, I modified "<name>general.classpaths</name>" in conf/accumulo-site.xml file. It works -:)
  <property>
    <name>general.classpaths</name>
    <!--
       Add the following for hadoop-2.0
       $HADOOP_PREFIX/share/hadoop/common/.*.jar,
       $HADOOP_PREFIX/share/hadoop/common/lib/.*.jar,
       $HADOOP_PREFIX/share/hadoop/hdfs/.*.jar,
       $HADOOP_PREFIX/share/hadoop/mapreduce/.*.jar,
       $HADOOP_PREFIX/share/hadoop/yarn/.*.jar,
    -->
    <value>
      $ACCUMULO_HOME/server/target/classes/,
      $ACCUMULO_HOME/lib/accumulo-server.jar,
      $ACCUMULO_HOME/core/target/classes/,
      $ACCUMULO_HOME/lib/accumulo-core.jar,
      $ACCUMULO_HOME/start/target/classes/,
      $ACCUMULO_HOME/lib/accumulo-start.jar,
      $ACCUMULO_HOME/fate/target/classes/,
      $ACCUMULO_HOME/lib/accumulo-fate.jar,
      $ACCUMULO_HOME/proxy/target/classes/,
      $ACCUMULO_HOME/lib/accumulo-proxy.jar,
      $ACCUMULO_HOME/lib/[^.].*.jar,
      $ZOOKEEPER_HOME/zookeeper[^.].*.jar,
      $HADOOP_CONF_DIR,
      $HADOOP_PREFIX/[^.].*.jar,
      $HADOOP_PREFIX/lib/[^.].*.jar,
      $HADOOP_PREFIX/lib/hadoop/lib/.*.jar,
      $HADOOP_PREFIX/lib/hadoop/client/.*.jar,

    </value>
    <description>Classpaths that accumulo checks for updates and class files.
      When using the Security Manager, please remove the ".../target/classes/" values.
    </description>
  </property>

Example:

[root@centos01 accumulo-1.5.0]# bin/accumulo init
Uncaught exception: null
java.lang.reflect.InvocationTargetException
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:606)
        at org.apache.accumulo.start.Main.main(Main.java:41)
Caused by: java.lang.NoClassDefFoundError: org/apache/commons/logging/LogFactory
        at org.apache.commons.vfs2.impl.DefaultFileSystemManager.<init>(DefaultFileSystemManager.java:120)
        at org.apache.accumulo.start.classloader.vfs.FinalCloseDefaultFileSystemManager.<init>(FinalCloseDefaultFileSystemManager.java:21)
        at org.apache.accumulo.start.classloader.vfs.AccumuloVFSClassLoader.generateVfs(AccumuloVFSClassLoader.java:227)
        at org.apache.accumulo.start.classloader.vfs.AccumuloVFSClassLoader.getClassLoader(AccumuloVFSClassLoader.java:201)
        ... 5 more
Caused by: java.lang.ClassNotFoundException: org.apache.commons.logging.LogFactory
        at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
        at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
        at java.security.AccessController.doPrivileged(Native Method)
        at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
        at org.apache.accumulo.start.classloader.AccumuloClassLoader$2.loadClass(AccumuloClassLoader.java:241)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
        ... 9 more
[root@centos01 accumulo-1.5.0]#
[root@centos01 accumulo-1.5.0]# cp conf/accumulo-site.xml conf/accumulo-site.xml.orig
[root@centos01 accumulo-1.5.0]# vi conf/accumulo-site.xml
[root@centos01 accumulo-1.5.0]# diff conf/accumulo-site.xml.orig conf/accumulo-site.xml
111a112,113
>       $HADOOP_PREFIX/lib/hadoop/lib/.*.jar,
>       $HADOOP_PREFIX/lib/hadoop/client/.*.jar,
[root@centos01 accumulo-1.5.0]# bin/accumulo init
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/usr/lib/hadoop/lib/slf4j-log4j12-1.6.1.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/usr/lib/hadoop/client/slf4j-log4j12-1.6.1.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
2013-11-10 19:51:06,618 [util.NativeCodeLoader] WARN : Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
2013-11-10 19:51:08,274 [util.Initialize] INFO : Hadoop Filesystem is hdfs://localhost:8020
2013-11-10 19:51:08,280 [util.Initialize] INFO : Accumulo data dir is /accumulo
2013-11-10 19:51:08,280 [util.Initialize] INFO : Zookeeper server is localhost:2181
2013-11-10 19:51:08,280 [util.Initialize] INFO : Checking if Zookeeper is available. If this hangs, then you need to make sure zookeeper is running


Warning!!! Your instance secret is still set to the default, this is not secure. We highly recommend you change it.


You can change the instance secret in accumulo by using:
   bin/accumulo org.apache.accumulo.server.util.ChangeSecret oldPassword newPassword.
You will also need to edit your secret in your configuration file by adding the property instance.secret to your conf/accumulo-site.xml. Without this accumulo will not operate correctly
2013-11-10 19:51:08,684 [util.Initialize] FATAL: It appears this location was previously initialized, exiting ...
[root@centos01 accumulo-1.5.0]#
[root@centos01 accumulo-1.5.0]# bin/start-all.sh
Starting monitor on localhost
WARN : Max files open on localhost is 1024, recommend 65536
Starting tablet servers .... done
Starting tablet server on localhost
WARN : Max files open on localhost is 1024, recommend 65536
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/usr/lib/hadoop/lib/slf4j-log4j12-1.6.1.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/usr/lib/hadoop/client/slf4j-log4j12-1.6.1.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
2013-11-10 19:51:27,036 [util.NativeCodeLoader] WARN : Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
2013-11-10 19:51:32,349 [server.Accumulo] INFO : Attempting to talk to zookeeper
2013-11-10 19:51:32,743 [server.Accumulo] INFO : Zookeeper connected and initialized, attemping to talk to HDFS
2013-11-10 19:51:33,227 [server.Accumulo] INFO : Connected to HDFS
Starting master on localhost
WARN : Max files open on localhost is 1024, recommend 65536
Starting garbage collector on localhost
WARN : Max files open on localhost is 1024, recommend 65536
Starting tracer on localhost
WARN : Max files open on localhost is 1024, recommend 65536
[root@centos01 accumulo-1.5.0]#
[root@centos01 accumulo-1.5.0]#
[root@centos01 accumulo-1.5.0]# bin/accumulo shell -u root
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/usr/lib/hadoop/lib/slf4j-log4j12-1.6.1.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/usr/lib/hadoop/client/slf4j-log4j12-1.6.1.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
2013-11-10 19:52:34,686 [util.NativeCodeLoader] WARN : Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
Password: ********

Shell - Apache Accumulo Interactive Shell
-
- version: 1.5.0
- instance name: opun
- instance id: e799232d-5638-4122-b4aa-bdfe81eff3f7
-
- type 'help' for a list of available commands
-
root@opun> tables
!METADATA
foo
hellotable
mytest
perDayCounts
test
wordCount
root@opun>
Related Posts:
Learned a bit - ACCUMULO
Learned a bit - ACCUMULO - generate_monitor_certificate.sh

No comments: