Monday, June 24, 2013

Learned Kiji in a few minutes

I had spent a bit time to learn "Introduction to Data Science". Today, I know the Kiji project. Kiji was built on Apache HBase and Apache Hadoop. I thought it's a good idea to learn HBase and Hadoop from it. You can read more about Architecture. So, I spent time a little bit about it. You can read on website, it's easy to install and use.
[surachart@linux01 ~]$ wget http://archive.kiji.org/tarballs/kiji-bento-albacore-1.0.5-release.tar.gz
[surachart@linux01 ~]$ tar xzf kiji-bento-*.tar.gz
[surachart@linux01 ~]$ cd kiji-bento-albacore
[surachart@linux01 kiji-bento-albacore]$ source bin/kiji-env.sh
Set KIJI_HOME=/home/surachart/kiji-bento-albacore/bin/..
Set KIJI_MR_HOME=/home/surachart/kiji-bento-albacore/bin/..
Set SCHEMA_SHELL_HOME=/home/surachart/kiji-bento-albacore/bin/../schema-shell
Set EXPRESS_HOME=/home/surachart/kiji-bento-albacore/bin/../express
Added kiji and kiji-schema-shell binaries to PATH.
Set BENTO_CLUSTER_HOME=/home/surachart/kiji-bento-albacore/cluster/bin/..
Set HADOOP_HOME=/home/surachart/kiji-bento-albacore/cluster/bin/../lib/hadoop-2.0.0-mr1-cdh4.1.2
Set HADOOP_CONF_DIR=/home/surachart/kiji-bento-albacore/cluster/bin/../lib/hadoop-2.0.0-mr1-cdh4.1.2/conf
Set HBASE_HOME=/home/surachart/kiji-bento-albacore/cluster/bin/../lib/hbase-0.92.1-cdh4.1.2
Set HBASE_CONF_DIR=/home/surachart/kiji-bento-albacore/cluster/bin/../lib/hbase-0.92.1-cdh4.1.2/conf
Added Hadoop, HBase, and bento-cluster binaries to PATH.

[surachart@linux01 kiji-bento-albacore]$ bento start
Running bento-cluster port configuration utility.

Hadoop configuration directory is: /home/surachart/kiji-bento-albacore/cluster/bin/../lib/hadoop-2.0.0-mr1-cdh4.1.2/conf

HBase configuration directory is: /home/surachart/kiji-bento-albacore/cluster/bin/../lib/hbase-0.92.1-cdh4.1.2/conf

Checking if default Hadoop/HBase ports are open...
Default Hadoop/HBase ports are open.
Using these in the Hadoop/HBase configuration for your cluster.

Writing bento-managed configuration.
Writing clean core-site.xml
Writing clean hdfs-site.xml
Writing clean mapred-site.xml
Writing clean hbase-site.xml
Configuration complete.

Starting bento-cluster...
Waiting for clusters to start...
bento-cluster started.

Cluster webapps can be visited at these web addresses:
HDFS NameNode:         http://localhost:50070
MapReduce JobTracker:  http://localhost:50030
HBase Master:          http://localhost:60010

Cluster services are available on the following ports:
HDFS NameNode:        8020
MapReduce JobTracker: 8021
Zookeeper:            2181

[surachart@linux01 kiji-bento-albacore]$ bin/kiji install
Creating kiji instance: kiji://localhost:2181/default/
Creating meta tables for kiji instance in hbase...
13/06/24 12:36:33 INFO org.kiji.schema.KijiInstaller: Installing kiji instance '     '.
13/06/24 12:36:41 INFO org.kiji.schema.KijiInstaller: Installed kiji instance 'kiji://localhost:2181/default/'.
Successfully created kiji instance: kiji://localhost:2181/default/
[surachart@linux01 kiji-bento-albacore]$
[surachart@linux01 kiji-bento-albacore]$ kiji-schema-shell
Kiji schema shell v1.0.0
Enter 'help' for instructions (without quotes).
Enter 'quit' to quit.
DDL statements must be terminated with a ';'
schema> CREATE TABLE users WITH DESCRIPTION 'A table for user names and email addresses'
     ->     ROW KEY FORMAT HASH PREFIXED(2)
     ->     WITH LOCALITY GROUP default WITH DESCRIPTION 'main storage' (
     ->       MAXVERSIONS = INFINITY,
     ->       TTL = FOREVER,
     ->       INMEMORY = false,
     ->       COMPRESSED WITH GZIP,
     ->       FAMILY info WITH DESCRIPTION 'basic information' (
     ->         name "string" WITH DESCRIPTION 'the user\'s name',
     ->         email "string"));
OK.
schema> show tables;
Table           Description
=========       ==========================================
users           A table for user names and email addresses
schema> describe users;
Table: users (A table for user names and email addresses)
Row key:
        key: STRING NOT NULL

Column family: info
        Description: basic information

        Column info:name (the user's name)
                Schema: "string"

        Column info:email ()
                Schema: "string"
schema> quit
Thank you for flying Kiji!
[surachart@linux01 kiji-bento-albacore]$ kiji ls
kiji://localhost:2181/default/users

[surachart@linux01 kiji-bento-albacore]$ kiji put     --target=kiji://.env/default/users/info:name     --entity-id='"surachart@gmail.com"'     --value='"Surachart Opun"'
[surachart@linux01 kiji-bento-albacore]$ kiji put     --target=kiji://.env/default/users/info:email     --entity-id='"surachart@gmail.com"'     --value='"surachart@gmail.com"'
[surachart@linux01 kiji-bento-albacore]$
[surachart@linux01 kiji-bento-albacore]$ kiji scan default/users
Scanning kiji table: kiji://localhost:2181/default/users/
entity-id=['surachart@gmail.com'] [1372056917172] info:name
                                 Surachart Opun
entity-id=['surachart@gmail.com'] [1372057025106] info:email
                                 surachart@gmail.com

[surachart@linux01 kiji-bento-albacore]$
Note: I think this open-source framework version was built from Coudera 4.1.2.  ^_______^
Read more!

No comments: