Create and Insert to HBase table example

Login into master node :

[ec2-user@ip-123-45-67-89 ~]$ sudo hbase shell

HBase Shell; enter ‘help<RETURN>’ for list of supported commands.
Type “exit<RETURN>” to leave the HBase Shell
Version 1.3.1, rUnknown, Fri Sep 22 21:28:57 UTC 2017

hbase(main):001:0> list tables

Create table :

Command stntax

create ‘<table_name>’, ‘<column_family>’

Example

hbase(main):004:0> create ’employee_hbase’, ‘cf1’

Insert data into above table :

hbase(main):008:0> put ’employee_hbase’, ‘r1’, ‘cf1:empid’, ‘111’
hbase(main):008:0> put ’employee_hbase’, ‘r1’, ‘cf1:name’, ‘sudhir’
hbase(main):008:0> put ’employee_hbase’, ‘r1’, ‘cf1:dept’, ‘RnD’

Show data from table ;

hbase(main):015:0> get ’employee_hbase’, ‘r1’
COLUMN CELL
cf1:dept timestamp=1517963999636, value=RnD
cf1:empid timestamp=1517963967215, value=111
cf1:name timestamp=1517963994900, value=sudhir
1 row(s) in 0.0080 seconds

Count rows :

hbase(main):025:0> count ’employee_hbase’, { INTERVAL => 1 }

Advertisements

Unzip Multiple Files from Linux Command Line

Problem :

[hadoop@spradhan]$ unzip *.zip

Archive: a.csv.zip
caution: filename not matched: b.csv.zip
caution: filename not matched: c.csv.zip
caution: filename not matched: d.csv.zip
caution: filename not matched: e.csv.zip

Solution :

[hadoop@spradhan]$ unzip ‘*.zip’

If you run in background,

[hadoop@spradhan]$ nohup unzip ‘*.zip’ &

Copy file or folder from amazon S3 to EC2

  1. Install aws cli in the ec2 instance [if not installed]
  2. $ sudo yum install aws-cli

  3. Configure aws cli
  4. $ aws configure
    AWS Access Key ID [None]: <your_access_key>
    AWS Secret Access Key [None]:<your_secret_key>
    Default region name [None]:
    Default output format [None]:

  5. Execute sync command in ec2 instance
  6. aws s3 sync s3://<path_to_file> <ec2_local_path>

Installing Maven using Yum on EC2 instance (Amazon Linux)

  1. sudo wget http://repos.fedorapeople.org/repos/dchen/apache-maven/epel-apache-maven.repo -O /etc/yum.repos.d/epel-apache-maven.repo
  2. sudo sed -i s/\$releasever/6/g /etc/yum.repos.d/epel-apache-maven.repo
  3. sudo yum install -y apache-maven
  4. mvn –version

And you all set to run the “mvn” command in ec2 instance.

[ec2-user@ip-123-45-67-89 sudhir]$ mvn –version

Apache Maven 3.3.9 (bb52d8502b132ec0a5a3f4c09453c07478323dc5; 2015-11-10T16:41:47+00:00)

Maven home: /usr/share/apache-maven

Java version: 1.7.0_151, vendor: Oracle Corporation

Java home: /usr/lib/jvm/java-1.7.0-openjdk-1.7.0.151.x86_64/jre

Default locale: en_US, platform encoding: UTF-8

OS name: “linux”, version: “4.9.43-17.39.amzn1.x86_64”, arch: “amd64”, family: “unix”