Skip to main content

HBase + Subversion + Eclipse + Windows

HBase + Subversion + Eclipse + Windows
(it should be easy to adapt for Linux)

Update : please note that since HBase-4336 / HBase 0.96 the source tree is split in more than one Maven module this post is no more relevant, i have created a new post on this subject : http://michaelmorello.blogspot.fr/2012/06/hbase-096-eclipse-maven.html

This is a simple setup in order to play with the source code of HBase under Microsoft Windows.
Since HBase use some Unix specific commands like chmod the only requirements here are Cygwin and a working Maven 3 environment. (It is obvious that you need Java and Eclipse, but you DON'T need anything else like the Eclipse Maven plugin or any SSH configuration)

1. Checkout the source code

The first step is to check out the source code from the Subversion repository. I did it under my cygwin home repository. In this example i want to play with the 0.90 branch :

svn co http://svn.apache.org/repos/asf/hbase/branches/0.90/ hbase-0.90


2. Generate the Eclipse configuration files
Go into the hbase-0.90 directory and run the following maven command :

mvn eclipse:eclipse

Maven will download half of the internet the first time and at the end you sould have .classpath and .project files at the root of the project directory.

3. Prepare your Eclipse workspace

Start Eclipse and select a new workspace :

Declare the M2_REPO variable :
Menu -> Window -> Preferences
Java -> Build Path -> Classpath Variable

4. Import the source code into your Eclipse workspace
Menu -> File -> Import ...
General -> Existing Projects into Workspace
In "Select root directory" select the directory where you have checked-out and executed the "mvn eclipse:eclipse" command

Now the project HBase should appear in your workspace :


5. Create a simple hbase-site.xml configuration file

In the "conf" directory edit the hbase-site.xml file (it should just contains the root xml element "configuration" for the moment) and add the following lines :

  <property>
    <name>hbase.defaults.for.version</name>
    <value>0.90.5-SNAPSHOT</value>
  </property>


  <property>
    <name>hbase.rootdir</name>
    <value>D:/cygwin/home/MORELLO/HBASE/WORK</value>
  </property>

The value of the hbase.rootdir should be an empty writable directory. It is used by HBase in replacement of the HDFS file system in order to store stuff like filestores, wal files, etc....

6. Prepare for ignition

Open the "Run Configurations ..." window from the menu Run -> Run configuration ...
     
        6.1 Main tab

  1. At the top replace "New_configuration" with "HBase (start)"
  2. Select the hbase project and in the Main class field enter : org.apache.hadoop.hbase.master.HMaster 

        6.2 Arguments tab

Add "start" as program argument

        6.3 Classpath tab

Add the "conf" directory in the bootstrap entries

The conf directory must appear in the bootstraps entries
        6.3 Environment tab

      As i said earlier HBase will use some Unix commands, so you must add the /bin directory of your Cygwin directory in your PATH environment variable :

7. Ignition !
Press the Run button, and if everything is ok you should see something like that :

You can also check the web ui with your favorite browser : http://localhost:60010/master.jsp


8. Play with the shell
In order to play with the shell create a new "Run configuration"
Menu -> Run -> Run Configurations

Create a new Run Configuration called HBase (Shell) :

Add org.jruby.Main as the main class

In the arguments tab add the absolute path to the hbase-0.90/bin/hirb.rb file
As in the 6.3 part add the conf directory to the bootstarp entries of the classpath

Press the Run button, and if everything is ok you should see something like that in the console :


In green are some commands that you can type in order to check that everything is ok.

9. Stop HBase
It is not recommended to press the red square in Eclipse and kill abruptly HBase. You can duplicate the "HBase (Start)" Run configuration into a "HBase (Stop)" one :

Just replace "start" by "stop"

Now if you select the "HBase (STOP)" run configuration and press the Run button HBase should be  gently stopped.

Small conclusion
I hope that this post will be useful to anybody that want to discover and play with HBase. It should be very easy to adapt for a Unix/Linux environment since you don't have to care about all the cygwin stuff.
Enjoy !

Comments

  1. hey Michael! Nice post, thanks for written it.

    I just want to precise something implicit in the second step "2. Generate the Eclipse configuration files". At the maven command 'mvn eclipse:eclipse' is executed, that should happen in Cygwin and never in Windows, otherwise the project will not compile.

    And second, if you decide to follow above steps with HBase 0.92, it is needed to at more properties in the file hbase-site.xml like:

    hbase.zookeeper.property.clientPort
    2181


    hbase.zookeeper.property.dataDir
    D:/cygwin/home/MORELLO/HBASE/zookeeper


    Cheers!

    ReplyDelete
    Replies
    1. Hi Nicolas,

      Thank you very much for your feedback !
      I will update the blog asap.

      Regards

      Delete
  2. Please Help me,

    When I run the Hbase(start) this warning appears in the console - hbase-default.xml file seems to be for and old version of HBase (0.92.0-SNAPSHOT), this version is 0.92.0

    ReplyDelete
    Replies
    1. Check that the 'conf' directory is in your classpath (step 6.3)
      In the hbase-site.xml file (in the 'conf' directory) replace the hbase.defaults.for.version value by 0.92.0 (step 5)

      Delete
  3. Merci beaucoup Michael pour ce tutoriel!

    ReplyDelete

Post a Comment

Popular posts from this blog

Orientée colonnes ?

Les bases NoSQL sont arrivées avec leur cortège de nouveautés et pour certaines d'entre elles une notion héritée de BigTable : celle de base de donnée orientée colonne. Cependant faire le lien entre l'article de Wikipedia et comprendre ce que permet réellement un base de donnée comme HBase n'est pas une chose évidente. En effet le simple fait de définir cette notion ne suffit pas toujours a bien comprendre quels sont les principes de conception du monde SQL qui peuvent être oubliés et ceux qui doivent être appris. Colonne or not colonne ? Prenons un modèle très simple de donnée et essayons de le transposer dans un modèle "orienté colonne": Comme on peut le voir on est passé d'un modèle à 2 dimensions (ligne x colonne) vers un modèle où une valeur est accédée au travers de 2  coordonnées qui sont ici (ligne, colonne) Cette notion de coordonnées est  importante  (c'est pour ça que je la met en gras 2 fois de suite) si l'on veut c

Row Count : HBase Aggregation example

With the coprocessors HBase 0.92 introduces a new way to process data directly on a region server. As a user this is definitively a very exciting feature : now you can easily define your own distributed data services. This post is not intended to help you how to define them (i highly recommend you to watch this presentation if you want to do so) but to quickly presents the new aggregation service shipped with HBase 0.92 that is built upon the endpoint coprocessor framework. 1. Enable AggregationClient coprocessor You have two choices : You can enable aggregation coprocessor on all your tables by adding the following lines to hbase-site.xml : <property> <name>hbase.coprocessor.user.region.classes</name> <value>org.apache.hadoop.hbase.coprocessor.AggregateImplementation</value> </property> or ...you can enable coprocessor only on a table throught the HBase shell : 1. disable the table hbase> disable ' mytable ' 2.