Tuesday, September 13, 2011

HBase + Subversion + Eclipse + Windows

HBase + Subversion + Eclipse + Windows
(it should be easy to adapt for Linux)

Update : please note that since HBase-4336 / HBase 0.96 the source tree is split in more than one Maven module this post is no more relevant, i have created a new post on this subject : http://michaelmorello.blogspot.fr/2012/06/hbase-096-eclipse-maven.html

This is a simple setup in order to play with the source code of HBase under Microsoft Windows.
Since HBase use some Unix specific commands like chmod the only requirements here are Cygwin and a working Maven 3 environment. (It is obvious that you need Java and Eclipse, but you DON'T need anything else like the Eclipse Maven plugin or any SSH configuration)

1. Checkout the source code

The first step is to check out the source code from the Subversion repository. I did it under my cygwin home repository. In this example i want to play with the 0.90 branch :

svn co http://svn.apache.org/repos/asf/hbase/branches/0.90/ hbase-0.90


2. Generate the Eclipse configuration files
Go into the hbase-0.90 directory and run the following maven command :

mvn eclipse:eclipse

Maven will download half of the internet the first time and at the end you sould have .classpath and .project files at the root of the project directory.

3. Prepare your Eclipse workspace

Start Eclipse and select a new workspace :

Declare the M2_REPO variable :
Menu -> Window -> Preferences
Java -> Build Path -> Classpath Variable

4. Import the source code into your Eclipse workspace
Menu -> File -> Import ...
General -> Existing Projects into Workspace
In "Select root directory" select the directory where you have checked-out and executed the "mvn eclipse:eclipse" command

Now the project HBase should appear in your workspace :


5. Create a simple hbase-site.xml configuration file

In the "conf" directory edit the hbase-site.xml file (it should just contains the root xml element "configuration" for the moment) and add the following lines :

  <property>
    <name>hbase.defaults.for.version</name>
    <value>0.90.5-SNAPSHOT</value>
  </property>


  <property>
    <name>hbase.rootdir</name>
    <value>D:/cygwin/home/MORELLO/HBASE/WORK</value>
  </property>

The value of the hbase.rootdir should be an empty writable directory. It is used by HBase in replacement of the HDFS file system in order to store stuff like filestores, wal files, etc....

6. Prepare for ignition

Open the "Run Configurations ..." window from the menu Run -> Run configuration ...
     
        6.1 Main tab

  1. At the top replace "New_configuration" with "HBase (start)"
  2. Select the hbase project and in the Main class field enter : org.apache.hadoop.hbase.master.HMaster 

        6.2 Arguments tab

Add "start" as program argument

        6.3 Classpath tab

Add the "conf" directory in the bootstrap entries

The conf directory must appear in the bootstraps entries
        6.3 Environment tab

      As i said earlier HBase will use some Unix commands, so you must add the /bin directory of your Cygwin directory in your PATH environment variable :

7. Ignition !
Press the Run button, and if everything is ok you should see something like that :

You can also check the web ui with your favorite browser : http://localhost:60010/master.jsp


8. Play with the shell
In order to play with the shell create a new "Run configuration"
Menu -> Run -> Run Configurations

Create a new Run Configuration called HBase (Shell) :

Add org.jruby.Main as the main class

In the arguments tab add the absolute path to the hbase-0.90/bin/hirb.rb file
As in the 6.3 part add the conf directory to the bootstarp entries of the classpath

Press the Run button, and if everything is ok you should see something like that in the console :


In green are some commands that you can type in order to check that everything is ok.

9. Stop HBase
It is not recommended to press the red square in Eclipse and kill abruptly HBase. You can duplicate the "HBase (Start)" Run configuration into a "HBase (Stop)" one :

Just replace "start" by "stop"

Now if you select the "HBase (STOP)" run configuration and press the Run button HBase should be  gently stopped.

Small conclusion
I hope that this post will be useful to anybody that want to discover and play with HBase. It should be very easy to adapt for a Unix/Linux environment since you don't have to care about all the cygwin stuff.
Enjoy !

6 comments:

  1. hey Michael! Nice post, thanks for written it.

    I just want to precise something implicit in the second step "2. Generate the Eclipse configuration files". At the maven command 'mvn eclipse:eclipse' is executed, that should happen in Cygwin and never in Windows, otherwise the project will not compile.

    And second, if you decide to follow above steps with HBase 0.92, it is needed to at more properties in the file hbase-site.xml like:

    hbase.zookeeper.property.clientPort
    2181


    hbase.zookeeper.property.dataDir
    D:/cygwin/home/MORELLO/HBASE/zookeeper


    Cheers!

    ReplyDelete
    Replies
    1. Hi Nicolas,

      Thank you very much for your feedback !
      I will update the blog asap.

      Regards

      Delete
  2. Please Help me,

    When I run the Hbase(start) this warning appears in the console - hbase-default.xml file seems to be for and old version of HBase (0.92.0-SNAPSHOT), this version is 0.92.0

    ReplyDelete
    Replies
    1. Check that the 'conf' directory is in your classpath (step 6.3)
      In the hbase-site.xml file (in the 'conf' directory) replace the hbase.defaults.for.version value by 0.92.0 (step 5)

      Delete
  3. Merci beaucoup Michael pour ce tutoriel!

    ReplyDelete