Tuesday, July 14, 2015

Hadoop Installation

Greetings!!!
Here in this post i will illustrate you about the installation of the Hadoop (Hadoop 1.0.4) on the Ubuntu system. Before reading this post you must be familiar with basic Unix command like
  • sudo apt-get update
  • sudo apt-get install
  • ls
So first and far-most you need to install the openjdk-7-jdk on your machine.
1. Open Console on your ubuntu machine and type the following command to install the openjdk-7-jdk
sudo apt-get install openjdk-7-jdk
 
2. Once you hit enter it will ask for the root password just provide the password and proceed. 

3. Once the installation is complete you can check it typing following commands (it should produce the information shown in the screenshot.)
wagh@wagh-Inspiron-N4010:/usr/local$ javac 

 

4. You need to check one more thing jps
wagh@wagh-Inspiron-N4010:/usr/local$ jps
The above command should produce output similar to below. Count may differ in your case.



4. Download the hadoop-1.0.4-bin.gz from the :- http://hadoop.apache.org/releases.html
 
5.  Now goto your /usr/local directory and paste the above downloaded hadoop-1.0.4-bin.gz into it. 

6. Now extract the hadoop-1.0.4-bin.gz in the same drive using the following command
$tar -xvf hadoop-1.0.4-bin.gz
 
7. Next stepp is you need to generate a ssh keys on your local system. Just use the following command generate the ssh keys
$ ssh-keygen
 
8. The keys will be generated in your home directory. In my case the home directory is /home/rahul/ so the keys would be /home/rahul/.ssh/



 
9. Create one folder name authorized_keys and paste the id_rsa.pub key into it from the .ssh folder /home/rahul/.ssh/
 
10. Now test your ssh server by using the following command
$ ssh localhost
it should produce the following output




 
11. Now you need to configure the JAVA_HOME in the hadoop-env.sh shell script to make the Hadoop up and running. So goto the your Hadoop installation directory and in my case it is /usr/local/hadoop-1.0.4/conf. Than goto the conf and open hadoop-env.sh than modify the JAVA_HOME variable as per the java installed on your system.

1
2
3
# The java implementation to use.  Required.
# export JAVA_HOME=/usr/lib/j2sdk1.5-sun
export JAVA_HOME=/usr/lib/jvm/java-7-openjdk-amd64
 
12. Once you are done with all the installation steps just run the following command see the output
$ Hadoop jar hadoop/hadoop-examples-1.0.4.jar
Final Output should be
Job Finished in 2.904 seconds
Estimated value of Pi is 3.14000000000000000000

 
Congratualtions you are able to configure the Hadoop on your local ubuntu machine..
 

9 comments:

  1. Thanks for the valid post, it's crystal clear step. The expansion of internet and other business intelligence leads to large volume of data. Industries are looking for talented professionals to maintain and process huge volume of data with latest tools available in the market. Taking Hadoop Training in Chennai | Big Data Training in Chennai will ensure better career prospects for talented professionals.

    ReplyDelete
  2. • Great information, I like this kind of blog information really very nice and more I can easily new skills are develop after reading that post.
    tib co training in chennai

    ReplyDelete
  3. Thank you for the useful post. It helps a lot in my training. I share you with my students. Keep posting more.
    Selenium Training in Chennai

    ReplyDelete
  4. This comment has been removed by the author.

    ReplyDelete
  5. Neat and clear steps for installation of the Hadoop installation thanks for helping to solve the issue in installation.
    msbi training in bangalore btm

    ReplyDelete