Monday, October 6, 2014

Install oozie on ubuntu 14.04 with hadoop-2.5.0

Assuming we use /home/hadoop/tools/ as the installation path for oozie-4.0.1 with hadoop-2.5.0 installed on /home/hadoop/tools/hadoop-2.5.0. 



1. download & build
cd /home/hadoop/tools/
curl http://apache.cs.utah.edu/oozie/4.0.1/oozie-4.0.1.tar.gz | tar -zx
cd /home/hadoop/tools/oozie-4.0.1
bin/mkdistro.sh -DskipTests -Dhadoopversion=2.5.0  -DjavaVersion=1.7 -DtargetJavaVersion=1.7

2. make a new folder for the build
cp -R /home/hadoop/tools/oozie-4.0.1/distro/target/oozie-4.0.1-distro/oozie-4.0.1 /home/hadoop/tools/oozie
cd /home/hadoop/tools/oozie/bin
echo "PATH=$PWD:\$PATH" >> ~/.bash_profile
source ~/.bash_profile
#now "/home/hadoop/tools/oozie" is the home path for oozie


3. prepare for oozie web console
cd /home/hadoop/tools/oozie/
mkdir libext
cp /home/hadoop/tools/oozie-4.0.1/hadooplibs/target/oozie-4.0.1-hadooplibs.tar.gz .
tar xzvf oozie-4.0.1-hadooplibs.tar.gz
cp oozie-4.0.1/hadooplibs/hadooplib-2.3.0.oozie-4.0.1/* libext/
cd libext
wget http://extjs.com/deploy/ext-2.2.zip
rm -fr /home/hadoop/tools/oozie/oozie-4.0.1  /home/hadoop/tools/oozie/oozie-4.0.1-hadooplibs.tar.gz

4. add these two lines to "core-site.xml" under "/home/hadoop/tools/hadoop-2.5.0/etc/hadoop"
property name "hadoop.proxyuser.hadoop.hosts", property value "*" property name "hadoop.proxyuser.hadoop.groups, property value "*"



5. Start oozie
oozie-setup.shoozie-setup.sh prepare-war
#oozie-setup.sh sharelib create -fs hdfs://localhost:8020
oozie-setup.sh db create –run
oozied.sh start
oozie admin –oozie http://localhost:11000 -status

6. Run an example
cd /home/hadoop/tools/oozie/
tar -zxvf oozie-examples.tar.gz find examples/ -name "job.properties" -exec sed -i "s/localhost/master/g" '{}' \; hdfs dfs -rm -f -r /user/hadoop/examples  hdfs dfs -put examples examples oozie job -oozie http://master:11000/oozie -config examples/apps/map-reduce/job.properties -run #now open a web browser and access "http://master:11000/oozie", you will see the submitted job.
#reference:
http://www.quora.com/Has-anyone-tried-installing-oozie-4-0-0-with-apache-hadoop-2-2-0-and-java1-7-0_45-on-ubuntu