Monday, October 6, 2014

Install oozie on ubuntu 14.04 with hadoop-2.5.0

Assuming we use /home/hadoop/tools/ as the installation path for oozie-4.0.1 with hadoop-2.5.0 installed on /home/hadoop/tools/hadoop-2.5.0. 

1. download & build
cd /home/hadoop/tools/
curl | tar -zx
cd /home/hadoop/tools/oozie-4.0.1
bin/ -DskipTests -Dhadoopversion=2.5.0  -DjavaVersion=1.7 -DtargetJavaVersion=1.7

2. make a new folder for the build
cp -R /home/hadoop/tools/oozie-4.0.1/distro/target/oozie-4.0.1-distro/oozie-4.0.1 /home/hadoop/tools/oozie
cd /home/hadoop/tools/oozie/bin
echo "PATH=$PWD:\$PATH" >> ~/.bash_profile
source ~/.bash_profile
#now "/home/hadoop/tools/oozie" is the home path for oozie

3. prepare for oozie web console
cd /home/hadoop/tools/oozie/
mkdir libext
cp /home/hadoop/tools/oozie-4.0.1/hadooplibs/target/oozie-4.0.1-hadooplibs.tar.gz .
tar xzvf oozie-4.0.1-hadooplibs.tar.gz
cp oozie-4.0.1/hadooplibs/hadooplib-2.3.0.oozie-4.0.1/* libext/
cd libext
rm -fr /home/hadoop/tools/oozie/oozie-4.0.1  /home/hadoop/tools/oozie/oozie-4.0.1-hadooplibs.tar.gz

4. add these two lines to "core-site.xml" under "/home/hadoop/tools/hadoop-2.5.0/etc/hadoop"
property name "hadoop.proxyuser.hadoop.hosts", property value "*" property name "hadoop.proxyuser.hadoop.groups, property value "*"

5. Start oozie prepare-war sharelib create -fs hdfs://localhost:8020 db create –run start
oozie admin –oozie http://localhost:11000 -status

6. Run an example
cd /home/hadoop/tools/oozie/
tar -zxvf oozie-examples.tar.gz find examples/ -name "" -exec sed -i "s/localhost/master/g" '{}' \; hdfs dfs -rm -f -r /user/hadoop/examples  hdfs dfs -put examples examples oozie job -oozie http://master:11000/oozie -config examples/apps/map-reduce/ -run #now open a web browser and access "http://master:11000/oozie", you will see the submitted job.