Monday, March 14, 2016

Install Spark 1.6.0 on Ubuntu 14.04


Environment:

Ubuntu 14.04 installed in VirtualBox

Steps:

1. Install JDK 8

###
# install oracle jdk 8
###
sudo apt-get -y install python-software-properties
sudo add-apt-repository ppa:webupd8team/java
sudo apt-get -y update
sudo apt-get -y install oracle-java8-installer

# find out the path of your Java installation:
sudo update-alternatives --config java

# for me it returns
# There is only one alternative in link group java (providing /usr/bin/java): /usr/lib/jvm/java-8-oracle/jre/bin/java

# edit /etc/environment
sudo gedit /etc/environment

# add this line into it, save and close
# JAVA_HOME="/usr/lib/jvm/java-8-oracle/jre/bin/java"
#
# then reload /etc/environment
source /etc/environment


2. Install scala 2.10

download scala 2.10.6 from http://www.scala-lang.org/download/2.10.6.html

extract it somewhere (I extract it to ~/Things/scala/scala-2.10.6)

again edit /etc/environment

add SCALA_HOME and PATH

after this the /etc/environment as below

PATH="/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/home/benbai/Things/scala/scala-2.10.6/bin"
JAVA_HOME="/usr/lib/jvm/java-8-oracle/jre/bin/java"
SCALA_HOME="/home/benbai/Things/scala/scala-2.10.6"


reload /etc/environment and check scala

source /etc/environment
scala -version
# for me it returns
# Scala code runner version 2.10.6 -- Copyright 2002-2013, LAMP/EPFL


3. Get Spark 1.6.0

download spark from http://spark.apache.org/downloads.html

I choose 1.6.0 pre-build for hadoop 2.6 and later

extract it somewhere (I extract it to ~/Things/spark/spark-1.6.0-bin-hadoop2.6)

go to spark directory and test it

cd ~/Things/spark/spark-1.6.0-bin-hadoop2.6
./bin/spark-shell
# then test some command


Testing result:



References:

How To Install Java on Ubuntu with Apt-Get

Scala Download Page

Scala Getting Started

Spark Quick Start

1 comment: