OpenLava – Platform LSF Alternative

In the computing world there are two main cluster workload scheduling software: Grid Engine and Platform LSF. Most companies will choose one or the other. In the semiconductor field many choose LSF which is what most tools support out of the box. Industry will choose the software which has a fair price and good support.

OpenLava started as a simplified version of LSF back in 2007. It was discontinued 4 years later. The open source community and Teraproc have continued development of the source code since then. OpenLava continues to be a great open source alternative to Platform LSF, Teraproc does offer enterprise support through per node licensing.

This article is a quick summary of how quickly one can deploy a completely free workload schedule on your hosts. For any small businesses and start-ups out there wanting to develop applications supporting the popular LSF commands OpenLava seems to have you covered.

Now for this installation I am using Ubuntu Server 15.10 which is the latest software release as of Jan 2016.

Install the prerequisites
sudo apt-get install tcl-dev ncurses-dev

Download the OpenLava package
Go to http://www.openlava.org/download/download.html for the latest tarball location

wget http://www.openlava.org/tarball/openlava-3.1.tar.gz
tar xzvf openlava-3.1.tar.gz


Compile the project and install it

cd openlava-3.1
./configure --prefix=/opt/openlava
make
sudo make install


lsf binaries have now been installed to:
/opt/openlava/bin

There is a bunch of configuration to go over in order to have a functional cluster. Here are some of the basic to get the hosts added. It might be future post to get the queues and permissions setup for various users.

# Create a user to run all the lsf processes
sudo useradd -r openlava


# Copy the default configuration files over
cd ~/openlava-3.1/config
sudo cp openlava /etc/init.d
sudo cp lsf.cluster.openlava lsf.conf lsf.shared lsb.params lsb.hosts /opt/openlava/etc/
sudo cp openlava.sh openlava.csh /etc/profile.d/
. openlava.sh
cd ../

# Add the hosts to the cluster config file
sudo sed -i ‘s/# yourhost/node1\t!\t!\t1\t-\t-\t\nnode2\t!\t!\t1\t-\t-\t\nnode3\t!\t!\t1\t-\t-\t\nnode4\t!\t!\t1\t-\t-\t\nnode5\t!\t!\t1\t-\t-\t\n# yourhost/g’ /opt/openlava/etc/lsf.cluster.openlava

# Add the hosts to the /ets/hosts file so they can connect by name
printf “33.33.33.31 node1\n33.33.33.32 node2\n33.33.33.33 node3\n33.33.33.34 node4\n33.33.33.35 node5” | sudo tee -a /etc/hosts

# Ensure the permissions are correct for the installation
sudo chown -R openlava:openlava /opt/openlava

# Ensure the service starts
sudo chmod +x /etc/init.d/openlava
sudo systemctl enable openlava
sudo systemctl start openlava
sudo systemctl status openlava

Since this is a bit laborious going to each host and performing the above here is a vagrant file which will start up 5 test hosts and provision them the same as above. I just thought it best to go over what is to be done before handing you over a cut-and-paste script for testing.

Filename: Vagrantfile

# -*- mode: ruby -*-
# vi: set ft=ruby :

# All Vagrant configuration is done below. The "2" in Vagrant.configure
# configures the configuration version (we support older styles for
# backwards compatibility). Please don't change it unless you know what
# you're doing.
Vagrant.configure(2) do |config|

  config.vm.define :node1 do |node|
    node.vm.box = "ubuntu/vivid64"
    node.vm.hostname = "node1"
    node.vm.network "private_network", ip: "33.33.33.31"
  end

  config.vm.define :node2 do |node|
    node.vm.box = "ubuntu/vivid64"
    node.vm.hostname = "node2"
    node.vm.network "private_network", ip: "33.33.33.32"
  end

  config.vm.define :node3 do |node|
    node.vm.box = "ubuntu/vivid64"
    node.vm.hostname = "node3"
    node.vm.network "private_network", ip: "33.33.33.33"
  end

  config.vm.define :node4 do |node|
    node.vm.box = "ubuntu/vivid64"
    node.vm.hostname = "node4"
    node.vm.network "private_network", ip: "33.33.33.34"
  end

  config.vm.define :node5 do |node|
    node.vm.box = "ubuntu/vivid64"
    node.vm.hostname = "node5"
    node.vm.network "private_network", ip: "33.33.33.35"
  end

  # Install OpenLava to hosts
  config.vm.provision "shell", privileged: false, path: "install_openlava.sh"
end

Filename: install_openlava.sh

sudo apt-get update
sudo apt-get upgrade -y
sudo apt-get install -y tcl-dev ncurses-dev autoconf libtool

wget http://www.openlava.org/tarball/openlava-3.1.tar.gz
tar xzvf openlava-3.1.tar.gz
cd openlava-3.1

./configure --prefix=/opt/openlava
make
sudo make install

# Create a openlava user to own all lsf processes
sudo useradd -r openlava

# Copy the configurations to the system
cd config
sudo cp openlava /etc/init.d
sudo cp lsf.cluster.openlava lsf.conf lsf.shared lsb.params lsb.hosts /opt/openlava/etc/
sudo cp openlava.sh openlava.csh /etc/profile.d/
. openlava.sh
cd ../

# Add the hosts to the cluster config file
sudo sed -i 's/# yourhost/node1\t!\t!\t1\t-\t-\t\nnode2\t!\t!\t1\t-\t-\t\nnode3\t!\t!\t1\t-\t-\t\nnode4\t!\t!\t1\t-\t-\t\nnode5\t!\t!\t1\t-\t-\t\n# yourhost/g' /opt/openlava/etc/lsf.cluster.openlava

# Add the hosts to the /ets/hosts file so they can connect by name
sudo printf "33.33.33.31 node1\n33.33.33.32 node2\n33.33.33.33 node3\n33.33.33.34 node4\n33.33.33.35 node5" | sudo tee -a /etc/hosts

# Ensure the permissions are correct for the installation
sudo chown -R openlava:openlava /opt/openlava

# Either add the new /opt/openlava/bin to the path or symlink the functions
sudo ln -s /opt/openlava/bin/* /usr/local/bin

# Start up the main service
sudo chmod +x /etc/init.d/openlava
sudo systemctl enable openlava
sudo systemctl start openlava
sudo systemctl status openlava

# Sanity check of install
lsid
lshosts
bhosts

bsub "sleep 10 && printf 'Completed Job'"
bsub -J "sampleArray[1-25]" sleep 10

With the ‘Vargrantfile’ in place in your working directly all that is required is to run:
vagrant up

Once that is complete you should have 5 VMs which are clustered together.

Log into a host and ensure you can see the others.

vagrant ssh node1
% lshosts