SwissTech Consulting

OpenLava
In the computing world there are two main cluster workload scheduling software: Grid Engine and Platform LSF. Most companies will choose one or the other. In the semiconductor field many choose LSF which is what most tools support out of the box. Industry will choose the software which has a fair price and good support.

OpenLava started as a simplified version of LSF back in 2007. It was discontinued 4 years later. The open source community and Teraproc have continued development of the source code since then. OpenLava continues to be a great open source alternative to Platform LSF, Teraproc does offer enterprise support through per node licensing.

This article is a quick summary of how quickly one can deploy a completely free workload schedule on your hosts. For any small businesses and start-ups out there wanting to develop applications supporting the popular LSF commands OpenLava seems to have you covered.

Now for this installation I am using Ubuntu Server 15.10 which is the latest software release as of Jan 2016.

Install the prerequisites
sudo apt-get install tcl-dev ncurses-dev

Download the OpenLava package
Go to http://www.openlava.org/download/download.html for the latest tarball location

wget http://www.openlava.org/tarball/openlava-3.1.tar.gz
tar xzvf openlava-3.1.tar.gz

Compile the project and install it

cd openlava-3.1
./configure --prefix=/opt/openlava
make
sudo make install

lsf binaries have now been installed to:
/opt/openlava/bin

There is a bunch of configuration to go over in order to have a functional cluster. Here are some of the basic to get the hosts added. It might be future post to get the queues and permissions setup for various users.

# Create a user to run all the lsf processes
sudo useradd -r openlava

# Copy the default configuration files over
cd ~/openlava-3.1/config
sudo cp openlava /etc/init.d
sudo cp lsf.cluster.openlava lsf.conf lsf.shared lsb.params lsb.hosts /opt/openlava/etc/
sudo cp openlava.sh openlava.csh /etc/profile.d/
. openlava.sh
cd ../

# Add the hosts to the cluster config file
sudo sed -i ‘s/# yourhost/node1\t!\t!\t1\t-\t-\t\nnode2\t!\t!\t1\t-\t-\t\nnode3\t!\t!\t1\t-\t-\t\nnode4\t!\t!\t1\t-\t-\t\nnode5\t!\t!\t1\t-\t-\t\n# yourhost/g’ /opt/openlava/etc/lsf.cluster.openlava

# Add the hosts to the /ets/hosts file so they can connect by name
printf “33.33.33.31 node1\n33.33.33.32 node2\n33.33.33.33 node3\n33.33.33.34 node4\n33.33.33.35 node5” | sudo tee -a /etc/hosts

# Ensure the permissions are correct for the installation
sudo chown -R openlava:openlava /opt/openlava

# Ensure the service starts
sudo chmod +x /etc/init.d/openlava
sudo systemctl enable openlava
sudo systemctl start openlava
sudo systemctl status openlava

 

Since this is a bit laborious going to each host and performing the above here is a vagrant file which will start up 5 test hosts and provision them the same as above. I just thought it best to go over what is to be done before handing you over a cut-and-paste script for testing.

Filename: Vagrantfile

# -*- mode: ruby -*-
# vi: set ft=ruby :

# All Vagrant configuration is done below. The "2" in Vagrant.configure
# configures the configuration version (we support older styles for
# backwards compatibility). Please don't change it unless you know what
# you're doing.
Vagrant.configure(2) do |config|

  config.vm.define :node1 do |node|
    node.vm.box = "ubuntu/vivid64"
    node.vm.hostname = "node1"
    node.vm.network "private_network", ip: "33.33.33.31"
  end

  config.vm.define :node2 do |node|
    node.vm.box = "ubuntu/vivid64"
    node.vm.hostname = "node2"
    node.vm.network "private_network", ip: "33.33.33.32"
  end

  config.vm.define :node3 do |node|
    node.vm.box = "ubuntu/vivid64"
    node.vm.hostname = "node3"
    node.vm.network "private_network", ip: "33.33.33.33"
  end

  config.vm.define :node4 do |node|
    node.vm.box = "ubuntu/vivid64"
    node.vm.hostname = "node4"
    node.vm.network "private_network", ip: "33.33.33.34"
  end

  config.vm.define :node5 do |node|
    node.vm.box = "ubuntu/vivid64"
    node.vm.hostname = "node5"
    node.vm.network "private_network", ip: "33.33.33.35"
  end

  # Install OpenLava to hosts
  config.vm.provision "shell", privileged: false, path: "install_openlava.sh"
end

 

Filename: install_openlava.sh

sudo apt-get update
sudo apt-get upgrade -y
sudo apt-get install -y tcl-dev ncurses-dev autoconf libtool

wget http://www.openlava.org/tarball/openlava-3.1.tar.gz
tar xzvf openlava-3.1.tar.gz
cd openlava-3.1

./configure --prefix=/opt/openlava
make
sudo make install

# Create a openlava user to own all lsf processes
sudo useradd -r openlava

# Copy the configurations to the system
cd config
sudo cp openlava /etc/init.d
sudo cp lsf.cluster.openlava lsf.conf lsf.shared lsb.params lsb.hosts /opt/openlava/etc/
sudo cp openlava.sh openlava.csh /etc/profile.d/
. openlava.sh
cd ../

# Add the hosts to the cluster config file
sudo sed -i 's/# yourhost/node1\t!\t!\t1\t-\t-\t\nnode2\t!\t!\t1\t-\t-\t\nnode3\t!\t!\t1\t-\t-\t\nnode4\t!\t!\t1\t-\t-\t\nnode5\t!\t!\t1\t-\t-\t\n# yourhost/g' /opt/openlava/etc/lsf.cluster.openlava

# Add the hosts to the /ets/hosts file so they can connect by name
sudo printf "33.33.33.31 node1\n33.33.33.32 node2\n33.33.33.33 node3\n33.33.33.34 node4\n33.33.33.35 node5" | sudo tee -a /etc/hosts

# Ensure the permissions are correct for the installation
sudo chown -R openlava:openlava /opt/openlava

# Either add the new /opt/openlava/bin to the path or symlink the functions
sudo ln -s /opt/openlava/bin/* /usr/local/bin

# Start up the main service
sudo chmod +x /etc/init.d/openlava
sudo systemctl enable openlava
sudo systemctl start openlava
sudo systemctl status openlava

# Sanity check of install
lsid
lshosts
bhosts

bsub "sleep 10 && printf 'Completed Job'"
bsub -J "sampleArray[1-25]" sleep 10

 

With the ‘Vargrantfile’ in place in your working directly all that is required is to run:
vagrant up

Once that is complete you should have 5 VMs which are clustered together.

Log into a host and ensure you can see the others.

vagrant ssh node1
% lshosts

 

New software solution designed of to aid in the development of microcontroller communication or for network admin’s configuring multiple routers. SwissTech Serial Terminal allows data to be transmitted and received through any communications port available on your computer. With simple user interface and scaleable advanced features, it suits various users and their connection needs. With the Macro Script feature you can customize transmission strings to be sent to a remote device to do such commands as setting baud rate of Bluetooth Modules using AT command set. This software integrates perfectly with microcontroller UART development for serial communication and configuration as it allows quick and simple connections to be made to multiple devices.

Currently we are developing a device to be marketed towards the mainstream public as well as to the business sector which would allow full monitoring of a building using existing door and alarm sensors. The device would be stand alone as well as computer configurable for monitoring to allow advanced automation tasks. Starting at 32 zones it is expandable by stacking master/slave units to create endless possibilities by linking buildings, sections or floors together. Outputs can be programmed to drive lamps, open electronic door locks or alarm outputs.

This is an undergraduate based project primarily interested in the progression of autonomous technology. Our goal is to fully automate the control of a helicopter in 3D space. Helicopters allow the precise control of camera views and are superior to fixed wing aircraft which do not have the capability to hover.

Our hope is that the ST Aerial Recon will be able to bridge the gap between cost effective monitoring and ease of use for anyone with basic point and click computer skills. With advanced 3-axis accelerometers, gyros and magnetometers we are able to relieve the remote operator of the difficult task of maintaining a stable flight.  With the computer keeping the unit stable it is just up to the user to program the flight path into the computer and decide what needs to be filmed.

aerialrecon_site_screenshot

Visit the site at swisstech.ca/aerialrecon.ca/

Latest News


OpenLava – Platform LSF Alternative

In the computing world there are two main cluster workload scheduling software: Grid Engine and Platform LSF. Most companies will choose one or the other. In the semiconductor field many choose LSF which is what most tools support out of the box. Industry will choose the software which has a…

SwissTech Serial Terminal

New software solution designed of to aid in the development of microcontroller communication or for network admin’s configuring multiple routers. SwissTech Serial Terminal allows data to be transmitted and received through any communications port available on your computer. With simple user interface and scaleable advanced features, it suits various users…

Security Home Automation

Currently we are developing a device to be marketed towards the mainstream public as well as to the business sector which would allow full monitoring of a building using existing door and alarm sensors. The device would be stand alone as well as computer configurable for monitoring to allow advanced…