How to run IDL programs on maya

Introduction

Now we’ll see how to run an IDL program on the cluster. Before proceeding, make sure you’ve read the How To Run tutorial first.

For more information about the software, see the IDL website.

IDL users may also be interested in GDL which is an open source alternative.

For users interested in the IDL Virtual Machine, please note that there were some issues getting it to work on the compute nodes of the cluster hpc. A workaround is described here for a different computing environment. Please contact hpc-support if you’re interested in further investigation into this workaround for maya.

Performing Calculations on the Cluster Nodes

Running IDL on the cluster is not much different than running MATLAB or other serial programs as long as you’re not making plots (we’ll cover that in the next section). There are a few minor modifications you must make to your batch script. First, we’ll need an example IDL program:

; sayhello.pro
pro sayhello,what
  print,'HELLO ',what
end

Download: ../code/sayhello-idl/sayhello.pro

We’ll also make a main program:

; main.pro
pro main
  sayhello,'WORLD'
end

Download: ../code/sayhello-idl/main.pro

Then we’ll create a batch script:

#!/bin/bash
#SBATCH --job-name=sayhello
#SBATCH --output=slurm.out
#SBATCH --error=slurm.err
#SBATCH --partition=develop

idl -e main

Download: ../code/sayhello-idl/run.slurm

For details on that script, see this page. Note that we’re requesting only one node since we didn’t specify a specific number here.

Now we can submit our job to the scheduler

[araim1@maya-usr1 sayhello-idl]$ sbatch run.slurm
[araim1@maya-usr1 sayhello-idl]$

Eventually the job will complete. When it does, it will create slurm.out and slurm.err files with the following output:

[araim1@maya-usr1 sayhello-idl]$ cat slurm.err
IDL Version 7.0 (linux x86_64 m64). (c) 2007, ITT Visual Information Solutions
Installation number: XXXXXX-X.
Licensed for use by: University of Maryland

% Compiled module: MAIN.
% Compiled module: SAYHELLO.
[araim1@maya-usr1 sayhello-idl]$ cat slurm.out
HELLO WORLD
[araim1@maya-usr1 sayhello-idl]$

Generating Plots on the Cluster Nodes

Generating plots without a desktop on IDL can be complicated if you don’t have functions that automate everything for you. A popular pair is plopen and plclose, developed at Goddard. For this tutorial, we will use imopen and imclose which are simply modified versions of Goddard’s plopen and plclose with support for additional image types:

Copy those to a directory on the machine and then create this file, the same directory:

pro testplot
  ; Generate a thousand numbers from 0 to 2*pi:
  zero_to_2pi=indgen(1001)/500.0*!pi

  ; Calculate the sine of each of the list of numbers:
  sine=sin(zero_to_2pi)

  ; Tell imopen to open a portable network graphics ("png") file "test.png":
  imopen,'png',fn='test'

  ; The 'png' tells imopen the file format.  The fn='test' tells imopen the
  ; file's basename -- the portion of the filename that comes before the last '.'
  ; Later, when you call imclose, imclose will generate the file "test.png".
  ; It makes that filename by appending a "." and the name of the file format
  ; ("png") to the end of your file's basename ("test" from fn='test').

  ; Plot sin(x) with x=0..2*pi:
  plot,zero_to_2pi,sine

  ; Tell imclose to copy everything we plotted between imopen and
  ; imclose to the test.png file and close that file:
  imclose
end

Download: ../code/testplot-idl/testplot.pro

Then create a new batch script:

#!/bin/bash
#SBATCH --job-name=testplot
#SBATCH --output=slurm.out
#SBATCH --error=slurm.err
#SBATCH --partition=develop

idl -e testplot

Download: ../code/testplot-idl/run.slurm

Note that all we’ve changed is that “main” has been replaced with “testplot”. Now submit that script:

[araim1@maya-usr1 testplot-idl]$ sbatch run.slurm
[araim1@maya-usr1 testplot-idl]$

When the job finishes, it should create slurm.out, slurm.err and test.png. The output files should contain the following:

[araim1@maya-usr1 testplot-idl]$ cat slurm.err
IDL Version 7.0 (linux x86_64 m64). (c) 2007, ITT Visual Information Solutions

Installation number: XXXXXX-X.
Licensed for use by: University of Maryland

% Compiled module: TESTPLOT.
% Compiled module: IMOPEN.
% Compiled module: IMCLOSE.
% Loaded DLM: PNG.
[araim1@maya-usr1 testplot-idl]$ cat slurm.out
[araim1@maya-usr1 testplot-idl]$

The test.png image should look like this:

test.png output

Any of the 2D IDL plotting routines should work. The visualization routines that rely on fancy 3D X11 applications will not work since they require an actual graphical display.

 

Steps for using Xvfb

First, start the virtual frame buffer using a random number generator:

$ export DISPLAY=:$RANDOM.0
$ /usr/X11R6/bin/Xvfb ${DISPLAY%.*} -screen 0 1152x900x8 &

Then set the current display to use the virtual frame buffer for graphics display:

$ setenv DISPLAY :1.0

Then you are ready to run IDL.