How to use OpenBLAS on taki

Note: This page has not been updated to reflect the move to taki from maya.

Introduction

OpenBLAS is an optimized implementation of the BLAS library. On this page, we will show how to start using it with with the GCC compiler on maya. Before you begin, make sure to read the tutorial for compiling C programs.

To use OpenBLAS, load the following modules

[araim1@maya-usr1 blas]$ module load gcc
[araim1@maya-usr1 openblas]$ module load openblas/dynamic
[araim1@maya-usr1 ~]$

Note that the “dynamic” OpenBLAS module will allow you to compile executables that will work on all maya compute nodes.

BLAS Example: Matrix Multiply

Consider three matrices A∈ℝm×k, B∈ℝk×n, and C∈ℝm×n, where all are represented as double precision floating point numbers. We will demonstrate the use of the “dgemm” function which computes

C=αAB+βC.

We will take α=1 and β=0 to compute the simple matrix multiplication C=AB. MKL has several versions of this function; we will use “cblas_dgemm”, which has bit friendlier “C-style” interface than other variants.

The most important line in the code is

cblas_dgemm(CblasColMajor, CblasNoTrans, CblasNoTrans,
   m, n, k, 1.0, A, m, B, k, 0.0, C, m);

Also notice in the Makefile that locations to OpenBLAS headers and libraries are specified. Because we are using the “dynamic” OpenBLAS module, we can compile once on the user node and run on any compute node.

#!/bin/bash
#SBATCH --job-name=openblas
#SBATCH --output=slurm.out
#SBATCH --error=slurm.err
#SBATCH --partition=develop
#SBATCH --constraint=hpcf2013

hostname
./matrix_multiply

Download: ../code/openblas/matrix-multply/run.slurm

#!/bin/bash
#SBATCH --job-name=openblas
#SBATCH --output=slurm.out
#SBATCH --error=slurm.err
#SBATCH --partition=develop
#SBATCH --constraint=hpcf2010

hostname
./matrix_multiply

Download: ../code/openblas/matrix-multply/run2.slurm

[araim1@maya-usr1 openblas]$ make clean
rm -f *.o matrix_multiply
[araim1@maya-usr1 openblas]$ make
gcc -I/cm/shared/apps/openblas/dynamic/0.2.6/include -O3 -std=c99
-c -o matrix_multiply.o matrix_multiply.c
gcc -I/cm/shared/apps/openblas/dynamic/0.2.6/include -O3 -std=c99
-I/cm/shared/apps/openblas/dynamic/0.2.6/include matrix_multiply.o
-o matrix_multiply -L/cm/shared/apps/openblas/dynamic/0.2.6/lib -lm -lopenblas
[araim1@maya-usr1 openblas]$ sbatch run.slurm 
Submitted batch job 7345
[araim1@maya-usr1 openblas]$ cat slurm.err 
[araim1@maya-usr1 openblas]$ cat slurm.out 
n1
Matrix A (3 x 5) is:
  1.0000  0.7290  0.5314  0.3874  0.2824
  0.9000  0.6561  0.4783  0.3487  0.2542
  0.8100  0.5905  0.4305  0.3138  0.2288

Matrix B (5 x 4) is:
  1.0000  0.5905  0.3487  0.2059
  0.9000  0.5314  0.3138  0.1853
  0.8100  0.4783  0.2824  0.1668
  0.7290  0.4305  0.2542  0.1501
  0.6561  0.3874  0.2288  0.1351

Matrix C (3 x 4) = AB is:
  2.5543  1.5083  0.8906  0.5259
  2.2989  1.3575  0.8016  0.4733
  2.0690  1.2217  0.7214  0.4260
[araim1@maya-usr1 openblas]$ sbatch run2.slurm 
Submitted batch job 7346
[araim1@maya-usr1 openblas]$ cat slurm.err 
[araim1@maya-usr1 openblas]$ cat slurm.out
n70
Matrix A (3 x 5) is:
  1.0000  0.7290  0.5314  0.3874  0.2824
  0.9000  0.6561  0.4783  0.3487  0.2542
  0.8100  0.5905  0.4305  0.3138  0.2288

Matrix B (5 x 4) is:
  1.0000  0.5905  0.3487  0.2059
  0.9000  0.5314  0.3138  0.1853
  0.8100  0.4783  0.2824  0.1668
  0.7290  0.4305  0.2542  0.1501
  0.6561  0.3874  0.2288  0.1351

Matrix C (3 x 4) = AB is:
  2.5543  1.5083  0.8906  0.5259
  2.2989  1.3575  0.8016  0.4733
  2.0690  1.2217  0.7214  0.4260
[araim1@maya-usr1 openblas]$