Numpy and Scipy with Intel MKL on Linux

Update:

This easiest way is to use the enthought python distribution. They already have MKL packaged within the software. But it seems to be slower than the version that i compiled.

The instruction on Scipy's website does not work with the newest version(10.3) of Intel MKL. With Intel MKL, one can have multi-thread support for many numerical functions. Note: if you want to enable multi-thread for MKL, set the os environment variable OMP_NUM_THREADS to the number of threads (e.g. number of cores) that MKL can use.

Compile and install numpy

My installation environment is 64bit Centos 5.5 and python2.6.

Install Intel® C++ Composer XE 2011 and Intel® Math Kernel Library (Intel® MKL)

Checkout the numpy source.

Create the site.cfg, you need to change the library_dirs and include_dirs to the MKL installation directory:

[mkl]
library_dirs = /local/scratch/ls418/intel/composerxe-2011.0.084/mkl/lib/intel64
include_dirs = /local/scratch/ls418/intel/composerxe-2011.0.084/mkl/include/
mkl_libs = mkl_def, mkl_intel_lp64, mkl_intel_thread, mkl_core, mkl_mc
lapack_libs = mkl_lapack95_lp64

In numpy/distutils/intelccompiler.py, set

cc_exe = 'icc -fPIC -O3 -openmp'

Add the library dirs to LD_LIBRARY_PATH

export LD_LIBRARY_PATH=/local/scratch/ls418/intel/composerxe-2011.0.084/mkl/lib/intel64/:$LD_LIBRARY_PATH

Then run the setup.py, set $INSTALL_DIR to your python package directory

python setup.py config --compiler=intel build_clib --compiler=intel build_ext --compiler=intel install --home=$INSTALL_DIR

Verify the numpy installation

>>> import numpy    
>>> numpy.show_config()
lapack_opt_info:
    libraries = ['mkl_lapack95_lp64', 'mkl_def', 'mkl_intel_lp64', 'mkl_intel_thread', 'mkl_core', 'mkl_mc', 'pthread']
    library_dirs = ['/local/scratch/ls418/intel/composerxe-2011.0.084/mkl/lib/intel64']
    define_macros = [('SCIPY_MKL_H', None)]
    include_dirs = ['/local/scratch/ls418/intel/composerxe-2011.0.084/mkl/include/']
blas_opt_info:
    libraries = ['mkl_def', 'mkl_intel_lp64', 'mkl_intel_thread', 'mkl_core', 'mkl_mc', 'pthread']
    library_dirs = ['/local/scratch/ls418/intel/composerxe-2011.0.084/mkl/lib/intel64']
    define_macros = [('SCIPY_MKL_H', None)]
    include_dirs = ['/local/scratch/ls418/intel/composerxe-2011.0.084/mkl/include/']
lapack_mkl_info:
    libraries = ['mkl_lapack95_lp64', 'mkl_def', 'mkl_intel_lp64', 'mkl_intel_thread', 'mkl_core', 'mkl_mc', 'pthread']
    library_dirs = ['/local/scratch/ls418/intel/composerxe-2011.0.084/mkl/lib/intel64']
    define_macros = [('SCIPY_MKL_H', None)]
    include_dirs = ['/local/scratch/ls418/intel/composerxe-2011.0.084/mkl/include/']
blas_mkl_info:
    libraries = ['mkl_def', 'mkl_intel_lp64', 'mkl_intel_thread', 'mkl_core', 'mkl_mc', 'pthread']
    library_dirs = ['/local/scratch/ls418/intel/composerxe-2011.0.084/mkl/lib/intel64']
    define_macros = [('SCIPY_MKL_H', None)]
    include_dirs = ['/local/scratch/ls418/intel/composerxe-2011.0.084/mkl/include/']
mkl_info:
    libraries = ['mkl_def', 'mkl_intel_lp64', 'mkl_intel_thread', 'mkl_core', 'mkl_mc', 'pthread']
    library_dirs = ['/local/scratch/ls418/intel/composerxe-2011.0.084/mkl/lib/intel64']
    define_macros = [('SCIPY_MKL_H', None)]
    include_dirs = ['/local/scratch/ls418/intel/composerxe-2011.0.084/mkl/include/']

Comile and install Scipy

Download and install Intel® Fortran Composer XE 2011, and add ifort to your PATH.

In the numpy package directory, the file numpy/distutils/fcompiler/intel.py in line around 49, change

pic_flags = ['-kPIC']

to

pic_flags = ['-fPIC'] 

In the file numpy/distutils/fcompiler/__init__.py, change line 426, so it does not raise exception when version cannot be found:

return 12.0

Use the same site.cfg as numpy

Again, add the library dirs to LD_LIBRARY_PATH.

Then run setup.py:

python setup.py config --compiler=intel --fcompiler=intel build_clib --compiler=intel --fcompiler=intel build_ext --compiler=intel --fcompiler=intel install --home=$INSTALL_DIR

 

 

 

Comments

[D]

Hi,
I was able to compile Numpy as you instructed here. But I have failed to compile Scipy. I wonder what causes to fail to compile it.

Thanks for the instructions.

Özhan