Home > Uncategorized > An OpenMPI, python and dlopen issue

An OpenMPI, python and dlopen issue

When trying to use a native python module which is using OpenMPI 1.6 I get this error:

mca: base: component_find: unable to open mca_paffinity_hwloc: perhaps a missing symbol, or compiled for a different version of Open MPI? (ignored)
mca: base: component_find: unable to open mca_carto_auto_detect: perhaps a missing symbol, or compiled for a different version of Open MPI? (ignored)
mca: base: component_find: unable to open mca_carto_file: perhaps a missing symbol, or compiled for a different version of Open MPI? (ignored)
mca: base: component_find: unable to open mca_shmem_mmap: perhaps a missing symbol, or compiled for a different version of Open MPI? (ignored)
mca: base: component_find: unable to open mca_shmem_posix: perhaps a missing symbol, or compiled for a different version of Open MPI? (ignored)
mca: base: component_find: unable to open mca_shmem_sysv: perhaps a missing symbol, or compiled for a different version of Open MPI? (ignored)
--------------------------------------------------------------------------
It looks like opal_init failed for some reason; your parallel process is
likely to abort.  There are many reasons that a parallel process can
fail during opal_init; some of which are due to configuration or
environment problems.  This failure appears to be an internal failure;
here's some additional information (which may only be relevant to an
Open MPI developer):

  opal_shmem_base_select failed
  --> Returned value -1 instead of OPAL_SUCCESS
--------------------------------------------------------------------------
[[INVALID],INVALID] ORTE_ERROR_LOG: Error in file runtime/orte_init.c at line 79
--------------------------------------------------------------------------
It looks like MPI_INIT failed for some reason; your parallel process is
likely to abort.  There are many reasons that a parallel process can
fail during MPI_INIT; some of which are due to configuration or environment
problems.  This failure appears to be an internal failure; here's some
additional information (which may only be relevant to an Open MPI
developer):

  ompi_mpi_init: orte_init failed
  --> Returned "Error" (-1) instead of "Success" (0)
--------------------------------------------------------------------------

A good explanation is given in the OpenMPI FAQ. Here are some workarounds:

  • Rebuild OpenMPI with --disable-dlopen
  • Run python with LD_PRELOAD=libmpi.so
  • Add this code in your python script before using MPI:
import ctypes
ctypes.CDLL("libmpi.so", mode=ctypes.RTLD_GLOBAL)
  • or this code:
from mpi4py import MPI

The last one is equivalent to using ctypes.CDLL, is more portable, but require to install mpi4py. The source code of the hack used in mpi4py is here.

Advertisements
  1. No comments yet.
  1. No trackbacks yet.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: