Opened 9 years ago

Closed 8 years ago

#853 closed task (complete)

boost::mpi (or MPI in general?) as a possible replacement of MSGQ

Reported by: shane Owned by: UnAssigned
Priority: high Milestone: Sprint-20110816
Component: ~msgq (obsolete) Version:
Keywords: Cc:
CVSS Scoring: Parent Tickets:
Sensitive: no Defect Severity: N/A
Sub-Project: Core Feature Depending on Ticket:
Estimated Difficulty: 5 Add Hours to Ticket: 0
Total Hours: 0 Internal?: no


This is subtask of #765, investigating if boost::mpi could be used as a replacement for our MSGQ. Note that MPI is an old, long-established messaging system, and may be useful without boost::mpi.


Change History (9)

comment:1 Changed 9 years ago by stephen

  • Defect Severity set to N/A
  • Milestone changed from Sprint-20110503 to Year 3 Task Backlog
  • Sub-Project set to Core

comment:2 Changed 8 years ago by stephen

  • Milestone changed from Year 3 Task Backlog to Sprint-20110802

comment:3 Changed 8 years ago by shane

  • Priority changed from major to critical

comment:4 Changed 8 years ago by shane

  • Owner set to shane
  • Status changed from new to accepted

comment:5 Changed 8 years ago by shane

MPI is a quite mature technology, from the 1990 originally. It is in heavy use by academia and governments. There are many implementations, and standards work continues to this day (although in a MPI-specific standards body).

Wikipedia has a longish article about it:

To run it, we would need a daemon or set of daemons. There are many implementations so we would need to choose one.

We would also need to choose a library to use in our applications. It might make sense to use the boost::mpi version, since we are already using Boost. However, we could not use the header-only mode for C++, so this is not a "slam dunk" option. Also, Boost has certain preferred daemon implementations:

Taken from:

I'll go through the points we came up with, then have discussion at the end.

  • Nice: Cross-machine

MPI is explicitly designed to work across a network. It is also designed to be high-performance, so often uses shared memory for local communication, and specialized network protocols for working with remote machines.

  • We have to be able to secure if it uses other computers.

This seems trickier. Some implementations can use SSH as the underlying mechanism, although MPI is designed more around running jobs. Think "batch processing".

  • Compatible license (not GPL)

BSD licensed versions of C++ and Python libraries exist. We may be able to use the boost::mpi for both C++ and Python, however we will not be able to use C++ in header-only mode. BSD licensed versions of the "actual" MPI install also exist.

  • Nice: send file descriptors

This does not appear to be supported.

  • Nice: minimize dependencies

Depending on which implementation we choose, this can be true. If we use boost::mpi then we have only added another use of Boost for the libraries. We'll need the MPI install too.

  • Nice: Client-side in Python and C++


  • Nice: Not have to declare types before sending

Types can be built at runtime (according to the Wikipedia article). I'm not sure how easy this is in practice, especially in C++, but it is possible.

  • Nice: high-performance

Used in supercomputers around the world.

  • Point-to-point and group communication (including no listeners)

Both are nicely supported.

  • Separate instances (to run multiple BIND 10 in parallel)


  • Multiple OS support


  • Nice: unit-tests for the code itself

Not sure... the boost::mpi presumably has the same testing as the rest of Boost. Some other libraries definitely do. The non-library part of MPI will have varying levels of testing, depending on the implementation. There are many!!!

  • Nice: mock test support

I did not see any.

I am pretty sure that MPI could replace msgq for us. We'd get increased performance and fewer bugs. However, MPI is not really a match conceptually for what we want to do. It is designed for batch processing, especially in the context of large computing efforts. We would need to spend an amount of time getting and testing our implementations together, and providing a way to install & configure these so the user does not have to learn the details.

So... I think MPI is a possibility, but probably not a strong one.

comment:6 Changed 8 years ago by shane

  • Owner changed from shane to UnAssigned
  • Status changed from accepted to reviewing

comment:7 Changed 8 years ago by vorner

Should this still sit in review, when it was talked about during the call? Shouldn't this ticket be closed?

comment:8 Changed 8 years ago by stephen

  • Estimated Difficulty changed from 0.0 to 5

comment:9 Changed 8 years ago by shane

  • Resolution set to complete
  • Status changed from reviewing to closed

I'd like to have someone else review my findings, but it seems like nobody is biting. Since we're going to have a discussion about this at the face to face meeting, I'll go ahead and mark this as closed.

Note: See TracTickets for help on using tickets.