The "challenge" is having 2 MPI process do the following in a completely non-blocking way (I just use non-blocking send and recv for exposition, but a similar example using MPI_IProbe could be constructed):
Process A:
non-blocking sends size of array
non-blocking send array
Process B:
non-blocking receive size of array
non-blocking allocate buffer
non-blocking receive array on buffer
*/
// In C++ pseudo-code:
// Process A:
std::vector data {1, 2, 3, 4};
auto request = nb_send(process_b, tag_0, data.size())
// the second send is scheduled as soon as the first
// completes
// }
request.get();// blocks until both sends complete
// Process B:
std::size_t size;
std::vector buffer;
auto request = nb_recv(process_a, tag_0, &size)
.then([&](){// schedules continuation
// allocate memory for the buffer before the second recv
buffer.resize(size);
b_recv(process_a, tag_1, buffer.data(), size);});
// application code {
// not a single call to test in here:
// as soon as the first recv completes, the continuation
// runs asynchronously and memory is allocated
// and the second recv is scheduled
// } request.get();
// blocks until both recvs complete
/*
The main net win is that one fires all the communication logic at the beginning, and blocks at the very end. That is, the unrelated application code in the middle doesn't need to e.g. call a function "every now and then" to just advance the continuation.
I don't see how to do this without spawning a different thread that continuously loops over the requests calling MPI test to advance to the next continuation. If one has to spawn a different thread anyways the advantage of non-blocking MPI calls is still there, but it is less important (they allow cool things like having a pool of request + continuations).
P.S: One could just have sent the array at once with send, but the receive end would still need to repeatedly call MPI_IProbe before executing the continuation (I just chose the two sends + two receives for exposition purposes).
prog.cpp:18:6: error: 'vector' in namespace 'std' does not name a type
std::vector data {1, 2, 3, 4};
^
prog.cpp:19:24: error: 'process_b' was not declared in this scope
auto request = nb_send(process_b, tag_0, data.size())
^
prog.cpp:19:35: error: 'tag_0' was not declared in this scope
auto request = nb_send(process_b, tag_0, data.size())
^
prog.cpp:19:42: error: 'data' was not declared in this scope
auto request = nb_send(process_b, tag_0, data.size())
^
prog.cpp:19:53: error: 'nb_send' was not declared in this scope
auto request = nb_send(process_b, tag_0, data.size())
^
prog.cpp: In lambda function:
prog.cpp:21:29: error: 'process_b' was not declared in this scope
nb_send(process_b, tag_1, data.data(), data.size()); });
^
prog.cpp:21:40: error: 'tag_1' was not declared in this scope
nb_send(process_b, tag_1, data.data(), data.size()); });
^
prog.cpp:21:47: error: 'data' was not declared in this scope
nb_send(process_b, tag_1, data.data(), data.size()); });
^
prog.cpp:21:71: error: 'nb_send' was not declared in this scope
nb_send(process_b, tag_1, data.data(), data.size()); });
^
prog.cpp: At global scope:
prog.cpp:29:1: error: 'request' does not name a type
request.get(); // blocks until both sends complete
^
prog.cpp:33:6: error: 'size_t' in namespace 'std' does not name a type
std::size_t size;
^
prog.cpp:34:6: error: 'vector' in namespace 'std' does not name a type
std::vector buffer;
^
prog.cpp:35:24: error: 'process_a' was not declared in this scope
auto request = nb_recv(process_a, tag_0, &size)
^
prog.cpp:35:35: error: 'tag_0' was not declared in this scope
auto request = nb_recv(process_a, tag_0, &size)
^
prog.cpp:35:43: error: 'size' was not declared in this scope
auto request = nb_recv(process_a, tag_0, &size)
^
prog.cpp:35:47: error: 'nb_recv' was not declared in this scope
auto request = nb_recv(process_a, tag_0, &size)
^
prog.cpp: In lambda function:
prog.cpp:38:16: error: 'buffer' was not declared in this scope
buffer.resize(size);
^
prog.cpp:38:30: error: 'size' was not declared in this scope
buffer.resize(size);
^
prog.cpp:39:23: error: 'process_a' was not declared in this scope
b_recv(process_a, tag_1, buffer.data(), size); });
^
prog.cpp:39:34: error: 'tag_1' was not declared in this scope
b_recv(process_a, tag_1, buffer.data(), size); });
^
prog.cpp:39:60: error: 'b_recv' was not declared in this scope
b_recv(process_a, tag_1, buffer.data(), size); });
^