python - mpi4py Communication error with rank 4 -
i have built 4 pi3 cluster using tutorial found here.
the tests available worked fine , nodes functioning described end.
i began following tutorial here.
the python code i'm using is:
from mpi4py import mpi comm = mpi.comm_world size=comm.get_size() rank-comm.get_rank() if rank == 0: data = [(x+1) ** x x in range (size)] print 'scattering data',data else: data = none data = comm.scatter(data,root=0) print 'rank',rank,'has data: ', data
my machinefile looks this:
node1:4 node2:4 node3:4 node4:4
if execute file command:
mpiexec -f machinefile -n 4 python scatter.py
i output:
pi@node1:~/cloud $ mpiexec -f machinefile -n 4 python scatter1.py scattering data [1, 2, 9, 64] rank 0 has data: 1 rank 1 has data: 2 rank 2 has data: 9 rank 3 has data: 64
but if try higher rank 4 falls over:
pi@node1:~/cloud $ mpiexec -f machinefile -n 5 python scatter1.py scattering data [1, 2, 9, 64, 625] rankrank 2 has data: 9 rank 3 has data: 64 1 has data: traceback (most recent call last): file "scatter1.py", line 12, in <module> 2 data = comm.scatter(data,root=0) file "mpi/comm.pyx", line 1286, in mpi4py.mpi.comm.scatter (src/mpi4py.mpi.c:109079) file "mpi/msgpickle.pxi", line 713, in mpi4py.mpi.pympi_scatter (src/mpi4py.mpi.c:48214) mpi4py.mpi.exception: unknown error class, error stack: pmpi_scatterv(386).........: mpi_scatterv(sbuf=0x76a32694, scnts=0x1659de0, displs=0x1656750, mpi_byte, rbuf=0x76a354f4, rcount=5, mpi_byte, root=0, mpi_comm_world) failed mpir_scatterv_impl(184)....: mpir_scatterv(108).........: mpic_isend(649)............: mpidi_eagercontigisend(573): failure occurred while attempting send eager message mpidi_ch3_isendv(34).......: communication error rank 4 ^c[mpiexec@node1] sending ctrl-c processes requested [mpiexec@node1] press ctrl-c again force abort [mpiexec@node1] hydu_sock_write (/home/pi/mpich2/mpich-3.2/src/pm/hydra/utils/sock/sock.c:286): write error (bad file descriptor) [mpiexec@node1] hyd_pmcd_pmiserv_send_signal (/home/pi/mpich2/mpich-3.2/src/pm/hydra/pm/pmiserv/pmiserv_cb.c:169): unable write data proxy [mpiexec@node1] ui_cmd_cb (/home/pi/mpich2/mpich-3.2/src/pm/hydra/pm/pmiserv/pmiserv_pmci.c:79): unable send signal downstream [mpiexec@node1] hydt_dmxu_poll_wait_for_event (/home/pi/mpich2/mpich-3.2/src/pm/hydra/tools/demux/demux_poll.c:76): callback returned error status [mpiexec@node1] hyd_pmci_wait_for_completion (/home/pi/mpich2/mpich-3.2/src/pm/hydra/pm/pmiserv/pmiserv_pmci.c:198): error waiting event [mpiexec@node1] main (/home/pi/mpich2/mpich-3.2/src/pm/hydra/ui/mpich/mpiexec.c:344): process manager error waiting completion
can help?
Comments
Post a Comment