Open MPI 1.2.6 BTL TCP NetPipe 3.7.1 Dual quad-core Xeon (E5345, 2.33GHz, C1E sleeping disabled) Myri-10G NICs (rx-usecs=1, myri10ge 1.4.1, IRQ to core #0), no switch Process on core #2 (just near core #0) Now starting the main loop 0: 1 bytes 6470 times --> 0.51 Mbps in 15.07 usec 1: 2 bytes 6634 times --> 1.01 Mbps in 15.11 usec 2: 3 bytes 6617 times --> 1.52 Mbps in 15.04 usec 3: 4 bytes 4431 times --> 2.01 Mbps in 15.15 usec 4: 6 bytes 4951 times --> 3.01 Mbps in 15.20 usec 5: 8 bytes 3288 times --> 4.06 Mbps in 15.02 usec 6: 12 bytes 4160 times --> 6.02 Mbps in 15.21 usec 7: 13 bytes 2740 times --> 6.51 Mbps in 15.24 usec 8: 16 bytes 3028 times --> 8.11 Mbps in 15.06 usec 9: 19 bytes 3736 times --> 9.62 Mbps in 15.07 usec 10: 21 bytes 4191 times --> 10.52 Mbps in 15.22 usec 11: 24 bytes 4379 times --> 12.09 Mbps in 15.14 usec 12: 27 bytes 4677 times --> 13.57 Mbps in 15.18 usec 13: 29 bytes 2928 times --> 14.43 Mbps in 15.33 usec 14: 32 bytes 3149 times --> 16.04 Mbps in 15.22 usec 15: 35 bytes 3489 times --> 17.50 Mbps in 15.26 usec 16: 45 bytes 3743 times --> 22.21 Mbps in 15.46 usec 17: 48 bytes 4312 times --> 23.93 Mbps in 15.30 usec 18: 51 bytes 4493 times --> 25.34 Mbps in 15.35 usec 19: 61 bytes 2554 times --> 30.00 Mbps in 15.52 usec 20: 64 bytes 3169 times --> 31.64 Mbps in 15.43 usec 21: 67 bytes 3341 times --> 33.21 Mbps in 15.39 usec 22: 93 bytes 3491 times --> 45.44 Mbps in 15.62 usec 23: 96 bytes 4269 times --> 47.32 Mbps in 15.48 usec 24: 99 bytes 4374 times --> 48.80 Mbps in 15.48 usec 25: 125 bytes 2349 times --> 61.05 Mbps in 15.62 usec 26: 128 bytes 3175 times --> 62.86 Mbps in 15.54 usec 27: 131 bytes 3268 times --> 64.35 Mbps in 15.53 usec 28: 189 bytes 3342 times --> 91.62 Mbps in 15.74 usec 29: 192 bytes 4235 times --> 93.54 Mbps in 15.66 usec 30: 195 bytes 4290 times --> 94.96 Mbps in 15.67 usec 31: 253 bytes 2225 times --> 121.44 Mbps in 15.89 usec 32: 256 bytes 3133 times --> 123.76 Mbps in 15.78 usec 33: 259 bytes 3193 times --> 124.95 Mbps in 15.81 usec 34: 381 bytes 3222 times --> 176.59 Mbps in 16.46 usec 35: 384 bytes 4050 times --> 179.66 Mbps in 16.31 usec 36: 387 bytes 4104 times --> 180.98 Mbps in 16.31 usec 37: 509 bytes 2090 times --> 231.36 Mbps in 16.79 usec 38: 512 bytes 2972 times --> 234.09 Mbps in 16.69 usec 39: 515 bytes 3008 times --> 235.61 Mbps in 16.68 usec 40: 765 bytes 3027 times --> 333.48 Mbps in 17.50 usec 41: 768 bytes 3809 times --> 337.41 Mbps in 17.37 usec 42: 771 bytes 3846 times --> 337.11 Mbps in 17.45 usec 43: 1021 bytes 1932 times --> 426.55 Mbps in 18.26 usec 44: 1024 bytes 2735 times --> 429.63 Mbps in 18.18 usec 45: 1027 bytes 2754 times --> 430.08 Mbps in 18.22 usec 46: 1533 bytes 2757 times --> 584.41 Mbps in 20.01 usec 47: 1536 bytes 3331 times --> 589.11 Mbps in 19.89 usec 48: 1539 bytes 3354 times --> 589.50 Mbps in 19.92 usec 49: 2045 bytes 1683 times --> 708.90 Mbps in 22.01 usec 50: 2048 bytes 2270 times --> 714.03 Mbps in 21.88 usec 51: 2051 bytes 2287 times --> 714.34 Mbps in 21.91 usec 52: 3069 bytes 2288 times --> 964.91 Mbps in 24.27 usec 53: 3072 bytes 2747 times --> 967.57 Mbps in 24.22 usec 54: 3075 bytes 2753 times --> 969.50 Mbps in 24.20 usec 55: 4093 bytes 1381 times --> 1087.36 Mbps in 28.72 usec 56: 4096 bytes 1740 times --> 1087.89 Mbps in 28.73 usec 57: 4099 bytes 1741 times --> 1087.63 Mbps in 28.75 usec 58: 6141 bytes 1741 times --> 1355.90 Mbps in 34.55 usec 59: 6144 bytes 1929 times --> 1353.58 Mbps in 34.63 usec 60: 6147 bytes 1925 times --> 1355.03 Mbps in 34.61 usec 61: 8189 bytes 964 times --> 1533.50 Mbps in 40.74 usec 62: 8192 bytes 1227 times --> 1528.42 Mbps in 40.89 usec 63: 8195 bytes 1223 times --> 1528.44 Mbps in 40.91 usec 64: 12285 bytes 1223 times --> 1811.24 Mbps in 51.75 usec 65: 12288 bytes 1288 times --> 1804.07 Mbps in 51.97 usec 66: 12291 bytes 1283 times --> 1810.61 Mbps in 51.79 usec 67: 16381 bytes 644 times --> 2276.81 Mbps in 54.89 usec 68: 16384 bytes 910 times --> 2271.70 Mbps in 55.02 usec 69: 16387 bytes 908 times --> 2281.27 Mbps in 54.80 usec 70: 24573 bytes 912 times --> 3062.08 Mbps in 61.23 usec 71: 24576 bytes 1088 times --> 3056.04 Mbps in 61.35 usec 72: 24579 bytes 1086 times --> 3069.65 Mbps in 61.09 usec 73: 32765 bytes 545 times --> 3533.86 Mbps in 70.74 usec 74: 32768 bytes 706 times --> 3531.34 Mbps in 70.79 usec 75: 32771 bytes 706 times --> 3531.71 Mbps in 70.79 usec 76: 49149 bytes 706 times --> 4178.45 Mbps in 89.74 usec 77: 49152 bytes 742 times --> 4171.98 Mbps in 89.89 usec 78: 49155 bytes 741 times --> 4181.66 Mbps in 89.68 usec 79: 65533 bytes 371 times --> 3899.21 Mbps in 128.23 usec 80: 65536 bytes 389 times --> 3918.25 Mbps in 127.61 usec 81: 65539 bytes 391 times --> 3905.14 Mbps in 128.04 usec 82: 98301 bytes 390 times --> 4789.20 Mbps in 156.60 usec 83: 98304 bytes 425 times --> 4785.25 Mbps in 156.73 usec 84: 98307 bytes 425 times --> 4794.31 Mbps in 156.44 usec 85: 131069 bytes 213 times --> 5401.57 Mbps in 185.13 usec 86: 131072 bytes 270 times --> 5381.70 Mbps in 185.82 usec 87: 131075 bytes 269 times --> 5392.64 Mbps in 185.44 usec 88: 196605 bytes 269 times --> 6268.64 Mbps in 239.28 usec 89: 196608 bytes 278 times --> 6260.51 Mbps in 239.60 usec 90: 196611 bytes 278 times --> 6269.64 Mbps in 239.25 usec 91: 262141 bytes 139 times --> 6811.06 Mbps in 293.64 usec 92: 262144 bytes 170 times --> 6843.26 Mbps in 292.26 usec 93: 262147 bytes 171 times --> 6812.36 Mbps in 293.59 usec 94: 393213 bytes 170 times --> 7361.52 Mbps in 407.52 usec 95: 393216 bytes 163 times --> 7354.33 Mbps in 407.92 usec 96: 393219 bytes 163 times --> 7362.41 Mbps in 407.48 usec 97: 524285 bytes 81 times --> 7194.13 Mbps in 556.01 usec 98: 524288 bytes 89 times --> 7123.49 Mbps in 561.52 usec 99: 524291 bytes 89 times --> 7187.24 Mbps in 556.55 usec 100: 786429 bytes 89 times --> 6961.03 Mbps in 861.94 usec 101: 786432 bytes 77 times --> 6852.26 Mbps in 875.62 usec 102: 786435 bytes 76 times --> 6962.44 Mbps in 861.77 usec 103: 1048573 bytes 38 times --> 6856.38 Mbps in 1166.79 usec 104: 1048576 bytes 42 times --> 6760.03 Mbps in 1183.43 usec 105: 1048579 bytes 42 times --> 6852.06 Mbps in 1167.54 usec 106: 1572861 bytes 42 times --> 6673.35 Mbps in 1798.19 usec 107: 1572864 bytes 37 times --> 6561.25 Mbps in 1828.92 usec 108: 1572867 bytes 36 times --> 6690.89 Mbps in 1793.49 usec 109: 2097149 bytes 18 times --> 6644.89 Mbps in 2407.86 usec 110: 2097152 bytes 20 times --> 6434.11 Mbps in 2486.75 usec 111: 2097155 bytes 20 times --> 6603.18 Mbps in 2423.08 usec 112: 3145725 bytes 20 times --> 6439.11 Mbps in 3727.22 usec 113: 3145728 bytes 17 times --> 6353.26 Mbps in 3777.59 usec 114: 3145731 bytes 17 times --> 6420.34 Mbps in 3738.12 usec 115: 4194301 bytes 8 times --> 6384.51 Mbps in 5012.12 usec 116: 4194304 bytes 9 times --> 6893.33 Mbps in 4642.17 usec 117: 4194307 bytes 10 times --> 6459.04 Mbps in 4954.30 usec 118: 6291453 bytes 10 times --> 6278.69 Mbps in 7644.90 usec 119: 6291456 bytes 8 times --> 7396.36 Mbps in 6489.68 usec 120: 6291459 bytes 10 times --> 6317.28 Mbps in 7598.21 usec 121: 8388605 bytes 4 times --> 6409.11 Mbps in 9985.77 usec 122: 8388608 bytes 5 times --> 6226.90 Mbps in 10277.99 usec 123: 8388611 bytes 4 times --> 6462.43 Mbps in 9903.40 usec