1.4.1 Redistributing vectors, matrix rows, and matrix columns

        We start by showing how matrix distributions that are induced by vector distributions naturally permit redistribution of rows and columns of matrices to the inducing vector distribution, as well as redistribution of matrix rows to columns, and vice versa.

Vector to matrix row, matrix row to vector: Consider a vector, x , distributed to nodes according to an inducing vector distribution for matrix A . Notice that the assignment of blocks of columns of A is determined by a projection of the indices of the corresponding sub-vectors of the inducing vector distribution. Thus, transforming a vector x into a row of A is equivalent to projecting onto that matrix row, or, equivalently, gathering   the sub-vectors of x within columns of nodes to the row of nodes that holds the desired row of A . Naturally, redistributing a row of the matrix to a vector reverses this process, requiring a scatter   within columns of A , as illustrated in Figure 1.3.

Vector to matrix column, matrix column to vector: The vector to matrix column operation is similar to the redistribution of a vector to a matrix row, except that the gather is performed within rows of nodes, as illustrated in Figure 1.3. Again, the matrix column to vector operation reverses this process.

Matrix row to matrix column, matrix column to matrix row: Redistributing a matrix row to become a matrix column, i.e.   transposing a matrix row to become a matrix column, can be achieved by redistributing from matrix row to inducing vector distribution, followed by a redistribution from vector distribution to matrix column, as illustrated in Figure 1.3. Naturally, reversing this process takes a matrix column to a matrix row.