Introduction to Parallel Programming with an example of P2P
By: Ali Kemal Ay
Tags: Arithmetic Operations, Image Processing, Parallel Programming

1. Introduction

The computers store the images as 3-D arrays. As it is well known, in the three dimensionalities, there are X, Y, and Z axes. It means that X, Y, and Z axes are the weight, height, and depth, respectively, of the image. For example, the size of a frame on any Full-HD video is 1920×1080 (width×height) pixels. The size of the image in the figure-1 is 4×5 pixels. You may consider that each red square in the image is a pixel. Each of the 20 pixels has three different color values. The image size was chosen tiny, deliberately, to much easier understand the subject. When it increases, the processes needed increase, as well. To illustrate, in the study fields worked on images, such as Deep Learning, the training stage requires a lot of processing. Some of such processes are about arithmetic operations. In the project, we work on the parallel multiplying of two different vectors that represent two images.

Figure-1 : Each red square means a pixel. To illustrate, the size of the image above is 4×5×3 pixel and each pixel has a depth(3) that stores the RGB values.

Figure-2 : The table shows the image above numerically. (W : Width, H : Height, D : Depth)

The table in figure-2 shows the depth values in each pixel of the image seen on the previous page. In the table, D1, D2, and D3 values mean red, green, and blue color codes, respectively. As shown in the table, each weight has specific heights, which have also three different depth values, too.

2. Problem

 * The width values must be equal to the number of cores.

 * The multiplying operation must be done according to the formula below. result= (rgb_value1 × rgb_value2) % 256

All cores must declare a result vector in 3-D(weight×height×depth) that represent the result image of multiplying.

The master core, whose rank is 0, must

declare two image vectors in 3-D, using the same weight, height, and depth.

assign randomly RGB values between 0 and 255 of two image vectors.

send the weight sub-vectors of two image vectors to every slave core by using the rank of the receiving core, individually. (For example, the master core must send third (3rd)weight sub-vectors of two image vectors to the core whose rank is 3.)

calculate the result of multiplying the own part, which is specified by rank, of two vectors.

receive all the results of multiplying from last slave.

The slave cores must

declare two image vectors in 2-D representing two different images, because it calculates just one of the weights vectors.

receive the values of two image vectors sent by master.

receive the result of the multiplying in previous slave core, if it is possible.

calculate the result of multiplying the own part, which is specified by rank, of two vectors.

send the result of the multiplying to next core, but in the last core to master.

3. Coding

Note that because of the length of some lines as though some the code lines have an indent error.

The width values must be equal to the number of cores.

#define height 4

#define depth 3 int rank, numberMachines, tag = 0, master = 0;



MPI_Init(&argc, &argv);



MPI_Comm_rank(MPI_COMM_WORLD, &rank);



MPI_Comm_size(MPI_COMM_WORLD, &numberMachines);



int width = numberMachines;

All cores must declare a result vector in 3-D (weight*height*depth) that represent the result image of multiplying.

vector<vector<vector<int>>> result_image(width);



for (int i = 0; i < width; i++) {

result_image[i] = vector<vector<int>>(height);

for (int j = 0; j < height; j++) {

result_image[i][j] = vector<int>(depth);

}

}

The master core, whose rank is 0, must

declare two image vectors in 3-D, using the same weight, height, and depth.

if (rank == master) {

vector<vector<vector<int>>> first_image(width);

vector<vector<vector<int>>> second_image(width);



for (int i = 0; i < width; i++) {

first_image[i] = vector<vector<int>>(height);

second_image[i] = vector<vector<int>>(height);

for (int j = 0; j < height; j++) {

first_image[i][j] = vector<int>(depth);

second_image[i][j] = vector<int>(depth);

}

}

assign randomly RGB values between 0 and 255 of two image vectors.

srand(time(NULL));



for (int i = 0; i < width; i++){

for (int j = 0; j < height; j++){

for (int k = 0; k < depth; k++){

first_image[i][j][k] = rand() % 255;

second_image[i][j][k] = rand() % 255;

}

}

}

send the weight sub-vectors of two image vectors to every slave core by using the rank of the receiving core, individually. (For example, the master core must send third (3rd) weight sub-vectors of two image vectors to the core whose rank is 3.)

for (int i = rank + 1; i < width; i++){

for (int j = 0; j < height; j++){

MPI_Send(&first_image[i][j][0], depth, MPI_INT, i, tag, MPI_COMM_WORLD);

MPI_Send(&second_image[i][j][0], depth, MPI_INT, i, tag, MPI_COMM_WORLD);

}

}

calculate the result of multiplying the own part, which is specified by rank, of two vectors.

The multiplying operation must be done according to the formula below. result = (rgb_value1 × rgb_value2) % 256

for (int i = 0; i < height; i++) {

for (int j = 0; j < depth; j++) {

result_image[rank][i][j] = (first_image[rank][i][j] * second_image[rank][i][j]) % 256;

}

}

receive all the results of multiplying from last slave.

for (int i = 1; i < width; i++) {

for (int j = 0; j < height; j++) {

MPI_Recv(&result_image[i][j][0], depth, MPI_INT, (numberMachines - 1), tag, MPI_COMM_WORLD, MPI_STATUSES_IGNORE);

}

}

The slave cores must

declare two image vectors in 2-D representing two different images, because it calculates just one of the weights vectors.

else {

vector<vector<int>> first_image(height);

vector<vector<int>> second_image(height);



for (int i = 0; i < height; i++) {

first_image[i] = vector<int>(depth);

second_image[i] = vector<int>(depth);

}

receive the values of two image vectors sent by master.

for (int j = 0; j < height; j++){

MPI_Recv(&first_image[j][0], depth, MPI_INT, master, tag, MPI_COMM_WORLD, MPI_STATUSES_IGNORE);

MPI_Recv(&second_image[j][0], depth, MPI_INT, master, tag, MPI_COMM_WORLD, MPI_STATUSES_IGNORE);

}

receive the result of the multiplying in previous slave core, if it is possible.

if (rank != master + 1) {

for (int i = 1; i < width; i++){

for (int j = 0; j < height; j++){

MPI_Recv(&result_image[i][j][0], depth, MPI_INT, rank - 1, tag, MPI_COMM_WORLD, MPI_STATUSES_IGNORE);

}

}

}

calculate the result of multiplying the own part, which is specified by rank, of two vectors.

The multiplying operation must be done according to the formula below. result = (rgb_value1 × rgb_value2) % 256

for (int i = 0; i < height; i++){

for (int j = 0; j < depth; j++){

result_image[rank][i][j] = (first_image[i][j] * second_image[i][j]) % 256;

}

}

send the result of the multiplying to next core, but in the last core to master.

int destCore = rank + 1;

if (rank == (numberMachines - 1))

destCore = master;



for (int i = 1; i < width; i++){

for (int j = 0; j < height; j++){

MPI_Send(&result_image[i][j][0], depth, MPI_INT, destCore, tag, MPI_COMM_WORLD);

}

}

}

MPI_Finalize();

4. Output