Vector 1 ghost get and ghost put

Ghost get and ghost put

This example shows more in details the functionalities of ghost_get and ghost_put for a distributed vector.

Inclusion

We activate the vector_dist functionalities

#include "Vector/vector_dist.hpp"

See also

inclusion

Initialization

Here we

• Initialize the library
• we create a Box that define our domain
• An array that define our boundary conditions
• A Ghost object that will define the extension of the ghost part in physical units

See also

Initialization

 
    // initialize the library
    openfpm_init(&argc,&argv);
 
    // Here we define our domain a 2D box with internals from 0 to 1.0 for x and y
    Box<2,float> domain({0.0,0.0},{1.0,1.0});
 
    // Here we define the boundary conditions of our problem
    size_t bc[2]={PERIODIC,PERIODIC};
 
    // extended boundary around the domain, and the processor domain
    Ghost<2,float> g(0.01);
    

Vector instantiation

Here we are creating a distributed vector following the previous example

See also

Vector instantiation

 
    vector_dist<2,float, aggregate<float,float[3],float[3][3]> > vd(4096,domain,bc,g);
 
    // the scalar is the element at position 0 in the aggregate
    const int scalar = 0;
 
    // the vector is the element at position 1 in the aggregate
    const int vector = 1;
 
    // the tensor is the element at position 2 in the aggregate
    const int tensor = 2;
 

Assign values

We get an iterator and we iterate across the 4096 particles where we define their positions and properties

See also

Assign position

 
    auto it = vd.getDomainIterator();
 
    while (it.isNext())
    {
        // Particle p
        auto p = it.get();
 
        // we define x, assign a random position between 0.0 and 1.0
        vd.getPos(p)[0] = (float)rand() / RAND_MAX;
 
        // we define y, assign a random position between 0.0 and 1.0
        vd.getPos(p)[1] = (float)rand() / RAND_MAX;
 
        // the scalar property
        vd.template getProp<scalar>(p) = vd.getPos(p)[0];
 
        // The vector
        vd.template getProp<vector>(p)[0] = 1.0;
        vd.template getProp<vector>(p)[1] = 2.0;
        vd.template getProp<vector>(p)[2] = 3.0;
 
        // The tensor
        vd.template getProp<tensor>(p)[0][0] = 1.0;
        vd.template getProp<tensor>(p)[0][1] = 2.0;
        vd.template getProp<tensor>(p)[0][2] = 3.0;
        vd.template getProp<tensor>(p)[1][0] = 4.0;
        vd.template getProp<tensor>(p)[1][1] = 5.0;
        vd.template getProp<tensor>(p)[1][2] = 6.0;
        vd.template getProp<tensor>(p)[2][0] = 7.0;
        vd.template getProp<tensor>(p)[2][1] = 8.0;
        vd.template getProp<tensor>(p)[2][2] = 9.0;
 
        // next particle
        ++it;
    }
 

Mapping particles

We redistribute the particles according to the underline decomposition

 
    vd.map();
 

ghost_get

Here we synchronize the ghosts in the standard way, updating the scalar and the tensor property.

Warning

Every ghost_get by default reset the status of the ghost so the previous information is lost

In the 2 images below we see what happen when we do a standard ghost_get Before and after. The blue arrows in the first image indicate the vector field for the real particles. In the second image instead the red arrow indicate the vector field for the real particle. The blue arrow indicate the ghosts. We can note that the blue arrow does not contain the correct vector. The reason is that when we used ghost_get we synchronized the scalar, and the tensor, but not the vector.

See also

Ghost

 
    // write in vtk to see the result before
    vd.write("before_ghost_get");
 
    // Here we synchronize the ghost get only the scalar and the tensor property
    vd.ghost_get<scalar,tensor>();
 
    // write in vtk to see the result after
    vd.write("after_ghost_get");
 

So ... how I have to put these ghost_get

The first thing to do is to place the ghost in a way that the program work in parallel for sure. In order to do this we can do the following reasoning: If we have a loop over particles we distinguish two type of loops:

• A loop that iterate over particles
• A loop that iterate over particles and neighborhood particles

If the loop is of the first type (you do not loop over the neighborhood particles) ghost_get is not necessary. If I am in the second case I need a ghost_get. The second point is which property I have to synchronize ghost_get<...>(), or more practically what I have to put in the ... . To answer this we have to check all the properties that we use from the neighborhood particles and pass it to ghost_get as a list. To summarize:

I am doing a simple loop over particles (1), or I am looping also over neighborhood particles (2)?
For the case (1) the answer is "I do not need ghost_get". For the case (2) the answer is "I need ghost_get"
 
if I am on the case (2) the second question is which parameters should I use ?
          The answer is look at all vd.getProp<...>(b) where b is a neighborhood particle. All ... properties should appear in
          ghost_get<...>()

This reasoning is always enough to have ghost_get function always placed correctly. For more fine tuning look at the options below

ghost_get KEEP_PROPERTIES

As described before every ghost_get reset the status of the ghost. In particular all the informations are lost. So for example doing a ghost_get<scalar> followed by a ghost_get<vector> will not preserve the information of the ghost_get<scalar>. This because in general ghost_get recompute the set of particles to send to the other processor based on their geometrical position. If the particles move between the ghost_get<scalar> and the the ghost_get<vector> the set of the particles sent can change between the 2 ghost_get. Merge the information between the two different set of particles would end up to be non trivial and would require to send additional information. This will make the communication inconveniently heavier. In the case the particles does not move on the other hand it would be possible to do a trivial merge of the information. The library is not able to detect automatically such cases. The information must be given explicitly. KEEP_PROPERTIES is an option that can be given to ghost_get to force to do a trivial merge in case the particles do not move. More in general such property is safe to be used in case that the particles move between ghost_get. What will happen is that the computation of the set of the particles to send will be fully skipped. OpenFPM will send the same set of particles from the previous ghost_get. Such functionality is important in case of usage of Verlet-List with radius+skin. This functionality is advanced and will be explained in a next example.

See also

Vector 5 molecular dynamic with symmetric Verlet list

Because the option KEEP_PROPERTIES has the functionality to keep the properties and skip the labelling. The option KEEP_PROPERTIES is also names SKIP_LABELLING. The two options are exactly equivalents.

In the picture below we can see what happen when from the previous situation, we do a ghost_get with KEEP_PROPERTIES for the vector field. The vector change pointing to the right direction. On the other hand also the scalar (The dot color)is kept and the information is not destroyed.

 
    vd.ghost_get<vector>(KEEP_PROPERTIES);
    
    // write in vtk to see the result before
    vd.write("after_ghost_get_kp");
 

ghost_get NO_POSITION

ghost_get in general send automatically the information about position. If we are sure that the particles position did not change its position we can use the option NO_POSITION, to avoid to send the positional informations. With the only purpose to show what happens we shift the particle position of the ghost parts by one. We also force the vector to point to the opposite direction.

Doing a ghost_get with KEEP_PROPERTY and NO_POSITION, it will updates the information of the vectors, but the position will remain the same.

We can restore the positioninformation doing a ghost_get without NO_POSITION

 
    it = vd.getGhostIterator();
 
    while (it.isNext())
    {
        // Particle p
        auto p = it.get();
 
        // we shift down he particles
        vd.getPos(p)[0] -= 1.0;
 
        // we shift
        vd.getPos(p)[1] -= 1.0;
 
        // The vector
        vd.template getProp<vector>(p)[0] = -1.0;
        vd.template getProp<vector>(p)[1] = -2.0;
        vd.template getProp<vector>(p)[2] = -3.0;
 
        // next particle
        ++it;
    }
 
    vd.ghost_get<vector>(KEEP_PROPERTIES | NO_POSITION);
 
    // write in vtk to see the result before
    vd.write("after_ghost_get_kp_no_pos");
 
    vd.ghost_get<vector>(KEEP_PROPERTIES);
 
    vd.write("after_ghost_get_kp_no_pos_restore");
 

ghost_put

ghost_put is another particular function. It work similarly to ghost_get but in the inverted direction. In particular it take the information from the ghost particles and it send the information back to the real particles. How to merge the information back to the real particles is defined by an operation.

In this particular case we scatter back the information present in the ghost.

Note

ghost_put does not re-compute the set of the particles to send. This mean that the set of particles communicated back from ghost_put is given by the set of particles received by the last ghost_get. The reason of this are similar to the ghost_get. If the set of particles is recomputed there is no trivial way for the other processor to merge the information. This mean that additional information must be sent and this is most of the time inconvenient.

Once the particles are received back we add_ such contributions to the real particles. A typical real application of such functionality is in the case of symmetric interactions

As we can see from the image below some of red particles near the ghost has a bigger arrow

See also

Vector 5 molecular dynamic with symmetric Verlet list

 
    vd.ghost_put<add_,vector>();
 
    // write in vtk to see the result before
    vd.write("after_ghost_put");
 

Finalize

At the very end of the program we have always de-initialize the library

 
    openfpm_finalize();
 

Full code

 
#include "Vector/vector_dist.hpp"
 
int main(int argc, char* argv[])
{
 
 
    // initialize the library
    openfpm_init(&argc,&argv);
 
    // Here we define our domain a 2D box with internals from 0 to 1.0 for x and y
    Box<2,float> domain({0.0,0.0},{1.0,1.0});
 
    // Here we define the boundary conditions of our problem
    size_t bc[2]={PERIODIC,PERIODIC};
 
    // extended boundary around the domain, and the processor domain
    Ghost<2,float> g(0.01);
    
 
 
    vector_dist<2,float, aggregate<float,float[3],float[3][3]> > vd(4096,domain,bc,g);
 
    // the scalar is the element at position 0 in the aggregate
    const int scalar = 0;
 
    // the vector is the element at position 1 in the aggregate
    const int vector = 1;
 
    // the tensor is the element at position 2 in the aggregate
    const int tensor = 2;
 
 
 
    auto it = vd.getDomainIterator();
 
    while (it.isNext())
    {
        // Particle p
        auto p = it.get();
 
        // we define x, assign a random position between 0.0 and 1.0
        vd.getPos(p)[0] = (float)rand() / RAND_MAX;
 
        // we define y, assign a random position between 0.0 and 1.0
        vd.getPos(p)[1] = (float)rand() / RAND_MAX;
 
        // the scalar property
        vd.template getProp<scalar>(p) = vd.getPos(p)[0];
 
        // The vector
        vd.template getProp<vector>(p)[0] = 1.0;
        vd.template getProp<vector>(p)[1] = 2.0;
        vd.template getProp<vector>(p)[2] = 3.0;
 
        // The tensor
        vd.template getProp<tensor>(p)[0][0] = 1.0;
        vd.template getProp<tensor>(p)[0][1] = 2.0;
        vd.template getProp<tensor>(p)[0][2] = 3.0;
        vd.template getProp<tensor>(p)[1][0] = 4.0;
        vd.template getProp<tensor>(p)[1][1] = 5.0;
        vd.template getProp<tensor>(p)[1][2] = 6.0;
        vd.template getProp<tensor>(p)[2][0] = 7.0;
        vd.template getProp<tensor>(p)[2][1] = 8.0;
        vd.template getProp<tensor>(p)[2][2] = 9.0;
 
        // next particle
        ++it;
    }
 
 
 
    vd.map();
 
 
 
    // write in vtk to see the result before
    vd.write("before_ghost_get");
 
    // Here we synchronize the ghost get only the scalar and the tensor property
    vd.ghost_get<scalar,tensor>();
 
    // write in vtk to see the result after
    vd.write("after_ghost_get");
 
 
 
    vd.ghost_get<vector>(KEEP_PROPERTIES);
    
    // write in vtk to see the result before
    vd.write("after_ghost_get_kp");
 
 
 
    it = vd.getGhostIterator();
 
    while (it.isNext())
    {
        // Particle p
        auto p = it.get();
 
        // we shift down he particles
        vd.getPos(p)[0] -= 1.0;
 
        // we shift
        vd.getPos(p)[1] -= 1.0;
 
        // The vector
        vd.template getProp<vector>(p)[0] = -1.0;
        vd.template getProp<vector>(p)[1] = -2.0;
        vd.template getProp<vector>(p)[2] = -3.0;
 
        // next particle
        ++it;
    }
 
    vd.ghost_get<vector>(KEEP_PROPERTIES | NO_POSITION);
 
    // write in vtk to see the result before
    vd.write("after_ghost_get_kp_no_pos");
 
    vd.ghost_get<vector>(KEEP_PROPERTIES);
 
    vd.write("after_ghost_get_kp_no_pos_restore");
 
 
 
    vd.ghost_put<add_,vector>();
 
    // write in vtk to see the result before
    vd.write("after_ghost_put");
 
 
 
    openfpm_finalize();
 
 
}