Vector 4 property serialization
This example show how we can use complex properties in a vector when we use a custom structure with a pointer inside. In particular we will show how to construct the serialization and de-serialization methods for the structure my_struct defined here
In order to make my_struct serializable we need 4 methods
- packRequest This method indicate how many byte are needed to serialize this structure
- pack This method serialize the data-structure
- unpack This method de-serialize the data structure
- noPointers() a static method that inform the OpenFPM sub-system that the structure has no pointers inside
packRequest
template<int ... prp> inline void packRequest(size_t & req) const
{
req += 2*sizeof(size_t) + size + str.size()+1;
v.packRequest(req);
}
This function calculate the size in byte to serialize this structure in this case we serialize 2 numbers so 2*sizeof(size_t). One C string of size "size" and one C++ string of size str.size(). Because std::string has a constructor from null terminating string we add the null terminator to easy construct an std::string. This mean that the C++ string will be serialized in str.size()+1 bytes. The result must be summed to counter req that is used to allocate a buffer big enough for serialization. The function is template and has a variadic argument "int ... prp" this can be ignored
pack
Here we serialize the object. The function give as argument
- ExtPreAlloc<Memory> mem The memory where we have to serialize the information
- Pack_stat unused
- int ... prp unused
The only important parameter is the object mem where we will serialize the information
We first pack the number that indicate the size of the C string. A convenient way is use the function
The function Packer<decltype(object),Memory>::pack(object,sts) work if object is a fundamental C++ type, a struct that does not contain pointers, any object that implement the interface pack,unpack,packRequest (like my_struct). Most openfpm data-structure like openfpm::vector and grid implement such interface and can be used directly with Packer<...>::pack(...). After packed the size of the string we have to serialize the content of the pointer string. unfortunately there is no Packer<...>::pack(...) function to do this and must be manually done. This can be done with the following steps
- Allocate the memory to copy the value of the string
- Get the pointer to the memory allocated
- copy the content of the string into the allocated memory
For the C++ string the concept is the same, the difference is that we put a null terminator at the end of the serialized string
memcpy(t_ptr2,str.c_str(),str.size());
t_ptr2[str.size()] = 0;
unpack
Here we de-serialize the object. The function take a reference to object
- ExtPreAlloc<Memory> mem, contain the serialized information that must be de-serialized
- Pack_stat contain the offset to the memory to de-serialize
- int ... prp unused
De-serialization must be in general done in the same order as we serialized We first unpack the number that indicate the size of the C string. A convenient way is to use the function
the variable size contain now the size of the packed string we can now
- create memory with new that store the C string
- Get the pointer to the serialized information
- copy the string from mem into the created memory
- update the offset pointer
ptr = new char[size];
char * ptr_src = (
char *)mem.getPointerOffset(ps.
getOffset());
memcpy(ptr,ptr_src,size);
For the C++ string is just the same
- We get the size of the string
- we create an std::string out of the null terminating serialized string
- we assign the created std::string to str
size_t cpp_size;
char * ptr_src2 = (
char *)mem.getPointerOffset(ps.
getOffset());
str = std::string(ptr_src2);
noPointers
This method inform the sub-system that the custom structure does not have pointers
static bool noPointers()
{
return false;
}
Constructor and destructor and operator=
Constructor and destructor are not releated to serialization and de-serialization concept. But on how my_struct is constructed and destructed in order to avoid memory leak/corruption. In the constructor we set ptr to NULL in the destructor we destroy the pointer (if different from NULL) to avoid memory leak
{
size = 0;
ptr = NULL;
}
{
this->operator=(my);
}
{
this->operator=(my);
}
{
if (ptr != NULL)
delete [] ptr;
}
{
ptr = new char[size];
return *this;
}
{
my.size = 0;
str.swap(my.str);
v.swap(my.v);
ptr = my.ptr;
my.ptr = 0;
return *this;
}
Initialization and vector creation
After we initialize the library we can create a vector with complex properties with the following line
float[3],
Point<3,double>,
openfpm::vector<float>,
openfpm::vector<A>,
vd(4096,domain,bc,g);
In this this particular case every particle carry two my_struct object
Assign values to properties
In this loop we assign position to particles and we fill the two my_struct that each particle contain. As demostration the first my_struct is filled with the string representation of the particle coordinates. The second my struct is filled with the string representation of the particle position multiplied by 2.0. The the vectors of the two my_struct are filled respectively with the sequence 1,2,3 and 1,2,3,4
while (it.isNext())
{
vd.
getPos(p)[0] = (float)rand() / RAND_MAX;
vd.
getPos(p)[1] = (float)rand() / RAND_MAX;
vd.
getProp<my_s1>(p).ptr =
new char[32];
pt = pt * 2.0;
vd.
getProp<my_s2>(p).ptr =
new char[32];
++it;
}
Mapping and ghost_get
Particles are redistributed across processors and we also synchronize the ghost
- See Also
- Mapping particles
-
Ghost
Output and VTK visualization
Vector with complex properties can be still be visualized, because unknown properties are automatically excluded
- See Also
- Visualization, write VTK files
Print 4 particles in the ghost area
Here we print that the first 4 particles to show that the two my_struct contain the right information
{
{
std::cout << "my_struct1:" << std::endl;
std::cout <<
"C-string: " << vd.
getProp<my_s1>(fg).ptr << std::endl;
std::cout <<
"Cpp-string: " << vd.
getProp<my_s1>(fg).str << std::endl;
for (
size_t i = 0 ; i < vd.
getProp<my_s1>(fg).v.size() ; i++)
std::cout <<
"Element: " << i <<
" " << vd.
getProp<my_s1>(fg).v.get(i) << std::endl;
std::cout << "my_struct2" << std::endl;
std::cout <<
"C-string: " << vd.
getProp<my_s2>(fg).ptr << std::endl;
std::cout <<
"Cpp-string: " << vd.
getProp<my_s2>(fg).str << std::endl;
for (
size_t i = 0 ; i < vd.
getProp<my_s2>(fg).v.size() ; i++)
std::cout <<
"Element: " << i <<
" " << vd.
getProp<my_s2>(fg).v.get(i) << std::endl;
}
}
Finalize
At the very end of the program we have always to de-initialize the library
Full code
#include "Vector/vector_dist.hpp"
{
public:
static bool pack() {
return true;}
{
size = 0;
}
{
}
{
}
{
}
{
return *this;
}
{
my.size = 0;
v.swap(my.v);
my.ptr = 0;
return *this;
}
template<
int ... prp>
inline void packRequest(
size_t & req)
const
{
req += 2*
sizeof(size_t) + size +
str.size()+1;
v.packRequest(req);
}
static bool noPointers()
{
return false;
}
{
memcpy(t_ptr2,
str.c_str(),
str.size());
}
{
memcpy(
ptr,ptr_src,size);
size_t cpp_size;
str = std::string(ptr_src2);
}
};
int main(int argc, char* argv[])
{
openfpm_init(&argc,&argv);
size_t bc[2]={PERIODIC,PERIODIC};
constexpr int my_s1 = 0;
constexpr int my_s2 = 1;
vd(4096,domain,bc,g);
while (it.isNext())
{
vd.
getPos(p)[0] = (float)rand() / RAND_MAX;
vd.
getPos(p)[1] = (float)rand() / RAND_MAX;
vd.
getProp<my_s1>(p).ptr =
new char[32];
pt = pt * 2.0;
vd.
getProp<my_s2>(p).ptr =
new char[32];
++it;
}
{
{
std::cout << "my_struct1:" << std::endl;
std::cout <<
"C-string: " << vd.
getProp<my_s1>(fg).ptr << std::endl;
std::cout <<
"Cpp-string: " << vd.
getProp<my_s1>(fg).str << std::endl;
for (
size_t i = 0 ; i < vd.
getProp<my_s1>(fg).v.size() ; i++)
std::cout <<
"Element: " << i <<
" " << vd.
getProp<my_s1>(fg).v.get(i) << std::endl;
std::cout << "my_struct2" << std::endl;
std::cout <<
"C-string: " << vd.
getProp<my_s2>(fg).ptr << std::endl;
std::cout <<
"Cpp-string: " << vd.
getProp<my_s2>(fg).str << std::endl;
for (
size_t i = 0 ; i < vd.
getProp<my_s2>(fg).v.size() ; i++)
std::cout <<
"Element: " << i <<
" " << vd.
getProp<my_s2>(fg).v.get(i) << std::endl;
}
}
openfpm_finalize();
}