graphlab-code/graphlab — pull request #137 by cerquide (2014-04-21 00:52:48+00:00)
Title: Big vertex fixes

As reported [here](http://forum.graphlab.com/discussion/148/non-integer-or-bigger-integer-type-for-vertex-ids), I am interested in extending graphlab so that it can be used with ids that are larger than the standard. This pull request does not change the vertex_id_type, but makes some minor changes which are necessary in the code that make it possible for someone who is interested to use a larger vertex id space.

### Description of the changes

Current version of graphlab is not prepared for larger vertex id types. Slight modifications are needed to the following files:

1. `graphlab/graph/graph_basic_types.hpp`: The type for local vertex id should be decoupled from the larger type (currently they are enforced to be the same type). We do this by introducing a new intermediate type called `standard_vertex_id_type`. For regular users this is just an implementation detail.
2. `graphlab/engine/distributed_chandy_misra.hpp`: The assessment of sequentialization keys misses a cast to `unsigned char`.
3. `graphlab/graph/distributed_graph.hpp` and `graphlab/graph/ingress/distributed_ingress_base.hpp`: `num_in_edges` and `num_out_edges` should have type `size_t`.
4. `graphlab/graph/graph_hash.hpp`: three casts from `vertex_id_type` to `size_t` are missing.

To the best of my knwoledge, those changes will not have any effect when the `vertex_id_type` is kept "standard" (that is, either `uint32_t` or `uint64_t`). Thus I push the changes to be accepted into the graphlab project.

### Steps to use a different `vertex_id_type` (assuming that the changes in this pull request are accepted into the project)

Here follows a step by step guide on how to use other types as `vertex_id_type`.

1. Decide on the type that you would prefer for your vertex id. Constraints on the type are:
1. It should be an arithmetic type implementing +, *, ^ and so on. Currently I have only tested with `boost::multiprecision::uint128_t`, and it fulfills all the needs in this sense.
2. It should be castable into a `size_t` and into an `unsigned char`. It our case this is also provided by the boost multiprecision library.
3. It should be graphlab serializable (for `boost::multiprecision::uint128_t`, I used out of place serialization (details below).
4. There should exist a function `size_t hash_value(const my_large_id_type& x)` in the same namespace where `my_large_id_type` is defined.
2. Modify `graphlab/graph/graph_basic_types.hpp` to convert your new type into the `vertex_id_type`.
`typedef boost::multiprecision::uint128_t vertex_id_type;`
3. Make sure that 1.iii) and 1.iv) are satisfied. In my case this meant including the following code
```
#include
#include
#include

BEGIN_OUT_OF_PLACE_SAVE(oarc, boost::multiprecision::uint128_t , x)
std::stringstream ss;
ss << "0x" << std::hex << x;
std::string s;
ss >> s;
oarc << s;
END_OUT_OF_PLACE_SAVE()

BEGIN_OUT_OF_PLACE_LOAD(iarc, boost::multiprecision::uint128_t , x)
std::string s;
iarc >> s;
x.assign(s);
END_OUT_OF_PLACE_LOAD()

namespace boost {
namespace multiprecision {
inline size_t hash_value (const boost::multiprecision::uint128_t &x)
{
// not a very good hash function, but I just want to get it working first!
return static_cast(x);
}
}
}
```
at the beginning of my graphlab C++ program. Maybe it could be done also in `graphlab/graph/graph_basic_types.hpp` but I just wanted minimal interference with the graphlab code base.