  This work presents a heterogeneous communication library for clusters of
processors and FPGAs. This library, Shoal, supports the Partitioned Global
Address Space (PGAS) memory model for applications. PGAS is a shared memory
model for clusters that creates a distinction between local and remote memory
access. Through Shoal and its common application programming interface for
hardware and software, applications can be more freely migrated to the optimal
platform and deployed onto dynamic cluster topologies.
  The library is tested using a thorough suite of microbenchmarks to establish
latency and throughput performance. We also show an implementation of the
Jacobi iterative method that demonstrates the ease with which applications can
be moved between platforms to yield faster run times. Through this work, we
have demonstrated the feasibility of using a PGAS programming model for
multi-node heterogeneous platforms.
