Package: pykokkos-base
pykokkos-baseAdditional Documentation can be found inWikiOverviewThis package contains the minimal set of bindings forKokkosinteroperability with Python:Free-standing function bindingsKokkos::initialize(...)Kokkos::finalize()Kokkos::is_initialized()Kokkos::deep_copy(...)Kokkos::create_mirror(...)Kokkos::create_mirror_view(...)Kokkos::Tools::profileLibraryLoaded()Kokkos::Tools::pushRegion(...)Kokkos::Tools::popRegion()Kokkos::Tools::createProfileSection(...)Kokkos::Tools::destroyProfileSection(...)Kokkos::Tools::startSection(...)Kokkos::Tools::stopSection(...)Kokkos::Tools::markEvent(...)Kokkos::Tools::declareMetadata(...)Kokkos::Tools::Experimental::set_<...>_callback(...)Data structuresKokkos::View<...>Kokkos::DynRankView<...>Kokkos_Profiling_KokkosPDeviceInfoKokkos_Profiling_SpaceHandleBy importing this package in Python, you can pass the supported Kokkos Views and DynRankViews
from C++ to Python and vice-versa. Furthermore, in Python, these bindings provide interoperability
with numpy and cupy arrays:importkokkosimportnumpyasnpview=kokkos.array([2,2],dtype=kokkos.double,space=kokkos.CudaUVMSpace,layout=kokkos.LayoutRight,trait=kokkos.RandomAccess,dynamic=False)arr=np.array(view,copy=False)Writing Kokkos in PythonIn order to write native Kokkos in Python, seepykokkos.InstallationYou can install this package via CMake or Python'ssetup.py. The important cmake options are:ENABLE_VIEW_RANKS(integer)ENABLE_LAYOUTS(bool)ENABLE_MEMORY_TRAITS(bool)ENABLE_INTERNAL_KOKKOS(bool)By default, CMake will enable the layouts and memory traits options if the Kokkos installation was not
built with CUDA support.
If Kokkos was built with CUDA support,ENABLE_MEMORY_TRAITSwill be disabled by default due to unreasonable
compilation times (> 1 hour).
TheENABLE_VIEW_RANKSoption (defaults to a value of 4) is the max number of ranks forKokkos::View<...>that can be returned to Python. For example, value of 4 means that
views of data typeT*,T**,T***, andT****can be returned to python butT*****and higher cannot. Increasing this value up to 7 can dramatically increase the length
of time required to compile the bindings.Kokkos InstallationIf theENABLE_INTERNAL_KOKKOSoption is not specified the first time CMake is run, CMake will try to
find an existing Kokkos installation. If no existing installation is found, it will build and install
Kokkos from a submodule. When Kokkos is added as a submodule, you can configure the submodule
as you would normally configure Kokkos. However, due to some general awkwardness configuring cmake
fromsetup.py(especially viapip install), CMake tries to "automatically" configure
reasonable default CMake settings for the Kokkos submodule.Here are the steps when Kokkos is added as a submodule:Doesexternal/kokkos/CMakeLists.txtexists?YES: assumes the submodule is already checked outIf compute node does not have internet access, checkout submodule before installing!NO: does.gitmodulesexist?YES:git submodule update --init external/kokkosNO:git clone -b master https://github.com/kokkos/kokkos.git external/kokkosSetBUILD_SHARED_LIBS=ONSetKokkos_ENABLE_SERIAL=ONfind_package(OpenMP)Was OpenMP found?YES: setKokkos_ENABLE_OPENMP=ONNO:find_package(Threads)Was Threads found?YES: setKokkos_ENABLE_THREADS=ON(if not Windows)find_package(CUDA)Was CUDA found?YES: set:Kokkos_ENABLE_CUDA=ONKokkos_ENABLE_CUDA_UVM=ONKokkos_ENABLE_CUDA_LAMBDA=ONConfiguring Options via CMakecmake -DENABLE_LAYOUTS=ON -DENABLE_MEMORY_TRAITS=OFF /path/to/sourceConfiguring Options viasetup.pyThere are three ways to configure the options:Via the Python argparse options--enable-<option>and--disable-<option>Setting thePYKOKKOS_BASE_SETUP_ARGSenvironment variable to the CMake optionsPassing in the CMake options after a--All three lines below are equivalent:python setup.py install --enable-layouts --disable-memory-traitsPYKOKKOS_BASE_SETUP_ARGS="-DENABLE_LAYOUTS=ON -DENABLE_MEMORY_TRAITS=OFF" python setup.py installpython setup.py install -- -DENABLE_LAYOUTS=ON -DENABLE_MEMORY_TRAITS=OFFConfiguring Options viapipPip does not handle build options well. Thus, it is recommended to use thePYKOKKOS_BASE_SETUP_ARGSenvironment variable noted above. However, using the--install-optionfor pip is possible but
each "space" must have it's own--install-option, e.g. all of the following are equivalent:
All three lines below are equivalent:pip install pykokkos-base --install-option=--enable-layouts --install-option=--disable-memory-traitspip install pykokkos-base --install-option=-- --install-option=-DENABLE_LAYOUTS=ON --install-option=-DENABLE_MEMORY_TRAITS=OFFpip install pykokkos-base --install-option={--enable-layouts,--disable-memory-traits}pip install pykokkos-base --install-option={--,-DENABLE_LAYOUTS=ON,-DENABLE_MEMORY_TRAITS=OFF}pip install pykokkos-basewill build against the latest release in the PyPi repository.
In order to pip install from this repository, usepip install --user -e .Differences vs. Kokkos C++Deep Copy and Host MirrorIf you are not familiar withKokkos::deep_copy(...),Kokkos::create_mirror(...),Kokkos::create_mirror_view(...), read thisKokkos Wiki entry.When Kokkos views are allocated on a non-host memory space, this data is not directly accessible in Python. Any
attempt to read or modify the data will result in a fatal error. In C++, Kokkos developers usually
perform two distinct operations: create a mirror or mirror-view and then execute a deep-copy, e.g.:// assume MemorySpace is Kokkos::Cuda or similarKokkos::View<int*,MemorySpace>a("a",10);// Allocate a view in HostSpace with the layout and padding of aautob=create_mirror(a);// This is always a memcopyKokkos::deep_copy(b,a);// This may not allocate a new view if a is in host spaceautoc=Kokkos::create_mirror_view(a);// This is a no-op if MemorySpace is HostSpaceKokkos::deep_copy(c,a)The python equivalent is available via standalone functions:# assume MemorySpace is kokkos.CudaSpace or similara=kokkos.array("a",shape=[10],space=MemorySpace)# Allocate a view in HostSpace with the layout and padding of ab=kokkos.create_mirror(a)# copy memorykokkos.deep_copy(b,a)# This may not allocate a new view if a is in host spacec=kokkos.create_mirror_view(a)# This is a no-op if MemorySpace is HostSpacekokkos.deep_copy(c,a)However, this makes it cumbersome to print data in python:# assume MemorySpace is kokkos.CudaSpace or similara=kokkos.array("a",shape=[10],space=MemorySpace)defprint_data(inp):v=kokkos.create_mirror_view(inp)kokkos.deep_copy(v,inp)foriinrange(v.shape[0]):print(f"v({i}) ={v[i]}")print_data(a)Thus, themember functionscreate_mirror()andcreate_mirror_view()accept a booleancopyargument whichdefaults to True, e.g.:a=kokkos.array("a",shape=[10],space=MemorySpace)# this:b=a.create_mirror()# is implicitly:b=a.create_mirror(copy=True)Thus, ourprint_datafunction above does not need handle mirror creation because
we can replaceprint_data(a)withprint_data(a.create_mirror())orprint_data(a.create_mirror_view()):# assume MemorySpace is kokkos.CudaSpace or similara=kokkos.array("a",shape=[10],space=MemorySpace)defprint_data(v):foriinrange(v.shape[0]):print(f"v({i}) ={v[i]}")print_data(a.create_mirror_view())In fact, the free-standingkokkos.create_mirror(...)andkokkoos.create_mirror_view(...)simply use this member function
and default thecopyargument toFalse:defcreate_mirror(src,copy=False):"""Performs Kokkos::create_mirror"""returnsrc.create_mirror(copy)defcreate_mirror_view(src,copy=False):"""Performs Kokkos::create_mirror_view"""returnsrc.create_mirror_view(copy)ExampleOverviewThis example is designed to emulate a work-flow where the user has code using Kokkos in C++ and writes python bindings to those functions. A python script is used as the"main":ex-numpy.pyimports the kokkos bindingsCalls a routine in the "users" python bindings to a C++ function which returns aKokkos::ViewThis view is then converted to a numpy array in python and printed via the numpy capabilities.Filesex-generate.cppThis is the python bindings to the user codeuser.cppThis is the implementation of the user's code which returns aKokkos::View<double**, Kokkos::HostSpace>ex-numpy.pyThis is the "main"ex-numpy.py#!/usr/bin/env pythonimportargparseimportnumpyasnp## The python bindings for generate_view are in ex-generate.cpp# The declaration and definition of generate_view are in user.hpp and user.cpp# The generate_view function will return a Kokkos::View and will be converted# to a numpy arrayfromex_generateimportgenerate_view,modify_view## Importing this module is necessary to call kokkos init/finalize and# import the python bindings to Kokkos::View which generate_view will# return#importkokkosdefprint_data(label,name,data):# write the type infoprint("{:12}:{}(ndim={}, shape={})".format(label,type(data).__name__,data.ndim,data.shape))# print the dataifdata.ndim==1:foriinrange(data.shape[0]):print("{:8}({}) ={}".format(name,i,data[i]))elifdata.ndim==2:foriinrange(data.shape[0]):print("{:8}({}) = [{}]".format(name,i," ".join("{}".format(data[i,j])forjinrange(data.shape[1])),))else:raiseValueError("only 2 dimensions are supported")defuser_bindings(args):# get the kokkos viewview=generate_view(args.ndim)print_data("Kokkos View","view",view.create_mirror_view())# modify view (verify that casting works)modify_view(view)print_data("Modify View","view",view.create_mirror_view())# wrap the buffer protocal as numpy array without copying the dataarr=np.array(view.create_mirror_view(),copy=False)print_data("Numpy Array","arr",arr)defto_numpy(args):# get the kokkos viewview=kokkos.array("python_allocated_view",[args.ndim],dtype=kokkos.double,space=kokkos.DefaultHostMemorySpace,)foriinrange(view.shape[0]):view[i]=i*(i%2)print_data("Kokkos View","view",view)# wrap the buffer protocal as numpy array without copying the dataarr=np.array(view,copy=False)print_data("Numpy Array","arr",arr)deffrom_numpy(args):arr=np.ones([args.ndim,args.ndim],dtype=np.int32)foriinrange(args.ndim):arr[i,i]=0print_data("Numpy Array","arr",arr)view=kokkos.array(arr,dtype=kokkos.int32,dynamic=True)print_data("Kokkos View","view",view)if__name__=="__main__":try:kokkos.initialize()parser=argparse.ArgumentParser()parser.add_argument("-n","--ndim",default=10,help="X dimension",type=int)args,argv=parser.parse_known_args()print("Executing to numpy...")to_numpy(args)print("Executing from numpy...")from_numpy(args)print("Executing user bindings...")user_bindings(args)kokkos.finalize()exceptExceptionase:importsysimporttracebackprint(f"{e}")exc_type,exc_value,exc_traceback=sys.exc_info()traceback.print_exception(exc_type,exc_value,exc_traceback)sys.exit(1)Build and Runmkdir buildcd buildcmake -DENABLE_EXAMPLES=ON ..makepython ./ex-numpy.pyExpected Output[user-bindings]> Generating View... Done.[user-bindings]> Modifying View... Done.Executing to numpy...Kokkos View  : KokkosView_float64_HostSpace_LayoutRight_1 (ndim=1, shape=[10])view    (0) = 0.0view    (1) = 1.0view    (2) = 0.0view    (3) = 3.0view    (4) = 0.0view    (5) = 5.0view    (6) = 0.0view    (7) = 7.0view    (8) = 0.0view    (9) = 9.0Numpy Array  : ndarray (ndim=1, shape=(10,))arr     (0) = 0.0arr     (1) = 1.0arr     (2) = 0.0arr     (3) = 3.0arr     (4) = 0.0arr     (5) = 5.0arr     (6) = 0.0arr     (7) = 7.0arr     (8) = 0.0arr     (9) = 9.0Executing from numpy...Numpy Array  : ndarray (ndim=2, shape=(10, 10))arr     (0) = [0 1 1 1 1 1 1 1 1 1]arr     (1) = [1 0 1 1 1 1 1 1 1 1]arr     (2) = [1 1 0 1 1 1 1 1 1 1]arr     (3) = [1 1 1 0 1 1 1 1 1 1]arr     (4) = [1 1 1 1 0 1 1 1 1 1]arr     (5) = [1 1 1 1 1 0 1 1 1 1]arr     (6) = [1 1 1 1 1 1 0 1 1 1]arr     (7) = [1 1 1 1 1 1 1 0 1 1]arr     (8) = [1 1 1 1 1 1 1 1 0 1]arr     (9) = [1 1 1 1 1 1 1 1 1 0]Kokkos View  : KokkosDynRankView_int32_HostSpace_LayoutRight (ndim=2, shape=[10, 10, 1, 1, 1, 1, 1])view    (0) = [0 1 1 1 1 1 1 1 1 1]view    (1) = [1 0 1 1 1 1 1 1 1 1]view    (2) = [1 1 0 1 1 1 1 1 1 1]view    (3) = [1 1 1 0 1 1 1 1 1 1]view    (4) = [1 1 1 1 0 1 1 1 1 1]view    (5) = [1 1 1 1 1 0 1 1 1 1]view    (6) = [1 1 1 1 1 1 0 1 1 1]view    (7) = [1 1 1 1 1 1 1 0 1 1]view    (8) = [1 1 1 1 1 1 1 1 0 1]view    (9) = [1 1 1 1 1 1 1 1 1 0]Executing user bindings...Kokkos View  : KokkosView_float64_HostSpace_LayoutRight_2 (ndim=2, shape=[10, 2])view    (0) = [-1.0 1.0]view    (1) = [-2.0 2.0]view    (2) = [-3.0 3.0]view    (3) = [-4.0 4.0]view    (4) = [-5.0 5.0]view    (5) = [-6.0 6.0]view    (6) = [-7.0 7.0]view    (7) = [-8.0 8.0]view    (8) = [-9.0 9.0]view    (9) = [-10.0 10.0]Modify View  : KokkosView_float64_HostSpace_LayoutRight_2 (ndim=2, shape=[10, 2])view    (0) = [-2.0 2.0]view    (1) = [-4.0 4.0]view    (2) = [-6.0 6.0]view    (3) = [-8.0 8.0]view    (4) = [-10.0 10.0]view    (5) = [-12.0 12.0]view    (6) = [-14.0 14.0]view    (7) = [-16.0 16.0]view    (8) = [-18.0 18.0]view    (9) = [-20.0 20.0]Numpy Array  : ndarray (ndim=2, shape=(10, 2))arr     (0) = [-2.0 2.0]arr     (1) = [-4.0 4.0]arr     (2) = [-6.0 6.0]arr     (3) = [-8.0 8.0]arr     (4) = [-10.0 10.0]arr     (5) = [-12.0 12.0]arr     (6) = [-14.0 14.0]arr     (7) = [-16.0 16.0]arr     (8) = [-18.0 18.0]arr     (9) = [-20.0 20.0]