Repository: spack/spack

Issue:
Installation issue: ucx with gdrcopy
### Steps to reproduce the issue

```console
$ spack install ucx@1.10.1+cuda+gdrcopy
```
When I tried to build UCX 1.10.1 with GDRCopy 2.2, I ran into the following error:

```
rocm_gdr_ep.c: In function 'uct_rocm_gdr_ep_put_short':
rocm_gdr_ep.c:47:15: error: implicit declaration of function 'gdr_copy_to_bar'; did you mean 'gdr_copy_to_mapping'? [-Werror=implicit-function-declaration]
   47 |         ret = gdr_copy_to_bar((void *)remote_addr, buffer, length);
      |               ^~~~~~~~~~~~~~~
      |               gdr_copy_to_mapping
rocm_gdr_ep.c:47:15: error: nested extern declaration of 'gdr_copy_to_bar' [-Werror=nested-externs]
rocm_gdr_ep.c: In function 'uct_rocm_gdr_ep_get_short':
rocm_gdr_ep.c:67:15: error: implicit declaration of function 'gdr_copy_from_bar'; did you mean 'gdr_copy_from_mapping'? [-Werror=implicit-function-declaration]
   67 |         ret = gdr_copy_from_bar(buffer, (void *)remote_addr, length);
      |               ^~~~~~~~~~~~~~~~~
      |               gdr_copy_from_mapping
rocm_gdr_ep.c:67:15: error: nested extern declaration of 'gdr_copy_from_bar' [-Werror=nested-externs]
cc1: all warnings being treated as errors
make[4]: *** [Makefile:622: libuct_rocm_gdr_la-rocm_gdr_ep.lo] Error 1
make[4]: *** Waiting for unfinished jobs....
```

The issues seems to come from incompatibility of `rocm_gdr` with newer versions of GDRCopy (> 2.0, I presume) which has been discussed [here](https://github.com/openucx/ucx/issues/4489). I could fix it by adding a new variant `rocm` to UCX package and setting the default to `False`.

Currently, Spack does not `rocm` as a variant and probably adding this variant and setting to `False` is a solution?

### Information on your system

* **Spack:** 0.16.3-5026-94a9733822
* **Python:** 3.7.3
* **Platform:** linux-debian10-broadwell
* **Concretizer:** clingo

### Additional information

```
==> ucx: Executing phase: 'autoreconf'
==> ucx: Executing phase: 'configure'
==> [2021-10-27-13:02:21.855882] 'contrib/configure-release' '--prefix=/home/mpaipuri/spack/opt/spack/linux-debian10-broadwell/gcc-9.3.0/ucx-1.10.1-4ghx7jadjpm6sgy6b4ingjkrpjors2sa' '--enable-mt' '--enable-cma' '--disable-params-check' '--with-avx' '--enable-optimizations' '--disable-assertions' '--disable-logging' '--with-pic' '--with-rc' '--with-ud' '--with-dc' '--with-mlx5-dv' '--with-ib-hw-tm' '--without-dm' '--without-cm' '--without-java' '--with-cuda=/home/mpaipuri/spack/opt/spack/linux-debian10-broadwell/gcc-9.3.0/cuda-11.4.0-ngdlgzyygj5tbrjhbdfplywzmbq7gev6' '--with-gdrcopy=/home/mpaipuri/spack/opt/spack/linux-debian10-broadwell/gcc-9.3.0/gdrcopy-2.2-lhghcojywjtsxcr56knumxq6zq7hmp4x' '--without-knem' '--without-xpmem'
checking for gcc... /home/mpaipuri/spack/lib/spack/env/gcc/gcc
checking whether the C compiler works... yes
checking for C compiler default output file name... a.out
checking for suffix of executables... 
checking whether we are cross compiling... no
checking for suffix of object files... o
checking whether we are using the GNU C compiler... yes
checking whether /home/mpaipuri/spack/lib/spack/env/gcc/gcc accepts -g... yes
checking for /home/mpaipuri/spack/lib/spack/env/gcc/gcc option to accept ISO C89... none needed
checking how to run the C preprocessor... /home/mpaipuri/spack/lib/spack/env/gcc/gcc -E
checking for grep that handles long lines and -e... /usr/bin/grep
checking for egrep... /usr/bin/grep -E
checking for ANSI C header files... yes
checking for sys/types.h... yes
checking for sys/stat.h... yes
checking for stdlib.h... yes
checking for string.h... yes
checking for memory.h... yes
checking for strings.h... yes
checking for inttypes.h... yes
checking for stdint.h... yes
checking for unistd.h... yes
checking minix/config.h usability... no
checking minix/config.h presence... no
checking for minix/config.h... no
checking whether it is safe to define __EXTENSIONS__... yes
checking for git... yes
checking for a BSD-compatible install... /usr/bin/install -c
checking whether build environment is sane... yes
checking for a thread-safe mkdir -p... /usr/bin/mkdir -p
checking for gawk... gawk
checking whether make sets $(MAKE)... yes
checking for style of include used by make... GNU
checking whether make supports nested variables... yes
checking whether UID '14592' is supported by ustar format... yes
checking whether GID '8000' is supported by ustar format... yes
checking how to create a ustar tar archive... gnutar
checking dependency style of /home/mpaipuri/spack/lib/spack/env/gcc/gcc... gcc3
checking whether make supports nested variables... (cached) yes
checking whether to enable maintainer-specific portions of Makefiles... no
checking for gcc... (cached) /home/mpaipuri/spack/lib/spack/env/gcc/gcc
checking whether we are using the GNU C compiler... (cached) yes
checking whether /home/mpaipuri/spack/lib/spack/env/gcc/gcc accepts -g... (cached) yes
checking for /home/mpaipuri/spack/lib/spack/env/gcc/gcc option to accept ISO C89... (cached) none needed
checking whether we are using the GNU C++ compiler... yes
checking whether /home/mpaipuri/spack/lib/spack/env/gcc/g++ accepts -g... yes
checking dependency style of /home/mpaipuri/spack/lib/spack/env/gcc/g++... gcc3
checking for /home/mpaipuri/spack/lib/spack/env/gcc/gcc option to support OpenMP... -fopenmp
checking dependency style of /home/mpaipuri/spack/lib/spack/env/gcc/gcc... gcc3
checking whether ln -s works... yes
checking for a sed that does not truncate output... /usr/bin/sed
checking build system type... x86_64-unknown-linux-gnu
checking host system type... x86_64-unknown-linux-gnu
checking how to print strings... printf
checking for a sed that does not truncate output... (cached) /usr/bin/sed
checking for fgrep... /usr/bin/grep -F
checking for ld used by /home/mpaipuri/spack/lib/spack/env/gcc/gcc... /home/mpaipuri/spack/lib/spack/env/ld
checking if the linker (/home/mpaipuri/spack/lib/spack/env/ld) is GNU ld... yes
checking for BSD- or MS-compatible name lister (nm)... /usr/bin/nm -B
checking the name lister (/usr/bin/nm -B) interface... BSD nm
checking the maximum length of command line arguments... 1572864
checking whether the shell understands some XSI constructs... yes
checking whether the shell understands "+="... yes
checking how to convert x86_64-unknown-linux-gnu file names to x86_64-unknown-linux-gnu format... func_convert_file_noop
checking how to convert x86_64-unknown-linux-gnu file names to toolchain format... func_convert_file_noop
checking for /home/mpaipuri/spack/lib/spack/env/ld option to reload object files... -r
checking for objdump... objdump
checking how to recognize dependent libraries... pass_all
checking for dlltool... no
checking how to associate runtime and link libraries... printf %s\n
checking for ar... ar
checking for archiver @FILE support... @
checking for strip... strip
checking for ranlib... ranlib
checking command to parse /usr/bin/nm -B output from /home/mpaipuri/spack/lib/spack/env/gcc/gcc object... ok
checking for sysroot... no
checking for mt... mt
checking if mt is a manifest tool... no
checking for dlfcn.h... yes
checking for objdir... .libs
checking if /home/mpaipuri/spack/lib/spack/env/gcc/gcc supports -fno-rtti -fno-exceptions... no
checking for /home/mpaipuri/spack/lib/spack/env/gcc/gcc option to produce PIC... -fPIC -DPIC
checking if /home/mpaipuri/spack/lib/spack/env/gcc/gcc PIC flag -fPIC -DPIC works... yes
checking if /home/mpaipuri/spack/lib/spack/env/gcc/gcc static flag -static works... yes
checking if /home/mpaipuri/spack/lib/spack/env/gcc/gcc supports -c -o file.o... yes
checking if /home/mpaipuri/spack/lib/spack/env/gcc/gcc supports -c -o file.o... (cached) yes
checking whether the /home/mpaipuri/spack/lib/spack/env/gcc/gcc linker (/home/mpaipuri/spack/lib/spack/env/ld -m elf_x86_64) supports shared libraries... yes
checking whether -lc should be explicitly linked in... no
checking dynamic linker characteristics... GNU/Linux ld.so
checking how to hardcode library paths into programs... immediate
checking whether stripping libraries is possible... yes
checking if libtool supports shared libraries... yes
checking whether to build shared libraries... yes
checking whether to build static libraries... yes
checking how to run the C++ preprocessor... /home/mpaipuri/spack/lib/spack/env/gcc/g++ -E
checking for ld used by /home/mpaipuri/spack/lib/spack/env/gcc/g++... /home/mpaipuri/spack/lib/spack/env/ld -m elf_x86_64
checking if the linker (/home/mpaipuri/spack/lib/spack/env/ld -m elf_x86_64) is GNU ld... yes
checking whether the /home/mpaipuri/spack/lib/spack/env/gcc/g++ linker (/home/mpaipuri/spack/lib/spack/env/ld -m elf_x86_64) supports shared libraries... yes
checking for /home/mpaipuri/spack/lib/spack/env/gcc/g++ option to produce PIC... -fPIC -DPIC
checking if /home/mpaipuri/spack/lib/spack/env/gcc/g++ PIC flag -fPIC -DPIC works... yes
checking if /home/mpaipuri/spack/lib/spack/env/gcc/g++ static flag -static works... yes
checking if /home/mpaipuri/spack/lib/spack/env/gcc/g++ supports -c -o file.o... yes
checking if /home/mpaipuri/spack/lib/spack/env/gcc/g++ supports -c -o file.o... (cached) yes
checking whether the /home/mpaipuri/spack/lib/spack/env/gcc/g++ linker (/home/mpaipuri/spack/lib/spack/env/ld -m elf_x86_64) supports shared libraries... yes
checking dynamic linker characteristics... (cached) GNU/Linux ld.so
checking how to hardcode library paths into programs... immediate
checking for ANSI C header files... (cached) yes
checking for cos in -lm... yes
checking for C/C++ restrict keyword... __restrict
checking whether strerror_r is declared... yes
checking for strerror_r... yes
checking whether strerror_r returns char *... yes
checking for pkg-config... /home/mpaipuri/spack/opt/spack/linux-debian10-broadwell/gcc-9.3.0/pkgconf-1.8.0-yndsolnup343y4rx3hncu5xseyt5ppg6/bin/pkg-config
checking if ln -s supports --relative... yes
checking for dot... yes
checking for doxygen... no
configure: WARNING: doxygen not found - will not generate any doxygen documentation
/tmp/mpaipuri/spack-stage/spack-stage-ucx-1.10.1-4ghx7jadjpm6sgy6b4ingjkrpjors2sa/spack-src/contrib/../configure: line 13896: doxygen: command not found
configure: WARNING: doxygen version  is bad. Required version: 1.8.6 and above
checking for perl... /usr/bin/perl
checking for size_t... yes
checking compiler flag -diag-error 10006... no
checking whether -diag-disable 1478 overrides deprecated declarations... no
checking whether -Wno-deprecated-declarations overrides deprecated declarations... yes
checking compiler flag -diag-disable 269... no
checking compiler flag -fmax-type-align=16... no
checking -mavx... yes
configure: Detected CPU implementation: 
configure: Detected CPU architecture: 
configure: Detected CPU variant: 
configure: Detected CPU part: 
checking for __attribute__(optimize)... 1
checking compiler flag -funwind-tables... yes
configure: compiling with unwind tables
checking if /home/mpaipuri/spack/lib/spack/env/gcc/g++ works... yes
checking c++11 support... yes
checking gnu++11 support... yes
checking whether _GLIBCXX_NOTHROW is declared... yes
checking compiler flag --display_error_number... no
checking compiler flag --diag_suppress 181... no
checking compiler flag --diag_suppress 381... no
checking compiler flag --diag_suppress 1215... no
checking compiler flag --diag_suppress 1901... no
checking compiler flag --diag_suppress 1902... no
checking compiler flag -pedantic... yes
checking compiler flag -Wno-missing-field-initializers... yes
checking compiler flag -Wno-unused-parameter... yes
checking compiler flag -Wno-unused-label... yes
checking compiler flag -Wno-long-long... yes
checking compiler flag -Wno-endif-labels... yes
checking compiler flag -Wno-sign-compare... yes
checking compiler flag -Wno-multichar... yes
checking compiler flag -Wno-deprecated-declarations... yes
checking compiler flag -Winvalid-pch... yes
checking compiler flag -Wno-pointer-sign... yes
checking compiler flag -Werror-implicit-function-declaration... yes
checking compiler flag -Wno-format-zero-length... yes
checking compiler flag -Wnested-externs... yes
checking compiler flag -Wshadow... yes
checking compiler flag -Werror=declaration-after-statement... yes
checking for working alloca.h... yes
checking for alloca... yes
checking for shm_open in -lrt... yes
checking for timer_create in -lrt... yes
checking libgen.h usability... yes
checking libgen.h presence... yes
checking for libgen.h... yes
checking whether asprintf is declared... yes
checking whether basename is declared... yes
checking whether fmemopen is declared... yes
checking sys/cpuset.h usability... no
checking sys/cpuset.h presence... no
checking for sys/cpuset.h... no
checking whether CPU_ZERO is declared... yes
checking whether CPU_ISSET is declared... yes
checking for cpu_set_t... yes
checking for cpuset_t... no
checking for sighandler_t... yes
checking for __sighandler_t... yes
checking pthread_np.h usability... no
checking pthread_np.h presence... no
checking for pthread_np.h... no
checking for library containing pthread_create... -lpthread
checking for library containing pthread_atfork... none required
checking for clearenv... yes
checking for malloc_trim... yes
checking for memalign... yes
checking for posix_memalign... yes
checking for mremap... yes
checking for sched_setaffinity... yes
checking for sched_getaffinity... yes
checking for cpuset_setaffinity... no
checking for cpuset_getaffinity... no
checking whether F_SETOWN_EX is declared... yes
checking whether ethtool_cmd_speed is declared... yes
checking whether SPEED_UNKNOWN is declared... yes
checking whether __ppc_get_timebase_freq is declared... no
checking sys/platform/ppc.h usability... no
checking sys/platform/ppc.h presence... no
checking for sys/platform/ppc.h... no
checking for using Google C++ Testing Framework... no
checking numa.h usability... yes
checking numa.h presence... yes
checking for numa.h... yes
checking numaif.h usability... yes
checking numaif.h presence... yes
checking for numaif.h... yes
checking for mbind in -lnuma... yes
checking for struct bitmask... yes
checking malloc hooks... yes
checking sys/capability.h usability... no
checking sys/capability.h presence... no
checking for sys/capability.h... no
checking whether PR_SET_PTRACER is declared... yes
checking for struct in6_addr.s6_addr32... yes
checking for struct in6_addr.__u6_addr.__u6_addr32... no
checking for struct iphdr.daddr.s_addr... no
checking for struct ip.ip_dst.s_addr... yes
checking for struct sigevent._sigev_un._tid... yes
checking for struct sigevent.sigev_notify_thread_id... no
checking for struct sigaction.sa_restorer... yes
checking sys/epoll.h usability... yes
checking sys/epoll.h presence... yes
checking for sys/epoll.h... yes
checking sys/eventfd.h usability... yes
checking sys/eventfd.h presence... yes
checking for sys/eventfd.h... yes
checking sys/event.h usability... no
checking sys/event.h presence... no
checking for sys/event.h... no
checking sys/thr.h usability... no
checking sys/thr.h presence... no
checking for sys/thr.h... no
checking malloc.h usability... yes
checking malloc.h presence... yes
checking for malloc.h... yes
checking malloc_np.h usability... no
checking malloc_np.h presence... no
checking for malloc_np.h... no
checking endian.h, usability... no
checking endian.h, presence... no
checking for endian.h,... no
checking sys/endian.h usability... no
checking sys/endian.h presence... no
checking for sys/endian.h... no
checking linux/mman.h usability... yes
checking linux/mman.h presence... yes
checking for linux/mman.h... yes
checking linux/ip.h usability... yes
checking linux/ip.h presence... yes
checking for linux/ip.h... yes
checking linux/futex.h usability... yes
checking linux/futex.h presence... yes
checking for linux/futex.h... yes
checking for net/ethernet.h... yes
checking for netinet/ip.h... yes
configure: Memory allocator is ptmalloc-2.8.6 version
checking for malloc_get_state... no
checking for malloc_set_state... no
checking whether MADV_FREE is declared... yes
checking whether MADV_REMOVE is declared... yes
checking whether POSIX_MADV_DONTNEED is declared... yes
checking whether SYS_mmap is declared... yes
checking whether SYS_munmap is declared... yes
checking whether SYS_mremap is declared... yes
checking whether SYS_brk is declared... yes
checking whether SYS_madvise is declared... yes
checking whether SYS_shmat is declared... yes
checking whether SYS_shmdt is declared... yes
checking whether SYS_ipc is declared... no
checking for __curbrk... yes
checking for tc_malloc in -ltcmalloc... no
configure: WARNING: Java support was explicitly disabled.
checking cuda.h usability... yes
checking cuda.h presence... yes
checking for cuda.h... yes
checking cuda_runtime.h usability... yes
checking cuda_runtime.h presence... yes
checking for cuda_runtime.h... yes
checking for cuDeviceGetUuid in -lcuda... yes
checking for cudaGetDeviceCount in -lcudart... yes
configure: ROCm path was not specified. Guessing ...
checking hsa.h usability... yes
checking hsa.h presence... yes
checking for hsa.h... yes
checking hsa_ext_amd.h usability... yes
checking hsa_ext_amd.h presence... yes
checking for hsa_ext_amd.h... yes
checking for hsa_init in -lhsa-runtime64... yes
checking for hipFree in -lhip_hcc... no
checking hip_runtime.h usability... yes
checking hip_runtime.h presence... yes
checking for hip_runtime.h... yes
checking for hipFree in -lamdhip64... yes
checking bfd.h usability... no
checking bfd.h presence... no
checking for bfd.h... no
configure: WARNING: binutils headers not found
checking for bfd_openr in -lbfd... no
configure: WARNING: bfd library not found
checking for dlopen in -ldl... yes
checking for main in -lintl... no
configure: WARNING: intl library not found
checking for struct dl_phdr_info... yes
checking whether bfd_get_section_flags is declared... no
checking whether bfd_section_flags is declared... no
checking whether bfd_get_section_vma is declared... no
checking whether bfd_section_vma is declared... no
checking bfd_section_size API version... 2-args API
configure: WARNING: detailed backtrace is not supported
checking __attribute__((constructor))... yes
configure: enabling builtin memcpy
checking for __clear_cache... yes
checking for __aarch64_sync_cache_range... no
checking gdrapi.h usability... yes
checking gdrapi.h presence... yes
checking for gdrapi.h... yes
checking for gdr_pin_buffer in -lgdrapi ... yes
checking whether gdr_copy_to_mapping is declared... yes
configure: Compiling with verbs support from /usr
checking infiniband/verbs.h usability... yes
checking infiniband/verbs.h presence... yes
checking for infiniband/verbs.h... yes
checking for ibv_get_device_list in -libverbs... yes
checking whether ibv_wc_status_str is declared... yes
checking whether ibv_event_type_str is declared... yes
checking whether ibv_query_gid is declared... yes
checking whether ibv_get_device_name is declared... yes
checking whether ibv_create_srq is declared... yes
checking whether ibv_get_async_event is declared... yes
checking infiniband/verbs_exp.h usability... no
checking infiniband/verbs_exp.h presence... no
checking for infiniband/verbs_exp.h... no
checking for struct ibv_exp_device_attr.exp_device_cap_flags... no
checking for struct ibv_exp_device_attr.odp_caps... no
checking for struct ibv_exp_device_attr.odp_caps.per_transport_caps.dc_odp_caps... no
checking for struct ibv_exp_device_attr.odp_mr_max_size... no
checking for struct ibv_exp_qp_init_attr.max_inl_recv... no
checking for struct ibv_async_event.element.dct... no
checking whether IBV_CREATE_CQ_ATTR_IGNORE_OVERRUN is declared... yes
checking whether IBV_EXP_CQ_IGNORE_OVERRUN is declared... no
configure: Checking for legacy bare-metal support
checking infiniband/mlx5_hw.h usability... no
checking infiniband/mlx5_hw.h presence... no
checking for infiniband/mlx5_hw.h... no
configure: Checking for DV bare-metal support
checking for mlx5dv_query_device in -lmlx5-rdmav2... no
checking for mlx5dv_query_device in -lmlx5... yes
checking for infiniband/mlx5dv.h... yes
checking whether mlx5dv_init_obj is declared... yes
checking whether mlx5dv_create_qp is declared... yes
checking whether mlx5dv_is_supported is declared... yes
checking whether mlx5dv_devx_subscribe_devx_event is declared... yes
checking whether MLX5DV_CQ_INIT_ATTR_MASK_CQE_SIZE is declared... yes
checking whether MLX5DV_QP_CREATE_ALLOW_SCATTER_TO_CQE is declared... yes
checking whether MLX5DV_UAR_ALLOC_TYPE_BF is declared... yes
checking whether MLX5DV_UAR_ALLOC_TYPE_NC is declared... yes
checking for struct mlx5dv_cq.cq_uar... yes
checking whether MLX5DV_OBJ_AH is declared... yes
checking whether MLX5DV_DCTYPE_DCT is declared... yes
checking whether ibv_alloc_td is declared... yes
checking whether ibv_alloc_td is declared... (cached) yes
checking whether MLX5DV_CONTEXT_FLAGS_DEVX is declared... yes
configure: Compiling with mlx5 bare-metal support
checking whether IBV_LINK_LAYER_INFINIBAND is declared... yes
checking whether IBV_LINK_LAYER_ETHERNET is declared... yes
checking whether IBV_EVENT_GID_CHANGE is declared... yes
checking whether ibv_create_qp_ex is declared... yes
checking whether ibv_create_srq_ex is declared... yes
checking whether ibv_query_device_ex is declared... yes
checking whether IBV_EXP_ACCESS_ALLOCATE_MR is declared... no
checking whether IBV_EXP_ACCESS_ON_DEMAND is declared... no
checking whether IBV_EXP_DEVICE_MR_ALLOCATE is declared... no
checking whether IBV_EXP_WR_NOP is declared... no
checking whether IBV_EXP_DEVICE_DC_TRANSPORT is declared... no
checking whether IBV_EXP_ATOMIC_HCA_REPLY_BE is declared... no
checking whether IBV_EXP_PREFETCH_WRITE_ACCESS is declared... no
checking whether IBV_EXP_QP_OOO_RW_DATA_PLACEMENT is declared... no
checking whether IBV_EXP_DCT_OOO_RW_DATA_PLACEMENT is declared... no
checking whether IBV_EXP_CQ_MODERATION is declared... no
checking whether IBV_EXP_DEVICE_ATTR_PCI_ATOMIC_CAPS is declared... no
checking whether ibv_exp_reg_mr is declared... no
checking whether ibv_exp_create_qp is declared... no
checking whether ibv_exp_prefetch_mr is declared... no
checking whether ibv_exp_create_srq is declared... no
checking whether ibv_exp_setenv is declared... no
checking whether ibv_exp_query_gid_attr is declared... no
checking whether ibv_exp_query_device is declared... no
checking whether ibv_exp_post_send is declared... no
checking whether IBV_EXP_WR_EXT_MASKED_ATOMIC_CMP_AND_SWP is declared... no
checking whether IBV_EXP_WR_EXT_MASKED_ATOMIC_FETCH_AND_ADD is declared... no
checking whether IBV_EXP_QP_INIT_ATTR_ATOMICS_ARG is declared... no
checking whether IBV_EXP_SEND_EXT_ATOMIC_INLINE is declared... no
checking whether IBV_EXP_DEVICE_ATTR_RESERVED_2 is declared... no
checking whether IBV_EXP_MR_INDIRECT_KLMS is declared... no
checking whether IBV_EXP_QP_CREATE_UMR is declared... no
checking for struct ibv_exp_qp_init_attr.umr_caps... no
checking whether IBV_EXP_MR_FIXED_BUFFER_SIZE is declared... no
checking for struct ibv_device_attr_ex.pci_atomic_caps... yes
configure: WARNING: Compiling without extended atomics support
checking for struct ibv_exp_masked_atomic_params.masked_log_atomic_arg_sizes_network_endianness... no
checking whether IBV_EXP_ODP_SUPPORT_IMPLICIT is declared... no
checking whether IBV_EXP_ACCESS_ON_DEMAND is declared... (cached) no
checking whether IBV_ACCESS_ON_DEMAND is declared... yes
checking whether IBV_EXP_ODP_SUPPORT_IMPLICIT is declared... (cached) no
checking whether IBV_ODP_SUPPORT_IMPLICIT is declared... yes
checking whether IBV_ACCESS_RELAXED_ORDERING is declared... yes
checking whether IBV_QPF_GRH_REQUIRED is declared... yes
checking whether ibv_exp_prefetch_mr is declared... (cached) no
checking whether ibv_advise_mr is declared... yes
checking for struct mlx5_wqe_av.base... no
checking for struct mlx5_grh_av.rmac... no
checking for struct mlx5_cqe64.ib_stride_index... no
checking whether IBV_EXP_QPT_DC_INI is declared... no
checking infiniband/tm_types.h usability... yes
checking infiniband/tm_types.h presence... yes
checking for infiniband/tm_types.h... yes
checking for struct ibv_exp_tmh.tag... no
checking for struct ibv_tmh.tag... yes
checking for struct ibv_tm_caps.flags... yes
checking whether ibv_cmd_modify_qp is declared... no
configure: Checking OFED valgrind libs /usr/lib64/mlnx_ofed/valgrind
checking /usr/include/rdma/rdma_cma.h usability... no
checking /usr/include/rdma/rdma_cma.h presence... no
checking for /usr/include/rdma/rdma_cma.h... no
configure: WARNING: RDMACM requested but required file (rdma/rdma_cma.h) could not be found in /usr
checking sys/uio.h usability... yes
checking sys/uio.h presence... yes
checking for sys/uio.h... yes
checking for process_vm_readv... yes
configure: WARNING: KNEM was explicitly disabled
checking cray-ugni... no
checking whether IPPROTO_TCP is declared... yes
checking whether SOL_SOCKET is declared... yes
checking whether SO_KEEPALIVE is declared... yes
checking whether TCP_KEEPCNT is declared... yes
checking whether TCP_KEEPIDLE is declared... yes
checking whether TCP_KEEPINTVL is declared... yes
checking compiler flag -fno-exceptions... yes
checking compiler flag -fno-rtti... yes
checking compiler flag --no_exceptions... no
checking compiler flag -fno-tree-vectorize... yes
checking compiler flag --diag_suppress 236... no
checking that generated files are newer than configure... done
configure: creating ./config.status
config.status: creating src/ucm/cuda/Makefile
config.status: creating src/ucm/rocm/Makefile
config.status: creating src/ucm/Makefile
config.status: creating src/uct/cuda/gdr_copy/Makefile
config.status: creating src/uct/cuda/Makefile
config.status: creating src/uct/ib/cm/Makefile
config.status: creating src/uct/ib/rdmacm/Makefile
config.status: creating src/uct/ib/Makefile
config.status: creating src/uct/rocm/gdr/Makefile
config.status: creating src/uct/rocm/Makefile
config.status: creating src/uct/sm/scopy/cma/Makefile
config.status: creating src/uct/sm/scopy/knem/Makefile
config.status: creating src/uct/sm/scopy/Makefile
config.status: creating src/uct/sm/mm/xpmem/Makefile
config.status: creating src/uct/sm/mm/Makefile
config.status: creating src/uct/sm/Makefile
config.status: creating src/uct/ugni/Makefile
config.status: creating src/uct/Makefile
config.status: creating src/tools/perf/lib/Makefile
config.status: creating src/tools/perf/cuda/Makefile
config.status: creating src/tools/perf/rocm/Makefile
config.status: creating src/tools/perf/Makefile
config.status: creating test/gtest/ucm/test_dlopen/Makefile
config.status: creating test/gtest/ucm/test_dlopen/rpath-subdir/Makefile
config.status: creating test/gtest/ucs/test_module/Makefile
config.status: creating test/gtest/Makefile
config.status: creating Makefile
config.status: creating docs/doxygen/header.tex
config.status: creating src/uct/api/version.h
config.status: creating ucx.spec
config.status: creating ucx.pc
config.status: creating contrib/rpmdef.sh
config.status: creating debian/rules
config.status: creating debian/control
config.status: creating debian/changelog
config.status: creating src/ucs/Makefile
config.status: creating src/ucp/Makefile
config.status: creating src/ucp/api/ucp_version.h
config.status: creating src/ucp/core/ucp_version.c
config.status: creating src/tools/info/Makefile
config.status: creating src/tools/profile/Makefile
config.status: creating test/apps/Makefile
config.status: creating test/apps/iodemo/Makefile
config.status: creating test/apps/sockaddr/Makefile
config.status: creating test/apps/profiling/Makefile
config.status: creating test/mpi/Makefile
config.status: creating bindings/java/Makefile
config.status: creating bindings/java/pom.xml
config.status: creating bindings/java/src/main/native/Makefile
config.status: creating examples/Makefile
config.status: creating test/mpi/run_mpi.sh
config.status: creating config.h
config.status: executing depfiles commands
config.status: executing libtool commands
configure: =========================================================
configure: UCX build configuration:
configure:       Build prefix:   /home/mpaipuri/spack/opt/spack/linux-debian10-broadwell/gcc-9.3.0/ucx-1.10.1-4ghx7jadjpm6sgy6b4ingjkrpjors2sa
configure: Preprocessor flags:   -DCPU_FLAGS="|avx" -I${abs_top_srcdir}/src -I${abs_top_builddir} -I${abs_top_builddir}/src
configure:         C compiler:   /home/mpaipuri/spack/lib/spack/env/gcc/gcc -O3 -g -Wall -Werror -mavx -funwind-tables -Wno-missing-field-initializers -Wno-unused-parameter -Wno-unused-label -Wno-long-long -Wno-endif-labels -Wno-sign-compare -Wno-multichar -Wno-deprecated-declarations -Winvalid-pch -Wno-pointer-sign -Werror-implicit-function-declaration -Wno-format-zero-length -Wnested-externs -Wshadow -Werror=declaration-after-statement
configure:       C++ compiler:   /home/mpaipuri/spack/lib/spack/env/gcc/g++ -O3 -g -Wall -Werror -mavx -funwind-tables -Wno-missing-field-initializers -Wno-unused-parameter -Wno-unused-label -Wno-long-long -Wno-endif-labels -Wno-sign-compare -Wno-multichar -Wno-deprecated-declarations -Winvalid-pch
configure:       Multi-thread:   enabled
configure:          MPI tests:   disabled
configure:      Devel headers:   no
configure:           Bindings:   < >
configure:        UCT modules:   < cuda ib rocm cma >
configure:       CUDA modules:   < gdrcopy >
configure:       ROCM modules:   < gdr >
configure:         IB modules:   < >
configure:        UCM modules:   < cuda rocm >
configure:       Perf modules:   < cuda rocm >
configure: =========================================================

```

Maintainer: @hppritcha 

PS: As I could successfully build it by modifying the UCX package file locally, I do not have logs anymore.

### General information

- [X] I have run `spack debug report` and reported the version of Spack/Python/Platform
- [X] I have run `spack maintainers <name-of-the-package>` and **@mentioned** any maintainers
- [X] I have uploaded the build log and environment files
- [X] I have searched the issues of this repo and believe this is not a duplicate


Discussion:
@mahendrapaipuri would you mind opening a pull request with your modifications?  
@hppritcha Sure, will do shortly!!

Resolving patch:
diff --git a/var/spack/repos/builtin/packages/ucx/package.py b/var/spack/repos/builtin/packages/ucx/package.py
--- a/var/spack/repos/builtin/packages/ucx/package.py
+++ b/var/spack/repos/builtin/packages/ucx/package.py
@@ -65,6 +65,8 @@ class Ucx(AutotoolsPackage, CudaPackage):
             description='Enable XPMEM support')
     variant('cma', default=False,
             description="Enable Cross Memory Attach")
+    variant('rocm', default=False,
+            description="Enable ROCm support")
     variant('rc', default=False,
             description="Compile with IB Reliable Connection support")
     variant('dc', default=False,
@@ -89,6 +91,8 @@ class Ucx(AutotoolsPackage, CudaPackage):
     depends_on('gdrcopy@1.3', when='@:1.6+gdrcopy')
     conflicts('+gdrcopy', when='~cuda',
               msg='gdrcopy currently requires cuda support')
+    conflicts('+rocm', when='+gdrcopy',
+              msg='gdrcopy > 2.0 does not support rocm')
     depends_on('xpmem', when='+xpmem')
     depends_on('knem', when='+knem')
     depends_on('binutils+ld', when='%aocc', type='build')
@@ -136,6 +140,7 @@ def configure_args(self):
         config_args.extend(self.with_or_without('ib-hw-tm'))
         config_args.extend(self.with_or_without('dm'))
         config_args.extend(self.with_or_without('cm'))
+        config_args.extend(self.with_or_without('rocm'))
         config_args.extend(self.with_or_without('java',
                                                 activation_value='prefix'))
         config_args.extend(self.with_or_without('cuda',


Test patch: