ROCm

HSAIL[7] was aimed at producing a middle-level, hardware-agnostic intermediate representation, that could be JIT-compiled to the eventual hardware (GPU, FPGA...) using the appropriate finalizer. This approach was dropped for ROCm: now it builds only GPU code, using LLVM, and its AMDGPU backend that was upstreamed,[8] although there is still research on such enhanced modularity with LLVM MLIR.[9]

AMD Instinct accelerators are the first-class ROCm citizens, alongside the prosumer Radeon Pro GPU series: they mostly see full support.

The only consumer-grade GPU that has relatively equal support is, as of January 2022, the Radeon VII (GCN 5 - Vega).

AMD ROCm product manager Terry Deem gave a tour of the stack.[21]

The main consumers of the stack are machine learning and high-performance computing/GPGPU applications.

Various Deep Learning frameworks have a ROCm backend:[22]

• PyTorch
• TensorFlow
• ONNX
• MXNet
• CuPy[23]
• MIOpen
• Caffe
• Iree (which uses LLVM Multi-Level Intermediate Representation (MLIR))

ROCm is gaining significant traction in the top 500.[24] ROCm is used with the Exascale supercomputers ElCapitan[25][26] and Frontier.

Some related software is to be found at AMD Infinity hub.

As of version 3.0, Blender can now use HIP compute kernels for its renderer Cycles.[27]

Julia has the AMDGPU.jl package,[28] which integrates with LLVM and selects components of the ROCm stack. Instead of compiling code through HIP, AMDGPU.jl uses Julia's compiler to generate LLVM IR directly, which is later consumed by LLVM to generate native device code. AMDGPU.jl uses ROCr's HSA implementation to upload native code onto the device and execute it, similar to how HIP loads its own generated device code.

AMDGPU.jl also supports integration with ROCm's rocBLAS (for BLAS), rocRAND (for random number generation), and rocFFT (for FFTs). Future integration with rocALUTION, rocSOLVER, MIOpen, and certain other ROCm libraries is planned.

ROCm software is currently spread across dozens of public GitHub repositories. Within the main public meta-repository, there is an xml manifest for each official release: using git-repo, a version control tool built on top of git, is the recommended way to synchronize with the stack locally.[29]

AMD starts distributing containerized applications for ROCm, notably scientific research applications gathered under AMD Infinity Hub.[30]

AMD distributes itself packages tailored to various Linux distributions.

There is a growing third-party ecosystem packaging ROCm.

Linux distributions are packaging officially (natively) ROCm, with various degrees of advancement: Arch,[31] Gentoo,[32] Debian and Fedora,[33] GNU Guix, NixOS.

There are spack packages.[34]

Support libraries implemented as LLVM bitcode. These provide various utilities and functions for math operations, atomics, queries for launch parameters, on-device kernel launch, etc.

The thunk is responsible for all the thinking and queuing that goes into the stack.

The ROC runtime is a set of APIs/libraries that allows the launch of compute kernals by host applications. It is AMD's implementation of the HSA runtime API.[39] It is different from the ROC Common Language Runtime.

ROCm code object manager is in charge of interacting with LLVM intermediate representation.

The common language runtime is an indirection layer adapting calls to ROCr on linux and PAL on windows. It used to be able to route between different compilers like the HSAIL-compiler. It is now being absorbed by the upper indirection layers (HIP, OpenCL).

ROCm ships its Installable Client Driver ICD loader and an OpenCL[40] implementation bundled together. As of January 2022, ROCm 4.5.2 ships OpenCL 2.2, and is lagging behind competition.[41]

The AMD implementation for its GPUs is called HIPAMD. There is also a CPU implementation mostly for demonstration purposes.

HIP builds a `HIPCC` compiler that either wraps Clang and compiles with LLVM open AMDGPU backend, or redirects to the NVIDIA compiler.[42]

HIPIFY is a source-to-source compiling tool, it translates CUDA to HIP and reverse, either using a clang-based tool, or a sed-like Perl script.

Like HIPIFY, GPUFORT is a tool compiling source code into other third-generation-language sources, allowing users to migrate from CUDA Fortran to HIP Fortran. It is also in the repertoire of research projects, even more so.[43]

ROCm high-level libraries are usually consumed directly by application software, such as machine learning frameworks. Most of the following libraries are in the General Matrix Multiply (GEMM) category, which GPU architecture excels at.

The majority of these user-facing libraries comes in dual-form: hip for the indirection layer that can route to Nvidia hardware, and roc for AMD implementation.[44]

rocBLAS and hipBLAS are central in high-level libraries, it is the AMD implementation for Basic Linear Algebra Subprograms. It uses the library Tensile privately.

This pair of libraries constitutes the LAPACK implementation for ROCm and is strongly coupled to rocBLAS.

• ROCm developer tools: Debug, tracer, profiler, System Management Interface, Validation suite, Cluster management.
• GPUOpen tools: GPU analyzer, memory visualizer...
• External tools: radeontop (TUI overview)

Nvidia is close-source until cuBLAS and such high-level libraries.
Nvidia vendors the Clang frontend and its Parallel Thread Execution (PTX) LLVM GPU backend as the Nvidia CUDA Compiler (NVCC).
There is an open-source layer above it, for example RAPIDS.

oneAPI is open source, and all the corresponding libraries are published on its GitHub Page.