OLD VERSION A quick look at Compute Languages

This is the old version. Please see the new version. This is just a quick look at the platforms available and the PC hardware they support put into one place (mostly for my own reference). I use Tensorflow as an example of how this situation plays out for a piece of software that utilises compute. There are a variety of different compute platforms available, targeting varying platforms with differing degrees of development, documentation and adoption.

The two well-known and longest developed platforms are CUDA and OpenCL. CUDA is developed by Nvidia exclusively for their devices, while OpenCL is developed by the Khronos group and is an open, royalty-free standard which can be implemented by any hardware manufacturer.

CUDA

Since its introduction in 2007 CUDA has been continuously developed by Nvidia. There are two different version numbers to pay attention to. Compute Capability refers to the instruction level which can be used on a particular device. A new generation of devices comes a newer version supporting extra operations which don’t exist on earlier devices. The CUDA development toolkit itself has a different version number system. Earlier versions of the toolkit supported all devices regardless of the Compute Capability version. However, since toolkit version 7 support for the oldest devices has been gradually removed when those devices became legacy, this is when they stop receiving updated drivers.

OpenCL and SYCL

Released 2 years after CUDA, OpenCL aims to be an open compute standard for any device which can implement it and is not vendor specific. Versions are simple in OpenCL as development is not as fast paced as in CUDA. Version 1.2 was finalised in 2011 and the latest Version 2.2 is still in development and not supported by many vendors. Due to the way that compute kernels are implemented, unlike CUDA, OpenCL doesn’t need its own compiler. Only C header and library files are needed to start development not a whole toolkit.

SYCL¹ is an abstraction layer for OpenCL intended to be more efficient than OpenCL which is lower level. However, it doesn’t appear to have seen as much adoption as OpenCL itself.

Vulkan

The Vulkan API, also developed by the Khronos group, is an open standard which is used as a replacement for Direct X/OpenGL in 3D game engines. But it can also be used for compute tasks⁹, the disadvantage is that it’s a very low-level API. An advantage would be that, due to being used in game engines, it is equally well supposed by Nvidia and AMD.

A complex situation with AMD

Historically AMD promoted OpenCL for compute applications on its GPUs. More recently this has evolved. AMD has released an entirely new open source driver and software stack called ROCm. This includes two high-level languages, HC and HIP. HC (Heterogeneous Compute)¹⁰ is a C++ API similar to C++ AMP, it targets only AMD devices. HIP (Heterogeneous Computing Interface for Portability)¹¹ is a API which is intentionally very similar to CUDA. It is intended to simplify the process of converting CUDA programs to run on AMD devices. AMD supply a tool to “hipify” existing CUDA code, converting it to HIP which can then be compiled to and run on AMD devices. AMD’s compiler also allows Nvidia devices to be targeted by HIP code. This is possible because internally HIP code can operate as an interfacing layer. When targeting AMD devices, the compiler (called hcc) compiles HIP directly for AMD devices. When targeting Nvidia devices, hcc converts the HIP code back to CUDA and then passes it to Nvidia’s compiler (nvcc) which then complies it as usual. The diagram below is a very simplified representation of the interactions between CUDA, HIP, HCC and NVCC.

Tensorflow as an example

Tensorflow⁴ is a well-known platform for machine learning and deep learning. It works as a library for python and aims to be simpler to use, by abstracting low level implementations away. It supports CPUs and GPUs as its back end. The downloadable binaries (installed through python pip) support standard CPUs and Nvidia GPUs which have at least Compute Capability 3.5, older versions supported CC 3. Nvidia support requires the CUDA toolkit and the cuDNN library to be installed.

There is no out of the box support for AMD graphics cards. AMD have created a separate port⁵ of Tensorflow which supports some of their latest GPUs. This port requires the ROCm runtime (explained above) to be installed, and a supported GPU to function. Benchmarks show performance to be similar to Nvidia cards.⁶ The main disadvantage is that fewer cards are supported and the version of Tensorflow is a version behind the latest release.

There is also an implementation of Tensorflow which uses OpenCL and SYCL.⁷ It requires OpenCL 1.2 and an extension called cl_khr_spir. This means that in theory it’s supported by most AMD cards. However, from what I can tell the performance is inferior⁸ to AMDs port which has an optimised implementation.

References