Compiling Julia for NVIDIA GPUs15 Jan 2015
For the few last months, I have been working on CUDA support for the Julia language. It is now possible to write kernels in Julia and without much hassle execute them on a NVIDIA GPU, but there are still many limitations. As I unexpectedly won’t have much time to work on this anymore, I’m publishing and documenting my work already.
The current state allows for code such as:
which is pretty neat I think :-)
I’ll start by giving a quick description of the modifications. Jump to the bottom of this post for usage instructions.
Compiling Julia for GPUs requires support at multiple levels. I’ve tried to avoid touching too much of core compiler; as a consequence most functionality is part of the CUDA.jl package. This should make it easier to maintain and eventually merge the code.
All of the relevant repositories are hosted at my
Github page, and contain
files. If you have any questions though, feel free to contact me.
Using the NVPTX back-end of LLVM, I have modified the Julia compiler so that it can generate PTX assembly. A non-exhaustive list of modifications:
@targetmacro for annotating functions with target information
- per-target compiler state (module, pass manager, etc)
- diverse small changes to generate suitable IR
- exported functions for accessing the PTX code
Most of the code churn comes from using an address-preserving bitcast, which is already being upstreamed thanks to Valentin Churavy.
CUDA.jl support package
Generating PTX assembly is only one part of the puzzle: hardware needs to be configured, code needs to be uploaded, etc. This functionality is exposed through the CUDA runtime driver, which already was conveniently wrapped in the CUDA.jl package.
I have extended this package with functionality required for GPU code generation, and developed user-friendly wrappers which should make it easier to interact with PTX code:
@cudamacro for invoking GPU kernels
- automatic argument management
- lightweight on-device arrays
- improved API consistency
- many new features
The significant part is obviously the
@cuda macro, allowing for seamless
execution of kernel functions on your GPU. The macro compiles the kernel you’re
calling to PTX assembly, and generates code for interacting with the driver
(creating a module, uploading code, managing arguments, etc).
The argument management is also pretty interesting. In function of the argument
type, it generates type conversions and/or memory operations in order to
mimic Julia’s pass-by-sharing convention. For example, if you pass an array to a
@cuda will automatically up- and download it when required1.
Most functionality of
@cuda is built using staged functions, and thus only
executes once without a recurring runtime cost. This means that it should be
possible to reach the same average performance of a traditional, precompiled
GPU Ocelot emulator
I have also forked the GPU Ocelot
project, which is a research project providing a dynamic compilation framework
(read: emulator) for CUDA hardware. By extending API support calls and fixing
certain bugs, you can use this as a drop-in replacement for
compatible with CUDA.jl.
In practice, I used this emulator for everyday development on a system without an NVIDIA GPU, while testing happened on real hardware.
The code is far from production ready: it is not cross-platform (Linux only), several changes should be discussed with upstream, and only a restricted subset of the language is supported. Most notable shortcomings:
allocin PTX mode (which breaks most of the language)
- can only pass
bitstypes, arrays or pointers
- standard library functionality is unavailable, because it lives in another LLVM module
In short: unless you’re only using relatively simple kernels with non-complex data interactions, this code is not yet usable for you.
Even though all code is pretty functional and well-maintained, you need some basic development skills to put the pieces together. Don’t expect a polished product!
Compile the modified compiler from source, using LLVM 3.5:
$ git clone https://github.com/maleadt/julia.git $ cd julia $ make LLVM_VER=3.5.0
Optionally, make sure Julia is not broken (this does not include GPU tests):
$ make LLVM_VER=3.5.0 testall
Note: the compiler will require
libdevice to link kernel binaries. This
library is only part of recent CUDA toolkits (version 5.5 or greater). If you
use an older CUDA release, you will need to get a hold of these
Afterwards, you can point Julia to them using the
GPU Ocelot emulator
If you don’t have any suitable CUDA hardware, you can use GPU Ocelot:
$ git clone --recursive https://github.com/maleadt/gpuocelot.git $ cd gpuocelot $ $PAGER README.md $ CUDA_BIN_PATH=/opt/cuda-5.0/bin \ CUDA_LIB_PATH=/opt/cuda-5.0/lib \ CUDA_INC_PATH=/opt/cuda-5.0/include \ python2 build.py --install -p $(realpath ../julia/usr)
Note: this probably will not build flawlessly. You’ll need at least the CUDA
toolkit2 (headers and tools, not the driver), gcc 4.6, scons, LLVM 3.5 and
Boost. Check the
Now if you load CUDA.jl and it doesn’t find
libcuda.so, it will look for
$ ./julia > using CUDA > CUDA_VENDOR "Ocelot"
CUDA.jl support package
Installing packages is easy3 (just make sure you use the correct
$ ./julia > Pkg.clone("https://github.com/maleadt/CUDA.jl.git")
Optionally, but recommended, test GPU support:
$ ./julia > Pkg.test("CUDA")
You tell me! I think this work can be a good start for future GPU endeavours in Julia land, even without most code being directly re-usable. For me at least it has been a very interesting project, but it’s in the hands of the community now.
You can influence this behaviour using the
CuInOutwrapper types. ↩
GPU Ocelot is only compatible with CUDA 5.0 or older. This means you’ll need to get
If you don’t want to pollute your main package directory with this experimental stuff, redefine the
JULIA_PKGDIRenvironment variable. ↩