CUDA Compatibility

The Friendli Engine supports CUDA-enabled NVIDIA GPUs, which means it relies on a specific version of CUDA and necessitates proper CUDA compute compatibilities. To utilize the Friendli Container effectively, ensure that you have the appropriate NVIDIA GPUs and an NVIDIA driver in place.

Currently, we publicly offer a single Friendli Container image (registry.friendli.ai/trial:latest) equipped with CUDA 12.4, targeting CUDA compute compatibility versions 8.0, 8.6, 8.9, and 9.0. To make the right choices regarding GPUs and driver versions, consult the required driver versions and GPUs for the CUDA toolkit and compute compatibility.