Install with pip
Binary Python wheels are published on
and can be directly installed with pip:
pipinstallctranslate2 The Python wheels have the following requirements:
OS: Linux (x86-64, AArch64), macOS (x86-64, ARM64), Windows (x86-64)
Python version: >= 3.9
pip version: >= 19.3 to support manylinux2014 wheels
GPU support
The Linux and Windows Python wheels support GPU execution. Install
12.x to use the GPU.
If you plan to run models with convolutional layers (e.g. for speech recognition), you should also install
for CUDA 12.x.
Note
On Windows
is required. It is installed in most of the systems, but if is not the case, you have to download it and install it.
Install with Docker
Docker images can be downloaded from the
:
dockerpullghcr.io/opennmt/ctranslate2:latest-ubuntu22.04-cuda12.8 The images include:
the NVIDIA libraries cuBLAS and cuDNN to support GPU execution
the C++ library installed in /opt/ctranslate2
the Python module installed in the Python system packages
the translator executable, which is the image entrypoint:
dockerrun--rmghcr.io/opennmt/ctranslate2:latest-ubuntu22.04-cuda11.2--help To update to the new version that supports CUDA 12.
Install from sources
Download the source code
Clone the CTranslate2 Git repository and its submodules.
gitclone--recursivehttps://github.com/OpenNMT/CTranslate2.git Compile the C++ library
Compiling the library requires a compiler supporting C++17 and CMake 3.15 or greater.
mkdirbuild&&cdbuild cmake.. make-j4 sudomakeinstall sudoldconfig By default, the library is compiled with the
backend which should be installed separately. See the
to select or add another backend.
Compile the Python wrapper
Once the C++ library is installed, you can compile the Python wrapper which uses
. This step requires the Python development libraries to be installed on the system.
cdpython pipinstall-rinstall_requirements.txt pythonsetup.pybdist_wheel pipinstalldist/*.whl Attention
If you installed the C++ library in a custom directory, you should configure additional environment variables:
When running setup.py, set CTRANSLATE2_ROOT to the CTranslate2 install directory.
When running your Python application, add the CTranslate2 library path to LD_LIBRARY_PATH.
Build options
The following options can be set with -DOPTION=VALUE during the CMake configuration:
CMake option
Values (default in bold)
Description
BUILD_CLI
OFF, ON
Compiles the command line clients
BUILD_TESTS
OFF, ON
Compiles the tests
CMAKE_CXX_FLAGS
compiler flags
Defines additional compiler flags
CMAKE_INSTALL_PREFIX
path
Defines the installation path of the library
CUDA_ARCH_LIST
Auto
List of CUDA architectures to compile for (see
in the CMake documentation)
CUDA_DYNAMIC_LOADING
OFF, ON
Enables the dynamic loading of CUDA libraries at runtime instead of linking against them (requires CUDA >= 11)
CUDA_NVCC_FLAGS
compiler flags
Defines additional compilation flags for nvcc
ENABLE_CPU_DISPATCH
OFF, ON
Compiles CPU kernels for multiple ISA and dispatches at runtime (should be disabled when explicitly targeting an architecture with the -march compilation flag)
ENABLE_PROFILING
OFF, ON
Enables the integrated profiler (usually disabled in production builds)
OPENMP_RUNTIME
INTEL, COMP, NONE
Selects the OpenMP runtime:
INTEL: Intel OpenMP
COMP: OpenMP runtime provided by the compiler
NONE: no OpenMP runtime (a custom threading implementation will be used)
WITH_CUDA
OFF, ON
Compiles with the CUDA backend
WITH_CUDNN
OFF, ON
Compiles with the cuDNN backend
WITH_DNNL
OFF, ON
Compiles with the oneDNN backend (a.k.a. DNNL)
WITH_MKL
OFF, ON
Compiles with the Intel MKL backend
WITH_ACCELERATE
OFF, ON
Compiles with the Apple Accelerate backend
WITH_OPENBLAS
OFF, ON
Compiles with the OpenBLAS backend
WITH_RUY
OFF, ON
Compiles with the Ruy backend
WITH_HIP
OFF, ON
Compiles with the AMD HIP GPU backend
Some build options require additional dependencies. See their respective documentation for installation instructions.
-DWITH_CUDA=ON requires
>= 11.0
-DWITH_CUDNN=ON requires
>= 8
-DWITH_MKL=ON requires
>= 2019.5
-DWITH_DNNL=ON requires
>= 3.0
-DWITH_ACCELERATE=ON requires
-DWITH_OPENBLAS=ON requires
-DWITH_HIP=ON requires
Multiple backends can be enabled for a single build, for example:
-DWITH_MKL=ON-DWITH_CUDA=ON: enable CPU and GPU support
-DWITH_MKL=ON-DWITH_DNNL=ON: during runtime, the library will select Intel MKL when running on Intel and oneDNN when running on AMD
-DWITH_OPENBLAS=ON-DWITH_RUY=ON: use Ruy for quantized models and OpenBLAS for non quantized models