CTranslate2 supports Transformer models trained with
. The conversion simply requires the PyTorch model path, e.g.:
pipinstallOpenNMT-py==2.* ct2-opennmt-py-converter--model_pathmodel.pt--output_dirct2_model Alternatively, you can also convert the model directly from OpenNMT-py with the model release script:
onmt_release_model--modelmodel.pt--formatctranslate2--outputct2_model Tip
See the
for a complete example using an OpenNMT-py model.
Text generation with transformer_lm
Decoder-only models using the transformer_lm decoder type are supported and can be converted with the same command line.
During generation, make sure to always include <s> in the start tokens, e.g.:
generator=ctranslate2.Generator(model_path)generator.generate_batch([["<s>","▁Hello"]])