Model Conversion Tools

Download scripts

Converting DistilBERT from PyTorch Transformer

The following command downloads the distilBERT model from pytorch-transformer, and converts the model to Gluon.

pip3 install pytorch-transformers
python3 --out_dir converted-model

Converting RoBERTa from Fairseq

The following command converts the roberta checkpoint <> from fairseq to Gluon. The converted Gluon model is saved in the same folder as the checkpoint’s.

pip3 install fairseq
# download the roberta checkpoint from the website, then do:
python3 --ckpt_dir ./roberta/roberta.base --model roberta_12_768_12