microsoft/codebert-base即梦al

CodeBERT-base

Pretrained weights for CodeBERT: A Pre-Trained Model for Programming and Natural Languages.制作ai的软件

Training Data

The model is trained on bi-modal data (documents & code) of CodeSearchNet有戏ai

Training Objective

This model is initialized with Roberta-base and trained with MLM+RTD objective (cf. the paper).百度ai智能云

Usage

Please see the official repository for scripts that support “code search” and “code-to-document generation”.ima是什么软件

Reference

1. CodeBERT trained with Masked LM objective (suitable for code completion)
2. ? Hugging Face’s CodeBERTa (small size, 6 layers)

Citation

@misc{feng2020codebert,
    title={CodeBERT: A Pre-Trained Model for Programming and Natural Languages},
    author={Zhangyin Feng and Daya Guo and Duyu Tang and Nan Duan and Xiaocheng Feng and Ming Gong and Linjun Shou and Bing Qin and Ting Liu and Daxin Jiang and Ming Zhou},
    year={2020},
    eprint={2002.08155},
    archivePrefix={arXiv},
    primaryClass={cs.CL}
}