dpr-question_encoder-multiset-base

Table of Contents

• Model Details
• How To Get Started With the Model
• Uses
• Risks, Limitations and Biases
• Training
• Evaluation
• Environmental Impact
• Technical Specifications
• Citation Information
• Model Card Authors

Model Details

Model Description:免费的ai工具 Dense Passage Retrieval (DPR) is a set of tools and models for state-of-the-art open-domain Q&A research. dpr-question_encoder-multiset-base is the question encoder trained using the Natural Questions (NQ) dataset, TriviaQA, WebQuestions (WQ), and CuratedTREC (TREC).

• Developed by:百度ai智能云 See GitHub repo for model developers
• Model Type:ai软件哪个比较好 BERT-based encoder
• Language(s):百度ai智能云 CC-BY-NC-4.0, also see Code of Conduct
• License:做al视频怎么赚钱 English
• Related Models:ai软件哪个比较好
  • dpr-ctx_encoder-multiset-base
  • dpr-reader-multiset-base
  • dpr-ctx_encoder-single-nq-base
  • dpr-question_encoder-single-nq-base
  • dpr-reader-single-nq-base
• Resources for more information:即梦下载官方
  • Research Paper
  • GitHub Repo
  • Hugging Face DPR docs
  • BERT Base Uncased Model Card

How to Get Started with the Model

Use the code below to get started with the model.al一键脱装入口

from Transformers即梦al import DPRQuestionEncoder, DPRQuestionEncoderTokenizer
tokenizer = DPRQuestionEncoderTokenizer.from_pretrained("facebook/dpr-question_encoder-multiset-base")
model = DPRQuestionEncoder.from_pretrained("facebook/dpr-question_encoder-multiset-base")
input_ids = tokenizer("Hello, is my dog cute ?", return_tensors="pt")["input_ids"]
embeddings = model(input_ids).pooler_output

Uses

Direct Use

dpr-question_encoder-multiset-base, dpr-ctx_encoder-multiset-base, and dpr-reader-multiset-base can be used for the task of open-domain question answering.

Misuse and Out-of-scope Use

The model should not be used to intentionally create hostile or alienating environments for people. In addition, the set of DPR models was not trained to be factual or true representations of people or events, and therefore using the models to generate such content is out-of-scope for the abilities of this model.ai是什么东西?

Risks, Limitations and Biases

CONTENT WARNING: Readers should be aware this section may contain content that is disturbing, offensive, and can propogate historical and current stereotypes.即梦al

Significant research has explored bias and fairness issues with language models (see, e.g., Sheng et al., 2021 and Bender et al., 2021). Predictions generated by the model can include disturbing and harmful stereotypes across protected classes; identity characteristics; and sensitive, social, and occupational groups.有戏ai

Training

Training Data

This model was trained using the following datasets:百度ai智能云

• Natural Questions (NQ) dataset元宝大模型 (Lee et al., 2019; Kwiatkowski et al., 2019)
• TriviaQA即梦al (Joshi et al., 2017)
• WebQuestions (WQ)ima是什么软件 (Berant et al., 2013)
• CuratedTREC (TREC)有戏ai (Baudiš & Šedivý, 2015)

Training Procedure

The training procedure is described in the associated paper:grok中文版下载

Given a collection of M text passages, the goal of our dense passage retriever (DPR) is to index all the passages in a low-dimensional and continuous space, such that it can retrieve efficiently the top k passages relevant to the input question for the reader at run-time.元宝大模型

Our dense passage retriever (DPR) uses a dense encoder EP(·) which maps any text passage to a d- dimensional real-valued vectors and builds an index for all the M passages that we will use for retrieval. At run-time, DPR applies a different encoder EQ(·) that maps the input question to a d-dimensional vector, and retrieves k passages of which vectors are the closest to the question vector.百度流畅ai制作

The authors report that for encoders, they used two independent BERT (Devlin et al., 2019) networks (base, un-cased) and use FAISS (Johnson et al., 2017) during inference time to encode and index passages. See the paper for further details on training, including encoders, inference, positive and negative passages, and in-batch negatives.制作ai的软件

Evaluation

The following evaluation information is extracted from the associated paper.百度ai智能云

Testing Data, Factors and Metrics

The model developers report the performance of the model on five QA datasets, using the top-k accuracy (k ∈ {20, 100}). The datasets were NQ, TriviaQA, WebQuestions (WQ), CuratedTREC (TREC), and SQuAD v1.1.al一键脱装入口

Results

Environmental Impact

Carbon emissions can be estimated using the Machine Learning Impact calculator presented in Lacoste et al. (2019). We present the hardware type and based on the associated paper.元宝大模型

• Hardware Type:百度流畅ai制作 8 32GB GPUs
• Hours used:al一键脱装入口 Unknown
• Cloud Provider:免费的ai工具 Unknown
• Compute Region:快问ai Unknown
• Carbon Emitted:ima是什么软件 Unknown

Technical Specifications

See the associated paper for details on the modeling architecture, objective, compute infrastructure, and training details.即梦al

Citation Information

  @inproceedings{karpukhin-etal-2020-dense,
    title = "Dense Passage Retrieval for Open-Domain Question Answering",
    author = "Karpukhin, Vladimir and Oguz, Barlas and Min, Sewon and Lewis, Patrick and Wu, Ledell and Edunov, Sergey and Chen, Danqi and Yih, Wen-tau",
    booktitle = "Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)",
    month = nov,
    year = "2020",
    address = "Online",
    publisher = "Association for Computational Linguistics",
    url = "https://www.aclweb.org/anthology/2020.emnlp-main.550",
    doi = "10.18653/v1/2020.emnlp-main.550",
    pages = "6769--6781",
}

Model Card Authors

This model card was written by the team at Hugging Face.有戏ai