model = AutoAdapterModel.from_pretrained("roberta-base") model.load_adapter("UKP-SQuARE/HotpotQA_Adapter_RoBERTa", source="hf")
This is the single-dataset adapter for the HotpotQA partition of the MRQA 2019 Shared Task Dataset. The adapter was created by Friedman et al. (2021) and should be used with the roberta-base
encoder.
The UKP-SQuARE team created this model repository to simplify the deployment of this model on the UKP-SQuARE platform. The GitHub repository of the original authors is https://github.com/princeton-nlp/MADE
This model contains the same weights as https://huggingface.co/princeton-nlp/MADE/resolve/main/single_dataset_adapters/HotpotQA/model.pt. The only difference is that our repository follows the standard format of AdapterHub. Therefore, you could load this model as follows:
from transformers import RobertaForQuestionAnswering, RobertaTokenizerFast
model = RobertaForQuestionAnswering.from_pretrained("roberta-base")
model.load_adapter("UKP-SQuARE/HotpotQA_Adapter_RoBERTa", source="hf")
model.set_active_adapters("HotpotQA")
tokenizer = RobertaTokenizerFast.from_pretrained('roberta-base')
pipe = pipeline("question-answering", model=model, tokenizer=tokenizer)
pipe({"question": "What is the capital of Germany?", "context": "The capital of Germany is Berlin."})
Note you need the adapter-transformers library https://adapterhub.ml
Friedman et al. report an F1 score of 78.5 on HotpotQA.
Please refer to the original publication for more information.
Single-dataset Experts for Multi-dataset Question Answering (Friedman et al., EMNLP 2021)
{ "adapter_residual_before_ln": false, "cross_adapter": false, "factorized_phm_W": true, "factorized_phm_rule": false, "hypercomplex_nonlinearity": "glorot-uniform", "init_weights": "bert", "inv_adapter": null, "inv_adapter_reduction_factor": null, "is_parallel": false, "learn_phm": true, "leave_out": [], "ln_after": false, "ln_before": false, "mh_adapter": true, "non_linearity": "swish", "original_ln_after": true, "original_ln_before": false, "output_adapter": true, "phm_bias": true, "phm_c_init": "normal", "phm_dim": 4, "phm_init_range": 0.0001, "phm_layer": false, "phm_rank": 1, "reduction_factor": 16, "residual_before_ln": true, "scaling": 1.0, "shared_W_phm": false, "shared_phm_rule": true, "use_gating": false }