Edit on GitHub 

model = AutoAdapterModel.from_pretrained("bert-base-multilingual-cased")
config = AdapterConfig.load("pfeiffer", non_linearity="gelu", reduction_factor=16)
model.load_adapter("wikiann/ja@ukp", config=config)

Description

Stacked adapter on top of Language adapter. MAD-X 2.0 style. The language adapters in the last layer (layer 11) are deleted.

Properties

Pre-trained model
bert-base-multilingual-cased
Adapter type
Task
Prediction Head
  Yes
Task
Wiki Ann NER
Dataset
Japanese

Dependencies

This adapter should be used together with these dependencies to achieve best results:

Architecture

Name
pfeiffer
Non-linearity
gelu
Reduction factor
16
{
  "ln_after": false,
  "ln_before": false,
  "mh_adapter": false,
  "output_adapter": true,
  "adapter_residual_before_ln": false,
  "non_linearity": "gelu",
  "original_ln_after": true,
  "original_ln_before": true,
  "reduction_factor": 16,
  "residual_before_ln": true
}

Author

  Name
Jonas Pfeiffer
  E-Mail
Jonas@Pfeiffer.ai
  Web
https://pfeiffer.ai
  GitHub
JoPfeiff
  Twitter
@PfeiffJo

Versions

Identifier Comment Score Download
1 DEFAULT
2
3
4
5

Citations

Adapter
@article{Pfeiffer21UNKs,
  author    = {Jonas Pfeiffer and
               Ivan Vuli\'{c} and
                Iryna Gurevych and
                Sebastian Ruder},
  title     = {{UNKs Everywhere: Adapting Multilingual Language Models to New Scripts}},
  journal   = {arXiv preprint},
  year      = {2021} ,
  url = {https://arxiv.org/abs/2012.15562}
}
Architecture
@misc{pfeiffer2020adapterfusion,
  title={AdapterFusion: Non-Destructive Task Composition for Transfer Learning},
  author={Jonas Pfeiffer and Aishwarya Kamath and Andreas Rücklé and Kyunghyun Cho and Iryna Gurevych},
  year={2020},
  eprint={2005.00247},
  archivePrefix={arXiv},
  primaryClass={cs.CL}
}
Task
@inproceedings{Pan2017wikiann,
  author    = {Xiaoman Pan and
               Boliang Zhang and
               Jonathan May and
               Joel Nothman and
               Kevin Knight and
               Heng Ji},
  title     = {Cross-lingual Name Tagging and Linking for 282 Languages},
  booktitle = {Proceedings of the 55th Annual Meeting of the Association for Computational
               Linguistics, {ACL} 2017, Vancouver, Canada, July 30 - August 4, Volume
               1: Long Papers},
  pages     = {1946--1958},
  year      = {2017},
  url       = {https://doi.org/10.18653/v1/P17-1178},
}
@inproceedings{Rahimi2019massively,
  author    = {Afshin Rahimi and
               Yuan Li and
               Trevor Cohn},
  title     = {Massively Multilingual Transfer for {NER}},
  booktitle = {Proceedings of the 57th Conference of the Association for Computational
               Linguistics, {ACL} 2019, Florence, Italy, July 28- August 2, 2019,
               Volume 1: Long Papers},
  pages     = {151--164},
  year      = {2019},
  url       = {https://doi.org/10.18653/v1/p19-1015},
}