Edit on GitHub

Description

Our adapters from the MultiCQA paper (https://arxiv.org/abs/2010.00980) trained on the different StackExchange forums (see "version") with self-supervised training signals of unlabeled questions.

Usage

model = BertForSequenceClassification.from_pretrained("bert-base-uncased")
config = AdapterConfig.load("pfeiffer", reduction_factor=12)
model.load_adapter("sts/stackexchange@ukp", "text_task", config=config)

Properties

Pre-trained model
bert-base-uncased
Adapter type
Task
Semantic Textual Similarity

Architecture

Name
pfeiffer
Non-linearity
relu
Reduction factor
12
{
    "ln_after": false,
    "ln_before": false,
    "mh_adapter": false,
    "output_adapter": true,
    "adapter_residual_before_ln": false,
    "non_linearity": "relu",
    "original_ln_after": true,
    "original_ln_before": true,
    "reduction_factor": 16,
    "residual_before_ln": true,
    "invertible_adapter": {
        "block_type": "nice",
        "non_linearity": "relu",
        "reduction_factor": 2
    }
}

Author

  Name
Andreas Rücklé
  GitHub
  Twitter

Versions

Identifier Comment Score Download
3dprinting_stackexchange_com
academia_stackexchange_com
ai_stackexchange_com
android_stackexchange_com
anime_stackexchange_com
apple_stackexchange_com
arduino_stackexchange_com
askubuntu_com
astronomy_stackexchange_com
aviation_stackexchange_com
avp_stackexchange_com
bicycles_stackexchange_com
bioinformatics_stackexchange_com
biology_stackexchange_com
bitcoin_stackexchange_com
blender_stackexchange_com
boardgames_stackexchange_com
bricks_stackexchange_com
buddhism_stackexchange_com
chemistry_stackexchange_com
chess_stackexchange_com
christianity_stackexchange_com
civicrm_stackexchange_com
codegolf_stackexchange_com
codereview_stackexchange_com
cogsci_stackexchange_com
computergraphics_stackexchange_com
cooking_stackexchange_com DEFAULT
craftcms_stackexchange_com
crypto_stackexchange_com
cs_stackexchange_com
cstheory_stackexchange_com
datascience_stackexchange_com
dba_stackexchange_com
devops_stackexchange_com
diy_stackexchange_com
drupal_stackexchange_com
dsp_stackexchange_com
earthscience_stackexchange_com
economics_stackexchange_com
electronics_stackexchange_com
elementaryos_stackexchange_com
ell_stackexchange_com
emacs_stackexchange_com
engineering_stackexchange_com
english_stackexchange_com
eosio_stackexchange_com
ethereum_stackexchange_com
expatriates_stackexchange_com
expressionengine_stackexchange_com
fitness_stackexchange_com
freelancing_stackexchange_com
gamedev_stackexchange_com
gaming_stackexchange_com
gardening_stackexchange_com
genealogy_stackexchange_com
gis_stackexchange_com
graphicdesign_stackexchange_com
ham_stackexchange_com
hardwarerecs_stackexchange_com
health_stackexchange_com
hermeneutics_stackexchange_com
hinduism_stackexchange_com
history_stackexchange_com
homebrew_stackexchange_com
hsm_stackexchange_com
interpersonal_stackexchange_com
islam_stackexchange_com
joomla_stackexchange_com
judaism_stackexchange_com
law_stackexchange_com
lifehacks_stackexchange_com
linguistics_stackexchange_com
literature_stackexchange_com
magento_stackexchange_com
martialarts_stackexchange_com
math_stackexchange_com
matheducators_stackexchange_com
mathematica_stackexchange_com
mechanics_stackexchange_com
monero_stackexchange_com
money_stackexchange_com
movies_stackexchange_com
music_stackexchange_com
musicfans_stackexchange_com
mythology_stackexchange_com
networkengineering_stackexchange_com
opendata_stackexchange_com
opensource_stackexchange_com
outdoors_stackexchange_com
parenting_stackexchange_com
patents_stackexchange_com
pets_stackexchange_com
philosophy_stackexchange_com
photo_stackexchange_com
physics_stackexchange_com
pm_stackexchange_com
poker_stackexchange_com
politics_stackexchange_com
puzzling_stackexchange_com
quant_stackexchange_com
quantumcomputing_stackexchange_com
raspberrypi_stackexchange_com
retrocomputing_stackexchange_com
reverseengineering_stackexchange_com
robotics_stackexchange_com
rpg_stackexchange_com
salesforce_stackexchange_com
scicomp_stackexchange_com
scifi_stackexchange_com
security_stackexchange_com
serverfault_com
sharepoint_stackexchange_com
sitecore_stackexchange_com
skeptics_stackexchange_com
softwareengineering_stackexchange_com
softwarerecs_stackexchange_com
sound_stackexchange_com
space_stackexchange_com
sports_stackexchange_com
sqa_stackexchange_com
stackapps_com
stackoverflow_com
stats_stackexchange_com
superuser_com
sustainability_stackexchange_com
tex_stackexchange_com
tor_stackexchange_com
travel_stackexchange_com
tridion_stackexchange_com
unix_stackexchange_com
ux_stackexchange_com
vi_stackexchange_com
webapps_stackexchange_com
webmasters_stackexchange_com
windowsphone_stackexchange_com
woodworking_stackexchange_com
wordpress_stackexchange_com
workplace_stackexchange_com
worldbuilding_stackexchange_com
writers_stackexchange_com

Citations

Adapter
@inproceedings{rueckle-etal-2020-multicqa,
  title = "{MultiCQA}: Zero-Shot Transfer of Self-Supervised Text Matching Models on a Massive Scale",
  author = {R{\"u}ckl{\'e}, Andreas  and
    Pfeiffer, Jonas and
    Gurevych, Iryna},
  booktitle = "Proceedings of The 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP-2020)",
  year = "2020",
  address = "Virtual Conference",
  url = "https://arxiv.org/abs/2010.00980",
}
Architecture
@misc{pfeiffer2020adapterfusion,
  title={AdapterFusion: Non-Destructive Task Composition for Transfer Learning},
  author={Jonas Pfeiffer and Aishwarya Kamath and Andreas Rücklé and Kyunghyun Cho and Iryna Gurevych},
  year={2020},
  eprint={2005.00247},
  archivePrefix={arXiv},
  primaryClass={cs.CL}
}
Task
@inproceedings{rueckle-etal-2020-multicqa,
  title = "{MultiCQA}: Zero-Shot Transfer of Self-Supervised Text Matching Models on a Massive Scale",
  author = {R{\"u}ckl{\'e}, Andreas  and
    Pfeiffer, Jonas and
    Gurevych, Iryna},
  booktitle = "Proceedings of The 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP-2020)",
  year = "2020",
  address = "Virtual Conference",
  url = "https://arxiv.org/abs/2010.00980",
}