Pre-trained model:
STS-B adapter (with head) trained using the `run_glue.py` script with an extension that retains the best checkpoint (out of 30 epochs).
Adapter in Houlsby architecture trained on the STS-Benchmark task for 20 epochs with early stopping and a learning rate of 1e-4. See https://arxiv.org/pdf/2007.07779.pdf.
Adapter in Pfeiffer architecture trained on the STS-Benchmark task for 20 epochs with early stopping and a learning rate of 1e-4. See https://arxiv.org/pdf/2007.07779.pdf.