Afrikaans

cc100 af/cc100

Language modeling for the Afrikaans language on the CC-100 corpus.

Albanian

cc100 sq/cc100

Language modeling for the Albanian language on the CC-100 corpus.

Amharic

cc100 am/cc100

Language modeling for the Amharic language on the CC-100 corpus.

wiki am/wiki

Language modeling for the Amharic language on Wikipedia.

Arabic

cc100 ar/cc100

Language modeling for the Arabic language on the CC-100 corpus.

wiki ar/wiki

Language modeling for the Arabic language on Wikipedia. Arabic is a Semitic language that first emerged in the 1st to 4th centuries CE. It is now the lingua franca of the Arab world.

Armenian

cc100 hy/cc100

Language modeling for the Armenian language on the CC-100 corpus.

wiki hy/wiki

Language modeling for the Armenian language on Wikipedia.

Azerbaijani

cc100 az/cc100

Language modeling for the Azerbaijani language on the CC-100 corpus.

Basque

cc100 eu/cc100

Language modeling for the Basque language on the CC-100 corpus.

wiki eu/wiki

Language modeling for the Basque language on Wikipedia.

Belarusian

cc100 be/cc100

Language modeling for the Belarusian language on the CC-100 corpus.

Bengali

cc100 bn/cc100

Language modeling for the Bengali language on the CC-100 corpus.

wiki bn/wiki

Language modeling for the Bengali language on Wikipedia.

Bhojpuri

wiki bh/wiki

Language modeling for the Bhojpuri language on Wikipedia.

Bulgarian

cc100 bg/cc100

Language modeling for the Bulgarian language on the CC-100 corpus.

Burmese

cc100 my/cc100

Language modeling for the Burmese language on the CC-100 corpus.

wiki my/wiki

Language modeling for the Burmese language on Wikipedia. The Burmese language is a Sino-Tibetan language spoken in Myanmar where it is an official language and the language of the Bamar people,...

Buryat

wiki bxr/wiki

Language modeling for the Buryat language on Wikipedia.

Cantonese

wiki zh_yue/wiki

Language modeling for the Cantonese language on Wikipedia.

Catalan

cc100 ca/cc100

Language modeling for the Catalan language on the CC-100 corpus.

Central Khmer

cc100 km/cc100

Language modeling for the Central Khmer language on the CC-100 corpus.

Chinese

cc100 zh/cc100

Language modeling for the Chinese (traditional) language on the CC-100 corpus.

wiki zh/wiki

Language modeling for the Chinese language on Wikipedia. Chinese is a family of East Asian analytic languages that form the Sinitic branch of the Sino-Tibetan languages.

Croatian

cc100 hr/cc100

Language modeling for the Croatian language on the CC-100 corpus.

Czech

cc100 cs/cc100

Language modeling for the Czech language on the CC-100 corpus.

wiki cs/wiki

Language modeling for the Czech language on Wikipedia.

Danish

cc100 da/cc100

Language modeling for the Danish language on the CC-100 corpus.

Dutch

cc100 nl/cc100

Language modeling for the Dutch language on the CC-100 corpus.

English

cc100 en/cc100

Language modeling for the English language on the CC-100 corpus.

wiki en/wiki

Language modeling for the English language on Wikipedia. English is a West Germanic language that was first spoken in early medieval England and eventually became a global lingua franca.

Erzya

wiki myv/wiki

Language modeling for the Erzya language on Wikipedia.

Esperanto

cc100 eo/cc100

Language modeling for the Esperanto language on the CC-100 corpus.

Estonian

cc100 et/cc100

Language modeling for the Estonian language on the CC-100 corpus.

wiki et/wiki

Language modeling for the Estonian language on Wikipedia. Estonian is a Uralic language of the Finnic branch spoken in Estonia.

Finnish

cc100 fi/cc100

Language modeling for the Finnish language on the CC-100 corpus.

wiki fi/wiki

Language modeling for the Finnish language on Wikipedia. Finnish, whose endonym is suomi or suomen kieli is a Uralic language of the Finnic branch spoken by the majority of the population in...

French

cc100 fr/cc100

Language modeling for the French language on the CC-100 corpus.

wiki fr/wiki

Language modeling for the French language on Wikipedia. French or la langue française is a Romance language of the Indo-European family.

Galician

cc100 gl/cc100

Language modeling for the Galician language on the CC-100 corpus.

Georgian

cc100 ka/cc100

Language modeling for the Georgian language on the CC-100 corpus.

wiki ka/wiki

Language modeling for the Georgian language on Wikipedia.

German

cc100 de/cc100

Language modeling for the German language on the CC-100 corpus.

wiki de/wiki

Language modeling for the German language on Wikipedia. German is a West Germanic language that is mainly spoken in Central Europe.

Greek

cc100 el/cc100

Language modeling for the Greek language on the CC-100 corpus.

wiki el/wiki

Language modeling for the Greek language on Wikipedia. Greek is an independent branch of the Indo-European family of languages, native to Greece, Cyprus, Albania and other parts of the Eastern...

Guarani

wiki gn/wiki

Language modeling for the Guarani language on Wikipedia. Guarani specifically the primary variety known as Paraguayan Guarani, is an indigenous language of South America that belongs to the...

Gujarati

cc100 gu/cc100

Language modeling for the Gujarati language on the CC-100 corpus.

Haitian Creole

wiki ht/wiki

Language modeling for the Haitian Creole language on Wikipedia. Haitian Creole is a French-based creole language spoken by 10–12 million people worldwide, and the only language of most Haitians.

Hausa

cc100 ha/cc100

Language modeling for the Hausa language on the CC-100 corpus.

Hebrew

cc100 he/cc100

Language modeling for the Hebrew language on the CC-100 corpus.

Hindi

cc100 hi/cc100

Language modeling for the Hindi language on the CC-100 corpus.

wiki hi/wiki

Language modeling for the Hindi language on Wikipedia. Hindi, or more precisely Modern Standard Hindi, is an Indo-Aryan language spoken in India.

Hungarian

cc100 hu/cc100

Language modeling for the Hungarian language on the CC-100 corpus.

wiki hu/wiki

Language modeling for the Hungarian language on Wikipedia.

Icelandic

cc100 is/cc100

Language modeling for the Icelandic language on the CC-100 corpus.

wiki is/wiki

Language modeling for the Icelandic language on Wikipedia. Icelandic is a North Germanic language spoken by about 314,000 people, the vast majority of whom live in Iceland where it is the national...

Ilocano

wiki ilo/wiki

Language modeling for the Ilocano language on Wikipedia. Ilocano (also Ilokano) is an Austronesian language spoken in the Philippines.

Indonesian

cc100 id/cc100

Language modeling for the Indonesian language on the CC-100 corpus.

wiki id/wiki

Language modeling for the Indonesian language on Wikipedia. Indonesian is the official language of Indonesia. It is a standardised variety of Malay, an Austronesian language that has been used as...

Irish

cc100 ga/cc100

Language modeling for the Irish language on the CC-100 corpus.

Italian

cc100 it/cc100

Language modeling for the Italian language on the CC-100 corpus.

wiki it/wiki

Language modeling for the Italian language on Wikipedia. Italian or lingua italiana, is a Romance language of the Indo-European language family. Italian descended from the Vulgar Latin of the...

Japanese

cc100 ja/cc100

Language modeling for the Japanese language on the CC-100 corpus.

wiki ja/wiki

Language modeling for the Japanese language on Wikipedia. Japanese is an East Asian language spoken by about 128 million people, primarily in Japan, where it is the national language.

Javanese

wiki jv/wiki

Language modeling for the Javanese language on Wikipedia. Javanese is the language of the Javanese people from the central and eastern parts of the island of Java, in Indonesia.

Kannada

cc100 kn/cc100

Language modeling for the Kannada language on the CC-100 corpus.

Kazakh

cc100 kk/cc100

Language modeling for the Kazakh language on the CC-100 corpus.

Kirghiz

cc100 ky/cc100

Language modeling for the Kirghiz language on the CC-100 corpus.

Komi

wiki kv/wiki

Language modeling for the Komi language on Wikipedia.

Korean

cc100 ko/cc100

Language modeling for the Korean language on the CC-100 corpus.

wiki ko/wiki

Language modeling for the Korean language on Wikipedia. The Korean language is an East Asian language spoken by about 77 million people.

Kurdish

cc100 ku/cc100

Language modeling for the Kurdish language on the CC-100 corpus.

Lao

cc100 lo/cc100

Language modeling for the Lao language on the CC-100 corpus.

Latin

cc100 la/cc100

Language modeling for the Latin language on the CC-100 corpus.

wiki la/wiki

Language modeling for the Latin language on Wikipedia.

Latvian

cc100 lv/cc100

Language modeling for the Latvian language on the CC-100 corpus.

wiki lv/wiki

Language modeling for the Latvian language on Wikipedia.

Lithuanian

cc100 lt/cc100

Language modeling for the Lithuanian language on the CC-100 corpus.

Macedonian

cc100 mk/cc100

Language modeling for the Macedonian language on the CC-100 corpus.

Malay

cc100 ms/cc100

Language modeling for the Malay language on the CC-100 corpus.

Malayalam

cc100 ml/cc100

Language modeling for the Malayalam language on the CC-100 corpus.

Marathi

cc100 mr/cc100

Language modeling for the Marathi language on the CC-100 corpus.

Meadow Mari

wiki mhr/wiki

Language modeling for the Meadow Mari language on Wikipedia. Meadow Mari or Meadow-Eastern Mari or Eastern Mari is a standardised dialect of the Mari language used by about half a million people...

Min Dong

wiki cdo/wiki

Language modeling for the Min Dong language on Wikipedia. Eastern Min or Min Dong, is a branch of the Min group of Sinitic languages of China.

Mingrelian

wiki xmf/wiki

Language modeling for the Mingrelian language on Wikipedia. Megrelian is a Kartvelian language spoken in Western Georgia (regions of Samegrelo and Abkhazia), primarily by the Megrelians.

Mongolian

cc100 mn/cc100

Language modeling for the Mongolian language on the CC-100 corpus.

Māori

wiki mi/wiki

Language modeling for the Māori language on Wikipedia. Māori, also known as te reo ('the language'), is an Eastern Polynesian language spoken by the Māori people, the indigenous population of New Zealand.

Nepali

cc100 ne/cc100

Language modeling for the Nepali language on the CC-100 corpus.

Northern Sami

wiki se/wiki

Language modeling for the Northern Sami language on Wikipedia.

Norwegian

cc100 no/cc100

Language modeling for the Norwegian language on the CC-100 corpus.

Oriya

cc100 or/cc100

Language modeling for the Oriya language on the CC-100 corpus.

Pashto

cc100 ps/cc100

Language modeling for the Pashto language on the CC-100 corpus.

Persian

cc100 fa/cc100

Language modeling for the Persian language on the CC-100 corpus.

wiki fa/wiki

Language modeling for the Persian language on Wikipedia.

Polish

cc100 pl/cc100

Language modeling for the Polish language on the CC-100 corpus.

Portuguese

cc100 pt/cc100

Language modeling for the Portuguese language on the CC-100 corpus.

wiki pt/wiki

Language modeling for the Portuguese language on Wikipedia.

Punjabi

cc100 pa/cc100

Language modeling for the Punjabi language on the CC-100 corpus.

Quechua

wiki qu/wiki

Language modeling for the Quechuan language on Wikipedia. Quechua, usually called Runasimi ("people's language") in Quechuan languages, is an indigenous language family spoken by the Quechua...

Romanian

cc100 ro/cc100

Language modeling for the Romanian language on the CC-100 corpus.

Russian

cc100 ru/cc100

Language modeling for the Russian language on the CC-100 corpus.

wiki ru/wiki

Language modeling for the Russian language on Wikipedia. Russian is an East Slavic language, which is an official language in Russia, Belarus, Kazakhstan, Kyrgyzstan, as well as being widely used...

Sanskrit

cc100 sa/cc100

Language modeling for the Sanskrit language on the CC-100 corpus.

Serbian

cc100 sr/cc100

Language modeling for the Serbian language on the CC-100 corpus.

Sinhala

cc100 si/cc100

Language modeling for the Sinhala language on the CC-100 corpus.

Slovak

cc100 sk/cc100

Language modeling for the Slovak language on the CC-100 corpus.

Slovenian

cc100 sl/cc100

Language modeling for the Slovenian language on the CC-100 corpus.

Somali

cc100 so/cc100

Language modeling for the Somali language on the CC-100 corpus.

Spanish

cc100 es/cc100

Language modeling for the Spanish language on the CC-100 corpus.

wiki es/wiki

Language modeling for the Spanish language on Wikipedia. Spanish, or Castilian, is a Romance language that originated in the Iberian Peninsula and today is a global language with more than 483...

Swahili

cc100 sw/cc100

Language modeling for the Swahili language on the CC-100 corpus.

wiki sw/wiki

Language modeling for the Swahili language on Wikipedia. Swahili, also known by its native name Kiswahili, is a Bantu language and the first language of the Swahili people.

Swedish

cc100 sv/cc100

Language modeling for the Swedish language on the CC-100 corpus.

Tagalog

cc100 tl/cc100

Language modeling for the Tagalog language on the CC-100 corpus.

Tamil

cc100 ta/cc100

Language modeling for the Tamil language on the CC-100 corpus.

wiki ta/wiki

Language modeling for the Tamil language on Wikipedia. Tamil, About this soundpronunciation is a Dravidian language predominantly spoken by the Tamil people of India and Sri Lanka, and by the...

Telugu

cc100 te/cc100

Language modeling for the Telugu language on the CC-100 corpus.

Thai

cc100 th/cc100

Language modeling for the Thai language on the CC-100 corpus.

wiki th/wiki

Language modeling for the Thai language on Wikipedia. Thai, Central Thai, is the official language of Thailand and the first language of the Central Thai people.

Turkish

cc100 tr/cc100

Language modeling for the Turkish language on the CC-100 corpus.

wiki tr/wiki

Language modeling for the Turkish language on Wikipedia. Turkish, also referred to as Istanbul Turkish or Turkey Turkish, is the most widely spoken of the Turkic languages, with around 70 to 80...

Turkmen

wiki tk/wiki

Language modeling for the Turkmen language on Wikipedia. Turkmen is an Oghuz Turkic language spoken by the Turkmens of Central Asia and the official language of Turkmenistan.

Ukrainian

cc100 uk/cc100

Language modeling for the Ukrainian language on the CC-100 corpus.

Urdu

cc100 ur/cc100

Language modeling for the Urdu language on the CC-100 corpus.

Uzbek

cc100 uz/cc100

Language modeling for the Uzbek language on the CC-100 corpus.

Vietnamese

cc100 vi/cc100

Language modeling for the Vietnamese language on the CC-100 corpus.

wiki vi/wiki

Language modeling for the Vietnamese language on Wikipedia. Vietnamese is an Austroasiatic language that originated in Vietnam, where it is the national and official language. Spoken natively by...

Welsh

cc100 cy/cc100

Language modeling for the Welsh language on the CC-100 corpus.

Wolof

wiki wo/wiki

Language modeling for the Wolof language on Wikipedia.

No tasks and datasets for

Add your language to AdapterHub,
it's super awesome!

Get started