OSCAR

Apr 27, 2016
Go to Project Site
Image credit: Alix Chagué
Corpus Linguistics
Pedro Ortiz Suarez
Pedro Ortiz Suarez
Senior Research Scientist

I’m a Senior Research Scientist at the Common Crawl Foundation.

Publications

Quality at a Glance: An Audit of Web-Crawled Multilingual Datasets
We audit 5 multilingual corpora, finding that lower-resource corpora have systematic issues.
Julia Kreutzer, Isaac Caswell, Lisa Wang, Ahsan Wahab, Daan van Esch, Nasanbayar Ulzii-Orshikh, Allahsera Tapo, Nishant Subramani, Artem Sokolov, Claytone Sikasote, Monang Setyawan, Supheakmungkol Sarin, Sokhar Samb, Benoît Sagot, Clara Rivera, Annette Rios, Isabel Papadimitriou, Salomey Osei, Pedro Ortiz Suarez, Iroro Orife, Kelechi Ogueji, Andre Niyongabo Rubungo, Toan Q. Nguyen, Mathias Müller, André Müller, Shamsuddeen Hassan Muhammad, Nanda Muhammad, Ayanda Mnyakeni, Jamshidbek Mirzakhalov, Tapiwanashe Matangira, Colin Leong, Nze Lawson, Sneha Kudugunta, Yacine Jernite, Mathias Jenny, Orhan Firat, Bonaventure F. P. Dossou, Sakhile Dlamini, Nisansa de Silva, Sakine Çabuk Ballı, Stella Biderman, Alessia Battisti, Ahmed Baruwa, Ankur Bapna, Pallavi Baljekar, Israel Abebe Azime, Ayodele Awokoya, Duygu Ataman, Orevaoghene Ahia, Oghenefego Ahia, Sweta Agrawal, Mofetoluwa Adeyemi
PDF Cite Project DOI TACL HAL arXiv
A Monolingual Approach to Contextualized Word Embeddings for Mid-Resource Languages
We explore the impact of the training corpus on contextualized word embeddings in five mid-resource languages.
PDF Cite Dataset Project Video DOI ACL Anthology ACL 2020 HAL arXiv
A Monolingual Approach to Contextualized Word Embeddings for Mid-Resource Languages
Asynchronous Pipeline for Processing Huge Corpora on Medium to Low Resource Infrastructures
We propose a new pipeline to filter, clean and classify Common Crawl by language, we publish the final corpus under the name OSCAR.
PDF Cite Code Dataset Project Slides DOI CMLC-7 Website HAL
Asynchronous Pipeline for Processing Huge Corpora on Medium to Low Resource Infrastructures