Full Papers

Preserving the memory of the first wave of COVID-19 pandemic: Crowdsourcing a collection of image search queries

Kalia Orphanou, Evgenia Christoforou, Jahna Otterbacher, Monica Lestari Paramita and Frank Hopfgartner

We don't need no expectations: on the observer-expectancy effect in crowdsourcing

Eddy Maddalena, Alessandro Checco and Elena Simperl

Designing human-in-the-loop approaches for closed deferral pipelines

Vijay Keswani, Matthew Lease and Krishnaram Kenthapadi

Short Papers

A Practical Framework for Building and Managing Digital Crowdsourcing Communities

Wuraola Fisayo Oyewusi, Olubayo Adekanmbi, Olawale Victor Abimbola, Chinazo Anebelundu, Oluwaseun Abel Ofinni and Sandra Oriji

Can Survey Methodology Improve Machine Learning? A Proposed Pilot Study Examining the Effect of Stratified Sampling on Color Naming Model Accuracy and Diversity

Anurata Hridi, Johanne Christensen and Benjamin Watson

Abstracts

Efficacy of Bengali Captioning Guideline in TV programs

Tanvir Ahmed and Akhter Al Amin

Mandating model cards and metrics before allowing to fork (pre-trained model) repos on Github

Sundaraparipurnan Narayanan

The Effect of Individual Cognitive Style when Mitigating Confirmation Bias during Search

Alisa Rieger and Nava Tintarev

Detecting Bias in Complex Annotations

Alexander Braylan and Matthew Lease

Investigating Biases in Crowdsourced Hate Speech Annotation

Danula Hettiachchi, Indigo Holcombe-James and Anjalee de Silva

A Quantum Approach to Crowdsourcing

Kevin Roitero

Bias in Computer Vision Algorithms for Natural Disaster Damage Assessment

Thomas Chen