Accepted Papers and Abstracts

Full Papers


Preserving the memory of the first wave of COVID-19 pandemic: Crowdsourcing a collection of image search queries

Kalia Orphanou, Evgenia Christoforou, Jahna Otterbacher, Monica Lestari Paramita and Frank Hopfgartner


We don't need no expectations: on the observer-expectancy effect in crowdsourcing

Eddy Maddalena, Alessandro Checco and Elena Simperl


Designing human-in-the-loop approaches for closed deferral pipelines

Vijay Keswani, Matthew Lease and Krishnaram Kenthapadi



Short Papers


A Practical Framework for Building and Managing Digital Crowdsourcing Communities

Wuraola Fisayo Oyewusi, Olubayo Adekanmbi, Olawale Victor Abimbola, Chinazo Anebelundu, Oluwaseun Abel Ofinni and Sandra Oriji


Can Survey Methodology Improve Machine Learning? A Proposed Pilot Study Examining the Effect of Stratified Sampling on Color Naming Model Accuracy and Diversity

Anurata Hridi, Johanne Christensen and Benjamin Watson



Abstracts


Efficacy of Bengali Captioning Guideline in TV programs

Tanvir Ahmed and Akhter Al Amin


Mandating model cards and metrics before allowing to fork (pre-trained model) repos on Github

Sundaraparipurnan Narayanan



The Effect of Individual Cognitive Style when Mitigating Confirmation Bias during Search

Alisa Rieger and Nava Tintarev


Detecting Bias in Complex Annotations

Alexander Braylan and Matthew Lease


Investigating Biases in Crowdsourced Hate Speech Annotation

Danula Hettiachchi, Indigo Holcombe-James and Anjalee de Silva


A Quantum Approach to Crowdsourcing

Kevin Roitero


Bias in Computer Vision Algorithms for Natural Disaster Damage Assessment

Thomas Chen