Towards Considering and Documenting Algorithmic Fairness in the Data Science Workflow
Galen Harrison, Julia Hanson, Blase Ur. Proceedings of the Workshop on Technology and Consumer Protection, 2019.
Abstract - Recent pushes to replace human decision-makers with machine learning models have surfaced concerns about algorithmic fairness.
These concerns have led to a quickly growing literature on defining fairness and making models transparent.
In practice, the data scientist building the model necessarily must make difficult tradeoffs choosing between imperfect models, balancing different definitions of fairness with accuracy and other considerations.
Because these choices have ethical dimensions, there is a need to better support these choices and both document and justify them for the public.
We outline a research agenda towards better visualizing difficult fairness-related tradeoffs between competing models, empirically quantifying societal norms about such tradeoffs, and documenting these decisions.
We outline how the best practices that result could enable a consumer protection framework for accountable fairness.