Where’s the Bias? Developing Effective Model Governance
Galen Harrison, Natasha Duarte, Joseph Lorenzo Hall. NIPS 2018 Workshop on Challenges and Opportunities for AI in Financial Services: the Impact of Fairness, Explainability, Accuracy, and Privacy. 2018
Abstract - Understanding and mitigating the discriminatory behavior of a machine learning algorithm and a system that uses a machine learning algorithm are related, but distinct tasks. We argue that regulatory procedures that are supposed to address discrimination in systems that use machine learning, but which only consider the algorithm or model will miss significant sources of discrimination. We support this argument by considering applications of machine learning to financial services, and showing how bias could arise both through areas of specific machine learning concern and broader issues of design. Because bias may extend from both these concerns, we argue that an effective model governance regime cannot limit itself to scrutinizing the model.