Reports  |  ,   |  October 2, 2020

Business Data Ethics: Emerging Trends in the Governance of Advanced Analytics and AI

Report written by Dennis Hirsch, Timothy Bartley, Aravind Chandrasekaran, Davon Norris, Srinivasan Parthasarathy and Piers Norris Turner. Published by The Ohio State University. 107 pages.


A few years ago Amazon, finding itself flooded with employment applications, developed an artificial intelligence (“AI”) tool to help it sort through the resumes. (Dastin 2018). It trained the tool on the resumes of its largely male workforce. As a result, the AI tool learned to penalize resumes that used the word “women’s,” as in “women’s tennis team.” According to media reports, Amazon’s recruiters looked at the tool’s recommendations when evaluating new hires, although they never relied entirely on the tool’s suggestions. Amazon subsequently spotted the gender bias problem and, unable to fix the AI tool, abandoned the project.

This brief story shows both the promise of advanced analytics and AI (here, sorting resumes quicker than a human could) and the hazards (here, perpetuating gender bias). It also shows the importance of governance mechanisms able to catch defects early and prevent them from hurting people.

In today’s algorithmic economy (Schneider 2018), it is vital that companies exercise such governance and so make their use of advanced analytics and AI fairer, more just, and more accountable. Government’s limited monitoring and enforcement resources make it unable to perform this vital task on its own. Effective corporate governance is an essential part of the solution.

The question is: how to motivate and achieve such governance? Here, the literature takes two main paths. Many authors focus on what it means for advanced analytics and AI to be “ethical.” Scholars (Floridi and Cowls 2019), think tanks and others have generated dozens of sets of ethical principles, and have encouraged businesses and other users of advanced analytics to align their practices with them. (Fjeld, Achten, Hilligoss, Nagy and Srikumar 2020). A second stream of writing proposes new forms of regulation that would require companies to bring their advanced analytics and AI into line with ethical or fairness standards (Balkin 2016; Calo 2014; Citron and Pasquale 2014; Hirsch 2020; Richards and Hartzog 2015; Barocas and Nissembaum 2014; Wachter and Mittelstadt 2019).

[ . . . ]

Table of Contents

Executive Summary

  • Advanced Analytics and AI Pose Threats
  • Companies See Data Ethics as Beyond Compliance Risk Mitigation
  • Companies Pursue Data Ethics to Further Their Interests and Values
  • A Corporate Data Ethics Program Has Three Main Components
  • Substantive Benchmarks Determine What is Ethical
  • Management Processes Achieve Substantive Goals
  • Technologies Reduce Potential Harms
  • Companies Can Use AI for the Social Good


  • Legal concepts and regulation
  • Normative principles
  • What is missing?


  • Interviews
  • Survey

Risks from Advanced Analytics

  • Privacy violations
  • Manipulation
  • Bias against protected classes
  • Increased power imbalances
  • Error
  • Opacity and procedural unfairness
  • Displacement of labor
  • Pressure to conform
  • Intentional, harmful use of analytics

What is “Corporate Data Ethics”?

Motivations – Why Do Companies Pursue Data Ethics?

  • Build reputation and sustain trust
  • Operating in the Shadow of the Law
  • European data protection law
  • Recruit and retain employees
  • Making risk-based decisions
  • Achieve competitive advantage
  • Fulfill corporate values

Drawing Substantive Lines

  • Published data ethics principles
  • Informal standards
  • Risk management frameworks
  • Formal principles in action
  • Policy: The missing middle layer

Managing For Data Ethics

  • Organizational infrastructure
    • Privacy office
    • Legal department
    • The Chief Data Ethics Officer
    • Philosophers in the corporate ranks
  • Spotting ethical issues
    • Touring the business units
    • Hub and spokes
    • External advisory group
    • Checklists
    • Sparking discussion about data ethics issues
    • Peer-to-peer conversations
  • Issue Resolution
    • Just in time data ethics
    • Triage and escalation
    • Cross-functional data ethics committee
    • Broader themes

Technological Solutions

  • Data Privacy and Anonymization
  • Algorithmic Fairness
  • The Clear and Pressing Need for Explainable Algorithms
  • Algorithmic Auditing of Data Use
  • Systems Technologies to Enable Governance

Pursuing the Social Good

The Research Team

  • Dennis Hirsch, The Ohio State University, Moritz College of Law (Principal Investigator)
  • Timothy Bartley, Washington University – St. Louis, Department of Sociology
  • Aravind Chandrasekaran, The Ohio State University, Fisher College of Business
  • Davon Norris, The Ohio State University, Department of Sociology
  • Srinivasan Parthasarathy, The Ohio State University, Department of Computer Science
  • Piers Norris Turner, The Ohio State University, Department of Philosophy