Books  |  ,   |  September 29, 2020

How Humans Judge Machines

Book by by Cesar A. Hidalgo, Diana Orghiain, Jordi Albo Canals, Filipa De Almeida and Natalia Martin.
Published by MIT Press.
256 pages.

How people judge humans and machines in scenarios involving labor displacement, algorithmic bias, policing, privacy violations, natural disasters, and more.

How would you feel about losing your job to a machine? How about a tsunami alert system that fails? Would you react differently to acts of discrimination depending on whether they were carried out by a machine or by a human? What about public surveillance?

How Humans Judge Machines compares people’s reactions to actions performed by humans and machines. Using data collected in dozens of experiments, this book reveals the biases that permeate human-machine interactions.

Are there conditions in which we judge machines unfairly? Is our judgment of machines affected by the moral dimensions of a scenario? Is our judgment of machine correlated with demographic factors such as education or gender?

César Hidalgo and colleagues use hard science to take on these pressing technological questions. Using randomized experiments, they create revealing counterfactuals and build statistical models to explain how people judge artificial intelligence and whether they do it fairly. Through original research, How Humans Judge Machines bring us one step closer to understanding the ethical consequences of AI.

Book introduction. Runtime 1 minute.
How Humans Judge Machines. Webinar with Cesar A. Hidalgo. September 9, 2020. Runtime 87 minutes.
Can You Judge Artificial Intelligence? Presentation by Cesar A. Hidalgo. September 12, 2019. Runtime 33 minutes.

Table of Contents

  • Executive Summary — A short summary of the main findings.
  • Chapter 0: Introduction — Why should we worry about the way in which people judge machines? Together with chapter 1, the introduction motivates the study and puts the experiments in the context of some of the relevant literature.
  • Chapter 1: The Ethics of Artificial Minds — This chapter introduces basic concepts used throughout the book: moral status and agency, intent, and the moral dimension of the moral foundations theory of social psychology. This chapter also introduces the basic experimental technique used in subsequent chapters.
  • Chapter 2: Unpacking the Ethics of AI — This chapter explores scenarios involving emergency decisions with uncertain outcomes, algorithmic creativity, self-driving cars, and patriotism. It provides the first examples of people judging humans and machines differently, exhibiting both bias in favor and against machines.
  • Chapter 3: Judged by Machines (Algorithmic Bias) — This chapter presents experiments related to algorithmic bias in human resources, university admissions, and police scenarios. This chapter also discusses basic ideas about fairness and theoretical work attempting on the mathematical foundations of fairer algorithms.
  • Chapter 4: In the Eye of the Machine (Privacy) — This chapter explores experiments related to privacy using scenarios involving camera systems and personal data. This chapter also discusses basic concepts in privacy, such as anonymity and privacy preserving data collection methods.
  • Chapter 5: Working Machines (Labor Displacement) — This chapter compares people’s reactions to labor displacement attribute to technology and humans. It also discusses recent literature on labor displacement, automation, and labor precarization.
  • Chapter 6: Moral Functions — This chapter uses statistics to model the behavior observed across experiments to uncover general principles governing differences in the way people judge humans and machines. It also explores the average effects of demographics (e.g. education, gender) in people’s judgments of humans and machines.
  • Chapter 7: Liable Machines — This chapter concludes by connecting the work presented before with ideas from science fiction, mathematics, and law.
  • Appendix and Additional Scenarios — The statistical appendix contains additional details about the data collection methodology plus dozens of new scenarios which were not discussed in the main text of the book.

Free digital pdf edition of the book available at

Hardcover print edition available in bookstores on February 2, 2021.

About the Authors

  • César A. Hidalgo leads the Collective Learning group at the Artificial and Natural Intelligence Toulouse Institute (ANITI) at the University of Toulouse. He also holds appointments at the University of Manchester and Harvard University. Hidalgo has authored dozens of peer reviewed publications and three books: Why Information Grows ,The Atlas of Economic Complexity, and How Humans Judge Machines. Between 2010 and 2019, Hidalgo led the Collective Learning group at MIT. He holds a Ph.D. in physics from the University of Notre Dame.
  • Diana Orghian is a Social Psychologist and Researcher at the University of Lisbon. Her research focuses on understanding how people perceive the personality and appearance of others, and on how people evaluate artificial agents. Orghian holds a Ph.D. in Psychology from the University of Lisbon (2017), was a visiting scholar at New York University (2015) and Harvard (2016), and a Postdoctoral Fellow at the the MIT (2017-2019).
  • Filipa de Almeida is a psychologist with a PhD in Social Cognition from the University of Lisbon. During her studies, she was a visiting student at the University College London and at MIT . Recently she completed a postdoctoral fellowship in consumer psychology at the University of Lisbon. Currently she is an invited assistant professor at Católica Lisbon School of Business and Economics, teaching at the undergraduate, masters and executive Masters levels, and supervising master students working in social power and decision making. Her research focuses on the effects of social power and on the perception of artificially intelligent agents.
  • Jordi Albo-Canals is a researcher and entrepreneur with expertise on human factors engineering, embodied social agents, AI-based education, health technologies, and cloud computing. He has worked on research and technology transfer at La Salle University, Tufts University, NTT DATA Corporation, and between the Barcelona Children’s Hospital Foundation, and the company he co-founded, Lighthouse-DIG, LLC.
  • Natalia Martin is a marketeer and publicist with a Master Degree from the ESIC Business & Marketing School. Martin has experience working for international companies, such as as NTT DATA, where she was part of the BXH team at the MIT Media Lab. Martin works on understanding how new marketing tools can influence a user’s perception of services and products.