Book chapter by Elana Zeide. Published in the book The Oxford Handbook of Ethics of AI.
The use of artificial intelligence (AI) in education immediately conjures the fantasy of robot teachers – as well as fears that robot teachers will replace their human counterparts. However, AI tools impact much more than instructional choices. Personalized learning systems take on a whole host of other educational roles, fundamentally reconfiguring education in the process. They not only perform the functions of “robot teachers,” but make pedagogical and policy decisions typically left to teachers and policymakers. Their design, affordances, analytical methods, and visualization dashboards construct a technological, computational, and statistical infrastructure that literally codifies what students learn, how they are assessed, and what standards they must meet.
However, school procurement and implementation of these systems are rarely part of public discussion. Students and parents cannot observe, let alone challenge, the many decisions that shape academic outcomes and subsequent life trajectories. Teacher and school officials cannot assess the accuracy, efficacy, and fairness of individual determinations or group outcomes. Private companies, rather than community members and public servants, set pedagogy and policy in practice. If they are to remain relevant to the educational process itself, as opposed to just its packaging and context, schools and their stakeholders must be more proactive in demanding information from technology providers and setting internal protocols to ensure effective and consistent implementation. Those who outsource instructional functions should do so with sufficient transparency mechanisms in place to ensure professional oversight guided by well-informed debate.