A new picture of AI corrigibility built on respect for authority
The author proposes restraining an AI from overstepping reserved human powers before it acts for its principal.
AGI & Safety#04
AGI & Safety
Image: AI Alignment Forum
The short version
- The author argues that existing definitions of corrigibility are deficient in one way or another.
- The article says the fundamental concept underlying corrigibility is respect for authority, using a healthcare proxy as an intuitive guide.
- The framework first restrains the agent from overstepping or interfering with reserved human authority, then has it act on the principal's behalf as the principal would judge best.
- The article presents an initial formalism and proves theorem-level guarantees in three increasingly realistic formal settings.
Why it matters
Corrigibility is central to theoretical AI alignment, and the article says no one can agree on what it means. The framework aims to match the original 2015 desiderata and to support the formal safety guarantees the concept needs. Developers working on powerful AI systems would find the formal settings relevant to how such goals could be specified.
“the fundamental concept underlying corrigibility is respect for authority”