OpenAI Blog · Jun 13, 2017
Learning from human preferences
Reviewed by Errol Vogt, Site support technician & online learning analyst · original summary · editorial policy
Learning from human preferences. One step towards building safe AI systems is to remove the need for humans to write goal functions, since using a simple proxy for a complex goal, or getting the complex goal a bit wrong, can lead to undesirable and even dangerous behavior. In collaboration with DeepMind’s safety team, we’ve developed an algorithm which can infer what humans want by being told which of two proposed behaviors is better. This update is relevant for small-office operators tracking changes in their tools.
Operator takeaway: For operators: review whether 'Learning from human preferences' affects your current setup before relying on it in production.
ai