OpenAI Blog · Apr 13, 2022
Measuring Goodhart’s law
Reviewed by Errol Vogt, Site support technician & online learning analyst · original summary · editorial policy
Measuring Goodhart’s law. Goodhart’s law famously says: “When a measure becomes a target, it ceases to be a good measure.” Although originally from economics, it’s something we have to grapple with at OpenAI when figuring out how to optimize objectives that are difficult or costly to measure. This update is relevant for small-office operators tracking changes in their tools.
Operator takeaway: For operators: review whether 'Measuring Goodhart’s law' affects your current setup before relying on it in production.
ai