Weak-to-strong generalization

We present a new research direction for superalignment, together with promising initial results: can we leverage the generalization properties of deep learning to control strong models with weak supervisors? A core challenge for aligning future superhuman AI systems (superalignment) is that humans will need to supervise AI systems much smarter than them. We study a simple analogy: can small models

[secondlife]

人間は将来能力がAIより低くなると予想されるが、その状態でもうまくAIを監督するには、をGPT-2(能力が低い)がGPT-4(能力が高い)を監督するアナロジーに見立ててシュミレーション学習。なるほど。