Phi-2: The surprising power of small language models

Key Insights Behind Phi-2 The massive increase in the size of language models to hundreds of billions of parameters has unlocked a host of emerging capabilities that have redefined the landscape of natural language processing. A question remains whether such emergent abilities can be achieved at a smaller scale using strategic choices for training, e.g., data selection. Our line of work with the P

[takc923]

2.7Bでsmallなのか

[secondlife]

良質な学習データを使い、2.7Bで7Bモデルを超える性能。HFで公開するのではなく、azure上で試せる、というのが流れを感じる。