KAN: Kolmogorov-Arnold Networks

Inspired by the Kolmogorov-Arnold representation theorem, we propose Kolmogorov-Arnold Networks (KANs) as promising alternatives to Multi-Layer Perceptrons (MLPs). While MLPs have fixed activation functions on nodes ("neurons"), KANs have learnable activation functions on edges ("weights"). KANs have no linear weights at all -- every weight parameter is replaced by a univariate function parametriz

[lyiase]

Transformerレイヤーで使われるMLPブロックの置換候補の提案

[misshiki]

論文“MLP の有望な代替手段としてコルモゴロフ-アーノルド ネットワーク（KAN）を提案。MLPはノード(ニューロン) に固定の活性化関数を持つが、KANはエッジ (重み) に学習可能な活性化関数を持つ。”