GPT-4 Architecture, Infrastructure, Training Dataset, Costs, Vision, MoE

OpenAI is keeping the architecture of GPT-4 closed not because of some existential risk to humanity but because what they’ve built is replicable. In fact, we expect Google, Meta, Anthropic, Inflection, Character, Tencent, ByteDance, Baidu, and more to all have models as capable as GPT-4 if not more capable in the near term. Don’t get us wrong, OpenAI has amazing engineering, and what they built is