Paper page - EMO: Pretraining Mixture of Experts for Emergent Modularity
… We’re releasing the model, matched baseline, code, technical report, and an interactive visualization of the learned expert clusters. 🧠 Models: https://huggingface.co/collections/allenai/emo 📝 Blog: https://allenai.org/blog/emo 📄 Tech report: https://allenai.org/papers/emo 📊 Visualization: https://… …