Has anyone else hit expert homogeneity collapse in small MoE models?

 (github.com)

1 points | by rfi-irfos 1 hour ago

1 comments