Shadowfetch News — AI news. Real coverage.

Models

Meta puts Llama 4 on a multimodal mixture-of-experts track

The official model cards identify Llama 4 Scout and Llama 4 Maverick as natively multimodal mixture-of-experts models, with stated parameter counts, context lengths, and training-data scale.

a close up of a piece of luggage with text on it
Photo by Google DeepMind on Unsplash

Meta's official Llama 4 model cards list April 5, 2025 as the model release date and describe the Llama 4 collection as natively multimodal models for text and multimodal experiences.

The model cards say Llama 4 Scout and Llama 4 Maverick use a mixture-of-experts architecture and incorporate early fusion for native multimodality.

The model cards identify Llama 4 Scout as a 17 billion-parameter model with 16 experts and Llama 4 Maverick as a 17 billion-parameter model with 128 experts.

The official Llama 4 Scout model card lists Scout at 17B activated parameters and 109B total parameters, with multilingual text and image input, multilingual text and code output, a 10M context length, about 40T training tokens, and an August 2024 knowledge cutoff.

The official Llama 4 Maverick model card lists Maverick at 17B activated parameters and 400B total parameters, with multilingual text and image input, multilingual text and code output, a 1M context length, about 22T training tokens, and an August 2024 knowledge cutoff.

The same model-card table says the training data mixes publicly available data, licensed data, and information from Meta's products and services, including publicly shared Instagram and Facebook posts and people's interactions with Meta AI.

The model cards say the Llama 4 collection supports Arabic, English, French, German, Hindi, Indonesian, Italian, Portuguese, Spanish, Tagalog, Thai, and Vietnamese.

The model cards describe the release as static models trained on an offline dataset and say future tuned-model versions may be released as Meta improves model behavior with community feedback.

Sources

  1. meta-llama/Llama-4-Scout-17B-16E base model card
  2. meta-llama/Llama-4-Scout-17B-16E-Instruct model card
  3. meta-llama/Llama-4-Maverick-17B-128E-Instruct model card

From Shadowfetch