Foundation Model
A large model trained on broad data that can be adapted to many downstream tasks via fine-tuning or prompting.
Definition
Coined by the Stanford HAI group in 2021, "foundation model" describes large AI models trained on broad, diverse datasets at scale that exhibit emergent capabilities and can be adapted to many tasks. They are the "foundation" on which many applications are built.
Foundation models demonstrate two key properties: emergent capabilities (behaviours that appear unexpectedly at scale, not present in smaller models) and homogenisation (many different applications built on the same base model). This raises both opportunities (shared investment, rapid iteration) and risks (common failure modes, concentration of power).
Examples span language (GPT-4, LLaMA), vision (CLIP, ViT), multimodal (GPT-4V, Gemini), audio (Whisper), and science (AlphaFold for proteins).
Examples
- GPT-4
- Claude 3
- Gemini
- LLaMA 3
- CLIP
- AlphaFold 2
Companies using this
Related Terms
Large Language Model (LLM)
A transformer-based AI system trained on billions of tokens of text, capable of generating, reasoning about, and transforming language.
Pre-training
The initial phase of training a large model on massive, general datasets before task-specific fine-tuning.
Fine-tuning
Continuing training of a pre-trained model on domain-specific data to specialise it for a particular task.
Multimodal AI
AI systems that process and generate multiple data types — text, images, audio, and video — simultaneously.