pre‐training

In machine learning, a training step that trains a general‐purpose model (sometimes called a foundation model) on publicly‐available data. Pre‐training is often followed by fine‐tuning to equip the model with task‐specific information.