OnlineLearning.md (1183B)
1 2 ML CH1 3 4 **Definition:** Online learning is the process of learning as a model is fed new data. 5 6 This paradigm is in contrast with [OfflineLearning](OfflineLearning.md) also known as batch learning where all data is trained on at the start and then the learned behavior is acted upon in a static way in perpetuity. 7 8 When using online learning, you can use either individual samples to train on or mini-batches which are groupings of samples. 9 10 This method can be used to train models on data where not all of the training data can fit in the machine's memory. This is referred to as out-of-core learning. When doing this, the algorithm loads in a mini-batch, runs a training step, and then repeats for all of the dataset. This may be confusing as the learning is done offline, but considering online learning more as incremental learning can help resolve this thought issue. 11 12 The rate at which these models adapt to new information is called the [LearningRate](LearningRate.md) When this is high they respond quickly to new data at the cost of losing old data faster. It is a balancing game. Counter to this, with a low learning rate we have more 'inertia' from old data in the set.