CategoricalCrossEntropy.md (834B)
1 # Categorical Cross Entropy 2 3 ML D6 4 5 **Definition:** Categorical cross entropy is a loss calculation used for classification algorithms. 6 7 Categorical cross entropy is calculated by summing the log of y_i log(p_i) and multiplying by -1 where y_i is the expected classification (1 is true 0 false) and p_i is the probability output of the model. 8 9 In essence, this is the negative sum of the logs of all probability outputs where the input should be a part of the class. All other classes are ignored so if another class has a .8 probability output it is multiplied by 0 thus not having an effect on the categorical cross entropy of the model. 10 11 Cross entropy is the idea that we want to have the difference between the true probability and the estimated probability. This can be stated more complexly, but in the end it always uses logs.