Discover published sets by community

Explore tens of thousands of sets crafted by our community.

Categories

Computer Science

Machine Learning

Activation Functions in Neural Networks

Bias-Variance Tradeoff

Computer Vision Fundamental Concepts

Cross-validation Techniques

Data Preprocessing for Machine Learning

Deep Learning Fundamentals

Data Imbalance Handling

Dimensionality Reduction Methods

Ensemble Learning Techniques

Evaluation Metrics for Classification

Feature Engineering Techniques

Generative Models

Gradient Descent Variants

Hyperparameter Tuning

Machine Learning Best Practices

Loss Functions Explained

Machine Learning Security and Privacy

Machine Learning Pipelines

Machine Learning Frameworks

Model Explainability and Interpretability

Multiclass Classification Strategies

Natural Language Processing Terminology

Model Selection Criteria

Neural Network Architectures

Performance Optimization in Machine Learning

Time Series Analysis Techniques

Reinforcement Learning Concepts

Statistical Tests for Model Comparison

Loss Functions Explained

Flashcards

0/10

Still learning

Kullback-Leibler Divergence (KLD)

KLD is a measure of how one probability distribution diverges from a second, expected probability distribution, used in various tasks like variational autoencoders.

KLD = \sum_i P(i) \log\left(\frac{P(i)}{Q(i)}\right)

Cross-Entropy Loss

Cross-Entropy quantifies the difference between two probability distributions for a given random variable or set of events, widely used in classification tasks.

CE = -\sum_{i} y_i \log(\hat{y}_i)

Mean Squared Error (MSE)

MSE measures the average squared difference between estimated values and the actual value, commonly used in regression problems.

MSE = \frac{1}{n}\sum_{i=1}^{n}(Y_i - \hat{Y}_i)^2

Huber Loss

Huber Loss is less sensitive to outliers in data than MSE and combines the properties of MSE and MAE. It's quadratic for small errors, and linear for large errors.

Huber = \begin{cases}\frac{1}{2}(Y_i - \hat{Y}_i)^2 & \text{for } |Y_i - \hat{Y}_i| \leq \delta,\\ \delta |Y_i - \hat{Y}_i| - \frac{1}{2}\delta^2 & \text{otherwise.}\end{cases}

Poisson Loss

Poisson Loss is used when modeling count data in regression problems. It assumes the target values, typically counts, are Poisson distributed and the loss is the log-likelihood of that distribution.

Poisson = \sum_i (\hat{Y}_i - Y_ilog(\hat{Y}_i))

Softmax Loss

Softmax Loss, or Multinomial Logistic Loss, extends Logistic Loss to multi-class classification problems. It applies the softmax function to the raw output scores, which are then used in the Cross-Entropy Loss.

Softmax = -\sum_{i} y_i \log(\frac{e^{z_i}}{\sum_{j} e^{z_j}})

Hinge Loss

Hinge loss is often used for binary classification tasks, such as Support Vector Machines. It's defined as the maximum of zero or the difference between one and the product of the true label and the predicted score.

Hinge = \max(0, 1 - y_i \cdot \hat{y}_i)

Logistic Loss

Also known as Binary Cross-Entropy Loss, it's used for binary classification problems. It measures the loss between predicted probabilities and target binary state.

Logistic = -y_i \log(\hat{y}_i) - (1 - y_i) \log(1 - \hat{y}_i)

Mean Absolute Error (MAE)

MAE measures the average of the absolute errors (i.e., the absolute differences between the predictions and the actual observations), commonly used in regression.

MAE = \frac{1}{n}\sum_{i=1}^{n}|Y_i - \hat{Y}_i|

Focal Loss

Focal Loss modifies Cross-Entropy Loss to focus more on hard-to-classify examples, it is particularly useful for imbalanced datasets in object detection.

Focal = -\alpha(1 - \hat{y}_i)^\gamma \log(\hat{y}_i)

Know

Still learning

Click to flip

Know