Confusion Matrix

Untitled

Question
  • Proportion of correct predictions(TP, TN) made by model out of a total number of cases (correct: TP, TN+in correct: FP, FN)?
  • Formula: (TP+TN)/(TP+TN+FP+FN)
  • Usage: Used in cases where classes are well balanced.
  • When not to use?: If the target variable is sparse
Precision
  • Proportion of model's correct Positive predictions(TP) out of total Positive predictions (TP+FP)
  • In my words: the model made positive predictions(TP, FP); I want to know what proportion of them were actually positive(TP)
  • Formula: TP/(TP+FP)
  • Usage: Use when you want to be very sure of your prediction. In case of stock investments, say we develop a model that determines if a stock will be a good investment or not. If we were to pour our entire net worth into this one stock, we would better hope that our model is right. Precision would be the best metric to use here because it determines the correctness of our model. We can afford to miss a few profitable stock investments here and there (so recall score is not as important), as long as our money is going to an appreciating stock correctly predicted by our model.
Recall
  • Proportion of actual positives(TP, FN) correctly classified as positives(TP)
Average Precision

Average Precision (AP) is defined as an area under the precision/recall curve.

The 5 Classification Evaluation metrics every Data Scientist must know

Multi-Class Metrics Made Simple, Part II: the F1-score

Confusion Matrix

Machine Learning Basics: Confusion Matrix & Precision/Recall Simplified | By Dr. Ry @Stemplicity

Based on Confusion Matrix

1. Acurracy

    $(TP+TN)/(TP+TN+FP+FN)$

2. Error rate
3. precision
4. Sensitivty = TPR(True Positive Rate)= Recall = TP/(TP+FN)
    - True positive rate or TPR is just the proportion of trues we are capturing using our algorithm.
    - TPR = TP/ALL POSITIVES
5. Specificity
6. 1- Specificity = FPR(False Positive Rate)= FP/(TN+FP)
    - False positive rate or FPR is just the proportion of false we are capturing using our algorithm.
    - FPR = FP/ALL NEGATIVES
7. F1 Score = 1/2(1/P + 1/R) = 2*(P*R)/(P+R)
    - It is the harmonic mean of Precision and Recall

[https://www.dataschool.io/simple-guide-to-confusion-matrix-terminology/](https://www.dataschool.io/simple-guide-to-confusion-matrix-terminology/)

[Machine Learning Basics: Confusion Matrix & Precision/Recall Simplified | By Dr. Ry @Stemplicity](https://www.youtube.com/watch?v=CYy0TZ6OIDw&ab_channel=Stemplicity)

https://www.dlology.com/blog/simple-guide-on-how-to-generate-roc-plot-for-keras-classifier/

  • Metrics

    accuracy is not to be used
im balanced class
prevelance - how much of positive vlaues in data
positive predictive power
npv
xg boost - parameters