ECS171-L5

📗 -> 10/09/25: ECS171-L5

---

[Lecture Slide Link]

🎤 Vocab

❗ Unit and Larger Context

Small summary

✒️ -> Scratch Notes

Probabilistic Classification Approach

Simplest prob classifier: Naive Bayes

• Generative method (as opposed to sigmoid)
  Three concepts:
• PRIOR prob
• POSTERIOR prob
• LIKELIHOOD
  However, not robust
• Classifier assumes features are independent of any other feature. Thats why the classifier is naive
  Despite simplicity, works well, especially when:
• Features are not highly correlated, or function is complex
• Number of data points is modest, so more complex classifiers risk over-fitting

Adressing Curse of Dimensionality:

1 Solution: Dimensionality Reduction
Another solution: Assume feature independence. Curse of dimensionality comes from combining features.

• Of course, not true so losing info. However, not losing THAT much.
  • He gave example of optimal classifier getting 80% while naive bayes gets 72%, while optimal classifier takes forever to train as compared to naive bayes taking seconds

Bayes Theorem

• - Prior
• - Posterior
• - Likelihood
• - Evident

In practice:

• • Substitute Y=1 for a class (say house with two floors) and x=5 for a feature (having 5 bedrooms)
    • It helps it make sense if you say it out loud

There was a really good image in the slides visualizing posterior ~ likelihood * prior, review it

Assuming independence, instead of doing a rough joint probability equation we get

Discriminant Function

• Probability of class 1 vs class 2

For more than 2 classes:

• Calculate for Class 1 vs Not Class 1, then repeat for all other classes
• Take the maximum probability as classification

Classification Performance

Cross-validation

Statistical Measures:

• True Positive Rate (TPR) = Recall = Sensitivity = TP/(TP+FN)
• 1-False Positive Rate (FDR) = Specificity = TN(TN+FP)
• Accuracy = (TP+TN) / (P+N)
• Precision = TP/(TP+FP)

Receiver Operating Characteristics (ROC Curves)
and Precision-Recall Curves

F1 score - Harmonic Mean of Precision and Recall

🧪 -> Refresh the Info

Did you generally find the overall content understandable or compelling or relevant or not, and why, or which aspects of the reading were most novel or challenging for you and which aspects were most familiar or straightforward?)

Did a specific aspect of the reading raise questions for you or relate to other ideas and findings you’ve encountered, or are there other related issues you wish had been covered?)

🔗 -> Links

Resources

• Put useful links here

Connections

• Link all related words