Machine Learning with Scikit-Learn
In traditional software development, programmers write explicit rules: Rules + Data = Answers. In Machine Learning (ML), algorithms learn statistical patterns from historical data: Data + Answers = Rules (Model)!
The 3 Core Machine Learning Paradigms:
- Supervised Learning: The training dataset includes both input Features ($X$) and ground-truth Target Labels ($y$).
- Regression: Predicting a continuous numeric value (e.g. house prices, stock prices, temperature).
- Classification: Predicting a discrete category (e.g. Spam vs Not Spam, Tumor Malignant vs Benign).
- Unsupervised Learning: The dataset contains only features ($X$) without labels ($y$). The model discovers hidden structures, groupings, or clusters (e.g. Customer Segmentation with K-Means).
- Reinforcement Learning: An agent learns optimal actions through trial-and-error rewards and penalties in an environment (e.g. self-driving cars, game AI).
# Complete End-to-End Supervised Learning Simulation (Linear Regression):
class SimpleLinearRegressionModel:
"""Manual implementation of Linear Regression (y = mx + c)."""
def __init__(self):
self.slope_m = 0.0
self.intercept_c = 0.0
def fit(self, X_train, y_train):
"""Learns optimal slope and intercept using Ordinary Least Squares."""
n = len(X_train)
mean_x = sum(X_train) / n
mean_y = sum(y_train) / n
# Calculate slope m:
num = sum((x - mean_x) * (y - mean_y) for x, y in zip(X_train, y_train))
den = sum((x - mean_x) ** 2 for x in X_train)
self.slope_m = num / den
self.intercept_c = mean_y - (self.slope_m * mean_x)
print(f"🤖 Model Trained: y = {self.slope_m:.2f}x + {self.intercept_c:.2f}")
def predict(self, X_test):
return [self.slope_m * x + self.intercept_c for x in X_test]
# 1. Dataset: Years of Experience vs Salary (in thousands):
X_experience = [1.0, 2.0, 3.0, 4.0, 5.0]
y_salary = [40.0, 50.0, 65.0, 75.0, 90.0]
# 2. Train Model:
regressor = SimpleLinearRegressionModel()
regressor.fit(X_experience, y_salary)
# 3. Predict Salaries for 6 and 8 Years of Experience:
unseen_candidates = [6.0, 8.0]
predictions = regressor.predict(unseen_candidates)
print("\n--- 📈 Salary Predictions for Unseen Data ---")
for exp, pred in zip(unseen_candidates, predictions):
print(f"• Experience: {exp} yrs -> Predicted Salary: ₹{pred:,.2f}k")In Scikit-Learn, every model follows the exact same 3-step interface: (1) model = LinearRegression(), (2) model.fit(X_train, y_train), and (3) y_pred = model.predict(X_test).
For classification tasks, Accuracy alone is often misleading (especially on imbalanced datasets where 99% of samples belong to one class):
| Metric | Formula | When It Matters Most |
|---|---|---|
| Accuracy | $\frac{TP + TN}{TP + TN + FP + FN}$ | Balanced datasets with equal class importance. |
| Precision | $\frac{TP}{TP + FP}$ | When False Positives are costly (e.g. Email Spam filter flagging a critical work email). |
| Recall (Sensitivity) | $\frac{TP}{TP + FN}$ | When False Negatives are catastrophic (e.g. Cancer detection: missing a sick patient). |
| F1-Score | $2 \times \frac{\text{Precision} \times \text{Recall}}{\text{Precision} + \text{Recall}}$ | Harmonic mean balancing Precision and Recall. |
# Classification Confusion Matrix & Metrics Calculation:
# True Positives, False Positives, False Negatives, True Negatives:
TP = 85 # Correctly predicted cancer
FP = 10 # Healthy patient incorrectly flagged
FN = 5 # Sick patient missed (CRITICAL!)
TN = 900 # Healthy patient correctly identified
accuracy = (TP + TN) / (TP + FP + FN + TN)
precision = TP / (TP + FP)
recall = TP / (TP + FN)
f1 = 2 * (precision * recall) / (precision + recall)
print("--- 🩺 Medical Diagnostic Model Evaluation Metrics ---")
print(f"• Accuracy: {accuracy * 100:.2f}%")
print(f"• Precision: {precision * 100:.2f}% (Out of flagged patients, {precision*100:.1f}% actually had illness)")
print(f"• Recall: {recall * 100:.2f}% (Model successfully caught {recall*100:.1f}% of all actual cases)")
print(f"• F1-Score: {f1:.4f}")Once a model is trained, save its learned weights using import joblib; joblib.dump(model, "model.joblib"). In your web server (FastAPI/Django), load it with model = joblib.load("model.joblib") for instantaneous inference without retraining.
Evaluating a model on the same data it was trained on produces falsely optimistic scores (a student memorizing exam answers). Always evaluate generalization performance strictly on a separate unseen test set generated via train_test_split(X, y, test_size=0.20).
Calculate the Mean Squared Error (MSE) between actual house prices and predicted values: actual = [50, 80, 120], predicted = [48, 85, 115].
actual = [50, 80, 120]
predicted = [48, 85, 115]
mse = sum((act - pred) ** 2 for act, pred in zip(actual, predicted)) / len(actual)
rmse = mse ** 0.5
print(f"Mean Squared Error (MSE): {mse:.2f}")
print(f"Root Mean Squared (RMSE): ₹{rmse:.2f} Lakhs average prediction error")Q What is the purpose of StandardScaler in Scikit-Learn?
StandardScaler scales each feature to have a mean of 0 and a standard deviation of 1. This prevents features with large numeric scales (like salary in ₹1,00,000) from dominating features with small scales (like age in 25) in distance-based algorithms like SVM, K-Means, and KNN.
Q What is K-Fold Cross Validation?
Cross-validation splits the training dataset into K equal folds, training on K-1 folds and validating on the remaining fold K times to ensure model stability across all data slices.
Q What is the difference between Bagging and Boosting in Ensemble Learning?
Bagging (e.g. Random Forest) trains multiple independent models in parallel and averages their predictions. Boosting (e.g. XGBoost, LightGBM) trains models sequentially, where each new model focuses on correcting the errors made by previous models.