Project 3: End-to-End Card Fraud Detection System

A classic ML project that stands out when done end to end: real-time features, a cost-based threshold, an API, explainability and drift monitoring. Perfect for banking, fintech and payments roles.

What you'll build

• A model trained on a public, highly imbalanced card-transaction dataset (e.g. the Kaggle credit-card fraud dataset or a synthetic simulator).
• A FastAPI /score endpoint returning probability, decision (approve / OTP / decline) and top 3 reason codes.
• A dashboard with PR curve, cost vs threshold, and a weekly drift (PSI) report.

Architecture

Raw transactions → feature pipeline (velocity, amount vs avg, time features) → train (LightGBM, out-of-time split)
   → calibrate → choose threshold by cost → save model + feature spec
API request → compute online features → model → policy (bands) → response {p, decision, reasons}
Monitoring job → PSI per feature, score distribution, alert rate → report

Tech stack

pandas, LightGBM or XGBoost, scikit-learn (calibration, metrics), SHAP, FastAPI, Docker, Evidently or custom PSI code, Streamlit.

Step by step

1. Explore: fraud rate, amount distribution, time patterns; there's only about 0.17% fraud in the Kaggle data.
2. Features: log amount, hour, card-level velocity (if IDs exist), amount vs rolling mean, plus the PCA features (Kaggle).
3. Split by time: train on the first 80%, test on the last 20%.
4. Train: LightGBM with scale_pos_weight; compare against logistic regression.
5. Evaluate: PR-AUC, recall at 1% alert rate, value-weighted recall.
6. Threshold: minimise the expected cost (FN = amount, FP = ₹50 review + friction); define three bands.
7. Explain: SHAP top features mapped to readable reasons.
8. Serve: FastAPI plus Docker; add input validation (Pydantic) and a latency test.
9. Monitor: compute PSI on new data weekly; alert if PSI > 0.25.

Starter code

import numpy as np, pandas as pd, shap, lightgbm as lgb
from sklearn.metrics import average_precision_score
df = pd.read_csv("creditcard.csv"); df["amount_log"] = np.log1p(df.Amount); df["hour"] = (df.Time // 3600) % 24
cut = int(len(df) * 0.8); tr, te = df.iloc[:cut], df.iloc[cut:]
X = [c for c in df.columns if c not in ("Class", "Time", "Amount")]
m = lgb.LGBMClassifier(n_estimators=500, learning_rate=0.03, scale_pos_weight=200).fit(tr[X], tr.Class)
p = m.predict_proba(te[X])[:, 1]; print("PR-AUC", average_precision_score(te.Class, p))
ts = np.linspace(.01, .99, 99)
cost = [((p < t) & (te.Class == 1)).dot(te.Amount) + ((p >= t) & (te.Class == 0)).sum() * 50 for t in ts]
T = ts[int(np.argmin(cost))]; print("best threshold", T)
explainer = shap.TreeExplainer(m)

# api.py
from fastapi import FastAPI; from pydantic import BaseModel
app = FastAPI()
class Txn(BaseModel): features: dict
@app.post("/score")
def score(t: Txn):
    x = pd.DataFrame([t.features])[X]; p = float(m.predict_proba(x)[0, 1])
    sv = explainer.shap_values(x)[0]; reasons = [X[i] for i in np.argsort(-sv)[:3]]
    decision = "decline" if p > 0.9 else "otp" if p > T else "approve"
    return {"p": p, "decision": decision, "reasons": reasons}

Deploy it

Run Docker on Render or Railway (API) plus a Streamlit dashboard. Include a Makefile, tests and a README with the metrics table and the cost-threshold chart.

Put it on your resume

• Built a real-time fraud scoring API (LightGBM + FastAPI) on a 0.17%-positive dataset, achieving PR-AUC 0.86 and catching 82% of fraud value at a 1% alert rate.
• Implemented cost-based thresholds, SHAP reason codes and weekly PSI drift monitoring; containerised with Docker.

Interview questions you'll get

• Why PR-AUC and not accuracy or ROC-AUC? How did you pick the threshold?
• How do you avoid leakage in velocity features?
• What do you do when labels (chargebacks) arrive 60 days late?
• How would you explain a decline to a customer or regulator?

Study the AI in Finance & Fraud track for the theory behind each step.

Stretch goals

• Graph features (shared devices or merchants) on a simulated dataset.
• Online learning or champion/challenger setup.
• A GenAI analyst assistant that summarises flagged cases.