Evaluation Playbook for fraud detection baseline #19

Code Example

from dataclasses import dataclass

@dataclass
class ExperimentResult:
  experiment: str
  objective: str
  score: float
  notes: str

def choose_candidate(results: list[ExperimentResult]):
  ranked = sorted(results, key=lambda item: item.score, reverse=True)
  best = ranked[0]
  return {
    "winner": best.experiment,
    "score": best.score,
    "objective": best.objective,
    "notes": best.notes,
  }

candidates = [
  ExperimentResult("baseline", "increase reproducibility", 0.74, "stable"),
  ExperimentResult("feature_set_b", "increase reproducibility", 0.79, "better recall"),
  ExperimentResult("regularized", "increase reproducibility", 0.77, "lower variance"),
]
print(choose_candidate(candidates))

Commands & References

• jupyter lab
• df.head()
• df.info()

Lab Steps

1. Prepare environment using: jupyter lab
2. Load a small sample dataset and validate schema.
3. Run the core code workflow and collect metrics.
4. Compare results and write one improvement note.

Exercises

• Change one hyperparameter and compare impact.
• Add one validation rule to reduce bad inputs.
• Document one failure mode and mitigation.