Data Science Blog

Twice I built a fix I loved. Twice a full run said no. On being wrong productively, and the fair baseline that makes measurement mean something.

Most of what looked like a retrieval problem was really a document problem, and it had to be solved before retrieval could mean anything.

Same model. Same test set. Two curves that tell almost opposite stories. A few years ago, while building a fraud detection model for insurance claims, I hit a wall that every data scientist eventually hits. My model came back with 99%+ accuracy. Leadership loved the number. I did not trust it for a second. Here is why. Fraud in insurance claims is rare, often less than 2% of all claims. A model that predicts "not fraud" for every single claim would already score above 98% acc