Out of 9 running in production, only 2 had a key element: verifier grounded in the real world
I looked at them as the same but I’ve realized that most of the industry thinks evals = offline against golden set. I use verifiers as the live production calibration element of the loop, but even that has evolved sine I wrote the post!
I looked at them as the same but I’ve realized that most of the industry thinks evals = offline against golden set. I use verifiers as the live production calibration element of the loop, but even that has evolved sine I wrote the post!