This post explores using semantic entropy as a training signal for calibrating confidence in language models.

Here is what I found:

1. Training on semantic entropy alone does not converge and leads to unstable behavior
2. In a data abundant/compute scarce regime, standard Brier score supervision achieves strong calibration on its …