This usually works for me, but it probably has no redeeming medical value whatsoever.

I usually gargle with hydrogen peroxide (someone above already suggested this) and then I use a q-tip and put Camphophenique on it, it tastes like ass, but it usually makes ulcers feel better and they seem to heal up rather quickly.

Zilactin-B works wonders as well, but yeah it burns like hell.