Trinity S³AI

PPO: how an agent learns from reward without lurching

Drag the probability ratio of a new policy over the old: PPO's objective follows r times the advantage until r moves more than eps the rewarded way, then it goes flat and stops pulling.