Trinity S³AI
An encoder block: what one transformer layer does
Three positions through one transformer block: attention, a residual, a feed-forward net and layer norm. Post-norm ends every row at mean 0 and variance 1; pre-norm does not.
Trinity S³AI
Three positions through one transformer block: attention, a residual, a feed-forward net and layer norm. Post-norm ends every row at mean 0 and variance 1; pre-norm does not.