Rounding and saturation: the error you choose, and its bias

Drag a value into a small Qm.n and switch between truncation, rounding and convergent rounding, and between wrap and saturate. The bars show which rounding is biased.

This page sets no cookie and sends nothing anywhere.

Where the numbers come from

gHashTag/t27 specs/fpga/dsp/fixed_point.t27 (t27#8985): every value on the page is its fn bodies, compiled to wasm (logic.js)
ARM CMSIS-DSP and the Q notation of DSP texts: Qm.n with m integer bits counting the sign (Q1.15 is a 16-bit word)
IEEE 754-2019, section 4.3.1: roundTiesToEven, the convergent rounding of this page

This widget's .t27 source · All widgets