Skip to main content

M02-1 | Observer Zoo: What Different Observers Compute from the Same Data

Open In Colab

An observer does only two things:

The affine quantization formula (extend the range to include 0, then split it evenly into 256 steps) was already derived in the M01 overview, so it is not repeated here. This notebook compares “how the range is chosen”:

import torch

from common.graph_utils import sqnr
from common.pt2e import (
    HistogramObserver,
    MinMaxObserver,
    MovingAverageMinMaxObserver,
    PerChannelMinMaxObserver,
    PlaceholderObserver,
    get_xnnpack_quantizer,
    quiet,
)
from common.viz import table

quiet()
_ = torch.manual_seed(0)

QMIN, QMAX = -128, 127


def minmax(symmetric=False):
    """int8 MinMaxObserver; symmetric=False means affine."""
    qscheme = torch.per_tensor_symmetric if symmetric else torch.per_tensor_affine
    return MinMaxObserver(dtype=torch.int8, qscheme=qscheme, quant_min=QMIN, quant_max=QMAX)


def qdq(x, scale, zp, qmin=QMIN, qmax=QMAX):
    """quantize → dequantize: quantize, then restore to float to see how much is lost."""
    q = (x / scale).round() + zp
    return (q.clamp(qmin, qmax) - zp) * scale

A. symmetric vs. affine

symmetric fixes the zero point at 0 and takes the range from whichever side has the larger absolute value:

\[ s = \frac{\max(|x_{\min}|,\ |x_{\max}|)}{(q_{\max} - q_{\min})/2},\qquad z = 0 \tag{1} \]

The denominator is 127.5: the 256 int8 steps are split in half, one half for each of the positive and negative axes. Affine instead uses all 256 steps on \([x_{\min}, x_{\max}]\) (extended to include 0).

x = torch.randn(20000)
aff, sym = minmax(), minmax(symmetric=True)
aff(x)
sym(x)
s_aff, z_aff = aff.calculate_qparams()
s_sym, z_sym = sym.calculate_qparams()
mn, mx = aff.min_val, aff.max_val

s_sym_hand = torch.maximum(-mn, mx) / ((QMAX - QMIN) / 2)  # Eq. (1)
table([["affine", f"{float(s_aff):.6f}", int(z_aff), "verified in M01"],
       ["symmetric", f"{float(s_sym):.6f}", int(z_sym), f"Eq. (1): {float(s_sym_hand):.6f}"]],
      ["scheme", "scale", "zero point", "by hand"])
scheme scale zero point by hand
affine 0.033117 3 verified in M01
symmetric 0.034065 0 Eq. (1): 0.034065
assert int(z_sym) == 0 and torch.allclose(s_sym, s_sym_hand)
assert s_sym > s_aff  # on the same data, symmetric steps are coarser

This data is roughly symmetric, so the two differ only a little: symmetric has a slightly larger scale (slightly coarser steps), because part of the shorter half-axis goes unused. The more asymmetric the data, the bigger the gap; the relu in part B is the extreme example.

B. Data Distribution × Observer

Three observers are compared: MinMax affine and symmetric, and HistogramObserver (as part E shows, the default XNNPACKQuantizer configuration used in this project uses it for activations in PTQ). The latter records a histogram of the value distribution and then searches for a range that lowers the quantization error, so it does not necessarily use min/max directly (the torchao docstring calls this “an approximation of L2 error minimization”). Why a smaller range can lower the error: the error splits into discretization (how coarse the steps are) and clipping (values outside the range get cut off); the larger the range, the larger the former and the smaller the latter. For the derivation and experiments see the blog post <Quantization Error (Case with Clipping)>; the earlier part, <Case without Clipping>, derives the discretization error.

Three kinds of data:

  • gaussian: roughly symmetric.
  • relu(gaussian): an activation after relu, all values ≥ 0.
  • heavy tail: Student-t (3 degrees of freedom); most values are small, with occasional very large ones. Real activations often look like this.

The “representable range” is the range of int8’s \([q_{\min}, q_{\max}]\) mapped back to floats; SQNR is the signal-to-quantization-noise ratio of the qdq result against the original values, higher is better.

data = {
    "gaussian": torch.randn(100_000),
    "relu(gaussian)": torch.relu(torch.randn(100_000)),
    "heavy tail t(3)": torch.distributions.StudentT(3.0).sample((100_000,)),
}
observers = {
    "MinMax affine": lambda: minmax(),
    "MinMax symmetric": lambda: minmax(symmetric=True),
    "Histogram affine": lambda: HistogramObserver(dtype=torch.int8, qscheme=torch.per_tensor_affine,
                                                  quant_min=QMIN, quant_max=QMAX),
}
result, hi_bound = {}, {}
rows = []
for dname, d in data.items():
    for oname, make in observers.items():
        obs = make()
        obs(d)
        s, z = (float(v) for v in obs.calculate_qparams())
        result[dname, oname] = sqnr(d, qdq(d, s, z))
        hi_bound[dname, oname] = (QMAX - z) * s
        rows.append([dname, oname, f"{s:.4f}", int(z),
                     f"[{(QMIN - z) * s:.2f}, {(QMAX - z) * s:.2f}]", f"{result[dname, oname]:.2f}"])
table(rows, ["data", "observer", "scale", "zp", "representable range", "SQNR (dB)"])
data observer scale zp representable range SQNR (dB)
gaussian MinMax affine 0.0346 -5 [-4.25, 4.57] 40.01
gaussian MinMax symmetric 0.0358 0 [-4.58, 4.54] 39.69
gaussian Histogram affine 0.0332 0 [-4.25, 4.22] 40.30
relu(gaussian) MinMax affine 0.0183 -128 [0.00, 4.66] 45.56
relu(gaussian) MinMax symmetric 0.0365 0 [-4.68, 4.64] 39.54
relu(gaussian) Histogram affine 0.0150 -128 [0.00, 3.83] 44.39
heavy tail t(3) MinMax affine 0.3740 -8 [-44.88, 50.49] 23.83
heavy tail t(3) MinMax symmetric 0.3973 0 [-50.85, 50.46] 23.29
heavy tail t(3) Histogram affine 0.3189 12 [-44.64, 36.67] 24.32
relu_gap = result["relu(gaussian)", "MinMax affine"] - result["relu(gaussian)", "MinMax symmetric"]
print(f"relu(gaussian): affine is {relu_gap:.2f} dB higher than symmetric")
assert 5 < relu_gap < 7
relu(gaussian): affine is 6.02 dB higher than symmetric
  • relu(gaussian): affine’s zero point is −128, so the whole int8 range is used on \([0, x_{\max}]\); symmetric wastes the entire negative half-axis, which amounts to losing 1 bit. Each lost bit costs about 6 dB of SQNR, consistent with the gap measured above. This is one reason activations often use affine. Weights, on the other hand, often use symmetric, because zero point = 0 removes one cross term from the integer arithmetic (M10).
  • heavy tail: MinMax does no clipping at all, so the scale is stretched by a few extreme values and most values get crowded into a few steps.

How much do Histogram’s range and SQNR differ from MinMax affine:

shrunk = all(hi_bound[d, "Histogram affine"] < hi_bound[d, "MinMax affine"] for d in data)
better = [d for d in data if result[d, "Histogram affine"] > result[d, "MinMax affine"]]
print(f"Is Histogram's upper bound smaller than MinMax's on all three datasets? {shrunk}")
print(f"Datasets with higher SQNR than MinMax affine:{better}")
assert shrunk and 0 < len(better) < len(data)
Is Histogram's upper bound smaller than MinMax's on all three datasets? True
Datasets with higher SQNR than MinMax affine:['gaussian', 'heavy tail t(3)']

Histogram pulls the range in a bit and clips a few extreme values, but its SQNR is not necessarily higher than MinMax’s; on some data it is lower. It approximates the quantization error with a histogram and then searches for a range, so it is an approximate solution and is not guaranteed to beat MinMax.

C. MinMax vs. MovingAverage: a Spike During Calibration

Feed 20 batches in a row, where the 3rd batch has 10 times the magnitude of the others. MovingAverageMinMaxObserver takes the first batch’s min/max directly, then updates with each later batch like this (\(c\) is averaging_constant):

\[ x_{\min} \leftarrow x_{\min} + c\,(x_{\min}^{\text{batch}} - x_{\min}) \tag{2} \]

\(x_{\max}\) is the same. MinMax simply keeps the running minimum/maximum.

mm = minmax()
c = 0.1
ma = MovingAverageMinMaxObserver(averaging_constant=c, dtype=torch.int8, qscheme=torch.per_tensor_affine,
                                 quant_min=QMIN, quant_max=QMAX)
hand_min = None
rows, mm_hist = [], []
for i in range(20):
    amp = 10.0 if i == 3 else 1.0
    batch = torch.randn(1000) * amp
    mm(batch)
    ma(batch)
    b_min = batch.min()
    hand_min = b_min if hand_min is None else hand_min + c * (b_min - hand_min)  # Eq. (2)
    mm_hist.append(float(mm.min_val))
    if i in (0, 2, 3, 4, 10, 19):
        rows.append([i, f"{amp:.0f}x", f"[{float(mm.min_val):.2f}, {float(mm.max_val):.2f}]",
                     f"[{float(ma.min_val):.2f}, {float(ma.max_val):.2f}]"])
table(rows, ["batch", "batch magnitude", "MinMax range", "MovingAverage range"])
batch batch magnitude MinMax range MovingAverage range
0 1x [-3.00, 3.22] [-3.00, 3.22]
2 1x [-3.25, 3.41] [-3.02, 3.21]
3 10x [-29.90, 35.58] [-5.70, 6.45]
4 1x [-29.90, 35.58] [-5.45, 6.10]
10 1x [-29.90, 35.58] [-4.47, 4.73]
19 1x [-29.90, 35.58] [-3.77, 3.79]
assert hand_min is not None and torch.allclose(ma.min_val, hand_min)  # Eq. (2) matches the observer
assert mm_hist[3:] == [mm_hist[3]] * 17                               # MinMax never changes after the spike
  • MinMax’s range can only grow, never shrink: a single spike widens it permanently.
  • MovingAverage drifts toward each new batch following Eq. (2), so a spike gets diluted by later batches. The cost is that the range depends on batch order, and later batches have more influence.

XNNPACK’s QAT configuration uses exactly moving average for activations (the table in part E). Inference (the torchao source does not give the reason): during training the weights keep changing and so do the activation ranges, which calls for an estimate that can “keep up”.

D. per-channel: the Scale Is a Vector

A conv weight has shape (out_ch, in_ch, kh, kw). Per-channel quantization gives each channel its own scale, and ch_axis specifies which dimension to slice along. Scale output channel 0 up by 5x to simulate channels whose ranges differ a lot:

conv_w = torch.randn(4, 3, 3, 3)
conv_w[0] *= 5.0
rows, per_axis = [], {}
for axis in (0, 1):
    obs = PerChannelMinMaxObserver(ch_axis=axis, dtype=torch.int8, qscheme=torch.per_channel_symmetric,
                                   quant_min=-127, quant_max=127)
    obs(conv_w)
    s, z = obs.calculate_qparams()
    shape = [1] * conv_w.dim()
    shape[axis] = -1  # broadcast the scale along axis
    per_axis[axis] = sqnr(conv_w, qdq(conv_w, s.view(shape), z.view(shape), -127, 127))
    rows.append([axis, "output channel" if axis == 0 else "input channel", tuple(s.shape), f"{per_axis[axis]:.2f}"])
table(rows, ["ch_axis", "along", "scale shape", "SQNR (dB)"])
ch_axis along scale shape SQNR (dB)
0 output channel (4,) 46.16
1 input channel (3,) 42.11
assert per_axis[0] > per_axis[1]

The scaled-up one is an output channel, so only ch_axis=0 lets that channel use its own large scale without dragging the other channels down. For PyTorch conv and linear weights, per-channel always means the output channel (ch_axis=0). A wrong axis raises no error; it just quietly gets worse.

E. Which Observer a Quantizer Really Uses: PTQ, QAT, and Dynamic Differ

A to D all compared observers that “record the range during calibration”. Real quantizers pick different classes depending on the quantization mode. The table below shows the class used for activations under three configurations of XNNPACKQuantizer (the torchao test version); the objects are built directly from the constructor in the quantizer’s configuration:

rows = []
for label, kw in [("PTQ (default)", {}), ("QAT", {"is_qat": True}), ("dynamic", {"is_dynamic": True})]:
    quantizer, _ = get_xnnpack_quantizer(is_per_channel=True, **kw)
    cfg = quantizer.global_config
    assert cfg is not None and cfg.input_activation is not None
    act = cfg.input_activation.observer_or_fake_quant_ctr()
    inner = getattr(act, "activation_post_process", None)  # a FakeQuantize class wraps an observer inside
    rows.append([label, type(act).__name__, type(inner).__name__ if inner is not None else "—"])
table(rows, ["config", "class used for activation", "wrapped observer"])
config class used for activation wrapped observer
PTQ (default) HistogramObserver —
QAT FusedMovingAvgObsFakeQuantize MovingAverageMinMaxObserver
dynamic PlaceholderObserver —
assert [r[1] for r in rows] == ["HistogramObserver", "FusedMovingAvgObsFakeQuantize", "PlaceholderObserver"]
assert rows[1][2] == "MovingAverageMinMaxObserver"
  • PTQ uses HistogramObserver (compared in part B).
  • QAT uses a fake quant class that wraps a MovingAverage observer (part C; for what fake quant is, see M02-2).
  • dynamic uses PlaceholderObserver, a new face, explained below.

This table is a choice made by this quantizer configuration, not a rule of PTQ. The observer kind is written in each QuantizationSpec’s observer_or_fake_quant_ctr, decided by whoever writes the quantizer, and the same backend can swap it out. Checked on torchao 0.18 and executorch 1.5.0: the default configurations of the executorch XNNPACKQuantizer and X86InductorQuantizer also use HistogramObserver for PTQ activations; the custom VPU quantizer in M10 instead uses a MinMax observer that rounds the scale to a power of 2.

Why dynamic Quantization Uses PlaceholderObserver

In dynamic quantization the activation scale is not decided at calibration time but computed at every run from the current input (M01-3 showed the choose_qparams after convert). So this edge does not need to record any range, only a marker: “quantize to some dtype here”. PlaceholderObserver is that marker. Compare it with MinMaxObserver:

y = torch.randn(1000)
mm_obs = minmax()
ph = PlaceholderObserver(dtype=torch.float32)


def qparams_or_error(obs):
    try:
        s, z = obs.calculate_qparams()
        return f"scale={float(s):.4f}, zp={int(z)}"
    except Exception as e:
        return f"error: {e}"


rows = []
for name, obs in [("MinMaxObserver", mm_obs), ("PlaceholderObserver", ph)]:
    out = obs(y)
    rows.append([name, torch.equal(out, y), ", ".join(b for b, _ in obs.named_buffers()) or "(none)",
                 qparams_or_error(obs)])
table(rows, ["observer", "forward output == input", "recording buffers", "calculate_qparams()"])
observer forward output == input recording buffers calculate_qparams()
MinMaxObserver True eps, min_val, max_val scale=0.0245, zp=6
PlaceholderObserver True (none) error: calculate_qparams should not be called for PlaceholderObserver
assert all(r[1] for r in rows)                          # both forwards pass the input through unchanged
assert rows[1][2] == "(none)" and rows[1][3].startswith("error")  # Placeholder records nothing and cannot compute a scale

Both forwards pass the input through unchanged, so a graph with observers inserted has unchanged outputs. The difference is in recording: MinMaxObserver records the range in min_val and max_val and can compute a scale; PlaceholderObserver records nothing, and calling calculate_qparams() raises an error directly. At convert it becomes a choose_qparams that computes the scale at run time, rather than a hard-coded constant.

Summary

Question Answer
How do symmetric and affine differ symmetric’s zero point is fixed at 0 and each half-axis gets half of the steps; when the data leans to one side (after relu) it wastes about 1 bit
How do Histogram and MinMax differ Histogram pulls the range in and clips a few extreme values, but its SQNR is not necessarily higher than MinMax’s
MinMax vs MovingAverage One spike widens MinMax permanently; MovingAverage dilutes it gradually, but depends on batch order
per-channel One scale per output channel (ch_axis=0); a wrong axis raises no error
Do observers change values No, forward passes through unchanged. What changes values is fake quant (M02-2)
Why does dynamic use PlaceholderObserver The scale is computed at run time, so no range needs recording at calibration; it only marks that this edge is to be quantized