import torch
from common.graph_utils import sqnr
from common.pt2e import (
HistogramObserver,
MinMaxObserver,
MovingAverageMinMaxObserver,
PerChannelMinMaxObserver,
PlaceholderObserver,
get_xnnpack_quantizer,
quiet,
)
from common.viz import table
quiet()
_ = torch.manual_seed(0)
QMIN, QMAX = -128, 127
def minmax(symmetric=False):
"""int8 MinMaxObserver; symmetric=False means affine."""
qscheme = torch.per_tensor_symmetric if symmetric else torch.per_tensor_affine
return MinMaxObserver(dtype=torch.int8, qscheme=qscheme, quant_min=QMIN, quant_max=QMAX)
def qdq(x, scale, zp, qmin=QMIN, qmax=QMAX):
"""quantize → dequantize: quantize, then restore to float to see how much is lost."""
q = (x / scale).round() + zp
return (q.clamp(qmin, qmax) - zp) * scaleM02-1 | Observer Zoo: What Different Observers Compute from the Same Data
An observer does only two things:
forward(x): record the numeric range ofx(min/max or a histogram), then passxon unchanged.calculate_qparams(): derive the scale and zero point from the range recorded so far.
The affine quantization formula (extend the range to include 0, then split it evenly into 256 steps) was already derived in the M01 overview, so it is not repeated here. This notebook compares “how the range is chosen”:
- A. symmetric vs. affine: how scale and zero point differ on the same data.
- B. Three data distributions × three observers (MinMax affine/symmetric, Histogram).
- C. MinMax vs. MovingAverage: an abnormal batch shows up during calibration.
- D. per-channel: the scale becomes a vector, and
ch_axismust be chosen correctly. - E. Which observer real quantizers use: PTQ, QAT, and dynamic each differ; why the
PlaceholderObserverused by dynamic records nothing.
A. symmetric vs. affine
symmetric fixes the zero point at 0 and takes the range from whichever side has the larger absolute value:
\[ s = \frac{\max(|x_{\min}|,\ |x_{\max}|)}{(q_{\max} - q_{\min})/2},\qquad z = 0 \tag{1} \]
The denominator is 127.5: the 256 int8 steps are split in half, one half for each of the positive and negative axes. Affine instead uses all 256 steps on \([x_{\min}, x_{\max}]\) (extended to include 0).
x = torch.randn(20000)
aff, sym = minmax(), minmax(symmetric=True)
aff(x)
sym(x)
s_aff, z_aff = aff.calculate_qparams()
s_sym, z_sym = sym.calculate_qparams()
mn, mx = aff.min_val, aff.max_val
s_sym_hand = torch.maximum(-mn, mx) / ((QMAX - QMIN) / 2) # Eq. (1)
table([["affine", f"{float(s_aff):.6f}", int(z_aff), "verified in M01"],
["symmetric", f"{float(s_sym):.6f}", int(z_sym), f"Eq. (1): {float(s_sym_hand):.6f}"]],
["scheme", "scale", "zero point", "by hand"])| scheme | scale | zero point | by hand |
|---|---|---|---|
| affine | 0.033117 | 3 | verified in M01 |
| symmetric | 0.034065 | 0 | Eq. (1): 0.034065 |
assert int(z_sym) == 0 and torch.allclose(s_sym, s_sym_hand)
assert s_sym > s_aff # on the same data, symmetric steps are coarserThis data is roughly symmetric, so the two differ only a little: symmetric has a slightly larger scale (slightly coarser steps), because part of the shorter half-axis goes unused. The more asymmetric the data, the bigger the gap; the relu in part B is the extreme example.
B. Data Distribution × Observer
Three observers are compared: MinMax affine and symmetric, and HistogramObserver (as part E shows, the default XNNPACKQuantizer configuration used in this project uses it for activations in PTQ). The latter records a histogram of the value distribution and then searches for a range that lowers the quantization error, so it does not necessarily use min/max directly (the torchao docstring calls this “an approximation of L2 error minimization”). Why a smaller range can lower the error: the error splits into discretization (how coarse the steps are) and clipping (values outside the range get cut off); the larger the range, the larger the former and the smaller the latter. For the derivation and experiments see the blog post <Quantization Error (Case with Clipping)>; the earlier part, <Case without Clipping>, derives the discretization error.
Three kinds of data:
- gaussian: roughly symmetric.
- relu(gaussian): an activation after relu, all values ≥ 0.
- heavy tail: Student-t (3 degrees of freedom); most values are small, with occasional very large ones. Real activations often look like this.
The “representable range” is the range of int8’s \([q_{\min}, q_{\max}]\) mapped back to floats; SQNR is the signal-to-quantization-noise ratio of the qdq result against the original values, higher is better.
data = {
"gaussian": torch.randn(100_000),
"relu(gaussian)": torch.relu(torch.randn(100_000)),
"heavy tail t(3)": torch.distributions.StudentT(3.0).sample((100_000,)),
}
observers = {
"MinMax affine": lambda: minmax(),
"MinMax symmetric": lambda: minmax(symmetric=True),
"Histogram affine": lambda: HistogramObserver(dtype=torch.int8, qscheme=torch.per_tensor_affine,
quant_min=QMIN, quant_max=QMAX),
}
result, hi_bound = {}, {}
rows = []
for dname, d in data.items():
for oname, make in observers.items():
obs = make()
obs(d)
s, z = (float(v) for v in obs.calculate_qparams())
result[dname, oname] = sqnr(d, qdq(d, s, z))
hi_bound[dname, oname] = (QMAX - z) * s
rows.append([dname, oname, f"{s:.4f}", int(z),
f"[{(QMIN - z) * s:.2f}, {(QMAX - z) * s:.2f}]", f"{result[dname, oname]:.2f}"])
table(rows, ["data", "observer", "scale", "zp", "representable range", "SQNR (dB)"])| data | observer | scale | zp | representable range | SQNR (dB) |
|---|---|---|---|---|---|
| gaussian | MinMax affine | 0.0346 | -5 | [-4.25, 4.57] | 40.01 |
| gaussian | MinMax symmetric | 0.0358 | 0 | [-4.58, 4.54] | 39.69 |
| gaussian | Histogram affine | 0.0332 | 0 | [-4.25, 4.22] | 40.30 |
| relu(gaussian) | MinMax affine | 0.0183 | -128 | [0.00, 4.66] | 45.56 |
| relu(gaussian) | MinMax symmetric | 0.0365 | 0 | [-4.68, 4.64] | 39.54 |
| relu(gaussian) | Histogram affine | 0.0150 | -128 | [0.00, 3.83] | 44.39 |
| heavy tail t(3) | MinMax affine | 0.3740 | -8 | [-44.88, 50.49] | 23.83 |
| heavy tail t(3) | MinMax symmetric | 0.3973 | 0 | [-50.85, 50.46] | 23.29 |
| heavy tail t(3) | Histogram affine | 0.3189 | 12 | [-44.64, 36.67] | 24.32 |
relu_gap = result["relu(gaussian)", "MinMax affine"] - result["relu(gaussian)", "MinMax symmetric"]
print(f"relu(gaussian): affine is {relu_gap:.2f} dB higher than symmetric")
assert 5 < relu_gap < 7relu(gaussian): affine is 6.02 dB higher than symmetric
- relu(gaussian): affine’s zero point is −128, so the whole int8 range is used on \([0, x_{\max}]\); symmetric wastes the entire negative half-axis, which amounts to losing 1 bit. Each lost bit costs about 6 dB of SQNR, consistent with the gap measured above. This is one reason activations often use affine. Weights, on the other hand, often use symmetric, because zero point = 0 removes one cross term from the integer arithmetic (M10).
- heavy tail: MinMax does no clipping at all, so the scale is stretched by a few extreme values and most values get crowded into a few steps.
How much do Histogram’s range and SQNR differ from MinMax affine:
shrunk = all(hi_bound[d, "Histogram affine"] < hi_bound[d, "MinMax affine"] for d in data)
better = [d for d in data if result[d, "Histogram affine"] > result[d, "MinMax affine"]]
print(f"Is Histogram's upper bound smaller than MinMax's on all three datasets? {shrunk}")
print(f"Datasets with higher SQNR than MinMax affine:{better}")
assert shrunk and 0 < len(better) < len(data)Is Histogram's upper bound smaller than MinMax's on all three datasets? True
Datasets with higher SQNR than MinMax affine:['gaussian', 'heavy tail t(3)']
Histogram pulls the range in a bit and clips a few extreme values, but its SQNR is not necessarily higher than MinMax’s; on some data it is lower. It approximates the quantization error with a histogram and then searches for a range, so it is an approximate solution and is not guaranteed to beat MinMax.
C. MinMax vs. MovingAverage: a Spike During Calibration
Feed 20 batches in a row, where the 3rd batch has 10 times the magnitude of the others. MovingAverageMinMaxObserver takes the first batch’s min/max directly, then updates with each later batch like this (\(c\) is averaging_constant):
\[ x_{\min} \leftarrow x_{\min} + c\,(x_{\min}^{\text{batch}} - x_{\min}) \tag{2} \]
\(x_{\max}\) is the same. MinMax simply keeps the running minimum/maximum.
mm = minmax()
c = 0.1
ma = MovingAverageMinMaxObserver(averaging_constant=c, dtype=torch.int8, qscheme=torch.per_tensor_affine,
quant_min=QMIN, quant_max=QMAX)
hand_min = None
rows, mm_hist = [], []
for i in range(20):
amp = 10.0 if i == 3 else 1.0
batch = torch.randn(1000) * amp
mm(batch)
ma(batch)
b_min = batch.min()
hand_min = b_min if hand_min is None else hand_min + c * (b_min - hand_min) # Eq. (2)
mm_hist.append(float(mm.min_val))
if i in (0, 2, 3, 4, 10, 19):
rows.append([i, f"{amp:.0f}x", f"[{float(mm.min_val):.2f}, {float(mm.max_val):.2f}]",
f"[{float(ma.min_val):.2f}, {float(ma.max_val):.2f}]"])
table(rows, ["batch", "batch magnitude", "MinMax range", "MovingAverage range"])| batch | batch magnitude | MinMax range | MovingAverage range |
|---|---|---|---|
| 0 | 1x | [-3.00, 3.22] | [-3.00, 3.22] |
| 2 | 1x | [-3.25, 3.41] | [-3.02, 3.21] |
| 3 | 10x | [-29.90, 35.58] | [-5.70, 6.45] |
| 4 | 1x | [-29.90, 35.58] | [-5.45, 6.10] |
| 10 | 1x | [-29.90, 35.58] | [-4.47, 4.73] |
| 19 | 1x | [-29.90, 35.58] | [-3.77, 3.79] |
assert hand_min is not None and torch.allclose(ma.min_val, hand_min) # Eq. (2) matches the observer
assert mm_hist[3:] == [mm_hist[3]] * 17 # MinMax never changes after the spike- MinMax’s range can only grow, never shrink: a single spike widens it permanently.
- MovingAverage drifts toward each new batch following Eq. (2), so a spike gets diluted by later batches. The cost is that the range depends on batch order, and later batches have more influence.
XNNPACK’s QAT configuration uses exactly moving average for activations (the table in part E). Inference (the torchao source does not give the reason): during training the weights keep changing and so do the activation ranges, which calls for an estimate that can “keep up”.
D. per-channel: the Scale Is a Vector
A conv weight has shape (out_ch, in_ch, kh, kw). Per-channel quantization gives each channel its own scale, and ch_axis specifies which dimension to slice along. Scale output channel 0 up by 5x to simulate channels whose ranges differ a lot:
conv_w = torch.randn(4, 3, 3, 3)
conv_w[0] *= 5.0
rows, per_axis = [], {}
for axis in (0, 1):
obs = PerChannelMinMaxObserver(ch_axis=axis, dtype=torch.int8, qscheme=torch.per_channel_symmetric,
quant_min=-127, quant_max=127)
obs(conv_w)
s, z = obs.calculate_qparams()
shape = [1] * conv_w.dim()
shape[axis] = -1 # broadcast the scale along axis
per_axis[axis] = sqnr(conv_w, qdq(conv_w, s.view(shape), z.view(shape), -127, 127))
rows.append([axis, "output channel" if axis == 0 else "input channel", tuple(s.shape), f"{per_axis[axis]:.2f}"])
table(rows, ["ch_axis", "along", "scale shape", "SQNR (dB)"])| ch_axis | along | scale shape | SQNR (dB) |
|---|---|---|---|
| 0 | output channel | (4,) | 46.16 |
| 1 | input channel | (3,) | 42.11 |
assert per_axis[0] > per_axis[1]The scaled-up one is an output channel, so only ch_axis=0 lets that channel use its own large scale without dragging the other channels down. For PyTorch conv and linear weights, per-channel always means the output channel (ch_axis=0). A wrong axis raises no error; it just quietly gets worse.
E. Which Observer a Quantizer Really Uses: PTQ, QAT, and Dynamic Differ
A to D all compared observers that “record the range during calibration”. Real quantizers pick different classes depending on the quantization mode. The table below shows the class used for activations under three configurations of XNNPACKQuantizer (the torchao test version); the objects are built directly from the constructor in the quantizer’s configuration:
rows = []
for label, kw in [("PTQ (default)", {}), ("QAT", {"is_qat": True}), ("dynamic", {"is_dynamic": True})]:
quantizer, _ = get_xnnpack_quantizer(is_per_channel=True, **kw)
cfg = quantizer.global_config
assert cfg is not None and cfg.input_activation is not None
act = cfg.input_activation.observer_or_fake_quant_ctr()
inner = getattr(act, "activation_post_process", None) # a FakeQuantize class wraps an observer inside
rows.append([label, type(act).__name__, type(inner).__name__ if inner is not None else "—"])
table(rows, ["config", "class used for activation", "wrapped observer"])| config | class used for activation | wrapped observer |
|---|---|---|
| PTQ (default) | HistogramObserver | — |
| QAT | FusedMovingAvgObsFakeQuantize | MovingAverageMinMaxObserver |
| dynamic | PlaceholderObserver | — |
assert [r[1] for r in rows] == ["HistogramObserver", "FusedMovingAvgObsFakeQuantize", "PlaceholderObserver"]
assert rows[1][2] == "MovingAverageMinMaxObserver"- PTQ uses
HistogramObserver(compared in part B). - QAT uses a fake quant class that wraps a MovingAverage observer (part C; for what fake quant is, see M02-2).
- dynamic uses
PlaceholderObserver, a new face, explained below.
This table is a choice made by this quantizer configuration, not a rule of PTQ. The observer kind is written in each QuantizationSpec’s observer_or_fake_quant_ctr, decided by whoever writes the quantizer, and the same backend can swap it out. Checked on torchao 0.18 and executorch 1.5.0: the default configurations of the executorch XNNPACKQuantizer and X86InductorQuantizer also use HistogramObserver for PTQ activations; the custom VPU quantizer in M10 instead uses a MinMax observer that rounds the scale to a power of 2.
Why dynamic Quantization Uses PlaceholderObserver
In dynamic quantization the activation scale is not decided at calibration time but computed at every run from the current input (M01-3 showed the choose_qparams after convert). So this edge does not need to record any range, only a marker: “quantize to some dtype here”. PlaceholderObserver is that marker. Compare it with MinMaxObserver:
y = torch.randn(1000)
mm_obs = minmax()
ph = PlaceholderObserver(dtype=torch.float32)
def qparams_or_error(obs):
try:
s, z = obs.calculate_qparams()
return f"scale={float(s):.4f}, zp={int(z)}"
except Exception as e:
return f"error: {e}"
rows = []
for name, obs in [("MinMaxObserver", mm_obs), ("PlaceholderObserver", ph)]:
out = obs(y)
rows.append([name, torch.equal(out, y), ", ".join(b for b, _ in obs.named_buffers()) or "(none)",
qparams_or_error(obs)])
table(rows, ["observer", "forward output == input", "recording buffers", "calculate_qparams()"])| observer | forward output == input | recording buffers | calculate_qparams() |
|---|---|---|---|
| MinMaxObserver | True | eps, min_val, max_val | scale=0.0245, zp=6 |
| PlaceholderObserver | True | (none) | error: calculate_qparams should not be called for PlaceholderObserver |
assert all(r[1] for r in rows) # both forwards pass the input through unchanged
assert rows[1][2] == "(none)" and rows[1][3].startswith("error") # Placeholder records nothing and cannot compute a scaleBoth forwards pass the input through unchanged, so a graph with observers inserted has unchanged outputs. The difference is in recording: MinMaxObserver records the range in min_val and max_val and can compute a scale; PlaceholderObserver records nothing, and calling calculate_qparams() raises an error directly. At convert it becomes a choose_qparams that computes the scale at run time, rather than a hard-coded constant.
Summary
| Question | Answer |
|---|---|
| How do symmetric and affine differ | symmetric’s zero point is fixed at 0 and each half-axis gets half of the steps; when the data leans to one side (after relu) it wastes about 1 bit |
| How do Histogram and MinMax differ | Histogram pulls the range in and clips a few extreme values, but its SQNR is not necessarily higher than MinMax’s |
| MinMax vs MovingAverage | One spike widens MinMax permanently; MovingAverage dilutes it gradually, but depends on batch order |
| per-channel | One scale per output channel (ch_axis=0); a wrong axis raises no error |
| Do observers change values | No, forward passes through unchanged. What changes values is fake quant (M02-2) |
Why does dynamic use PlaceholderObserver |
The scale is computed at run time, so no range needs recording at calibration; it only marks that this edge is to be quantized |