handwired.wiki

Soft switch: an if/else made of ReLUs

The ternary a > 4 ? 10 : 12 becomes above4 = relu(a − 4) − relu(a − 5) and result = above4 × 10 + (1 − above4) × 12. Drag a — every number below comes from a real Handwired Circuit.

your code

const a = 5
const result = a > 4
  ? 10
  : 12
circuit10
plain JS10

turns into live network

Live circuit: input a feeds relu(a−4) and relu(a−5); their difference above4 selects between the on and off branch. Nodes light up with their activation. a 5 relu(a−4) 1 relu(a−5) 0 above4 1 on 10 off 12 result 10 both branches always run — the bit picks one

Exact match

relu(a − 4) relu(a − 5) above4 = difference · drag the plot or use the slider ·
5

Why does this work?

For an integer a, relu(a − 4) − relu(a − 5) is 0 when a ≤ 4 and 1 when a ≥ 5 — an exact bit. emitSelect then computes bit × on + (1 − bit) × off, so both branches run and the bit picks one. Set a to a fraction like 4.5 and the bit becomes 0.5, blending the branches: that's why upstream's emitCompare relies on declared ranges and grids to guarantee exact bits.

What does the circuit look like?

Two reglu operations, one affine difference, and the select macro. See it drawn on the code → network page.

Loading Handwired…