Auto-Editor's whole job starts with a sequence of *levels*: for every unit of the timebase, how loud is the audio, how much motion is on screen, and so on. A threshold turns that sequence into keep/cut decisions. We also cache those levels on disk so re-running the same edit is instant. For a long video, "one value per frame" is a *lot* of values. A two-hour clip at a 30fps timebase is over 200,000 audio levels alone. Every one of them was a `float32`. Four bytes both in memory and in the cache file on disk. The thing is, almost none of these numbers need 32 bits of range. Loudness and motion are normalized to `[0.0, 1.0]`. Opacity is `[0.0, 1.0]`. Brightness is `[-1.0, 1.0]`. These are *bounded* quantities, and a 32-bit float spends most of its bits representing values these numbers will never take. ## Why not just use float16? The obvious move is to reach for a half-precision float. `float16` is two bytes, so it halves the storage just the same. But it's the wrong tool here, for four reasons. First, most CPUs don't do `float16` arithmetic natively; any math means widening to `float32` first. Nim doesn't even have it in its type system. Second, **it spends its bits on range we don't have.** A float divides its bits between an exponent (range) and a mantissa (precision). `float16` can represent values out past 65000 and denormals down near `10⁻⁸` — an enormous dynamic range that a number clamped to `[0, 1]` will never touch. Every bit in that exponent is a bit *not* spent on precision. Third, **float precision is nonuniform**, and that matters when you only have a handful of bits to begin with. Floats are dense near zero and sparse near one. Up around `1.0`, consecutive `float16` values are about `0.00049` apart, while down near `0.04` (where the default audio threshold sits) they're far finer. So a half-float would give you wildly different resolution depending on how loud the moment is. Fourth, floating point is noncanonical. Zero is represented by both `+0.0` and `-0.0`, and the all-ones exponent gives you ~2046 distinct `NaN` bit patterns plus two infinities. None of that is a *large* fraction of the 65536 codes, but it means two values that compare equal can have different bytes. That bites later, when the cache blits these straight to disk and trusts the bytes — a canonical, every-pattern-is-a-real-value type is simply safer to store. ## Borrowing from graphics GPUs solved this problem a long time ago. Texture formats have a pair of types called **UNORM** and **SNORM**, "unsigned normalized" and "signed normalized" respectively. A `unorm16` is just a `uint16` that you agree to interpret as a fraction: `0` means `0.0`, `65535` means `1.0`, and everything in between is linear. A `snorm16` does the same with a `int16` over `[-1.0, 1.0]`. So I created `dnorm16.nim` with a pair of [distinct types](https://nim-lang.org/docs/manual.html#types-distinct-type): ```nim type Unorm16* = distinct uint16 type Snorm16* = distinct int16 func toUnorm16*(f: float32): Unorm16 = let c = max(0.0'f32, min(1.0'f32, f)) Unorm16(uint16(c * 65535.0'f32 + 0.5'f32)) func toFloat32*(u: Unorm16): float32 = uint16(u).float32 * (1.0'f32 / 65535.0'f32) ``` Using `distinct` improves safety by making sure the compiler won't let you accidentally add a `Unorm16` to a raw `uint16`, or pass a snorm where an unorm is expected. The operators that I want unchanged from the base type are pulled in Nim's `borrow` pragma: ```nim func `==`*(a, b: Unorm16): bool {.borrow.} func `<=`*(a, b: Unorm16): bool {.borrow.} func `+`*(a, b: Unorm16): Unorm16 {.borrow.} func `-`*(a, b: Unorm16): Unorm16 {.borrow.} ``` That gives me a type that's two bytes wide, compares as cheaply as an integer, and can't be confused with anything else. ## First stop: the action bytecode The first place this landed was opacity. Auto-Editor serializes actions into a compact byte stream, and opacity was being stored as a tag byte plus a 4-byte float: ```nim of actOpacity: var u = toUnorm16(a.val) copyMem(addr base[i + 1], addr u, sizeof(Unorm16)) i += 3 ``` Five bytes became three. Opacity is a fraction between 0 and 1, perfect for `unorm16`. Brightness followed as a `snorm16`, since it ranges over `[-1.0, 1.0]`. ## The cache gets a codec byte Next was the on-disk level cache. The trick here is that a cache file needs to be *self-describing* — if the format changes, or you read a unorm file expecting floats, you want it to fail cleanly and recompute rather than silently hand back garbage. So every cache file now starts with a one-byte codec tag: ```nim type CacheCodec = enum ccFloat32 = 0'u8 # raw float32 ccUnorm16 = 1'u8 # [0.0, 1.0] ccSnorm16 = 2'u8 # [-1.0, 1.0] ``` At first I picked the codec from a string (`"audio"` and `"motion"` → unorm). That worked, but it was a lookup that could drift out of sync with reality. The better version lets the *type* decide: ```nim func codecOf(T: typedesc): CacheCodec = when T is Unorm16: ccUnorm16 elif T is Snorm16: ccSnorm16 elif T is float32: ccFloat32 else: {.error: "cache: unsupported element type".} ``` Once the cache was generic over `T`, save and load stopped iterating value-by-value and just blasted the whole buffer to disk in one shot: ```nim proc saveCache[T](filename: string, data: seq[T]) = ... fs.write(uint8(codecOf(T))) fs.write(data.len.int32) if data.len > 0: fs.writeData(unsafeAddr data[0], data.len * sizeof(T)) ``` Loading checks the codec byte matches the type you asked for, then reads the bytes straight back into the seq. A mismatch or a truncated file raises, the `Option` comes back empty, and the levels are simply recomputed. ## Keeping it 16-bit in memory too The early commits converted to `unorm16` only at the disk boundary — levels were still `seq[float32]` while the program held them. That meant doubling the memory the moment you loaded a cache. So the next step was to keep them narrow all the way through: thresholding now operates on `Unorm16` directly. ```nim const defaultAudioThres = toUnorm16(0.04) defaultMotionThres = toUnorm16(0.02) proc orWithThreshold(result: var seq[bool], levels: seq[Unorm16], t: Unorm16) = ... result[i] = levels[i] >= t ``` Because the threshold is itself quantized into a `Unorm16`, the comparison is an integer compare against integer data — no per-sample float conversion just to decide keep-or-cut. The in-memory footprint of the levels is halved, and for the cache, the file on disk is halved too. ## The annoying part `1/65535` is not a round number. When you quantize `0.5` into a unorm16 and read it back, you get `0.5000076`. If you ever need to turn one of these back into text — and Auto-Editor does, because action parameters get serialized into a human-readable form — naive formatting splatters that quantization noise everywhere. The fix is to find the *shortest decimal string that re-quantizes to the same bucket*. Walk up the precision until a rounded decimal snaps back to exactly the value you started with: ```nim func `$`*(u: Unorm16): string = let n = uint16(u).int const d = 65535 for prec in 1 .. 5: let scale = 10 ^ prec let rounded = (n * scale + d div 2) div d # round(n/d * 10^prec) let f = rounded.float32 / scale.float32 if toUnorm16(f) == u: # ...format `rounded` with a decimal point and trim trailing zeros... return result ``` The first version of this leaned on `formatFloat` + `parseFloat` to do the round-tripping. It worked, but it was doing string formatting and parsing in a hot path. The faster rewrite does the whole thing in integer arithmetic and only builds a string once, at the end, for the precision that actually round-trips. So `0.5` prints as `"0.5"`, not `"0.5000076"`, and it gets there without touching the float formatter. ## Letting actions hold dnorms directly The last bit of cleanup was removing a round-trip I'd left in by accident. Actions were storing a `float32` and converting `unorm16 → float32 → string` every time they were serialized. Once the storage type *is* the dnorm, the action can hold it directly and skip the hop — fewer conversions, and the value you print is exactly the value you stored. A small set of dnorm literals and tighter range guards (opacity must be in `[0, 1]`, brightness in `[-1, 1]`) finished it off. ## Was it worth it? For the data that dominates Auto-Editor's work, yes. Memory and cache size for levels sequences dropped by half, and the threshold loop got *cheaper* in the process because it compares integers instead of floats. The quantization is lossless for the purpose: 16 bits over `[0, 1]` gives a resolution of about `0.0000153`, far finer than any threshold a human would set or any difference that changes a keep/cut decision. You give up precision you were never using, and you get back half your bytes and a faster inner loop. Pick the data structure that fits the data. A loudness value between 0 and 1 never needed to be a 32-bit float. It just took borrowing a 30-year-old idea from graphics hardware to notice.