Português

The pyramid

A year of readings once a second is 31.5 million points. A screen is about 2,000 pixels wide. Reading all 31.5 million points to draw 2,000 columns, sixty times a second as a person drags, is too slow — and sending them over a network, far too slow. The pyramid is how Nonius draws that year from a few thousand numbers, and still draws exactly the line all the points would.

A summary: the bucket

A bucket summarises the points in a stretch of time in eight numbers:

Two buckets side by side merge into one: the first’s first, the second’s last, the least of the two minimums, the greatest of the two maximums, the counts and weights added. So a bucket of a minute can be made from four of fifteen seconds without going back to the points.

Levels

The pyramid is levels of buckets. Level 0’s buckets are the base wide — say one second. Each level above is four times as wide: 4 s, 16 s, 64 s, and so on. Buckets are aligned to 1970, so every client and server cuts them in the same places. A stretch with no points has no bucket.

What a view reads

A view is a time range on a number of pixel columns. Each column spans a stretch of time; the chart picks the coarsest level whose buckets fit at least 16 to a column, and merges them into one summary per column. When even level 0 is too coarse — fewer than 16 base widths a column, deep in a zoom — it reads the raw points instead.

So a frame reads somewhere between 16 and 64 buckets a column, a few thousand numbers for the whole screen, whatever the length of the history. And the line is exact: a column drawn from its least to its greatest value, joined to the next from its last value to the next’s first, is the same, pixel for pixel, as the line drawn from every point. The pyramid changes how much is read, never what is drawn.

Choosing the base and the levels

Two numbers, set where the pyramid is built:

• The base: about the spacing of your points where they are densest. One reading a second, a base of one second. Points in bursts — a thousand a second for ten seconds an hour — take the spacing inside a burst, a millisecond. A base much finer than your points costs memory (64 bytes a bucket); much coarser, and a deep zoom reads more raw points a column than it needs.

• The levels: enough for your widest view. A view reads at most 64 buckets a column while the top level is fine enough; past it, the top level’s buckets pile up. Take enough levels that

  base × 4^(levels − 1) ≥ widest span ÷ (64 × plot width)

  with the plot’s width in device pixels (CSS pixels × the pixel ratio). Take at least 2, even when the rule gives 1: a server keeps the top level always, so a single level 0 of one point a bucket is kept in full (What it costs). Three days of readings every 10 seconds, base 10 s, cost 88 bytes a point as a pyramid file with 1 level and 40 with 2. More levels than needed cost little — 44 bytes with 3, 46 with 9 — so when in doubt, take more.

Worked through: points a second apart, a year shown on 2,000 pixels. The widest span is 31,536,000 s; divided by 64 × 2,000 it is about 246 s. With a base of 1 s, the top level must be at least 246 s wide: 4⁴ = 256 s is level 4, so 5 levels.

Neither number is a promise you cannot change: a series file holds the points, not a pyramid, which is built from them when the chart or the server loads the file. (A server’s pyramid file keeps one built, and is built again to change it.)

Where they are set

The defaults cover years of points a second apart. nonius-server’s millisecond base suits fast data; for a sensor that reads every few seconds, pass --base-ns. A chart fed by tiles takes the server’s pyramid, from its metadata: its own settings are for points you push.

In Rust, building a pyramid is two calls:

#![allow(unused)]
fn main() {
// Summaries from one millisecond up, eight levels.
let mut pyramid = Pyramid::new(Width::new(1_000_000)?, 8)?;
pyramid.extend_sorted(series.iter())?;
}

What it costs

A bucket is 64 bytes. Level 0 of a one-second base over a year of one-second points has 31.5 million of them — as many as the points, at almost three times their size. That level earns nothing: a bucket of one point is the point. nonius-server therefore releases a level whose buckets hold fewer than 4 points on average (--sparse 4, the default) — any level but the top one, which it always keeps — and answers a view at that depth from the points themselves. Each level above has a quarter of the buckets of the one below, so the rest of the pyramid together costs about a third of its first kept level.

In Serving’s example that is: 24 bytes a point for the points themselves, 16 for level 1 (a 64-byte bucket for every 4 points), and about 5 for the levels above it — 45 bytes a point.

Serving shows a server building a pyramid and reporting what it kept.