Keyboard shortcuts

Press ← or → to navigate between chapters

Press S or / to search in the book

Press ? to show this help

Press Esc to hide this help

Português

Serving

A server holds a series and answers small requests about it: its description, tiles for a chart in a browser, and images. The protocol is plain HTTP GET, so any client, cache and proxy understands it. nonius-server is its reference server, a program you install with Cargo:

cargo install nonius-server

This chapter serves the readings of Your data: one point a millisecond for 100 seconds, in a CSV.

From a CSV to a running server

nonius-server csv readings.csv readings.nsb
nonius-server build readings.nsb readings.pyr.nsb --base-ns 1000000 --levels 4
nonius-server verify readings.pyr.nsb

The first line writes the series file. The second builds a pyramid file: the series and its pyramid, stored as the protocol’s tiles are, so a tile is read straight from it. The third checks its hashes and builds the pyramid again to compare. Each says what it did:

readings.nsb: 100000 points from readings.csv; 2 decimal places, the most any value has; in time order
readings.pyr.nsb: 100000 points, 3 levels kept from level 1, 32810 buckets; 4.5 MB, 45.0 B a point

The points are a millisecond apart, so the base is a millisecond. Level 0 would hold one point a bucket, and was released (see What it costs): three levels are kept.

A server can serve the plain series file too, building the pyramid in memory as it starts. A pyramid file is built once and then mapped: the server starts at once, whatever its size, and only the parts a request reads are brought into memory.

nonius-server readings.pyr.nsb --id readings --addr 127.0.0.1:8080 &

The & leaves it running in the background, for the commands below. A series’ id is its file’s name unless --id names it; several files serve several series.

Asking it

The series’ description, its metadata:

curl -s http://127.0.0.1:8080/v1/series/readings

{"protocol":1,"series":"readings","base_ns":"1000000","levels":4,"ratio":4,"tile_columns":2048,"raw_tile_ns":"4096000000","max_k":4096,"v_exponent":2,"rows":"100000","first_t":"1760000000000000000","last_t":"1760000099999000004","last":{"t":"1760000099999000004","v":"10099","w":"1"}}

The 64-bit integers are strings, because JSON numbers lose precision past 2⁵³. A browser’s attachTiles reads this first, then asks for tiles: /v1/series/{id}/columns/{level}/{k}/{i} for summaries and /v1/series/{id}/raw/{i} for points. Their addresses and contents are the protocol’s sections 4 and 5; a chart computes them itself, and you never write one by hand.

An image of the whole 100 seconds, light, with a title and a unit:

image='http://127.0.0.1:8080/v1/series/readings/png?from=1760000000000000000&to=1760000100000000000&width=1600&height=600&ratio=2&theme=light&title=Readings&unit=%C2%B0C'
curl -s -o readings.png "$image"

from and to are the time shown, [from, to): to is not included, so the whole series ends at its last point’s time plus one, or later. width and height are the PNG’s own pixels; ratio=2 draws labels and lines for a screen of two device pixels a CSS pixel, so this image fills a box of 800 × 300 on such a screen. The unit °C is percent-encoded, as a browser’s URLSearchParams writes it. The theme is dark unless theme=light. Every parameter is on nonius-server’s page; one the server does not know is ignored, so a misspelling (timezone= for tz=) quietly falls back to the default.

Caching

A tile or an image never changes: the same address always answers the same bytes. The server says so — Cache-Control: public, max-age=31536000, immutable — and gives each answer an ETag, a hash of its bytes. A client that already has it asks with the tag and gets 304 Not Modified, no body:

etag=$(curl -s -o /dev/null -w '%header{etag}' "$image")
curl -s -o /dev/null -w '%{http_code}\n' -H "If-None-Match: $etag" "$image"

That prints 304. So a browser, a proxy or a CDN can keep tiles and images for as long as it likes. The metadata is the exception: it says no-cache, because a series that grows changes its last point.

A page on another origin

A page served from one address reads a server at another only if the server allows it. nonius-server allows any page: every answer carries Access-Control-Allow-Origin: *, exposes its ETag and its image headers to scripts (Access-Control-Expose-Headers), and Cross-Origin-Resource-Policy: cross-origin, so a page that isolates itself for the live feed’s shared memory can still read it.

An image also says where its plot is: Nonius-Plot: x y width height, the plot’s rectangle in the image’s pixels, and Nonius-Range: lo hi, the values on its bottom and top rows. A page showing the PNG turns a click into a time and a value with them, and asks the server’s /pick for the point under it.

In production

nonius-server is the protocol’s reference and a test fixture, not a production server: no TLS, no limit on connections, one thread a connection. In front of users, put it behind a reverse proxy that terminates TLS and limits connections; since every answer but the metadata is immutable, the proxy can cache them all.

For a server of your own, two ways:

  • The protocol inside your Rust program, from the crate’s library:
#![allow(unused)]
fn main() {
let served = Served::new("day", (t, v, w), 2, Options::default())?;
}

served.answer(&request) answers a request with no socket, and serve puts it on a TcpListener; the crate’s page shows both.

  • The protocol in any language. It is written to be implemented without Nonius’s code: the protocol gives every address, byte and rule, and the images are drawn by the .NET package as by Rust.