From a scan to 16 grays

When a comic is downloaded, each page is decoded to gray, shrunk to the screen, given back its contrast, dithered to the panel's 16 grays and packed two pixels to a byte in a .pg4 file, so that turning a page only has to unpack it.

A comic page arrives from the server as a JPEG, PNG, WebP or JPEG XL file, often in color, often much larger than the screen. The panel shows 1404×1872 pixels in 16 grays16 grays: The InkPad 3 shows 16 levels from black to white, 17 apart on the 0–255 scale: 0, 17, 34, … 255., and it renders mid grays lighter than a computer screen does. Every page has to be translated into the panel's terms.

The translation is slow on the reader's small ARM processor: decoding a big JPEG alone takes a noticeable part of a second. A page turn already waits half a second for the e-ink panel; it cannot also wait for that. So Inkshelf does the work once, when the book is downloaded, and saves the result in a file that needs no decoder.

🍱

It is meal prep. On Sunday you wash, chop, cook and portion everything into boxes. On Wednesday evening, dinner is just "open a box and heat it". Downloading is Sunday; turning a page is Wednesday.

One page, four stages

All of it is one function, pageimage.Prepare, which runs four stages in turn. Here is a page going through them, computed in your browser with a faithful copy of the Go code:

InteractiveA page through pageimage.Prepare

A 300×420 color page, as Komga sends it (a JPEG, PNG, WebP or JPEG XL file).

Try this: Click the stages from left to right. At stage 3 the sky and the grass get darker; at stage 4, look closely at the sky: it is made of only a few grays, mixed.

The rest of the chapter takes the stages one by one. The last one, ditherAndPack, also packs the pixels into the file as it goes.

1. Decode straight to gray

DecodeGray (internal/graphics/pageimage/decode.go) opens the file and keeps only how bright each pixel is. For a color pixel that is its luma, a weighted sum of red, green and blue that matches how bright the eye finds each color: 0.299 R + 0.587 G + 0.114 B (green looks brightest, blue darkest). A JPEG already stores brightness apart from color (its Y plane), so for a JPEG the gray image is simply taken as it is.

Transparent pixels are often stored as black. Shown as is, a transparent PNG would come out as a black page, so transparency is laid over white, the color of paper.

2. Fit: average, don't pick

The page must shrink to fit the screen, keeping its shape. fit (internal/graphics/pageimage/fit.go) first computes the output size (fitSize): as wide as the screen, or as tall, whichever comes first. Then each output pixel covers a small box of source pixels, and takes their average. That is a box filterBox filter: Shrinking an image by making each new pixel the average of the source pixels it covers..

The obvious shortcut, taking one source pixel per box, is fast but loses thin lines: an ink stroke one pixel wide falls between the picked pixels and vanishes. Averaging keeps it, as a lighter gray.

InteractiveEach output pixel averages its box

source: 12×9

output: 5×3

output (1, 0) = 936 ÷ 6 pixels = 156
columns 2–3, rows 0–2

Output width5 pxPick one pixel instead of averaging

Try this: Hover (or tap) the output pixels and read the sum. Set the width to 5 and turn on “Pick one pixel”: the thin diagonal breaks up. Notice that the boxes are not all the same width.

The boxes come from integer division: output column c of n covers source columns from c × width ÷ n to (c + 1) × width ÷ n (boxSpan), at least one. While it writes each output pixel, fit also counts how many pixels have each gray: the image's histogramHistogram: A count of how many pixels have each gray level. It shows at a glance whether a scan is faded., which the next stage needs.

3. Tone: stretch, then bend

Old scans are "gray on gray": yellowed paper that is not white, faded ink that is not black. toneTable (internal/graphics/pageimage/tone.go) builds a Tone curveTone curve: A table that maps each input gray to an output gray: here it stretches the contrast and darkens the mid-tones for e-ink. from the histogram, in two moves:

1. Stretch. Find the gray below which the darkest 0.5% of the pixels lie, and the gray above which the lightest 0.5% lie (clippedShare = 0.005). The first becomes black, the second white, and everything in between is spread out over the whole range.
2. Bend. Raise the result to the power 1.8, the gammaGamma: A curve that brightens or darkens the mid-grays while keeping black and white. Inkshelf uses 1.8 to darken the grays of page images for e-ink; text gets none. (gamma = 1.8). That darkens the mid grays, which e-ink shows lighter than a computer screen: a gray at half intensity comes out at 0.5^1.8, about 29%.

Why clip 0.5% and not take the very darkest and lightest pixel? Because one speck of dust is enough to be pure black or pure white, and then nothing would stretch.

InteractiveThe tone table of a faded scan

before: gray paper, faded ink

after toneTable (not dithered yet)

out = 255 × ((in − 75) ÷ (204 − 75))^1.8, kept between 0 and 255

Clip each end0.5%Gamma1.8

The histogram uses square roots so the few ink pixels show next to the many paper ones.

Try this: Set “Clip each end” to 0%: a single dust speck at 0 and a scratch at 255 decide the stretch, and the page stays dull. Put it back, then play with gamma.

The table has 256 entries, one per possible input gray, so the next stage spends a single lookup per pixel on it.

4. Dither: hide the missing grays

Now each pixel must become one of the 16 panel grays: 0, 17, 34, … 255 (grayStep = 17). Rounding each pixel to the nearest one is easy, but a smooth shade then turns into flat bands with hard edges.

DitheringDithering: Hiding the lack of grays by spreading each pixel’s rounding error onto its neighbours, so the eye sees the right average. does better. Round a pixel, note how much it was rounded by (its error), and hand that error on to the neighbours not drawn yet, so they make up for it. Floyd–Steinberg dithering, the classic recipe that ditherAndPack (internal/graphics/pageimage/dither.go) follows, splits the error in sixteenths: 7 to the right, 3 below left, 5 below, 1 below right.

InteractiveFloyd–Steinberg, pixel by pixel

pixel 1/49

A gradient from 100 to 136. Each cell shows its gray; red numbers are errors carried to it from pixels already drawn. Press play, or step with ›.

the gradient

plain rounding: bands

Floyd–Steinberg: mixed

Try this: Step through the first row with ›: watch the red numbers (carried errors) appear on the next pixels. Then press Play and compare the three small grids at the bottom.

A few details, exactly as in the Go code: the rows always run left to right (no back-and-forth); the errors are kept multiplied by 16, and divided (rounding toward zero) only when a pixel reads them; the nearest gray is (value + 8) / 17, kept between 0 and 15.

From afar, the eye averages the grain, and the shade looks smooth again. With 16 grays the grain is very fine; with fewer it shows:

InteractiveRounding against dithering

4 grays, dithered: the bands are gone, and from afar the average gray is right.

Grays4

Try this: Choose 2 or 3 grays and switch between Round and Dither. Then go up to 16, the panel's number, and look at the gradient at the top.

Then pack: two pixels per byte

A panel gray is a number from 0 to 15: four bits, half a byte, a NibbleNibble: Half a byte: 4 bits, a number from 0 to 15. Exactly one of the 16 grays.. So ditherAndPack writes two pixels in each byte, the left one in the high half, straight into the output as it goes. The result is a .pg4.pg4: Inkshelf’s page image file: a small header, then two pixels per byte (4 bits each), ready to draw. file (internal/graphics/pageimage/pg4.go):

Offset	Size	What
0	4 bytes	PG4\x01: the format's name and version
4	2 bytes	the width, little-endian (low byte first)
6	2 bytes	the height, little-endian
8	(width + 1) ÷ 2 bytes per row	the rows, top to bottom

InteractiveInside a .pg4 file

a 5×3 page; the numbers are gray indexes 0–15

left pixel	15 × 17 = 255	1111
right pixel	10 × 17 = 170	1010
one byte	0xFA	11111010

Pixel (0, 0)gray 15

5047340105000300FA4080C6F39005BF20

50 47 34 01 is “PG4” and version 1; 05 00 03 00 are 5 and 3, low byte first; then 3 rows of (5 + 1) ÷ 2 = 3 bytes. The last byte of a row only uses its high half.

A full page, 1404 × 1872, one byte a pixel	2,628,288 bytes
The same page as .pg4: 8 + 702 × 1872	1,314,152 bytes
A 200-page comic as .pg4 (pages that fill the screen)	263 MB

Try this: Click a pixel and move the slider: watch its four bits change, then its byte in the stream. Hover a byte of the stream to find its two pixels. Look at the last byte of each row.

Half the size of one byte per pixel, and no decoder: reading it back is a table lookup.

At page turn: just unpack

When you turn a page, Store.LoadPage (internal/shelf/store.go) reads 0007.pg4 and pageimage.Unpack expands it. Unpack uses a table, gray4, built once: for each of the 256 possible bytes, the grays of its two pixels. One lookup per byte gives two pixels. The app then copies the image to the screen and refreshes the panel. It even unpacks the pages next to the one you read ahead of time, so most turns only draw.

internal/graphics/pageimage/pg4.go

// gray4 maps a byte of a .pg4 page to the grays of its two pixels.
var gray4 = func() (table [256][2]uint8) {
	for packed := range table {
		table[packed] = [2]uint8{uint8(packed>>4) * grayStep, uint8(packed&15) * grayStep}
	}
	return table
}()

The same Prepare serves more than comics: book covers, the pages of a fixed-layout EPUB (drawn first, then prepared with PrepareDecoded), and pictures inside an EPUB's text, which are prepared when the page is drawn, at the size the layout gave them.

Going deeper

• Two teams of workers. Store.Download (internal/shelf/download.go) runs 3 fetchers (fetchWorkers) that ask the server for pages, and preparers that run Prepare: one per core, at least 2, at most 8 (prepareWorkers), because each holds a decoded page of 20 to 40 MB. A small queue between them keeps memory bounded.
• Hot paths. fit, ditherAndPack, rgbaToGray and Unpack run on every page and are marked HOT PATH. They were tuned by reading the ARM assembly: helpers small enough to inline, slices shaped so the compiler drops bounds checks. Changing one needs its benchmark (BenchmarkFitDither, BenchmarkUnpack…) before and after, and the same inlining for ARM.
• Dividing without dividing. ARM is slow at division, so fit replaces "divide by the box's area" by "multiply by 2³² ÷ area and keep the high 32 bits" (reciprocalTable, divide). That is exact as long as the area stays below 4096 (maxReciprocalArea); larger boxes divide normally.
• A test twin. fit must give exactly what fitRef, a plain version of the same box filter in fit_test.go, gives.
• Huge JPEG XL images. Their header can claim any size, so one above maxJXLPixels (2²³, about 8.4 million pixels) is refused before decoding, and the decoder's panics are recovered.
• The widgets are exact. The TypeScript in src/lib/pageimage.ts follows the Go code step by step; on the same input, it produces the same .pg4 bytes as PrepareDecoded.
• Never change the format. Books already on readers are .pg4 files: the layout above, the magic and the 16 grays stay as they are (golden tests check it). Books from older versions have NNNN.png pages, which LoadPage still reads.

Where it lives in the code

• internal/graphics/pageimage/prepare.go · Prepare, PrepareDecodedRuns the stages in turn; PrepareDecoded skips the decoding.
• internal/graphics/pageimage/decode.go · DecodeGray, toGray, lumaAny image file to gray, transparency over white.
• internal/graphics/pageimage/fit.go · fit, fitSize, boxSpanThe box filter down to the screen size, and the histogram.
• internal/graphics/pageimage/tone.go · toneTable, clippedShare, gammaStretch with 0.5% clipped at each end, then gamma 1.8.
• internal/graphics/pageimage/dither.go · ditherAndPackFloyd–Steinberg to 16 grays, written straight as nibbles.
• internal/graphics/pageimage/pg4.go · newPG4, Unpack, gray4The .pg4 layout, and reading it back with a 256-entry table.
• internal/shelf/download.go · fetchWorkers, prepareWorkersDownloads pages and prepares them in parallel.
• internal/shelf/store.go · LoadPageReads a page back at page turn.

The full story: The image pipeline.

Next: the app has its pixels, on a canvas. How does a Go program with no SDK get them onto the glass? That is Running Go on a PocketBook.

Check yourself

Why is a comic page prepared when it is downloaded, not when you turn to it?

Why does fit average a box of source pixels instead of picking one?

A pixel worth 120 is rounded to the panel gray 119. What does Floyd–Steinberg do with the difference?

How many bytes does a row of a 1404-pixel-wide .pg4 page take?