Three servers, one interface

Each kind of server is one package that turns its own API into the same Go interface, servers.Server; the app only ever sees that interface, and only the two programs know which package is behind it.

Inkshelf reads from three very different serversServer: Where books come from: Komga (comics), Mealie (recipes) or Calibre-Web (EPUB and CBZ). The app sees them through one Go interface.. Komga serves comics as JSON and page images. Mealie serves recipes, which have no pages at all. Calibre-Web serves ebooks through Atom feeds. Yet the app shows all three with the same screens: a list of series, the books of a series, a book to read.

This chapter shows how. The constraint behind it: the app should not grow a branch for each server. A fourth kind of server (and a Kobo port, one day) must not touch the screens.

🔌

Travel adapters. Every country has its own socket, but your charger has one plug. Each adapter does the translation on its side; the charger never learns where it is.

One interface and a neutral model

What the app needs from any server fits in one Go interface:

internal/servers/server.go (trimmed)

type Server interface {
	Series(ctx) ([]Series, error)                       // sorted by title
	Libraries(ctx) ([]Library, error)
	Books(ctx, seriesID) ([]Book, error)                // in series order
	InProgress(ctx, count) ([]Book, error)              // last read first
	OpenBook(ctx, bookID, pageCount) (Pages, error)     // a book of page images
	OpenFile(ctx, bookID) (io.ReadCloser, int64, error) // a CBZ or an EPUB, whole
	SetProgress(ctx, bookID, page, completed) error
	// … FirstPage, SeriesThumbnail, BookThumbnail, ClearProgress
}

Its types are neutral: a Series has a title and counts of books read; a Book has a number in its series, a page count and a Format. The format says how the book travels:

• PageImages: page by page, through OpenBook, each page a JPEG, PNG or WebP;
• ComicArchive: a CBZ, a zip of page images, fetched whole by OpenFile;
• Reflowable: an EPUBEPUB: A zip of web pages (XHTML), stylesheets, images and fonts, plus a package file that says in what order to read them., fetched whole and laid out while reading.

A server that cannot do something does not pretend. It returns an error wrapping errors.ErrUnsupported, and the app does without: no library filter, no "continue reading" row, progress kept on the reader alone. Never an empty list: the app would take it for "nothing there" and, for progress, for "unread".

Three translations

Each kind of server is one package (servers/komga, servers/mealie, servers/calibreweb) with the same parts: a Client that speaks the server's API call by call, and a Server (adapter.go) that turns it into servers.Server. Each doc.go explains its server.

InteractiveWhat each server says, and what the app sees

What the server says

GET /api/v1/series/{id}/books
X-API-Key: …

{"content": [{"id": "0A1…", "name": "Tome 1",
  "media": {"pagesCount": 46},
  "readProgress": {"page": 12}}], …}

What the app sees

Series: a Komga series

Book: a volume or a chapter

Format: PageImages: fetched page by page, /api/v1/books/{id}/pages/{n}

The only kind that keeps reading progress: the app reads it with the books and PATCHes it back.

servers.Server, method by method

Series	✓
Libraries	✓
Books	✓
InProgress	✓
OpenBook	✓
OpenFile	ErrUnsupported
FirstPage	✓
SeriesThumbnail	✓
BookThumbnail	✓
SetProgress	✓
ClearProgress	✓

Try this: Switch between the three. Compare the right-hand table: only Komga keeps progress, only Calibre-Web sends whole files, and Mealie answers Unsupported to almost everything but still reads like a comic.

• Komga is the closest to what the app wants. It authenticates with an X-API-Key header, answers in JSON, and sends each page as an image. It is the only kind that remembers how far you read: the app reads the progress with the books and sends it back with a PATCH (Syncing reading progress). One subtlety, pageConversions: Komga can convert any page to JPEG or PNG on the fly, but its PNG of a JPEG is much bigger, so the app asks for JPEG, PNG and WebP pages as they are, and the rest as PNG.
• Mealie is the strangest. A category becomes a series, a recipe a book. But a recipe has no pages, so the Mealie package makes them, on the reader: recipepages.Layout places the name, the photo, the ingredients and the steps on 1404 × 1872 pages, and Render draws each as a PNG. To the rest of the app, a recipe is a book of page images, like a comic. The page count is only known after layout, so Books lays out every recipe it lists and keeps the result (laidOut), so that OpenBook later agrees on the count.
• Calibre-Web is read through its OPDSOPDS: A catalog of books as Atom feeds, a page at a time. Inkshelf reads Calibre-Web through it. catalog only: Atom feeds, with HTTP Basic auth. (Its JSON API needs a login session.) The catalog is read whole; a book's number in its series is found in its entry's text ("SERIES: Eragon [2.5]"). An EPUB is Reflowable, a CBZ ComicArchive, and a book with both is read as the EPUB. Files can be large, so they come through a copy of the HTTP client with no timeout (fileClient), watched by the download's WatchdogWatchdog: A timer that gives up on a download when no data has arrived for 30 seconds. instead.

config.json and the registry

The user lists their servers in config.json, on the reader:

applications/inkshelf/config.json

[
  {"name": "Manga", "type": "komga", "server": "http://nas:25600", "apiKey": "…"},
  {"name": "Cuisine", "type": "mealie", "server": "http://nas:9925", "apiKey": "…"},
  {"name": "Livres", "type": "calibreweb", "server": "http://nas:8083", "username": "…", "password": "…"}
]

servers.LoadConfig reads it, and is strict where a mistake would hurt later. The name is also the folder of that server's books on the reader, so it must be a valid FAT folder name, and two names may not differ only by case (FAT would see one folder).

Who turns "type": "mealie" into the Mealie package? Only servers/registry. Its Opener switches on the type and calls the package's NewServer; KindOf gives the kind's name, logo and words ("category" and "recipe" for Mealie, so the screens say the right thing). An empty type means Komga, the first kind there was, and calibre-web is accepted for calibreweb.

The RegistryRegistry: The one package that knows every kind of server: it opens the right one for each entry of config.json. Only the programs in cmd/ import it. is imported only by cmd/inkshelf and cmd/emulator, which hand registry.Opener to app.New. That is the trick that keeps app and shelf from importing any one server (The layer cake shows the go list command that checks it).

Sorting titles

Every server's series and books are sorted the same way, by servers.CompareTitles, because a plain string sort gets shelves wrong:

• "Tome 10" sorts before "Tome 2", since "1" comes before "2";
• "Le Trône de fer" files under L, with everything else that starts with an article;
• "Élégance" sorts after "Z", since "É" is a byte above every ASCII letter.

CompareTitles compares the sort keysSort key: What a title sorts by: without accents or a leading article, its numbers compared as numbers. "Le Tome 2" files under T, before "Tome 10". instead. SortKey spells apostrophes and dashes one way, drops a leading French or English article (le, la, les, l', un, une, des, the, a, an) and folds the accents away. Then naturalorder.Compare compares the keys with runs of digits as numbers. The details are French: "L'" is an article only before a vowel or an h, and in "A la croisée des mondes" the A is a "à" without its accent, kept.

InteractiveSorting a shelf

1. A la croisée des mondes → a la croisee des mondes
2. L’Arabe du futur → arabe du futur
3. Astérix → asterix
4. Élégance du hérisson → elegance du herisson
5. The Hobbit → hobbit
6. Œufs brouillés → oeufs brouilles
7. Les Rois maudits → rois maudits
8. Tome 2 → tome 2
9. Tome 10 → tome 10
10. Le Trône de fer → trone de fer

Try this: Switch to the plain sort and watch Tome 10 jump ahead of Tome 2. Add “Tome 9”, “Les Misérables”, “L’Homme” and “La 2e vie”.

Fake servers

You do not need a Komga server to work on Inkshelf. Each kind has a fake, in its own package: fakekomga, fakemealie and fakecalibreweb. Each is an http.Handler that answers like the real server, with fixed, generated content (comics with numbered pages, recipes, EPUBs that test right-to-left text, MathML, links). The EmulatorEmulator: A stand-in for the reader in a web page on your computer: the real event loop and screens, with a fake firmware. starts them with -fake and -fake-type, on local ports, and writes a matching config.json; the real client code then runs against them. Because they never change, the emulator's screenshots compare byte for byte.

make emu-fake                                                            # fake Komga
make shots SCRIPT=emulator/scripts/mealie.txt EMUFLAGS="-fake-type mealie"

Tests that do not need HTTP use servertest.Fake, a Server in memory whose content is in its fields and which records what it was asked (PageRequests, SentProgress).

Going deeper

• One HTTP client. httpclient.New makes the client every request goes through: a one-minute timeout and a few kept-alive connections per host. A response that is not 2xx becomes a *httpclient.StatusError with its code, so callers tell a 401 (wrong key) from a network error with errors.As.
• Read status. servers.SeriesStatus computes unread, reading or read from the counts, with the same rules as Komga's filter, so the series list filters every kind alike.
• Adding a kind of server is a checklist in the reference: a package with its Client, Server, Kind and logo, a fake, a case in both switches of the registry, an emulator script and its screenshots.

Where it lives in the code

• internal/servers/server.go · Server, Pages, Book, FormatWhat the app needs from any server, and its neutral model.
• internal/servers/config.go · Config, LoadConfigconfig.json, read strictly.
• internal/servers/registry/registry.go · Opener, KindOfThe switch on the type; imported only by cmd/.
• internal/servers/komga/adapter.go · Server, pageConversionsKomga: pages as images, progress kept.
• internal/servers/mealie/recipepages/layout.go · LayoutRecipes laid out into pages on the reader.
• internal/servers/calibreweb/opds.go · seriesPattern, formatPatternCalibre-Web’s OPDS feeds, read whole.
• internal/servers/sorting.go · SortKey, CompareTitlesTitles sorted by key, numbers as numbers.
• cmd/emulator/main.go · fakeHandlers, startFakeServersThe fake servers the emulator runs.

The full story: Servers.

Next: a book picked from a server has to come to the reader: Downloads and the shelf.

Check yourself

Mealie keeps no reading progress. What should its SetProgress return?

Where are a recipe’s pages drawn?

Which package may import servers/calibreweb?

How does CompareTitles order “Tome 10” and “Le Tome 2”?