Scanning and analysis

Two distinct flows populate the database: scanning registers what is on disk (GraphQL mutation scanLibraries(libraryId?), ScannerController), metadata refresh enriches it with metadata from external providers (refreshMetadata(mode, libraryId?) and the per-item refresh* mutations, MetadataRefreshController).

Startup bootstrap

disk/.../StartupTasks handles Spring's ContextRefreshedEvent — no RabbitMQ events are sent at startup. It creates or updates NodeEntity, LibraryEntity and DirectoryEntity rows from the configuration properties (disk.yaml / env vars), creates the cache directories on disk, and validates the multi-node configuration. See the startup diagram.

Library scan

See the scan-flow diagram. scanLibraries() sends NEW_DIRECTORIES_SCAN_REQUEST per directory; the disk handler walks the filesystem and emits one FILE_SCAN_REQUESTED per file. FileScanRequestedHandle routes on extension (and library type). After the walk the rows of files that were not seen again are deleted (ScannedCache), and for a MUSIC library OrphanTrackCleanupService then removes the tracks left without a media file and the albums left without tracks. A moved or renamed file comes back as a new track, so before an orphan is deleted its watch status, play-queue and playlist items and rating are handed over to the track by the same artist with the same title that still has a file. Artists nothing refers to any more (no album, track, track credit, film credit, book, chapter or series) are removed too — typically a split collaboration credit such as "A & B". The credits are rewritten by the analysis that follows the scan, so such an artist disappears on the next scan. Every removed track, album and artist gets a delete in the search index.

HandleMediaFileFound does its ffprobe/ffmpeg work (streams, duration, crop detection, episode boundaries, the background still) outside any transaction and persists the result in one short TransactionTemplate block. On a slow disk a single file takes minutes; holding a connection "idle in transaction" that long starved the pool for every other handler. The events that need the new rows (subtitle extraction, intro detection) are sent after that commit. The extension lists are exact and short (PathObject): images are jpg/png, video is mkv/mp4/webm/m4v/flv/avi, subtitles are srt — a .jpeg or .wmv is simply not picked up. Which scanners run at all depends on the library type: a COMIC library uses only ComicScanner + ImageScanner; MUSIC uses audio/image/nfo, BOOK adds EpubScanner; only movie/show libraries run the SubtitleScanner and MediaFileScanner.

File	Event	Handler work
Video	MEDIA_FILE_FOUND	ffprobe streams + duration, extract embedded subs to SRT, screenshot as backdrop
Audio	AUDIO_FILE_FOUND	ffprobe, ID3 tags (title/track no, track credits from the artist tag — primary artist plus feat. guests — with the path artist as fallback), embedded cover, clear the HLS cache
.epub (BOOK library)	EPUB_FILE_FOUND	OPF title/language/description, media overlays from content, cover from the zip
.cbz/.pdf/.epub (COMIC library)	COMIC_FILE_FOUND (epubs reuse EPUB_FILE_FOUND)	page count, ComicInfo.xml, cover extraction
.srt	SUBTITLE_FILE_FOUND	link SRT to episode as an EXTERNAL_SUBTITLE stream
Image	IMAGE_FOUND	save ImageEntity, link to show/movie/episode/etc.
.nfo	NFO_FILE_FOUND	parse XML: title, description, release date, biography/review

Entity creation goes through ScannerHelperService.getOrCreate*, which also fires the *_FOUND enrichment events and the search-index creation events.

getOrCreatePerson looks a person up on the normalized name (PersonNames.normalize: lower-case, collapsed whitespace — mirrored by the generated person_entity.name_normalized column), so "ABBA" on one album and "Abba" on the next are one artist. The lookup is scoped per library, with a fallback to a library-less person (a TMDB actor, say) that is then attached to the library — not a single global lookup. The stored name keeps the spelling seen first, as the display value. ArtistTagParser splits a feat./ft./featuring tag into the primary artist and its guests. A collaboration credit — "A & B", "A, B", "A / B", "A + B" or the "A;B" ffprobe makes of a multi-valued tag — is split only when every part is a person the server already knows (in the library, or library-less like a TMDB actor) and the whole name is not an album artist itself: "Victoria Justice & Elizabeth Gillies" becomes two credits, while "Mumford & Sons" ("Sons" is nobody) and "Selena Gomez & The Scene" (owns albums) stay one act. The first part becomes the track artist, the rest are credited as featured.

Storage kinds and the S3 scan

DirectoryEntity.storageKind is LOCAL (an absolute path on exactly one owning node) or S3 (s3://bucket/prefix, no owner, N attached nodes in directory_node). Rows of an S3 directory store s3://bucket/key in the same path column, so the path parsers — pure string parsers that strip the root and split on / — and the (directory, path) uniqueness work unchanged. The two walkers share one per-file decision: AnalyzerSimpleFileVisitor (a java.nio adapter) and S3LibraryScanner (a delimiter-based recursive ListObjectsV2) both prune with DirectoryPruner and hand ScanEntrys to ScanEntryDispatcher, which asks the scanners and publishes FILE_SCAN_REQUESTED (its path is a string, with a tolerant setter for the file: URI older producers sent). HandleNewDirectoriesScanRequested takes a transaction-scoped advisory lock per directory, since an S3 directory's scan can be picked up by any attached node.

Handlers never touch java.nio on a row's path directly. core/.../storage provides the seams: ObjectStore (one per app.ister.s3.connections[n], held by ObjectStoreRegistry), FileAccess (stat/open/delete by directory + path), LocalCopy (an LRU-cached download for zip/PDF tools), CacheStore via CacheDirectoryResolver (every derived-file writer), and TmpStoreProvider (the shared transcode store, chapter 4). MediaFileInputResolver decides what ffmpeg reads: a local path, another node's tokenized download URL, or — for an S3 file on an attached node — the node's own /mediaFile/{id}/download over loopback (FileController proxies the object with Range), or a presigned URL when app.ister.s3.ffmpeg-direct is on.

Multi-episode files

A filename may carry an episode range — s04e06-e07.mkv, s04e06-08.mkv, s04e06e07.mkv — for a file holding up to three consecutive episodes. PathObject parses the range (an implausible range, backwards or longer than three, falls back to the first episode) and MediaFileScanner creates one EpisodeEntity per episode, so each gets its own TMDB metadata and watch status. The file's episode_entity_id FK always points at the first episode; every contained episode (including the first) additionally gets a media_file_episode_entity link row carrying its start/duration slice within the file. No link rows means "normal single-episode file" — all existing FK-based queries stay correct, and playback paths resolve files via MediaFileEpisodeService.filesForEpisode.

The slice boundaries are computed in HandleMediaFileFound (MediaFileFoundEpisodeBoundaries) once ffprobe knows the file duration: one MKV chapter per episode is used directly; with more chapters (scene markers) the chapter nearest to each equal-split point wins, unless that would make an episode implausibly short; otherwise the duration is split equally. Each episode also gets its own backdrop still, taken at the midpoint of its slice. Files scanned before multi-episode support are backfilled by a normal library rescan: the scanner notices an existing file whose path parses as a range but has no link rows, creates the missing episodes and links, and re-sends MEDIA_FILE_FOUND so the boundaries and stills get computed.

Sidecar files (NFO, local images, external subtitles) attach to the first episode of the range, as before.

Crop detection

Some rips carry baked-in black bars. HandleMediaFileFound runs a crop-detection step (MediaFileFoundDetectCrop): ffmpeg's cropdetect filter samples a handful of moments in the file, and the converged crop rectangle is stored on the video's MediaFileStreamEntity (crop_* columns, V37). This decodes a few dozen frames per sample, so it is not free — it runs as part of the file analysis, not on every scan. Files analyzed before the feature existed are caught by a scanner-side backfill: app.ister.server.crop-detect-backfill (default true) re-sends MEDIA_FILE_FOUND on a rescan for video files whose streams have no crop values yet. The consumer of the rectangle is the player, via the GraphQL crop fields — the transcoder deliberately leaves the bars in place (chapter 4).

Subtitle extraction

Embedded text subtitle streams (subrip, ass/ssa, mov_text, webvtt) become SRT files in the owner's cache directory, one SUBTITLE_EXTRACT_REQUESTED event per stream fired after the file's analysis commits (HandleSubtitleExtractRequested → SubtitleExtractionProcessor → SubtitleExtractor). It is a plain ffmpeg remux, but one that reads the whole container — hence a separate, non-transactional event rather than a step inside MEDIA_FILE_FOUND, and one message per stream so each stays well under RabbitMQ's consumer timeout. The result is an EXTERNAL_SUBTITLE row whose path is the owner-local SRT; a stream whose extraction fails is flagged extractionFailed so a redelivered event does not retry it (a re-analysis rewrites the stream rows and does). Bitmap subtitles (Blu-ray PGS, DVD VobSub) get no event and no row: they are never turned into text — the transcoder serves their pictures as sprite sheets when playback is set up, see chapter 4. The family is helper-capable: a helper node reads the source through the owner's download URL, extracts into its own tmp dir, uploads the SRT with POST /cache/upload/{fileName} and records the owner's path, so the row is indistinguishable from a local extraction. Because extraction now finishes after the analysis, a master playlist generated in between lists the SRT rendition only once that file's playlist cache is regenerated.

Intro/outro detection

Recurring intros and closing credits are found by comparing audio across a season: the disk module (on the owner, or on a helper node listing the directory for DETECT_SEGMENTS — the reader takes a local path or the owner's ranged download URL alike) decodes short windows (first 10 / last 4 minutes of each episode's slice) to mono PCM, fingerprints them (ChromaFingerprinter, a chromaprint-style 32-bit gradient hash per 128 ms — loudness-invariant, no external library), and SegmentMatcher finds the longest shared run between an episode and up to four season neighbours. Because a lag between two episodes that falls between hash frames shears every frame pair apart (the shared run then fragments below the minimum length), the comparison side is fingerprinted at four quarter-hop phase shifts (0/32/64/96 ms) and the phase with the longest run wins. The median bounds over agreeing pairs (at least two, one for two-episode seasons) become media_file_segment_entity rows (INTRO/OUTRO) in absolute file time; in a multi-episode file each slice gets its own rows, disambiguated by episode_entity_id. The player reads them as MediaFile.segments for its skip-intro / next-episode buttons. An intro must be 10–150 s long and start within the first 8 minutes — the bounds are deliberately loose because many intros carry per-episode voice-over over the same music, leaving only part of the audio identical, and long cold opens push the intro well past the five-minute mark.

Intro matching runs in two stages. Pending episodes are ordered from the season's ends inward (first, last, second, second-to-last, …), pairwise-matched as above. Once three episodes of the season carry a confirmed intro, the remaining episodes switch to template matching: their window is matched against up to three confirmed intros (first, middle, last of the season, so a mid-season variant switch still contributes both variants). A template is a known-good intro, so a single ≥ 8 s match suffices where the pairwise stage demands two agreeing neighbours — that rescues episodes whose intro variant none of their four neighbours share — and sliding a ~20 s template over a window is far cheaper than aligning two whole windows against each other. When the season's last chunk finishes, episodes that failed the pairwise stage get one template retry. Outros always use the pairwise stage (their 4-minute window keeps that cheap).

Because detection is cross-episode it cannot run inside the per-file MEDIA_FILE_FOUND handler: that handler instead fires a season-scoped DETECT_SEGMENTS event after its transaction commits, on the same directory-scoped queue family, so detection runs on the node owning the files. HandleDetectSegments is idempotent — files whose media_file_entity.segment_detector_version already equals the current detector version are only used as comparison material — so one event per analyzed episode is fine: the last episode's event does the real work. The version column is also the "ran but found nothing" sentinel; null means detection never ran, and the scanner's backfill (app.ister.server.segment-detect-backfill, default on, once per season per run) sends DETECT_SEGMENTS for such files on a rescan. Per-item reanalysis wipes the segment rows and resets the version, and bumping SegmentDetectionChunkProcessor.DETECTOR_VERSION re-runs detection everywhere via the same backfill. Known limitation: a season spread over multiple nodes only pairs the episodes local to each node — a node holding a single stray episode detects nothing for it.

While an admin upload into the directory is still running (upload_session_entity ACTIVE), HandleMediaFileFound does not fire the event: detection would compete with the transfer for CPU and, worse, compare the first episodes that landed against one or two siblings, find nothing and stamp them as done. UploadSessionService publishes one DETECT_SEGMENTS per season when the session finishes (UploadedFilePublisher.publishSegmentDetection); an episode whose analysis completes after that fires its own event as usual.

All ffmpeg/ffprobe work in this chapter runs through BackgroundFfmpeg (core): wrapper scripts that start the binaries under nice -n 10 (BACKGROUND_NICE), and HandleDetectSegments has its own listener concurrency (SEGMENT_DETECT_CONCURRENCY, default 1) instead of the shared helper value, so detection takes about one core and yields it to playback and uploads.

That idempotence is serial, not concurrent. Recomputing a file is a delete-then-insert, and two transactions doing it for the same season neither see nor cancel each other's rows, so a re-analysis sweep — one message per file, all for the same few seasons — stored every segment once per consumer. A chunk therefore claims its season with a non-blocking advisory lock (pg_try_advisory_xact_lock, namespace 1, key hashtext(seasonId)); a message that does not get the claim is dropped rather than retried, because the holder's chunk chain covers the whole season anyway. Blocking instead would idle a listener thread for the minutes a season takes and walk into the same consumer_timeout. A unique index on (file, type, episode) with NULLS NOT DISTINCT (V39, which also de-duplicated the existing rows) is the backstop.

One message detects at most app.ister.server.segment-detect.chunk-size episodes (default 4): fingerprinting a whole season in one go can outlast RabbitMQ's consumer_timeout (default 30 minutes), which closes the channel and requeues the message forever. Like the blur-hash sweep, SegmentDetectionChunkProcessor runs one chunk in its own transaction (stretched so slices of one multi-episode file never straddle a chunk boundary), and HandleDetectSegments publishes a successor message for the same season only after that commit. The version column is the cursor, and it is stamped even when decoding fails, so the chain always terminates.

Games

A GAME library is platform-first (<dir>/<platform>/<Title (Year)>/…, parsed by GamePathObject; the platform table — folder aliases, launchable extensions, companion extensions, default launcher — is the GamePlatform enum). GameScanner reacts to exactly two kinds of file: the game file of a single-file form (a ROM and its .bin companions, a PC zip, a Linux installer or tar — each becomes a MediaFileEntity with gameEntity, like comic volumes) and the marker of a folder form (PS3_GAME/PARAM.SFO, appmanifest_<appid>.acf). Everything else inside a PS3 or Steam tree is deliberately not scannable: such a game is 5k–100k files, which would swamp the media-file table and the scan cache, so it is one game_entity row with rootPath, sized by walking the tree (GameTreeLister, LOCAL and S3) and re-sized on every rescan of the marker. DirectoryPruner descends into a game's sub-folders only for those two platforms. A Steam marker is parsed on the spot (VdfParser: app id, name, installdir, build id) and the game only counts when the installdir folder exists next to the .acf. Creation publishes GAME_FOUND (global queue); the handler is described in chapter 3.

Admin upload

See the upload-flow diagram. LibraryUploadController (disk module, /library-upload/**, admin only, bearer JWT only) lets the player write files INTO a library directory. It lives in disk because the path parsers do, and it is REST because no module but api hosts GraphQL.

• One decision, three callers. Scanners.forLibrary / analyzableBy is the single place that knows which scanners look at a library type and how each is asked. The scan walk (ScanEntryDispatcher), the FILE_SCAN_REQUESTED handler and UploadPreviewService all use it, and the preview additionally applies DirectoryPruner.shouldDescend to every ancestor folder. A file the preview calls recognised is therefore a file the scan picks up. The parsed details shown to the admin come from the path objects' explicit three-argument constructors (the directory flag): the dot heuristic of the two-argument form misreads R.E.M. and J.K. Rowling.
• Paths. LibraryPathValidator replaces SafeFilename here (real media names have spaces, parentheses and unicode): it rejects by rule — traversal, separators, control characters, dot-prefixed segments (the scan skips them, and it keeps .ister-upload out of reach), reserved names — NFC-normalizes, and asserts containment with PathStrings.isUnder. LocalLibraryWriteStore additionally resolves the real path of the nearest existing parent before creating anything, so a symlinked folder cannot carry a write out of the library.
• Storage seam. LibraryWriteStore (core/.../storage/) is the write counterpart of FileAccess. LOCAL appends chunks to <root>/.ister-upload/<session>/<file>.part and completes with an atomic move on the same filesystem. S3 maps a file to one multipart upload and a chunk to one part; the upload id and the part ETags live in upload_file / upload_part, because listing pending multipart uploads by prefix is not something every S3 server does (MinIO only answers for an exact key). complete is safe to repeat.
• Transactions. A chunk is "short transaction, stream, short transaction" (UploadSessionService, explicit TransactionTemplate): nothing holds a database transaction while bytes come in over a slow uplink. A semaphore bounds the chunk requests per node (each one still pins a connection through open-in-view), and a per-file lock keeps two writers off one part file.
• Into the pipeline. UploadedFilePublisher publishes after commit. A new file is one FILE_SCAN_REQUESTED on <base>.<directoryName> — what the scan walk would have sent. A replaced file cannot go that way (the scanners see a row and stop), so its analysis event is published directly, after dropping what was derived from the old bytes (the HLS tmp dir, the shared tmp store entry, the LocalCopy of an S3 object).
• Multi-node. Nothing is forwarded: the client sends chunks to Directory.servingNode (LibraryWriteStoreResolver.servingNode, which the GraphQL field delegates to), and a request that reaches a node that cannot write the directory gets 421.
• Cleanup. UploadCleanupScheduler (hourly, every node) expires idle sessions whose storage it can reach and removes staging folders without an active session.

Game trees in the upload

Because only the marker of a PS3/Steam game is scannable, the preview would call every other file of the tree unsupported. UploadPreviewService therefore groups the entries per game folder: when the group holds the marker, every file under that folder is RECOGNISED (GAME_TREE_FILE); without it they are IGNORED with GAME_INCOMPLETE and a detail naming the missing marker. On completion UploadedFilePublisher publishes no event for tree files at all and defers the marker's FILE_SCAN_REQUESTED to the end of the session (publishGameMarkers), so the scanner sizes a complete tree. max-files-per-session defaults to 100 000 for the same reason.

Metadata backfill

See the refresh-flow diagram. refreshMetadata(MISSING) sends one global METADATA_BACKFILL_REQUESTED event (consumed by exactly one worker, so the backfill runs once cluster-wide; it used to run per node, which duplicated all globally-queried book/comic/ music/person work on multi-node installs). MetadataBackfillHandle finds everything missing metadata, artwork or TMDB enrichment (movies/shows also match when their tmdbId was never filled — the V45 backfill marker) and fans out: SHOW_FOUND / EPISODE_FOUND / MOVIE_FOUND (TMDB), PERSON_FOUND / ALBUM_FOUND (MusicBrainz + NFO lookup on the disk side), AUDIO_FILE_FOUND for tracks, BOOK_FOUND/EPUB_FILE_FOUND for books, COMIC_SERIES_FOUND/ COMIC_FILE_FOUND for comics, and an optional libraryId scopes everything to one library. The controller sends UPDATE_IMAGES_REQUESTED per directory for the BlurHash sweep itself (those queues are directory-scoped, so the work lands on the owning node). The steps run in separate transactions in MetadataBackfillService; the book-series heuristic runs once cluster-wide in its own write transaction. The per-type pipelines are covered in chapter 3.

Force refresh (per item or per library)

refreshMetadata(FORCE, libraryId) and the per-item mutations (refreshShow(id), refreshMovie(id), …) send ANALYZE_DATA, consumed by two handlers: AnalyzeDataHandle (worker) wipes the item's metadata/images/streams and cascades — a library fans out per type (shows, movies, artists, book authors, comic series), a show to its episodes, a person to their albums and books, an album to its tracks — re-firing the *_FOUND events (persons get both the global send for the external enrichment and the node-scoped send for the artist.nfo re-parse); HandleAnalyzeDataDisk (disk) clears the HLS cache and re-emits the file-level events (MEDIA_FILE_FOUND/AUDIO_FILE_FOUND, NFO_FILE_FOUND, SUBTITLE_FILE_FOUND). Deleted image rows leave their cache files behind (unlinking cross-node is unsafe); the daily cache cleanup reclaims them.

The *_FOUND events are published after the wipe has committed (AfterCommitPublisher.publishAfterCommit): their consumers check for existing metadata/image rows and would otherwise still see the doomed rows and skip the refetch, leaving the item permanently without covers. For albums, disk-side HandleAlbumFound additionally re-emits FILE_SCAN_REQUESTED for local artwork (cover.jpg and friends) in the album directory — album analysis wipes those image rows too, and unlike movies/episodes no directory rescan follows, so the files are re-ingested explicitly (deduped by ImageScanner on the existing (directory, path) row).

The BlurHash sweep

HandleImageFound deliberately saves images without a BlurHash: encoding one is CPU-expensive and made that handler the bottleneck of large scans. The hashes are filled afterwards by the UPDATE_IMAGES_REQUESTED sweep, per directory — including the cache directory, which holds the downloaded artwork and therefore the vast majority of images.

Each message processes at most app.ister.server.blur-hash.chunk-size images, then publishes a successor message carrying a keyset cursor (afterId). One sweep over a whole library in a single message used to exceed RabbitMQ's consumer_timeout (30 minutes), so the message was requeued and the sweep restarted endlessly without ever committing. The chunk size alone does not rule that out on a busy node, so a chunk is also time-boxed (app.ister.server.blur-hash.chunk-seconds, default 120): when the budget is spent the images hashed so far are committed and the successor continues after the last of them; only a chunk that finishes with fewer images than the chunk size ends the sweep (Chunk.exhausted). Images are downscaled to at most 96 pixels on the long side before encoding: a BlurHash captures a few gradients, so hashing a full-size poster is wasted CPU.

Two subtleties:

• The cursor is a keyset on id — not an offset and not "next row without a hash". An image that can never be hashed (a corrupt file) keeps blur_hash NULL; a naive LIMIT query would re-select such rows every round and never terminate. PostgreSQL orders uuid unsigned while java.util.UUID.compareTo compares signed, so both the ORDER BY and the id > comparison must run in the database, never in Java.
• The successor message is published only after the chunk's transaction commits (BlurHashChunkProcessor). The other order would let a failed commit leave a cursor pointing past work that was never saved.

Images are decoded through RasterImageDecoder, not ImageIO.read directly. A steady trickle of JPEGs — TMDB serves plenty — carries an ICC profile whose component count disagrees with the actual number of raster bands, typically a Photoshop export tagged CMYK over three-band data. ImageIO builds its colour model from that profile and refuses the file outright, so those images used to keep a null hash for good. The decoder falls back to reading the raw raster, which bypasses colour management, and converts by hand: readRaster performs no conversion at all, so a plain JFIF JPEG arrives as YCbCr rather than RGB, and the Adobe APP14 marker decides whether four bands are CMYK or YCCK.

Related scheduled jobs

CacheCleanupScheduler (disk) and TmpTranscodeCleanupScheduler (transcoder) run a daily zombie sweep of the image cache and transcode tmp dirs, deleting files no database row references, and expire old podcast downloads. app.ister.server.cache-cleanup.dry-run defaults to true — the cleanup only logs until that flag is switched off. A third, much more frequent sweep exists next to these two: HlsTranscodeService.cleanupOldFiles runs every 15 minutes over the HLS cache dirs and honors each dir's keep_until deadline — see chapter 4 for how the two HLS sweeps divide the work.