Build internals

Reading time 5 min

The pinned builder is a Node process in build.js. It reads saved files and produces a static reader. It does not start the optional editor, authenticate readers, validate every authored link or upload output to a host. This page describes current internals, not a stable extension API.

Inputs and ownership

Input Selection
Markdown --content, else engine content/
Output --out, else engine dist/
Runtime data --data, environment, then default
Static files Installed engine static/
Templates Installed engine templates/
File policy Installed siteconfig.json

Folder arguments resolve from the working directory. Runtime precedence is explicit --data, MARKSTACK_DATA_DIR, parent of MARKSTACK_USERS_FILE, then engine .markstack/. Content/output arguments do not relocate static files, templates or configuration. Commands and Environment variables describe operational inputs.

The process reads file configuration when it starts. Malformed JSON prints a warning and starts from defaults; validated runtime site overrides are then applied. Raw HTML permission is captured from the owner’s file before those overrides, so Admin site settings cannot enable it. Do not interpret fallback defaults as successful configuration recovery.

Full build sequence

Stage Result
Reset Clear route map and search entries
Register Scan and reject duplicate routes
Clear Remove selected output recursively
Copy Copy installed static tree
Header Render chosen links/copy icons
Pages Render Markdown and index sections
Special Replace home; generate 404
Search Write search-index.json

Route registration runs before deleting existing output. A duplicate URL therefore fails without replacing the last output. That protection is narrow: a later Markdown/render/filesystem failure can leave partially rebuilt output. Direct build is not a transactional release. Use isolated output and review before deployment; editor Publish adds its own staged checks.

The builder does not protect arbitrary source paths from an unsafe --out choice. Never choose the engine, source, runtime data, a valuable existing directory or a parent containing them. Build workflow explains the safe operational path.

Routes and collisions

Folder titles come from _index.md when supplied, otherwise formatted directory names. Page titles come from frontmatter or filename. Each folder/page title becomes a lowercase ASCII slug: punctuation and non-ASCII letters are removed; whitespace becomes hyphens; repeated/edge hyphens are collapsed/trimmed. A folder’s slug prefixes its children.

The map records resolved source paths and claims normalized routes, ignoring trailing slashes for collision checks. Different source files with the same generated route fail with both source names. Do not depend on filename uniqueness alone. Title changes can change URLs even if files do not move.

Dotted entries are skipped. Folder _index.md maps to its folder route; ordinary pages map to their title route. The fallback for unmapped sources uses relative file paths. The builder does not add redirect rules for renamed titles or files. Non-Latin/empty slugs need actual fixture testing, not guessed transliteration. Frontmatter covers author-facing rules.

Item Ordering/visibility
Folders Integer order, then title
Ordinary files Title alphabetically
Folder landing Folder item, not child file
Hidden folder Omitted from sidebar subtree
Generated output Still built when hidden

Folders precede files. A page’s order does not set ordinary-file position. Expanded state follows the current URL; breadcrumbs follow source folders but use mapped title URLs. Previous/next cards flatten the visible sidebar order, including folder landings. A hidden page outside that list may have no previous/next cards.

hidden: true on folder metadata is navigation presentation, not access control. Route registration, content traversal and search still see that saved content. Protect private readers at the hosting layer or keep private source out of the public build.

Markdown to page

The YAML-only wrapper refuses alternate/executable frontmatter languages. markdown-it renders core Markdown with linkification and typographer. Plugins add heading anchors, footnotes, task boxes and five alerts. Supported code languages use Highlight.js; other code is escaped. Mermaid fences become escaped source containers for lazy browser rendering, not build-time SVG.

processMarkdownFile combines title, description, rendered body, mapped URL and navigation. generatePage fills the base template, applies header links and adds reader furniture. H2 headings form the on-this-page rail when at least two exist. Reading time excludes code/scripts/styles/SVG and rounds prose words at 200/minute, minimum one minute. Updated date uses updated, then date, formatted in UTC when parseable. These are display estimates, not editorial verification.

The base prefix is added to generated navigation/asset references where implemented. Authored root-relative links are not rewritten by a general link resolver. Canonical metadata combines configured site URL and route. Review subdirectory deployment explicitly.

Homepage has two passes

The ordinary scan renders root _index.md and adds its page/headings to search. Later generateHomepage replaces root HTML with optional configured hero and rendered _index.md body. Its document title is Home; its description can come from root frontmatter. Hero values are configuration, not an extra search-index pass.

Consequently, final home presentation and the root search entry are not identical construction paths. A root title/hero change needs inspection of both homepage and actual search results. The special generated home and 404 are not added as extra search entries by their generators. The build’s progress/log totals are not a reliable count of unique public routes.

Search construction

Entry Indexed text
Page Introduction before first H2-H4
Section Prose after heading to next H2-H4
Context Folder trail and owning page
Exclusions Code, scripts, styles, SVG

Heading entries use the actual rendered IDs. H1/H5/H6 do not become section entries. The index strips permalink markers, selected rendered elements and tags, decodes selected entities and normalizes whitespace. The count printed as “pages” is page plus heading entries, not Markdown-page count.

search-core.js and reader app.js consume this JSON in the browser. Code/diagram source is not searchable prose; write explanatory text beside it. Index generation does not test query relevance, network reachability, reader latency or memory. See Search for reader behavior and Validation limits for measurement boundaries.

Watch mode and change review

Watch mode performs the same full build for change/add/unlink events in content, static and templates. It is not incremental compilation. Runtime data and configuration are not watcher targets; process-start configuration remains in memory. Restart/rebuild deliberately when those inputs change rather than assuming watch picked them up.

Source areas to review together are build.js, lib/frontmatter.js, lib/site-settings.js, lib/links-store.js, templates/base.html, static/js/app.js and static/js/search-core.js. Preserve exact route/anchor/search contracts in tests and inspect actual browser pixels after content or renderer changes.