Storage and formats

Reading time 7 min

Keep source, generated artifact, private runtime and browser recovery separate. Some runtime text renders publicly; that does not authorize publishing its whole storage folder. Restore each layer deliberately and verify its actual consumer.

Storage audiences

Layer Typical contents
Source Markdown/frontmatter, authored static inputs
Engine Configuration, templates, code/vendor assets
Reader artifact HTML, search JSON, copied media/icons
Private runtime Accounts, settings, links, audit, marker
Browser Session cookie, recovery drafts, reader preferences
Process memory Sessions, login limiters, current Publish job

A static artifact is not a source/runtime backup. Browser recovery is not server history. Memory is not persisted across restart. Backup and restore defines operational recovery.

Markdown and routes

Source is UTF-8 Markdown with optional YAML-only frontmatter. The parser accepts ordinary YAML/YML labels, BOM/string/Buffer input and refuses executable/alternate frontmatter engines. Writer preview’s lightweight strip/parser is not identical to builder. Frontmatter gives field types and delimiters.

Root and folder _index.md are landing inputs; normal lowercase .md files create articles. Dot-prefixed names are skipped by normal scans, not a secure storage policy. Title-derived ASCII route mapping differs from source filename and heading fragments. Unknown metadata is data, not a new feature. Review duplicate/empty/non-Latin routes before release.

Static files copy independently. A private PDF or source backup under static can become public even if Markdown scan ignores its source page. Template substitution/config are engine inputs, not browser-account preferences.

Generated reader

Normal routes use directory index.html. Root home and 404 are generated separately. Search JSON and all required CSS/scripts/fonts/vendor assets must deploy with HTML. The engine does not automatically generate sitemap, robots, RSS, PDF or host redirects; authored static versions are independent copied files.

Code/diagram containers are rendered in the browser where appropriate. Build file existence is not proof of real SVG/clipboard/assets or correct host status. Keep reviewed artifact manifest and real host checks separate from source save success.

Account file

users.json contains an object with users entries. Account records include name, role, section assignments, disabled/forced-change state, password hash, auth version and timestamps. Public account summaries omit password hashes. It remains private even when names appear in local administrative UI/logs.

Writes use owner-only mode 600 temporary file and rename. Cross-process change lock is adjacent .lock, with default five-second acquisition timeout and stale dead-owner handling after thirty seconds. Reads reload on inode/size/time stamp changes. This is local filesystem coordination, not distributed database replication or general Markdown concurrency control.

Damaged account JSON can stop reads/init rather than silently become authenticated. Do not repair by pasting hashes/accounts into public documentation. Restore/reconcile privately. Platform ACLs, parent-directory permissions and actual operating-system behavior still need review; a requested POSIX mode is not Windows certification.

Password hash

Stored format is scrypt$N$r$p$saltB64$hashB64. Current defaults are N=65536, r=8, p=1, 16-byte random salt and 32-byte derived key. Input is NFKC-normalized for derivation. Hash stores costs so later defaults can coexist.

Verification rejects unsuitable cost ranges/format before derivation and compares expected bytes with timing-safe equality. This is a description, not guidance to hand-edit credentials or tune production costs casually. Policy is 12-256 characters plus small common-password list. Use account workflows and approved secret handling.

Exact local formats and bounds

Value Implementation bound
Account temp Users filename plus process ID and .tmp
Lock Users filename plus .lock
Lock retry wait 20 ms synchronous wait
Session secret 32 random bytes, base64url cookie
Stored session key SHA-256 of supplied session token
CSRF secret 24 random bytes, base64url
Audit ID 24 lowercase hex characters
Audit event Lowercase letter first, up to 64 allowed characters
Audit actor/target Truncated to 64/200 characters
Publish audit ID Truncated to 64 characters

Session lookup refuses nonstring/very short tokens; hashing the token does not mean the in-memory CSRF value is absent. Idle/absolute expiry uses elapsed time greater than the configured limit, not equality. Session summary sweeps expiry without refreshing activity. Account versions invalidate old sessions on subsequent lookup.

Scrypt verification requires six stored fields with scrypt label, a power-of-two N between 1024 and 1048576, r between 1 and 16, p between 1 and 4, decoded salt at least 8 bytes and expected key at least 16 bytes. Stored defaults remain 65536/8/1 with 16-byte salt and 32-byte key. These checks are not a complete arbitrary-file schema or permission to replace hashes manually. Malformed data can still cause a derivation/storage exception. Restore through the approved private account workflow.

Audit event grammar is ^[a-z][a-z0-9-]{0,63}$. IDs are random record identities, not time-sortable sequence numbers. Imported/hand-edited lines are accepted by reads only with object/string ID/time/event checks; read does not fully revalidate every saved field or cryptographically authenticate it. There is no audit event for every ordinary Markdown save/create/delete. A missing such event is expected, not proof of failed source mutation.

Settings file

site-settings.json stores only supported safe branding overrides. Missing file gives no overrides; malformed JSON or invalid saved values can throw. Base effective defaults are combined with validated supported config fields, then saved overrides. It does not store base prefix, raw-HTML policy or process environment controls.

Write validates fields, compares expected SHA-256 revision and returns conflict when changed. New file uses mode 640 temp/rename. General/Appearance share a form/save; this revision behavior is not ordinary Markdown Save’s contract. Settings and links gives field limits and partial/persistence boundaries.

links.json has version 1 and a links list. Entries include 12-hex ID, title, full HTTP/HTTPS URL, show flag and validated favicon filename. Limits: 30 links, 40-character title and 2000-character address. Title whitespace collapses; credentials in URL are refused.

Read missing/damaged file gives empty list; invalid entries are skipped. Save validates whole list, replaces invalid/duplicate IDs and writes mode 640 temp/rename. There is no settings-style revision guard. A whole-list save can overwrite another administrator’s changes.

favicons/ holds validated ID filenames with supported image extensions. Saving links removes unreferenced matching icon files best-effort. Selected cached icons copy into reader output; link visibility/cleanup is not a secure deletion or reader access service. Fetched origin/resources have separate SSRF/network limits documented in Settings and links.

Local activity history

audit.jsonl is one JSON record per line, with ID/time/event and bounded actor/target fields. Publish records add summarized job outcome, page counts and failed stage. Not full source revisions, passwords, usable tokens or general arbitrary free-text storage.

Before append, file at least 1 MiB rotates. Three archives .1, .2, .3 are retained, plus current file. The threshold is not an exact total-byte cap: append can take current file beyond threshold until next write. New append file uses mode 600.

Reads skip malformed lines, reverse retained records, apply exact event/actor/target/publish filtering and paginate with a before-ID. Unknown before-ID returns no older records; result limit clamps 1-100, default 50. Write/read permission failures differ from a legitimate empty history.

This is bounded operational evidence, not immutable compliance storage or source undo. Operators can edit/delete it. Keep real retention/export/tamper requirements outside assumed app guarantees. Security and history describes UI checks.

Publish state and temporary output

publish-state.json beside users file records source fingerprint, publication time and page count. Fingerprint hashes paths/sizes/rounded modification times and folders, not file contents. A missing marker can show unpublished even if CLI-built reader exists. Equal-size/timestamp edits can evade it. It is not approval, a cryptographic artifact manifest or remote deployment state.

Publisher uses sibling output names for temporary build, source/runtime snapshot, old live folder and previous artifact. It snapshots Markdown and runtime links/settings/icons; static/templates/siteconfig remain installed-engine inputs. Verification checks required files, valid search JSON and page count, not all authored links/private data/browser results.

Two renames switch output, so a request between them can see a missing route. Failure restoration attempts are reported; do not universalize “all failures leave site untouched”. Backup rotation or marker recording can warn after new output is already live. Inspect actual reported folders before cleanup. Publishing stages owns that procedure.

Browser and memory

Reader preferences are local to origin/profile. Writer drafts are scoped to browser origin/account/path and can disappear through clearing, quota/privacy limits or file actions. They are not collaboration locks or guaranteed recovery of deletion. Drafts and saving explains exact Save/recovery boundaries.

Session cookie holds random token; server stores its SHA-256 hash and separate CSRF token in memory. Restart ends sessions. Login limiters and known-address state are also memory-only; restoring files does not restore them or a running publication job.

Recovery matrix

Loss Recovery source
Unsaved text Browser draft if actually retained
Deleted Markdown Reviewed source history/backup
Wrong reader Prior reviewed artifact/host rollback
Accounts/settings/links Private runtime backup and verification
Restarted sessions New sign-in, not copied memory
Audit older than retention Your external retained evidence if arranged

Verify restored source, account access, build/search/media and actual reader audience. Do not upload private runtime merely because restore succeeds.