Internals

Transpilation Pipeline

Bascik transforms source HTML into deployable HTML by replacing every custom component tag with its resolved, scoped content. The pipeline runs in two nested phases: the page phase and the component phase.

Overview

Every time a source page or component file changes, pageProcessing(filePath) in processing.ts is called. This is the top-level entry point for the whole pipeline.

Build scripts (<script data-bascik-build>) run first, before component resolution, so their output can contain component tags. Body HTML minification runs after component resolution so that whitespace-sensitive content from resolved components (e.g. <pre> blocks from <code-block>) is preserved intact.

Lifecycle and Pipeline Ordering Guarantee

Bascik guarantees strict, deterministic execution ordering across all compilation phases:

1. Output Directory Cleaning: In full builds and dev startup, directory.out is wiped cleanly before executing any lifecycle scripts.
2. Pre-phase Lifecycle Scripts: All pipeline.exec scripts configured with phase: 'pre' (or defaulting to 'pre') execute sequentially in array order and are fully awaited before any page or component compilation starts. This includes any watched scripts: their startup run is owned by the phase runner, so the later watcher registration does not re-run already-completed work.
3. Parallel Lifecycle Scripts: All pipeline.exec scripts configured with phase: 'parallel' launch alongside page compilation in both dev and build. Build joins all parallel tasks and the compilation/post branch before metadata finalization or success. Dev observes every outcome without blocking startup. Completion never recompiles pages or sends a reload. Later source edits select matching scripts and associated pages through one pre, parallel/compile, post cycle.
4. Dev Server Initialization (dev mode): In development mode, the dev HTTP/HTTPS server starts and registers watched exec handlers (startExecDev). Registering the watcher does not re-run startup work.
5. Page and Component Compilation: File watchers and initial multi-page transpilation (watchFiles() / processAllPages()) execute, compiling pages, evaluating page-level <script data-bascik-build> blocks, resolving components, injecting scoped CSS/JS, and writing output files.
6. Post-phase Lifecycle Scripts: All pipeline.exec scripts configured with phase: 'post' execute sequentially in array order after all page transpilation has completed.
7. Post-build Artifacts: Sidecar registration (server-scripts.json), CSP hash collection (csp-hashes.json), and the build manifest (manifest.json) are finalized and written to disk.

This guarantee ensures that pre scripts can reliably generate source files or data dependencies needed by build scripts and components, while post scripts can safely inspect or transform final compiled assets in dist/.

Artifact Accounting Ownership

Artifact accounting has a single owner: the main-thread disk writer. Collector state is process-local, so worker-local collectors can never reach the main thread. Every emitted artifact funnels through one owner so serial and worker builds record identical manifest and CSP metadata.

• transpilePage computes each page's inline script/style CSP hashes from its final emitted HTML (computePageCspHashes) before the bytes are transferred. It never records to a global collector itself, because the compiling agent (a worker or the main thread) is not the accounting owner.
• The single owner is writeTranspiledPage (in processing.ts). It writes the file and, only after a successful write, records the page's manifest entry and CSP hashes. A failed write throws before recording, so a failed write is never accounted as emitted.
• In the worker path (processAllPages with pipeline.workers: true), page-worker.ts passes deferDiskWrite: true to transpilePage, so the worker computes and transfers the bytes but never writes or records. The main thread writes the transferred bytes (a Buffer view over the transferred ArrayBuffer, no decode) and records manifest and CSP from them.
• Serial builds go through the same writeTranspiledPage owner, so both modes produce byte-identical artifact metadata.
• Raw-copied assets (copyReplicatePath) are recorded at the destination in dist/ after a successful copy, including hash-equal no-op copies. Recording the source path was the raw-asset defect: the manifest collector rejects paths outside dist/, so those assets silently vanished from the manifest. A failed copy throws before recording.
• The manifest, csp-hashes.json, server-scripts.json, and ownership.json remain explicitly excluded from being served and are regenerated each build, so no stale entries survive between builds.

Targeted Build Owned Artifact Transactions

A bascik --build --only … (targeted) build runs in a fresh CLI process. Its process-local collectors and in-memory route caches are empty, so without a durable source-to-output ownership inventory a targeted build used to silently:

• drop untouched B's server-sidecar entry (the sidecar writer serialized only the current process's registry), and
• leave a route removed from a dynamic template behind (the removed-route knowledge lived only in the in-memory templateToGeneratedRelativePaths map).

Bascik persists a versioned source-to-output ownership inventory at dist/.bascik/ownership.json (see lib/ownership.ts). Every output file and server-sidecar script is owned by exactly one source page. The inventory is the single source of ownership truth; it is never inferred from filename prefixes or mutable traversal order.

• Rebuilding a page exactly replaces that owner's recorded outputs and scripts with what the current build produced.
• A source page not in the targeted set is an untouched owner: its prior outputs and scripts are retained. This is what keeps untouched B's server script resolvable after A is rebuilt.
• A dynamic template that is rebuilt and now emits fewer routes (down to zero) is a rebuilt owner whose shrunken output set prunes the removed routes. A template that emits zero routes is recorded as a rebuilt owner with an empty set so its prior outputs are pruned too.
• Removed outputs are computed only for rebuilt owners from the difference between their prior and current output sets, so an obsolete output is never deleted based on a filename guess.

One transaction for HTML, sidecar, manifest, and CSP

finalizeOwnedArtifacts (in lib/ownership.ts) reconciles the sidecar, manifest.json, and csp-hashes.json from the same merged ownership inventory as the HTML prune. Individual collectors no longer implement their own merge rules; the coordinator is the single merge owner. Non-page files (raw assets, sitemap, robots) are retained across targeted builds because they are not owned by a page.

• Manifest and CSP entries for a still-owned page output come from the current build or the retained prior entry; entries for a pruned output are removed.
• The sidecar survives only for owned script ids: a rebuilt owner's scripts come from this build, an untouched owner's scripts are carried over from the prior sidecar.
• A full build cleans dist/ at startup, so the coordinator publishes the current process's state fresh with no merge.

Staging, validation, and rollback

Metadata commits are staged and atomic: the entire reconciled artifact set (ownership, manifest, CSP, sidecar) is first written to temp siblings and validated (each staged file must parse back to the intended JSON object), and only after every artifact staged successfully are the temps renamed over their targets. Obsolete outputs are deleted only after those commits succeed.

• A failure while writing or validating any staged artifact aborts before any rename, so the previous valid artifact set is left completely untouched.
• A rename failure surfaces and aborts before deleting any obsolete output, so a failed targeted update never deletes a file the committed metadata still references.
• Corrupt or schema-incompatible ownership.json (or a missing one) is treated as no reliable ownership: the targeted build degrades to the safe additive merge, logs a warning, and writes a fresh inventory so future builds are correct. This is the explicit safe first-targeted-build behavior.
• Full removal of a source page is only pruned when that page is rebuilt. A page deleted without being in a later targeted build is retained until a full build because a targeted build cannot know whether a source was deleted versus merely not rebuilt.

Contract: why global artifacts still need a full build

Whole-site artifacts (sitemap.xml and robots.txt) are not owned by a single page, so a targeted build keeps the documented behavior: it warns and skips regenerating them rather than publishing a partial site map. Regenerate them with a full bascik --build. This is a deliberate contract of targeted builds, not a bug workaround.

Multi-Page Startup: processAllPages

On startup (and whenever a component is added), the watch system calls processAllPages() instead of invoking pageProcessing() once per file. This avoids redundant I/O:

1. Hoist shared computation. listComponents() and resolveInlineStylesHtml() each run once, in parallel, before any page is processed. The results are passed to every page rather than re-computed per page.
2. Prioritize open pages first. In dev mode, partitionByOpenPages() separates pages into those currently open in active browser tabs and background pages. Active pages are transpiled, committed to MemoryStore, and have their "transpiled" reload events emitted immediately, so open browser windows update without waiting for the full site to compile.
3. Transpile each page. By default, pages are transpiled on the main thread via processPageBatch(). Main-thread dispatch approximates per-page synchronous work by accumulating execution segments around asynchronous pauses. The logged transpiled: ... in Nms duration excludes time parked at Bascik's explicit await boundaries, but it is not a measurement of total CPU time and does not include asynchronous callee continuations that run while the page is paused. When pipeline.workers: true is configured in bascik.config.ts, processAllPages() uses a worker pool instead (see Worker Pool Architecture).
4. Apply side effects on the main thread. As each page finishes transpilation, the main thread runs mem.storePage() and emits the "transpiled" event. Brotli compression inside storePage() runs in the background and does not block the page from being marked ready or served.
5. Write HTML without delaying dev serving. Build mode awaits each write to dist/. In dev mode, Bascik first commits the page to MemoryStore, then starts the dist/ write asynchronously. The server can return the updated page while that disk write is still pending.
6. Publish artifacts from the single owner. In build mode, the main thread is the single owner of disk publication and artifact accounting. A worker transfers the page bytes (UTF-8 ArrayBuffer, no structured-clone copy) and the main thread writes them and records the manifest and CSP. See Artifact Accounting Ownership.

Generation Ownership: Monotonic Dev Publication

Concurrent invalidation entrypoints (processPageBatch, pageProcessing, and the worker path in processAllPages) can race for the same page. Without coordination, an older, slower batch could finish after a newer direct edit and overwrite the newer state in memory, on disk, in the dependency indexes, or in the reload stream.

Bascik assigns every page a monotonically increasing generation at publication time. A job carries the generation it was claimed under and, when it completes, applies its side effects (memory store, disk write, sidecar record, transpiled event) only if that generation is still the latest for the page:

• processPageBatch claims a fresh generation at enqueue time, and pageProcessing claims one when it is about to publish, so a newer edit always supersedes an older one regardless of completion order.
• The worker path in processAllPages captures the page's current generation without bumping, so a broad worker rebuild can never become "newer" than a concurrent specific page edit. It publishes only if that captured generation is still current when the worker result lands.
• removePage bumps the generation, so in-flight work for a deleted page cannot resurrect it.
• Dynamic route templates claim one generation shared by their generated route set, so a route is never dropped by a stale half of a template rebuild.

The disk-write queue is versioned: a queued write carries its page's generation and is skipped if that generation is superseded. This guarantees the displayed, on-disk, and reloaded content all describe the same newest generation.

Phase 1: Page Phase (pageProcessing)

The page phase prepares the source HTML document and orchestrates the component phase:

For bracket-parameter pages, <script data-bascik-routes> runs before page build scripts. Its genuine relative and import-root ESM specifiers are rewritten (relative against the containing page or external routes script file, @/ against scripts.importRoot) before the cached temporary module executes.

1. Execute build scripts. Any <script data-bascik-build> blocks are run as Node.js ESM modules. A lexical scanner resolves genuine relative ESM specifiers against the source page, component, or external script directory, and @/ import-root specifiers against scripts.importRoot, before temporary-module execution. A bare leading / specifier is rejected with a located error. Generic quoted data paths retain process.cwd() semantics. Deferred page-aware component scripts carry their component source identity through expansion, so their relative imports never fall back to the consuming page. Script stdout replaces the script tag. The result can contain component tags, these will be resolved in step 4. Output is cached on disk so unchanged scripts skip the child-process spawn on subsequent builds (see Build Script Output Cache below).
2. Extract body and head. The inner content of <body> and <head> are extracted separately so component injection can happen in both zones independently.
3. Obtain component list. On the multi-page startup path (processAllPages), the list is pre-computed once and passed in. On a single-page re-transpilation, it is loaded from src/components/ at this point.
4. Run component phase. recursivelyTranspile is called on both the body and head HTML strings. Each call returns a TranspileResult containing the resolved HTML and the list of components that were used.
5. Collect and deduplicate CSS. All CSS from used components is gathered. Since multiple instances of the same component share identical scoped class names, scoping.deduplicateCss emits a single <style> block regardless of how many times a component appears on the page. Any global stylesheets configured via assets.inlineStyles are also injected into <head> at this stage.
6. Rewrite base paths. When base is not /, root-relative HTML and CSS URLs receive the normalized prefix. This runs after component ID-reference rewriting, so href="#id" and url(#id) are already final and remain untouched, and before minification. Markup-like text in comments and raw-text elements is shielded. With the default /, the step returns the original bytes immediately.
7. Inject live-reload script. In dev mode only, a small <script> that opens a Server-Sent Events connection to /bascik-live-reload is appended to the body.
8. Extract server scripts to sidecar. Any <script data-bascik-server> blocks have their source recorded in the server sidecar registry and are replaced with inert placeholders (<script type="text/bascik-server" data-bascik-server-id="...">). This keeps server source code out of the emitted HTML.
9. Minify. HTML comments are stripped and excess whitespace is collapsed via minifyHtml. This runs after component resolution so that whitespace-sensitive content inside resolved components (e.g. <pre> blocks from <code-block>) is preserved intact.
10. Reassemble HTML. The resolved body and head are placed back into the original HTML document structure. An idempotent whole-document base-path pass then covers URL-bearing attributes on the html, head, and body opening tags.
11. Store and write output. In build mode, the finished HTML is written to dist/ before transpilation completes. In dev mode, the result is stored in the in-memory page store first, then written to dist/ asynchronously so serving never waits for file I/O. Writes for repeated edits to the same page are serialized.
12. Emit transpiled event. eventEmitter.emit("transpiled") triggers live-reload for any connected browser.

Output Directory Lifecycle

Dev and build runs remove directory.out before pre-phase lifecycle scripts execute, then repopulate it from the current source tree. This prevents deleted pages, renamed assets, and removed dynamic routes from surviving as stale deployment output. Cleaning happens before pre-phase scripts so files those scripts intentionally generate in dist/ remain available to the rest of the run. Production server mode (bascik --server) reads an existing build and never cleans it.

During a dev session, the active file watcher also removes corresponding output files when it receives deletion events (unlink and unlinkDir).

Build Script Output Cache & Batch Execution

Uncached <script data-bascik-build> blocks are executed by Node.js, which carries a ~50–150 ms V8 startup cost when spawning individual processes. Bascik optimizes cold builds by batching all uncached scripts on a page into a single harness runner process, and eliminates startup overhead entirely on warm builds via disk caching for scripts whose inputs have not changed.

Page-Level Batch Execution

When a page contains multiple uncached <script data-bascik-build> blocks:

1. Bascik groups all uncached tasks into a single batch.
2. A lightweight ESM runner harness dynamically imports each script sequentially in one child process. Each task retains its own BASCIK_SOURCE_FILE value and source map identity, including scripts authored by components and deferred until the page phase.
3. During evaluation, process.stdout.write and process.stderr.write are intercepted per script block to ensure clean output separation.
4. Outputs are mapped back to their corresponding tags, cached on disk, and spliced into the page simultaneously.

This reduces Node child process spawns from N scripts to 1 per page during cold builds.

Location

Cache entries live under node_modules/.cache/bascik/script-cache/ as individual JSON files named by their cache key:

node_modules/.cache/bascik/script-cache/<sha256>.json

Each file contains { "v": <version>, "output": "<html>" }. The v field is a hard-coded integer in build-scripts.ts; bumping it at the source level immediately invalidates every existing entry across all projects.

Cache key

The key is the SHA-256 hex digest of:

1. The cache version integer.
2. The authored script content before relative import specifier rewriting.
3. "1" or "0" for build vs. dev mode (isBuild), since the same script may produce different output in each mode via the BASCIK_BUILD env var.
4. The source file path (BASCIK_SOURCE_FILE).
5. The page file path (BASCIK_PAGE_FILE).
6. The page route path (BASCIK_PAGE_PATH), which guarantees page-aware component scripts derive distinct cache keys per page.
7. The site URL (BASCIK_SITE_URL), since it can influence output and changes rarely.
8. The normalized deployment base (BASCIK_BASE), since scripts can emit base-aware output.
9. The dynamic route payload (BASCIK_ROUTE), if applicable.
10. The normalized path and full content of every detected local dependency. Relative ESM imports are followed recursively from each containing module, while generic quoted data paths are rooted at process.cwd().
11. The resolved package identity for every external (bare, scoped, subpath) package the script imports: resolved entry content and manifest, plus the bounded transitive package graph (package-identity.ts). A script with no package imports contributes nothing for this item.

File references are extracted by extractScriptDeps() (exported from build-scripts.ts). It lexically identifies genuine relative and import-root (@/) ESM specifiers and path-like string arguments used in code, so editing an alias-imported helper invalidates the cache and triggers a dev rebuild. The following paths are illustrative examples, not an exhaustive list of supported directories or extensions:

'./content/foo.md'          → included in key
'scripts/md-renderer.ts'  → included in key

If the script contains no detectable references, items 1 through 9 still contribute to the key.

Invalidation

Because the content of every referenced file is hashed into the key, editing a content file produces a new key for any script that references it, giving a cache miss. Scripts on other pages that do not reference that file keep their old keys and continue to hit the cache.

The resolved package graph is also part of the key. Upgrading an npm package, editing a workspace/file:-linked package in place, or changing a transitive package a build script imports all produce a new key without any script edit or manual cache clear (see package-identity.ts). To bust the entire cache manually, for example after an unexpected external change, delete the cache directory:

rm -rf node_modules/.cache/bascik/script-cache

Interaction with pipeline.workers

When pipeline.workers: true is set, worker threads share the same filesystem and therefore the same cache directory. On the first (cold) build, multiple workers may independently get a cache miss for the same script, spawn child processes, and write the same entry. Because every worker writes the same content for the same key, the last write wins harmlessly. On subsequent builds all workers benefit from the cached entries.

Phase 2: Component Phase (recursivelyTranspile)

The component phase recurses until no custom component tags remain in the HTML string. On each pass it finds the first component tag, fully resolves it, and substitutes it. It then repeats until no more tags are found.

Fast-path string guards and raw-text masking mechanics

Component tags are valid markup only outside of raw-text elements. Text like <my-card> inside a <script> (for example, a JSON-LD schema string), a <style> comment, or a <textarea> is content rather than component usage. Resolving tags inside raw-text elements would inject component markup, including stray </script> closing tags, into script or style content and corrupt the document.

To prevent corruption while maintaining high performance, every tag search operates on a masked copy of the HTML generated by maskRawTextContent:

1. Fast-path string existence guards: Before running regex operations, maskRawTextContent performs a fast string check (!htmlString.includes("<!--") && !/<(?:script|style|textarea)\b/i.test(...)). Pages that do not contain comments or raw-text tags bypass masking regexes entirely. Similarly, DOM selector script rewrites (prefixElementAttribute), CSS custom property/layer/container scoping, prop injection, and slot replacements use .includes() guards to skip regex execution when target identifiers or markers are absent.
2. Space-masking and index alignment: The mask replaces the inner text of <script>, <style>, and <textarea> elements with an equal number of space characters. Because space characters preserve exact string lengths and byte offsets, character indices found in the masked copy (getFirstComponent, findOpenTag, self-closing fallbacks, and findMatchingClose depth counters) match the original string precisely. The compiler slices and splices the original string at those exact indices without allocating secondary offset maps or corrupting raw-text content.
3. Unresolved tag scanner: The unresolved-tag warning scanner uses the same masking approach, stripping raw-text content before checking for leftover hyphenated custom tags.

For each component tag found:

Step 1: Scoping pipeline

A deterministic instanceId (an 8-character hex hash derived from page path, component name, and occurrence ordinal via deriveInstanceId in names.ts) is generated for this occurrence of the component. An ordered list of transform functions is assembled and applied in a pipeline (each step receives the output of the previous):

1. prefixElementAttribute(c, "id", instanceId): scopes id attributes and all corresponding JS DOM selector references.
2. prefixElementAttribute(c, "name", instanceId): scopes name attributes and getElementsByName calls.
3. prefixElementAttribute(c, "class", instanceId): scopes class names in HTML attributes, CSS, and JS selector calls.
4. namespaceScriptTags(c): wraps every inline <script> in an IIFE, preserves line positioning, and appends a //# sourceURL comment for browser DevTools source attribution.

Each step is skipped if disabled in bascik.config.ts.

Step 2: Template resolution

1. Props. injectProps replaces every data-bascik-prop-* placeholder in the component template with the corresponding attribute value from the usage tag.
2. Named slots. replaceNamedSlots fills each data-bascik-slot="name" zone in the template with the matching <div data-bascik-slot="name"> content from the usage site.
3. Default slot. The inner content of the usage tag is placed into the element carrying data-bascik-slot (no value). If the usage tag has no inner content, the template's fallback content is preserved.
4. Attribute inheritance. mergeAttributesOntoRoot copies pass-through attributes (aria-*, data-*, class, etc.) from the usage tag onto the component's root element. If the component template contains multiple root elements (or leading comments, <script>, or <style> blocks), attributes are merged onto the first root HTML element.

Step 3: Substitution

replaceTag replaces the original usage tag in the parent HTML string with the fully resolved component HTML. The outer loop runs again on the updated string. Because component templates can themselves contain other component tags, this naturally handles any depth of nesting.

Termination

The recursion terminates when getFirstComponent no longer finds any custom tag in the HTML string, i.e., when all recognized component names have been replaced with vanilla HTML.

Selective Re-transpilation

When a component file changes during dev, Bascik does not reprocess every page. The memory store maintains a reverse index mapping each component name to the set of pages that use it. selectivelyProcessPages uses this index to retranspile only the affected pages.

Scoped Name Format

All scoped attribute names follow this pattern:

bascik__<componentName>__<instanceId>__<originalName>

# id and name attributes include instanceId for uniqueness:
bascik__my-nav__a1b2c3d4__search-input

# class attributes use component name only (no instanceId):
bascik__my-nav__toggle-btn

When minify.identifiers is enabled (the default for builds), each full scoped name is hashed to a short alphanumeric string using SHA-256 with Base62 encoding before being written to the output, e.g. b2Y4G9eD1K8b. See Scoping System for full details.