Warning
This is an internal project, and is not intended for public use. No support or stability guarantees are provided.
This page explains the overall architecture of @mui/internal-docs-infra, including data flows, key design decisions, and how the various pieces fit together.
The package is built around one core principle: move locally-derived computation to build time. The cache only updates whenever those files change.
┌─────────────────────────────────────────────────────────────────┐
│ BUILD TIME │
│ ┌──────────────┐ ┌──────────────┐ ┌──────────────┐ │
│ │ TypeScript │ │ Source │ │ Markdown │ │
│ │ Sources │ │ Files │ │ Files │ │
│ └──────┬───────┘ └──────┬───────┘ └──────┬───────┘ │
│ │ │ │ │
│ ▼ ▼ ▼ │
│ ┌──────────────┐ ┌──────────────┐ ┌──────────────┐ │
│ │ typescript- │ │ Starry Night │ │ Remark │ │
│ │api-extractor │ │ Highlighter │ │ Plugins │ │
│ └──────┬───────┘ └──────┬───────┘ └──────┬───────┘ │
│ │ │ │ │
│ └───────────────────┼───────────────────┘ │
│ ▼ │
│ ┌──────────────┐ │
│ │ HAST │ (JSON-serializable AST) │
│ └──────┬───────┘ │
│ │ │
│ ▼ │
│ ┌──────────────┐ │
│ │ precompute │ (injected into source) │
│ └──────┬───────┘ │
└───────────────────────────┼─────────────────────────────────────┘
│
┌───────────────────────────┼─────────────────────────────────────┐
│ ▼ RUNTIME │
│ ┌──────────────┐ │
│ │ hastToJsx │ (HAST → React.ReactNode) │
│ └──────┬───────┘ │
│ │ │
│ ▼ │
│ ┌──────────────┐ │
│ │ Rendered │ │
│ │ Components │ │
│ └──────────────┘ │
└─────────────────────────────────────────────────────────────────┘
Why this architecture?
When you write a demo with createDemo():
demos/basic/index.ts (createDemo call)
│
▼ [loadPrecomputedCodeHighlighter loader]
│
┌────┴────┐
│ Parse │ Find createDemo() call, extract variants
│ Factory │
└────┬────┘
│
▼
┌─────────┐
│ Load │ Read each variant file + dependencies
│ Files │ Track all files for webpack watching
└────┬────┘
│
▼
┌─────────┐
│ Starry │ Syntax highlight with grammar scopes
│ Night │ Add line gutters
└────┬────┘
│
▼
┌─────────┐
│Transform│ TypeScript → JavaScript (optional)
│ TS→JS │ via Babel standalone
└────┬────┘
│
▼
┌─────────┐
│ Inject │ Insert { precompute: { ... } } into source
│ HAST │
└────┬────┘
│
▼
createDemo(import.meta.url, Component, { precompute: { ... } })
Key files:
When you write types with createTypes():
types.ts (createTypes call)
│
▼ [loadPrecomputedTypes loader]
│
┌────┴────┐
│ Parse │ Find createTypes() call
│ Factory │
└────┬────┘
│
▼ [Worker Thread - preserves TS language service cache]
│
┌──────────┐
│typescript│ Parse TypeScript AST
│ -api- │ Extract exports, components, props
│extractor │
└────┬─────┘
│
▼
┌─────────┐
│ Format │ Format props, find DataAttributes/CssVars enums
│Component│ Parse JSDoc descriptions as markdown
└────┬────┘
│
▼
┌─────────┐
│ Sync │ Update types.md file (for validation)
│ types.md│
└────┬────┘
│
▼
┌─────────┐
│Highlight│ Syntax highlight type strings
│ Types │ Convert to HAST
└────┬────┘
│
▼
createTypes(import.meta.url, Component, { precompute: { ... } })
Key files:
[Out of band - transformMarkdownMetadata]
│
page.mdx files ──────────────────┼──▶ index page.mdx files
(title, description, sections) │ (aggregated metadata)
│
─────────────────────────────────────┼─────────────────────────
│
app/sitemap/index.ts (createSitemap call)
│
▼ [loadPrecomputedSitemap loader]
│
┌────┴────┐
│ Parse │ Find createSitemap() call
│ Factory │ Extract page imports
└────┬────┘
│
▼
┌─────────┐
│ Load │ Import each index page.mdx
│ Indexes │ Read pre-extracted metadata
└────┬────┘
│
▼
┌─────────┐
│ Build │ Create Orama-compatible schema
│ Index │ Index all searchable content
└────┬────┘
│
▼
{ schema: {...}, data: {...} } // Injected as precompute
│
▼ [Runtime - Client]
│
┌──────────┐
│ useSearch│ Create Orama instance from precomputed data
│ hook │ Provide search API to components
└──────────┘
Key files:
All precomputable features use the same pattern:
// 1. Define a factory in your app
export const createDemo = createDemoFactory({ DemoContent });
// 2. Use the factory in index.ts files
export const DemoCheckbox = createDemo(import.meta.url, Checkbox);
// 3. Webpack loader detects the factory call
// 4. Loader processes and injects precompute
// 5. Factory receives precomputed data at runtime
Why import.meta.url?
demos/basic/index.ts)See Built Factories for details.
Type processing uses a singleton worker thread to preserve the TypeScript language service cache:
┌─────────────────────────────────────────────────────────────┐
│ Main Thread (Webpack) │
│ │
│ Loader 1 ──┐ │
│ Loader 2 ──┼──▶ WorkerManager ──▶ Worker Thread │
│ Loader 3 ──┘ │ │ │
│ │ │ │
│ │ ┌────────────────┐ │
│ │ │ TS Language │ │
│ │ │ Service Cache │ │
│ │ └────────────────┘ │
│ │ │
│ Request/Response via postMessage │
└─────────────────────────────────────────────────────────────┘
Why a worker?
Symbol.for() on process to ensure singleton across Turbopack contextsSee workerManager.ts for implementation.
| Module | Environment | Purpose |
|---|---|---|
CodeHighlighter/ | Isomorphic | Main code display component |
CodeProvider/ | Client | Client-side code loading context |
abstractCreate*/ | Server | Factory utilities ('server-only') |
use*/ | Client | React hooks (state/effects) |
pipeline/loadPrecomputed* | Build | Webpack loaders |
pipeline/loadCodeVariant | Isomorphic | Runtime code loading/transforms |
pipeline/loadServer* | Server | RSC data loading |
pipeline/syncTypes | Build | TypeScript parsing, worker threads |
withDocsInfra/ | Build | Next.js configuration plugin |
dangerouslySetInnerHTML and its XSS risks| Operation | When | Typical Time |
|---|---|---|
| TypeScript parsing (cold) | Build | 500-2000ms |
| TypeScript parsing (warm) | Build | 50-200ms |
| Syntax highlighting | Build | 10-50ms per file |
| HAST → JSX conversion | Runtime | under 10ms |
| Search index creation | Runtime (once) | 50-100ms |
| Search query | Runtime | under 10ms |
The worker thread architecture makes TypeScript parsing ~10x faster after the first file by preserving the language service cache.