Warning
This is an internal project, and is not intended for public use. No support or stability guarantees are provided.
The loadCodeVariant module provides utilities for loading, processing, and transforming code variants with support for syntax highlighting, TypeScript-to-JavaScript transformation, extra files, and metadata management.
Note
This module is primarily used internally by
CodeHighlighterfor runtime code loading and processing. Most users won't need to call these functions directly.
import { loadCodeVariant } from '@mui/internal-docs-infra/pipeline/loadCodeVariant';
// Load and process a single variant
const result = await loadCodeVariant(
'file:///app/components/button/Button.tsx',
'default',
{
fileName: 'Button.tsx',
url: 'file:///app/components/button/Button.tsx',
},
{
sourceParser: createParseSource(),
loadSource: myLoadSourceFunction,
sourceTransformers: [typescriptToJavaScript],
},
);
// Returns:
// {
// code: { fileName: 'Button.tsx', source: /* HAST nodes */, transforms: { ... } },
// dependencies: ['file:///app/components/button/Button.tsx'],
// externals: {}
// }
The function:
Extra files can be specified directly in the variant, or returned automatically by your loadSource function:
// Option 1: Specify extraFiles in the variant
const result = await loadCodeVariant(
'file:///app/components/button/Button.tsx',
'default',
{
fileName: 'Button.tsx',
source: buttonSource,
extraFiles: {
'Button.module.css': 'file:///app/components/button/Button.module.css',
'../utils/helpers.ts': 'file:///app/components/utils/helpers.ts',
},
},
options,
);
// Option 2: Return extraFiles from loadSource (automatically loaded)
const loadSource = async (url) => {
const source = await readFile(url);
const extraFiles = await findDependencies(url); // e.g., CSS imports, relative imports
return {
source,
extraFiles: {
'Button.module.css': 'file:///app/components/button/Button.module.css',
'../utils/helpers.ts': 'file:///app/components/utils/helpers.ts',
},
};
};
// Both approaches result in all files loaded and highlighted recursively
// result.code.extraFiles contains processed CSS and helpers
Add shared dependencies (like package.json or global configs) to all variants:
const result = await loadCodeVariant(
'file:///app/components/button/Button.tsx',
'default',
variant,
{
...options,
globalsCode: [
{ fileName: 'package.json', source: packageJsonContent },
'file:///app/tsconfig.json',
],
},
);
// Global files are merged with metadata flag for filtering
Convert code objects to HAST for rendering:
import { parseCode } from '@mui/internal-docs-infra/pipeline/loadCodeVariant';
const code = {
default: {
fileName: 'Example.tsx',
source: 'const x = 1;', // String source
},
};
const parsed = parseCode(code, parseSource);
// Converts string sources to HAST nodes for rendering
Use sourceEnhancers to modify HAST nodes after parsing. Enhancers receive the parsed HAST root, any comments extracted from the source (via loadServerSource), and the file name:
import type { SourceEnhancer, SourceComments } from '@mui/internal-docs-infra/CodeHighlighter';
import type { HastRoot } from 'hast';
// SourceComments is Record<number, string[]> - line numbers to comment arrays
// Example: Add line numbers based on comment sections
const addSectionMarkers: SourceEnhancer = (root, comments, fileName) => {
// comments is Record<number, string[]> mapping line numbers to comment arrays
// Use comments to enhance the HAST tree, e.g., add section markers
if (comments) {
for (const [lineNumber, lineComments] of Object.entries(comments)) {
// Process comments to add markers, highlights, etc.
}
}
return root;
};
// Example: Async enhancer that fetches additional data
const addTypeInfo: SourceEnhancer = async (root, comments, fileName) => {
const typeData = await fetchTypeInfo(fileName);
// Modify root with type information
return root;
};
const result = await loadCodeVariant(
'file:///app/components/button/Button.tsx',
'default',
variant,
{
sourceParser: createParseSource(),
loadSource: myLoadSourceFunction,
sourceEnhancers: [addSectionMarkers, addTypeInfo], // Run in order
},
);
Enhancers run sequentially after parsing, each receiving the output of the previous enhancer. They are applied to both main files and extra files.
Convert nested variant structure to flat file paths:
import { flattenCodeVariant } from '@mui/internal-docs-infra/pipeline/loadCodeVariant';
const variant = {
url: 'file:///src/components/button/Button.tsx',
fileName: 'Button.tsx',
source: buttonSource,
extraFiles: {
'../shared/utils.ts': { source: utilsSource },
'../../package.json': { source: pkgSource, metadata: true },
},
};
const flattened = flattenCodeVariant(variant);
// Returns:
// {
// 'src/components/button/Button.tsx': { source: buttonSource },
// 'src/components/shared/utils.ts': { source: utilsSource },
// 'src/package.json': { source: pkgSource, metadata: true }
// }
Determine if you have enough data to render without loading:
import { maybeCodeInitialData } from '@mui/internal-docs-infra/pipeline/loadCodeVariant';
const { initialData, reason } = maybeCodeInitialData(
['typescript', 'javascript'],
'typescript',
code,
undefined,
true, // needsHighlight
false, // needsAllFiles
false, // needsAllVariants
);
if (initialData) {
// Ready to render immediately
return <CodeDisplay {...initialData} />;
} else {
// Need to load more data
console.log('Missing data:', reason);
}
Use loadCodeFallback to load the minimal data needed for initial render:
import { loadCodeFallback } from '@mui/internal-docs-infra/pipeline/loadCodeVariant';
const fallback = await loadCodeFallback(
'file:///app/demos/example/index.ts',
'default', // initialVariant
existingCode, // may be undefined
{
loadCodeMeta,
loadVariantMeta,
loadSource,
sourceParser,
variants: ['default', 'typescript'],
fallbackUsesExtraFiles: true,
fallbackUsesAllVariants: false,
},
);
// Returns minimal data for initial render:
// {
// code: { default: { ... } },
// initialFilename: 'Example.tsx',
// initialSource: /* HAST nodes */,
// allFileNames: ['Example.tsx', 'Example.module.css'],
// }
Apply TypeScript-to-JavaScript transforms to rendered code:
import { applyCodeTransform } from '@mui/internal-docs-infra/pipeline/loadCodeVariant';
const jsSource = applyCodeTransform(
tsSource, // Original TypeScript HAST
transforms, // From variant.transforms
'javascript', // Transform key
);
// Returns JavaScript version of the source as HAST
Position metadata files (package.json, configs) relative to source files:
import { mergeCodeMetadata } from '@mui/internal-docs-infra/pipeline/loadCodeVariant';
const merged = mergeCodeMetadata(
variant,
{
'package.json': { source: pkgJson },
'tsconfig.json': { source: tsConfig },
},
{ metadataPrefix: 'src/' },
);
// Positions metadata files at correct depth based on source structure
Analyze variant structure for path resolution:
import { examineCodeVariant } from '@mui/internal-docs-infra/pipeline/loadCodeVariant';
const context = examineCodeVariant(variant);
// Returns:
// {
// hasUrl: true,
// hasMetadata: true,
// maxSourceBackNavigation: 2,
// urlDirectory: ['src', 'components', 'button'],
// rootLevel: 'src',
// pathInwardFromRoot: 'components/button',
// actualUrl: 'file:///src/components/button/Button.tsx'
// }
loadVariantMetaloadSource if not providedsourceParsersourceEnhancers sequentially with comments from sourceWhen sourceEnhancers are provided, they process the HAST after parsing:
// Enhancement flow
const enhancedRoot = await sourceEnhancers.reduce(async (rootPromise, enhancer) => {
const root = await rootPromise;
return enhancer(root, comments, fileName);
}, Promise.resolve(parsedRoot));
Each enhancer receives:
root - The current HAST root (output of previous enhancer)comments - Comments extracted by loadSource (e.g., from parseImportsAndComments)fileName - The file name for contextEnhancers can be synchronous or asynchronous and are applied to both main files and extra files.
Extra files support relative paths that are resolved recursively:
Button.tsx
├── Button.module.css (same directory)
├── ../utils/helpers.ts (parent directory)
└── ../../package.json (grandparent, metadata)
The loader:
When sourceTransformers are provided:
.ts, .tsx, etc.)transforms objectTransforms are applied on-demand during rendering via applyCodeTransform.
The metadataPrefix option positions source files within a directory while keeping metadata files at the project root:
Without metadataPrefix:
├── Button.tsx (source)
├── utils.ts (source)
├── package.json (metadata)
└── tsconfig.json (metadata)
With metadataPrefix: 'src/':
├── src/
│ ├── Button.tsx (source)
│ └── utils.ts (source)
├── package.json (metadata - stays at root)
└── tsconfig.json (metadata - stays at root)
This ensures metadata files remain accessible at the project root while source files are scoped within their directory.
When NOT to use:
CodeHighlighterparseSource directly for basic needsThe loader uses Promise.all to load extra files in parallel:
// Loads all files simultaneously
extraFiles: {
'styles.css': 'url1',
'utils.ts': 'url2',
'types.ts': 'url3',
}
// All three load in parallel, not sequentially
loadSource calls are cached within a single loadCodeVariant execution to prevent duplicate requests for the same file.
The loader tracks loaded files and throws errors on circular dependencies:
// This would throw an error:
// A.tsx imports ../B.tsx
// B.tsx imports ../A.tsx
output: 'hastJson' - JSON stringified HAST (development)output: 'hastGzip' - Compressed HAST (production, smaller bundles)Circular dependency detected:
Error: Circular dependency detected: file:///path/to/file.ts
Fix: Restructure imports to remove circular references.
Invalid extraFiles:
Error: Invalid extraFiles from loadSource: key "file:///..." appears to be an absolute path.
Fix: Use relative paths as keys in extraFiles.
Missing loadSource:
Error: "loadSource" function is required when source is not provided
Fix: Provide loadSource in options or include source inline.
Transform not found:
Error: Transform "javascript" not found in transforms
Fix: Ensure sourceTransformers generated the requested transform.
loadCodeVariantLoads a variant with support for recursive extra file loading. The loadSource function can now return extraFiles that will be loaded recursively. Supports both relative and absolute paths for extra files. Uses Promise.all for efficient parallel loading of extra files.
| Parameter | Type | Description |
|---|---|---|
| url | string | undefined | File URL for the variant |
| variantName | string | Name of the variant (used for error messages) |
| variant | VariantCode | string | undefined | Variant data object or URL string |
| options | LoadVariantOptions | undefined | Loading and processing options (source parser, transformers, enhancers, etc.) |
Promise<{ code: VariantCode; dependencies: string[]; externals: Externals }>loadCodeFallbackLoads minimal data needed for fallback rendering. Returns code, initial filename, initial source, extra files, all file names, and processed globals code.
| Parameter | Type | Description |
|---|---|---|
| url | string | File URL for the variant |
| initialVariant | string | Name of the initial variant to load |
| loaded | Code | undefined | Previously loaded Code object, if any |
| options | LoadFallbackCodeOptions | undefined | Optional loading configuration |
Promise<FallbackVariants>parseCodePure function to parse code variants and their extraFiles. Converts string sources to HAST nodes and handles hastJson parsing.
| Parameter | Type | Description |
|---|---|---|
| code | { [key: string]: string | VariantCode | undefined } | |
| parseSource | ((source: string, fileName: string, language?: string) => HastRoot) |
{ [key: string]: string | VariantCode | undefined }flattenCodeVariantFlatten a VariantCode into a flat files structure Resolves relative paths and handles metadata file scoping Uses addPathsToVariant for path resolution logic
| Parameter | Type | Description |
|---|---|---|
| variant | VariantCode |
{ [filePath: string]: FlatFile }mergeCodeMetadata| Parameter | Type | Description |
|---|---|---|
| variant | VariantCode | |
| metadataFiles | VariantExtraFiles | undefined | |
| options | MergeMetadataOptions | undefined |
VariantCodeexamineCodeVariantCreate path context for processing files with extended information
| Parameter | Type | Description |
|---|---|---|
| variant | VariantCode |
maybeCodeInitialDataType guard function that determines if we have sufficient data to render a code highlighter component immediately, or if we need to start loading data first.
This function acts as a validation layer to ensure we have the minimal required data to render either a fallback state or the actual code content, helping to prevent rendering errors and provide better user experience.
This function is used in two main scenarios:
Server-side rendering (CodeHighlighter): Determines if we can render with initial
source content immediately, or if we need to load fallback data via CodeInitialSourceLoader
Client-side hydration (CodeHighlighterClient): Within useInitialData hook to determine
if we should trigger loading effects or if we can render with available data
The function checks data availability in this order:
needsAllVariants is true)needsAllFiles is true)needsHighlight is true)This function operates synchronously and only validates existing data — it never triggers any loading operations. This design is crucial for performance and rendering strategies:
initialData, avoiding async components entirelyWhen initialData: false is returned, the calling component is responsible for initiating
asynchronous loading operations (e.g., loadCodeFallback, CodeInitialSourceLoader).
| Parameter | Type | Description |
|---|---|---|
| variants | string[] | Array of all available variant names for this code block (e.g., [‘javascript’, ‘typescript’]) |
| variant | string | The specific variant we want to display (must exist in variants array) |
| code | Code | undefined | The code object containing all variant data (may be undefined if not loaded) |
| fileName | string | undefined | Optional specific file to display. Resolution logic:
|
| needsHighlight | boolean | undefined | Whether the code needs to be syntax highlighted (source must be highlighted object, not string) |
| needsAllFiles | boolean | undefined | Whether all extra files must be loaded before rendering (checks that all extraFiles have source content) |
| needsAllVariants | boolean | undefined | Whether all variants must be available before rendering (validates using hasAllVariants) |
| Key | Type | Description |
|---|---|---|
| initialData | | |
| reason | string | undefined |
hasAllVariantsDetermines if all code variants are fully loaded and ready to render the complete content component.
This function validates that we have all necessary data to transition from fallback/loading state to the full interactive code highlighter. It checks both main files and extra files for all variants.
Used primarily to determine when to show the full Content component instead of ContentLoading fallback, ensuring a smooth user experience without rendering errors.
| Parameter | Type | Description |
|---|---|---|
| variants | string[] | Array of variant names that must all be ready (e.g., [‘javascript’, ‘typescript’]) |
| code | { [key: string]: string | VariantCode | undefined } | The code object containing variant data |
| needsHighlight | boolean | undefined | Whether all sources need to be syntax highlighted (hast nodes, not strings) |
booleanTrue if all variants and their files are loaded and ready for full rendering
enhanceCodeAsync function to enhance parsed code variants and their extraFiles. Applies sourceEnhancers to HAST nodes, using comments stored in the variant.
| Parameter | Type | Description |
|---|---|---|
| code | { [key: string]: string | VariantCode | undefined } | |
| sourceEnhancers | SourceEnhancer[] |
Promise<Code>applyCodeTransformApplies a specific transform to a variant source and returns the transformed source
| Parameter | Type | Description |
|---|---|---|
| source | VariantSource | The original variant source (string, HastNodes, or hastJson object) |
| transforms | { [key: string]: { delta: Delta; fileName?: string } } | Object containing all available transforms |
| transformKey | string | The key of the specific transform to apply |
VariantSourceThe transformed variant source in the same format as the input
addPathsToVariantAdd flat paths to all files in a variant
| Parameter | Type | Description |
|---|---|---|
| variant | VariantCode |
VariantCodetype FallbackVariants = {
code: Code;
initialFilename: string | undefined;
initialSource: VariantSource;
initialExtraFiles?: VariantExtraFiles;
allFileNames: string[];
processedGlobalsCode?: Code[];
};type FileWithPath = { source?: VariantSource; metadata?: boolean; path: string };type FlatFile = { source: string; metadata?: boolean };type FlattenedFiles = { [filePath: string]: FlatFile };type PathContext = PathContextWithUrl | PathContextWithoutUrl;loadPrecomputedCodeHighlighter - Build-time alternativeparseSource - Syntax highlighting functionloadServerSource - Server-side source loadingtransformTypescriptToJavascript - TypeScript transformer