Runtime Catalog and Artifact Cache Invalidation

This document explains the revised runtime catalog architecture.

The editor first loads a lightweight remote catalog containing component metadata and preview images. Full package artifacts are loaded only when the user adds a component.

Each artifact is cached independently with Next.js unstable_cache, and UI Builder can invalidate only the changed artifact through a protected host API.

Remote Catalog Structure

The remote catalog contains lightweight component information:

{
  "defaultExternals": [
    "react",
    "react-dom",
    "react/jsx-runtime",
    "react/jsx-dev-runtime",
    "next",
    "next/image",
    "next/link",
    "next/navigation",
    "@xsite/common",
    "@xsite/common/client",
    "@xsite/common/server",
    "@xsite/core",
    "lucide-react"
  ],
  "packages": [
    {
      "id": "about1",
      "label": "About 1",
      "package": "@xsite/ui-about1",
      "version": "1.0.8",
      "category": "about",
      "previewImage": "https://raw.githubusercontent.com/xsite-io/xsite/main/packages/ui-about1/preview.png"
    }
  ]
}

The initial catalog is used to:

List available components
Show component labels
Group components by category
Display preview images
Resolve package names and versions
Know the default external dependencies

At this stage, the host does not fetch:

descriptor.json
data.json
schema.json
puck.json
meta.json

The remote ESM module is also not compiled or imported yet.

Revised Architecture

The flow has two main stages.

Stage 1: Lightweight catalog loading
Remote catalog
→ component list
→ labels, categories, previews
→ display component picker

Stage 2: Package loading on component insertion
User adds component
→ call host runtime catalog API
→ fetch package artifacts
→ build CatalogEntry
→ pass module URL to remote loader

Overall Flow

Editor starts
  ↓
Fetch lightweight remote catalog
  ↓
List about1, about2, and other packages
  ↓
Display previewImage in component picker
  ↓
User clicks Add About1
  ↓
GET /api/runtime-catalog/about1
  ↓
Fetch cached descriptor.json
  ↓
Fetch cached data.json
  ↓
Fetch cached schema.json
  ↓
Fetch cached puck.json
  ↓
Fetch cached meta.json
  ↓
Build CatalogEntry
  ↓
Create Puck fields and initial props
  ↓
Pass module URL to remote component loader
  ↓
Compile and import About1
  ↓
Add About1 to the page

1. Load the Lightweight Remote Catalog

When the editor starts:

useEffect(() => {
  fetchRemoteCatalog()
    .then(setCatalog)
    .catch(setError);
}, []);

The remote catalog only contains the data needed by the component picker.

Example package type:

type RemoteCatalogPackage = {
  id: string;
  label: string;
  package: string;
  version: string;
  category: string;
  previewImage?: string;
};

Example picker rendering:

{catalog.packages.map((pkg) => (
  <button
    key={pkg.id}
    onClick={() =>
      addRemoteComponent(pkg.id)
    }
  >
    {pkg.previewImage ? (
      <img
        src={pkg.previewImage}
        alt={pkg.label}
      />
    ) : null}
 
    <span>{pkg.label}</span>
  </button>
))}

No package JSON artifacts are fetched during this step.

2. User Adds About1

When the user selects About1, the editor calls the host API:

GET /api/runtime-catalog/about1

The host looks up the package from the lightweight catalog:

{
  id: "about1",
  label: "About 1",
  package: "@xsite/ui-about1",
  version: "1.0.8",
  category: "about"
}

The host uses the package name and version to construct artifact URLs.

3. Resolve Package Artifact URLs

Assume:

ARTIFACTS_ORIGIN=https://pub-xxxxxxxxxxxxxxxxxxx.r2.dev

For About1:

Package: @xsite/ui-about1
Version: 1.0.8

The base artifact URL is:

https://pub-xxxxxxxxxxxxxxxxxxx.r2.dev/@xsite/[email protected]

Artifact URLs:

https://pub-xxxxxxxxxxxxxxxxxxx.r2.dev/@xsite/[email protected]/descriptor.json

https://pub-xxxxxxxxxxxxxxxxxxx.r2.dev/@xsite/[email protected]/data.json

https://pub-xxxxxxxxxxxxxxxxxxx.r2.dev/@xsite/[email protected]/schema.json

https://pub-xxxxxxxxxxxxxxxxxxx.r2.dev/@xsite/[email protected]/puck.json

https://pub-xxxxxxxxxxxxxxxxxxx.r2.dev/@xsite/[email protected]/meta.json

The module URL is generated separately:

https://esm.xsite.live/@xsite/[email protected]?target=es2022&external=...

4. Artifact Responsibilities

Artifact	Responsibility
`descriptor.json`	Runtime package and module information
`data.json`	Default props for newly inserted components
`schema.json`	Component data and validation schema
`puck.json`	Puck field configuration
`meta.json`	Label, status, preview, owner, category, and related metadata
Module URL	Remote ESM component implementation

5. Separate Cache for Every Artifact

Each artifact must have:

Its own cache key
Its own cache value
Its own artifact-specific tag
A shared package-level tag

Recommended artifact tags:

runtime-artifact:about1:descriptor.json
runtime-artifact:about1:data.json
runtime-artifact:about1:schema.json
runtime-artifact:about1:puck.json
runtime-artifact:about1:meta.json

Recommended package tag:

runtime-package:about1

The package-level tag allows all About1 artifacts to be invalidated together.

6. Cache Tag Helpers

type ArtifactName =
  | "descriptor.json"
  | "data.json"
  | "schema.json"
  | "puck.json"
  | "meta.json";
 
export function artifactTag(
  packageId: string,
  artifactName: ArtifactName,
) {
  return (
    `runtime-artifact:` +
    `${packageId}:` +
    `${artifactName}`
  );
}
 
export function packageTag(
  packageId: string,
) {
  return `runtime-package:${packageId}`;
}

Examples:

artifactTag(
  "about1",
  "data.json",
);

Returns:

runtime-artifact:about1:data.json

packageTag("about1");

Returns:

runtime-package:about1

7. Cached Artifact Fetcher

A reusable cached fetcher can be implemented with unstable_cache.

import {
  unstable_cache,
} from "next/cache";
 
type ArtifactName =
  | "descriptor.json"
  | "data.json"
  | "schema.json"
  | "puck.json"
  | "meta.json";
 
export async function getCachedPackageArtifact<T>(
  packageId: string,
  version: string,
  artifactName: ArtifactName,
  url: string,
): Promise<T | null> {
  const cachedFetch =
    unstable_cache(
      async () => {
        const response =
          await fetch(url, {
            cache: "no-store",
          });
 
        if (response.status === 404) {
          return null;
        }
 
        if (!response.ok) {
          throw new Error(
            `Failed to fetch ` +
            `${artifactName} for ` +
            `${packageId}: ` +
            `${response.status}`,
          );
        }
 
        return (
          response.json() as Promise<T>
        );
      },
      [
        "runtime-artifact-fetch",
        packageId,
        version,
        artifactName,
      ],
      {
        tags: [
          artifactTag(
            packageId,
            artifactName,
          ),
          packageTag(packageId),
        ],
      },
    );
 
  return cachedFetch();
}

The cache key includes:

runtime-artifact-fetch
package ID
package version
artifact name

Example:

runtime-artifact-fetch
about1
1.0.8
data.json

This ensures that different artifact files cannot return the same cached value.

8. Load About1 Artifacts

When the user adds About1:

const [
  descriptor,
  data,
  schema,
  puck,
  meta,
] = await Promise.all([
  getCachedPackageArtifact(
    "about1",
    "1.0.8",
    "descriptor.json",
    urls.descriptor,
  ),
 
  getCachedPackageArtifact(
    "about1",
    "1.0.8",
    "data.json",
    urls.data,
  ),
 
  getCachedPackageArtifact(
    "about1",
    "1.0.8",
    "schema.json",
    urls.schema,
  ),
 
  getCachedPackageArtifact(
    "about1",
    "1.0.8",
    "puck.json",
    urls.puck,
  ),
 
  getCachedPackageArtifact(
    "about1",
    "1.0.8",
    "meta.json",
    urls.meta,
  ),
]);

The requests can run in parallel because each artifact is independent.

9. Package Artifact Result

The loaded package can use this shape:

type PackageArtifacts = {
  descriptor: PackageDescriptor;
 
  data:
    | Record<string, unknown>
    | null;
 
  puck: {
    $source?: string;
    title?: string;
    type?: string;
    properties?:
      Record<string, unknown>;
  } | null;
 
  meta:
    | Record<string, unknown>
    | null;
 
  schema:
    | Record<string, unknown>
    | null;
 
  urls: {
    descriptor: string;
    data: string;
    puck: string;
    meta: string;
    schema: string;
    module: string;
  };
};

10. Build the CatalogEntry

The host combines:

Lightweight catalog package data
Descriptor
Data
Schema
Puck configuration
Meta information
Module URL
Artifact URLs

Example result:

{
  "id": "about1",
  "label": "About 1",
  "package": "@xsite/ui-about1",
  "version": "1.0.8",
  "category": "about",
  "renderMode": "client",
  "module": {
    "path": "./index.mjs",
    "url": "https://esm.xsite.live/@xsite/[email protected]?target=es2022&external=react,react-dom,react/jsx-runtime,next,@xsite/common"
  },
  "schema": {
    "path": "./json/schema.json",
    "url": "https://pub-xxxxxxxxxxxxxxxxxxx.r2.dev/@xsite/[email protected]/schema.json"
  },
  "puck": {
    "path": "./json/puck.json",
    "url": "https://pub-xxxxxxxxxxxxxxxxxxx.r2.dev/@xsite/[email protected]/puck.json"
  },
  "data": {
    "path": "./json/data.json",
    "url": "https://pub-xxxxxxxxxxxxxxxxxxx.r2.dev/@xsite/[email protected]/data.json"
  },
  "meta": {
    "path": "./json/meta.json",
    "url": "https://pub-xxxxxxxxxxxxxxxxxxx.r2.dev/@xsite/[email protected]/meta.json"
  },
  "preview": {
    "url": "https://raw.githubusercontent.com/xsite-io/xsite/main/packages/ui-about1/preview.png"
  }
}

11. Handoff to the Remote Module Loader

After the CatalogEntry is created, the host reads:

entry.module.url

The URL is passed to the remote component loader:

CatalogEntry.module.url
  ↓
loadRemoteComponent()
  ↓
GET /api/remote-component
  ↓
compileRemoteComponent()
  ↓
createEsmHttpPlugin()
  ↓
createHostRuntimePlugin()
  ↓
compiled browser ESM
  ↓
dynamic import()
  ↓
render About1

Artifact Invalidation

The main benefit of this architecture is that each JSON artifact can be invalidated independently.

12. UI Builder Changes Only `data.json`

Assume UI Builder updates:

about1/data.json

The package version remains:

1.0.8

After the new file is uploaded to R2, UI Builder calls:

POST /api/revalidate

Request body:

{
  "packageId": "about1",
  "artifact": "data.json"
}

The host invalidates:

runtime-artifact:about1:data.json

The other About1 artifact caches remain valid:

runtime-artifact:about1:descriptor.json
runtime-artifact:about1:schema.json
runtime-artifact:about1:puck.json
runtime-artifact:about1:meta.json

When the next user adds About1:

data.json
→ fetched again from R2

schema.json
→ returned from cache

puck.json
→ returned from cache

meta.json
→ returned from cache

descriptor.json
→ returned from cache

13. UI Builder Changes Only `schema.json`

UI Builder sends:

{
  "packageId": "about1",
  "artifact": "schema.json"
}

The host invalidates:

runtime-artifact:about1:schema.json

The next About1 load receives the new schema while reusing the other cached artifacts.

14. Invalidate All About1 Artifacts

To invalidate the complete package artifact set:

{
  "packageId": "about1"
}

The host invalidates:

runtime-package:about1

Because every About1 artifact cache includes the package-level tag, these are all invalidated:

descriptor.json
data.json
schema.json
puck.json
meta.json

Other packages such as About2 remain cached.

15. Protected Revalidation API

Use a protected POST route instead of a public query-string endpoint.

// app/api/revalidate/route.ts
 
import {
  revalidateTag,
} from "next/cache";
 
import {
  NextRequest,
  NextResponse,
} from "next/server";
 
const ALLOWED_ARTIFACTS =
  new Set([
    "descriptor.json",
    "data.json",
    "schema.json",
    "puck.json",
    "meta.json",
  ]);
 
export async function POST(
  request: NextRequest,
) {
  const secret =
    request.headers.get(
      "x-revalidation-secret",
    );
 
  if (
    !process.env
      .REVALIDATION_SECRET ||
    secret !==
      process.env
        .REVALIDATION_SECRET
  ) {
    return NextResponse.json(
      {
        error: "Unauthorized",
      },
      {
        status: 401,
      },
    );
  }
 
  const body =
    await request.json();
 
  const packageId =
    typeof body.packageId ===
    "string"
      ? body.packageId
      : null;
 
  const artifact =
    typeof body.artifact ===
    "string"
      ? body.artifact
      : null;
 
  if (!packageId) {
    return NextResponse.json(
      {
        error:
          "packageId is required",
      },
      {
        status: 400,
      },
    );
  }
 
  if (artifact) {
    if (
      !ALLOWED_ARTIFACTS
        .has(artifact)
    ) {
      return NextResponse.json(
        {
          error:
            "Invalid artifact name",
        },
        {
          status: 400,
        },
      );
    }
 
    const tag =
      artifactTag(
        packageId,
        artifact as ArtifactName,
      );
 
    revalidateTag(tag, "max");
 
    return NextResponse.json({
      revalidated: true,
      packageId,
      artifact,
      tag,
    });
  }
 
  const tag =
    packageTag(packageId);
 
  revalidateTag(tag, "max");
 
  return NextResponse.json({
    revalidated: true,
    packageId,
    tag,
  });
}

Environment variable:

REVALIDATION_SECRET=replace-with-a-strong-secret

16. UI Builder Invalidation Script

type ArtifactName =
  | "descriptor.json"
  | "data.json"
  | "schema.json"
  | "puck.json"
  | "meta.json";
 
export async function invalidateHostArtifact(
  packageId: string,
  artifact?: ArtifactName,
) {
  const response =
    await fetch(
      `${process.env.HOST_APP_URL}` +
      `/api/revalidate`,
      {
        method: "POST",
 
        headers: {
          "Content-Type":
            "application/json",
 
          "x-revalidation-secret":
            process.env
              .HOST_REVALIDATION_SECRET!,
        },
 
        body: JSON.stringify({
          packageId,
          artifact,
        }),
      },
    );
 
  if (!response.ok) {
    const message =
      await response.text();
 
    throw new Error(
      `Host cache invalidation ` +
      `failed: ${message}`,
    );
  }
 
  return response.json();
}

Environment variables:

HOST_APP_URL=https://host.xsite.live
HOST_REVALIDATION_SECRET=replace-with-the-same-secret

17. Publish and Invalidate Order

The upload must happen before invalidation.

Correct order:

Generate updated artifact
  ↓
Upload artifact to R2
  ↓
Verify upload succeeded
  ↓
Call host /api/revalidate
  ↓
Host invalidates artifact cache

Example:

await uploadArtifactToR2(
  "about1",
  "1.0.8",
  "data.json",
);
 
await invalidateHostArtifact(
  "about1",
  "data.json",
);

Do not invalidate before the R2 upload completes.

Otherwise, the host may immediately fetch:

The old object
A missing object
A partially published artifact

Complete Update Flow

UI Builder changes about1/data.json
  ↓
Generate new data.json
  ↓
Upload data.json to R2
  ↓
POST /api/revalidate
  {
    packageId: "about1",
    artifact: "data.json"
  }
  ↓
Invalidate:
runtime-artifact:about1:data.json
  ↓
Next user adds About1
  ↓
Host fetches new data.json
  ↓
Other About1 artifacts remain cached

New vs Existing Components

New About1 Component

User selects About1
  ↓
Host loads current data.json
  ↓
Default props are copied into new page component
  ↓
Component props are saved in CMS or IndexedDB

Existing About1 Component

Load saved page
  ↓
Find component type About1
  ↓
Load compatible runtime component
  ↓
Render using saved page props

The latest data.json must not automatically overwrite existing user-edited props.

Central rule:

data.json
→ default props for new instances

Saved page data
→ source of truth for existing instances

Recommended Tag Structure

runtime-catalog

runtime-package:about1

runtime-artifact:about1:descriptor.json

runtime-artifact:about1:data.json

runtime-artifact:about1:schema.json

runtime-artifact:about1:puck.json

runtime-artifact:about1:meta.json

Final Mental Model

Remote catalog
  ↓
List component metadata and previews
  ↓
User selects About1
  ↓
Host runtime catalog API
  ↓
Individually cached JSON artifacts
  ↓
Build CatalogEntry
  ↓
Pass module URL to remote loader
  ↓
Compile and render About1

Update behavior:

UI Builder changes one artifact
  ↓
Upload changed artifact to R2
  ↓
Invalidate only that artifact tag
  ↓
Next component insertion receives fresh data

Responsibility split:

Remote catalog
→ component discovery and previews

R2
→ package JSON artifacts

Next.js host cache
→ artifact-level caching

UI Builder
→ artifact publication and invalidation trigger

Remote module loader
→ component ESM compilation and browser import

Saved page data
→ existing component instance props

Remote Esm Component Loading Architecture Upload Catalog R2

Runtime Catalog and Artifact Cache Invalidation

Remote Catalog Structure

Revised Architecture

Overall Flow

1. Load the Lightweight Remote Catalog

2. User Adds About1

3. Resolve Package Artifact URLs

4. Artifact Responsibilities

5. Separate Cache for Every Artifact

6. Cache Tag Helpers

7. Cached Artifact Fetcher

8. Load About1 Artifacts

9. Package Artifact Result

10. Build the CatalogEntry

11. Handoff to the Remote Module Loader

Artifact Invalidation

12. UI Builder Changes Only data.json

13. UI Builder Changes Only schema.json

14. Invalidate All About1 Artifacts

15. Protected Revalidation API

16. UI Builder Invalidation Script

17. Publish and Invalidate Order

Complete Update Flow

New vs Existing Components

New About1 Component

Existing About1 Component

Recommended Tag Structure

Final Mental Model

12. UI Builder Changes Only `data.json`

13. UI Builder Changes Only `schema.json`