How to Compose Push Data for a Search Index

Problem Statement

Uniform combines component structure with external data via edgehancers (Uniform code running on the CDN edge). To feed a search index, you can use the Uniform SDK to pull the final edgehanced composition JSON for every page and flatten it into documents to push.

Solution

1. Create the indexing script (static pages)

The script retrieves all project map nodes, fetches the composition for each, and combines a JSON document from simple parameter values such as Text. Adjust retrieveContent to your composition structure.

import { RouteClient } from '@uniformdev/canvas';
import { ProjectMapClient } from '@uniformdev/project-map';
import dotenv from 'dotenv';
dotenv.config();

const projectMapId = '40535f31-cdc1-4ac3-bb4d-2a008493b431';

const getRouteClient = () => {
  const apiKey = process.env.UNIFORM_API_KEY;
  const edgeApiHost = process.env.UNIFORM_CLI_BASE_EDGE_URL || 'https://uniform.global';
  const projectId = process.env.UNIFORM_PROJECT_ID;

  if (!apiKey) {
    throw new Error('apiKey is not specified. RouteClient cannot be instantiated: ' + apiKey);
  }
  if (!edgeApiHost) throw new Error('edgeApiHost is not specified. RouteClient cannot be instantiated');
  if (!projectId) throw new Error('projectId is not specified. RouteClient cannot be instantiated.');

  return new RouteClient({ apiKey, projectId, edgeApiHost, bypassCache: true });
};

const projectMapClient = new ProjectMapClient({
  apiKey: process.env.UNIFORM_API_KEY,
  projectId: process.env.UNIFORM_PROJECT_ID,
});

const getPaths = async (): Promise<{ path: string }[]> => {
  const { nodes } = await projectMapClient.getNodes({ projectMapId });

  if (!nodes) {
    console.warn('No project map nodes found');
    return [];
  }

  return nodes.filter(node => node.compositionId).map(node => ({ path: node.path }));
};

const routeClient = getRouteClient();

const getCompositionDataByPath = async (path: string) => {
  const res = await routeClient.getRoute({
    path,
    projectMapId,
    state: 0,
    withComponentIDs: true,
  });

  if (res.type !== 'composition') throw new Error('Requested route is not a composition');

  const { composition } = res.compositionApiResponse;
  return composition;
};

const findTextParameterValues = (inputArray: any[]) => {
  const textValues: any[] = [];

  function traverseArray(arr: any[]) {
    arr.forEach(obj => {
      if (obj.parameters) {
        traverseObject(obj.parameters);
      }
    });
  }

  function traverseObject(obj: { [x: string]: any }) {
    for (const key in obj) {
      if (obj[key].type === 'text') {
        textValues.push(obj[key].value);
      } else if (typeof obj[key] === 'object') {
        traverseObject(obj[key]);
      }
    }
  }

  traverseArray(inputArray);
  return textValues.join(', ');
};

const retrieveContent = async (composition: any) => {
  const { parameters, slots } = composition;
  // content-specific code below
  const title = parameters?.pageTitle?.value;
  const pageContent = findTextParameterValues(slots?.pageContent);

  return { title, pageContent };
};

const composeDataForIndex = async () => {
  console.log('Composing the data to index...');
  const pathsToIndex = await getPaths();

  const dataToIndex = await Promise.all(
    pathsToIndex.map(async p => {
      const compositionDataToAppend = await getCompositionDataByPath(p.path);
      const processedComposition = await retrieveContent(compositionDataToAppend);
      return { ...p, ...processedComposition };
    })
  );

  console.log('Data is ready');
  return dataToIndex;
};

composeDataForIndex().then(data => console.log(data));

2. Run it after the build

Save the script as scripts/rebuild-index.ts and call it on postbuild:

"postbuild": "npx ts-node scripts/rebuild-index.ts",

3. Support dynamic pages (optional)

With Uniform dynamic pages, Uniform doesn't know the values used to compose the URL, so fetch the slugs from your CMS and expand the dynamic paths. This example pulls slugs from Contentstack — replace getSlugsFromCMS with your CMS call, then update getPaths:

const getSlugsFromCMS = async () => {
  // here comes the CMS-specific code
  const url =
    'https://cdn.contentstack.io/v3/content_types/blog_post/entries?environment=production&query={}&skip=0&limit=100&include_count=true';
  const apiKey = 'xxx';
  const accessToken = 'yyy';

  try {
    const response = await fetch(url, {
      method: 'GET',
      headers: {
        'Content-Type': 'application/json',
        api_key: apiKey,
        access_token: accessToken,
      },
    });

    if (!response.ok) {
      throw new Error(`HTTP error!`);
    }

    const data = await response.json();
    return data.entries.map((entry: any) => ({ slug: entry.url }));
  } catch (error: any) {
    console.error('Error fetching data:', error.message);
    throw error;
  }
};

const getPaths = async (): Promise<{ path: string }[]> => {
  const { nodes } = await projectMapClient.getNodes({ projectMapId });

  if (!nodes) {
    console.warn('No project map nodes found');
    return [];
  }

  const staticPaths = nodes.filter(node => node.compositionId).map(node => ({ path: node.path }));

  const newPaths: { path: string }[] = [];
  const slugsFromCms = await getSlugsFromCMS();
  const tokenToReplace = ':article-title';

  staticPaths.forEach((node: { path: string }) => {
    if (node.path.includes(tokenToReplace)) {
      newPaths.push(
        ...slugsFromCms.map((slug: { slug: string }) => ({
          path: node.path.replace(tokenToReplace, slug.slug),
        }))
      );
    } else {
      newPaths.push({ path: node.path });
    }
  });

  return newPaths;
};

Troubleshooting

Verify it works: run npx ts-node scripts/rebuild-index.ts — the console prints an array of { path, title, pageContent } documents ready to push to your search index.

Build takes too long: this is a full index rebuild. Hosting providers may limit Next.js build time, so for larger sites (hundreds of pages) trigger the rebuild outside the Next.js build instead of in postbuild.

Resources

• Dynamic compositions