Batch steps

A company step sees one company. Some questions need them all: is this company also on the list under another name? Which ten are the best fit this week? A batch step runs once over every company that reached it, and returns a result for each.

  - { id: twins, kind: batch, title: Find duplicates, run: steps/twins.ts, crm: { read: [companies] } }

Write one

defineBatchStep gets the companies and returns an object keyed by each company’s key, with the same results a company step returns:

import { defineBatchStep, drop, type StepReturn } from "@motionskit/sdk";

export default defineBatchStep({
  async run(companies, ctx) {
    const out: Record<string, StepReturn> = {};
    const firstByDomain = new Map<string, string>();

    for (const c of companies) {
      const domain = c.core.domain?.replace(/^www\./, "");
      const first = domain ? firstByDomain.get(domain) : undefined;
      if (!domain || !first) {
        if (domain) firstByDomain.set(domain, c.key);
        out[c.key] = { facts: { twin_of: null } };
        continue;
      }

      const answer = ctx.answer("duplicate", c.key);
      if (answer === "same") out[c.key] = { ...drop("duplicate"), facts: { twin_of: first } };
      else if (answer) out[c.key] = { facts: { twin_of: null } };
      else
        out[c.key] = {
          facts: { twin_of: first },
          questions: [
            {
              type: "duplicate",
              prompt: `${c.core.name} has the same domain, ${domain}, as another company on this list. Is it the same company?`,
              options: [
                { id: "same", text: "Same company: take this row out" },
                { id: "different", text: "Different companies: keep both" },
              ],
              recommended: "same",
              blocking: true,
            },
          ],
        };
    }
    return out;
  },
});

When it runs

A batch step waits until no company is still on its way to it, then runs once with every company that reached it. Companies that were dropped or held earlier aren’t included. Parked companies don’t hold it up.

Questions and re-runs

A batch step can’t use ctx.ask, which waits for one company. Return questions in a company’s result instead, with blocking: true to make that company wait.

When someone answers, the step runs again for that company only, with every company still in view so it can compare again. ctx.rerun lists the keys whose results count on this run; results for the others are ignored. Read the answer with the company’s key: ctx.answer("duplicate", c.key).

Keep the comparison stable

Because a re-run sees the same companies, a batch step that sorts or compares deterministically gives the same verdicts the second time, and only the answered company’s result changes. Avoid anything that depends on the order of the list, unless you sort it first.

Cost and limits

A batch step’s cap.per_batch_usd covers the whole run. It has the same limits as a company step, for the whole run: 30 seconds of CPU and 500 source and CRM calls. For lists in the hundreds, keep per-company work, such as source lookups, in company steps, and use the batch step only to compare.

Testing

runStep takes companies for a batch step and returns results keyed by company. Pass rerun to test a re-run after an answer:

const { results } = await runStep(twins, {
  companies: [
    { key: "a", core: { name: "Aurora", domain: "aurora.example" } },
    { key: "b", core: { name: "Aurora Metais", domain: "www.aurora.example" }, answers: { duplicate: "same" } },
  ],
  rerun: ["b"],
  fixtures: [],
});
expect(results.b).toMatchObject({ status: "ok", outcome: { lane: "drop", reason: "duplicate" } });