---
sidebar_label: Items and data flow
toc_max_heading_level: 3
doc_id: ffc3367d-36fa-4fe6-a562-f4f700d7890b
description: >-
  How data moves through a nullplatform workflow: items, per-item versus
  whole-batch steps, when the engine fans a step out on its own, and what
  happens when several steps feed one.
keywords:
  - workflows
  - items
  - executeMode
  - fan-out
  - join strategy
  - maxStepParallelism
  - data flow
  - nullplatform
---

# Items and data flow

Every step in a workflow receives an array and emits an array. The elements are called **items**, and an item is a plain JSON object: `{"scope_id": 1234, "ami": "ami-0abc"}`. There is no envelope, no wrapper, no special binary type.

Keep that rule in mind and the rest of the page follows from it: why a step sometimes runs five times, why a summary step sees everything at once, and how a fan-out over a thousand scopes behaves.

## Read a step's results with `items` or `outputs`

When a step finishes, the engine records its items. You read them with two expressions:

- `steps.detect.items` returns the full array.
- `steps.detect.outputs.total` returns the `total` field of the first item. It's shorthand for `steps.detect.items[0].total`.

Use `outputs` when a step produces a single result, which is the common case. Use `items` when a step returns a collection: `outputs` still works there, but it silently gives you only the first item.

```yaml
inputs:
  total:    "${{ steps.detect.outputs.total }}"   # one field of the first item
  findings: "${{ steps.detect.items }}"           # every item
```

## How many times a step runs: per item or once per batch

Each plugin declares one of two modes, and the engine invokes it accordingly.

In **per-item** mode the engine calls the plugin once per input item. Three items in means three invocations, and the three results are collected back into one array. Inside each invocation the step sees `$item` (the object it's working on), `$itemIndex` (its position) and `$itemsLength` (how many items are in the batch).

In **whole-batch** mode the engine calls the plugin once and hands it the entire array as `$items`. One invocation, one result. Use this mode for anything that aggregates, counts, or summarizes.

The defaults cover the common case without any configuration:

| Plugin | Mode | Why |
|---|---|---|
| `code-exec`, `log`, `set-variable` | whole batch | Aggregating is the usual reason to reach for them |
| `sub-workflow` | per item | One child run per item is nearly always the intent |
| `http-request`, and most others | per item | One call per item is usually correct |

To change the mode on one step, set `metadata.fanOutPerItem`: `true` makes a whole-batch plugin run once per item, and `false` makes a per-item plugin run once with the whole batch.

```yaml
- id: mark_processed
  type: module
  plugin_type: code-exec
  metadata:
    fanOutPerItem: true      # code-exec takes the whole batch by default
  config:
    code: |
      var r = $item || {};
      return { ...r, processed: true };
```

### Why `$items` is unavailable in a per-item step

A per-item invocation gets `$item`, `$itemIndex` and `$itemsLength`, and never `$items`. If a per-item step references `$items`, graph validation fails with `FANOUT_ITEMS_UNAVAILABLE` and the workflow can't be saved.

The reason is size. Copying the whole batch into every per-item invocation multiplies the payload by the number of items, so a hundred-item page of scopes becomes a hundred copies of that page. That is what ended a production run mid-sweep, and it's why the engine now rejects it before the run starts.

If a step needs the whole batch, run it in whole-batch mode: put the aggregation in a `set-variable` or `code-exec` step, which already work that way.

## Automatic fan-out, and what turns it off

A step doesn't need a `forEach` block to run many times. When the step before it hands over several items and its plugin runs per item, the engine dispatches it once per item automatically, up to 5 at a time by default.

Four things turn that automatic fan-out off. Check them first when a step you expected to run per item ran only once:

- **More than one upstream edge carries items.** That is a join, not a fan-out: the plugin runs once with every upstream item concatenated. This is also what keeps re-entry loops from firing forever.
- **The step declares `forEach`.** You asked for explicit iteration, so the engine doesn't add its own.
- **`error_handling.retry_policy.max_attempts` is greater than 1.** A retry policy on a step disables its automatic fan-out. Put the retry on the steps inside a sub-workflow instead.
- **`metadata.fanOutPerItem: false`.** The explicit switch.

To control how many items run at once, set `metadata.maxParallelism` on the step. If it's not set, the engine uses the plugin's `config.maxParallelism`, and failing that the default of 5.

## Run independent branches in parallel

A branch is a path that leaves a step: two steps connected to the same predecessor, or the `true` and `false` ports of a decider. Nothing in the graph forces an order between branches, but by default the engine runs them one after another anyway. It starts one ready step, waits for it to finish, and only then starts the next.

That default is safe rather than fast. Branches written expecting to run in series often share a rate-limited API, or write the same variable, and when two branches write the same variable the last write wins. Running them side by side could change what the run produces.

To run branches in parallel, first check that they are independent: no shared variable, no API that would throttle two callers at once. Then set `maxStepParallelism` on the workflow, which caps how many ready steps run at the same time:

```yaml
metadata:
  maxStepParallelism: 3   # 1 is the default, 32 the maximum
```

Whatever the value, ready steps always start in step id order, and when several fail at once the run reports the failure from the lowest step id. A run stays predictable no matter which branch happens to finish first.

This setting is for branches only. To run the items of one step in parallel, use `metadata.maxParallelism` on that step, as described in the previous section, or a `forEach` block with `parallel: true`.

## How deciders split items between ports

A decider in per-item mode evaluates its expression once per item and sends each item out the matching port, so one input batch becomes two output batches:

```
in:  [{value: 5}, {value: 15}, {value: 8}, {value: 20}]

conditional  (value > 10)
  true  →  [{value: 15}, {value: 20}]
  false →  [{value: 5},  {value: 8}]
```

Both ports always carry an array. A branch that matched nothing receives an empty array rather than nothing at all.

## Join several steps into one with a join strategy

When a step has more than one incoming connection, the engine needs a rule for when to start it. That rule is the **join strategy**, set with `join_strategy` on the step:

| Strategy | Fires when |
|---|---|
| `all` (default) | Every predecessor has reached a terminal state. A failed predecessor counts as terminal |
| `any` | The first predecessor **completes**. Failed, skipped, and cancelled ones never satisfy it |
| `allSettled` | Every predecessor is terminal, success or failure alike. Skipped ones count as settled |
| `count` | At least `join_count` predecessors have completed. Failed and skipped ones don't count toward the quorum |

Whatever arrives from the predecessors is concatenated into one input array.

Two cases come up in almost every workflow that joins:

**A loop back into a node needs `join_strategy: any`.** A node that a later step loops back into has two incoming edges: the first entry and the re-entry. With the default `all` it waits on both forever and the loop deadlocks. Every published suite that loops sets `any` on that node.

**Give success and failure paths separate nodes.** A node fed by one completed edge and one failure edge can't start under any strategy: `any` ignores the failure edge and waits for a success that never comes, and `all` waits on the edge that was neither settled nor skipped. Point each path at its own node with a single predecessor, where the default `all` is satisfied as soon as that predecessor finishes:

```yaml
- id: resolve_ok        # single edge from classify_ok:true
- id: resolve_failed    # single edge from the failure lane
```

This is why [error handling](/docs/workflows/building-blocks/error-handling) uses one resolve node per lane instead of a shared final node.

## Next steps

- [Loops and iteration](/docs/workflows/building-blocks/loops): `forEach`, batches, and paging through a large source
- [Error handling](/docs/workflows/building-blocks/error-handling): retries, fallbacks, and the failure lane
- [Expressions](/docs/workflows/building-blocks/expressions): `$item`, `steps.*`, and the rest of what you can reference
- [Limits](/docs/workflows/building-blocks/limits): what a fan-out costs in bytes