PipelineStep

Shared request and response schema. Request: Pipeline step accepted in a definition. Exactly one of script or transform must be set. Transform configuration refers to inputs and outputs by port key, and transform steps cannot use secrets. Response: Pipeline step returned in a definition. Exactly one of script or transform is set. Transform configuration refers to inputs and outputs by port key, and transform steps do not use secrets.

nameName (string)필수

Step name

alias object

하나 이상(anyOf)

string
null

string

description object

Step-owned description, independent of the referenced Code's description so steps sharing one Code can be labeled distinctly.

하나 이상(anyOf)

string
null

string

script object

Code resource ID to execute (code step)

하나 이상(anyOf)

string
null

string

transform object

Native transform spec (transform step)

하나 이상(anyOf)

object
null

op필수

Possible values: [join, stages, union]

join
stages
union

versionVersion (integer)

상수 값: 1

기본값: 1

config object필수

Two-input equi-join configuration. left_input and right_input are pipeline port keys, keys contains at least one equality pair, and columns explicitly selects the output. Other join predicates are not accepted.

left_inputLeft Input (string)필수

Left input port key

right_inputRight Input (string)필수

Right input port key

join_typeJoin Type (string)필수

inner | left | right | full

Possible values: [inner, left, right, full]

keys object[]필수

Equi-join column pairs (non-empty)

Array [

leftLeft (string)필수

Join column on the left input

rightRight (string)필수

Join column on the right input

]

columns object[]필수

Output projection (non-empty)

Array [

inputInput (string)필수

Source input port key (left_input or right_input)

nameName (string)필수

Source column name on that input

alias object

Output column name (defaults to name)

하나 이상(anyOf)

string
null

string

]

versionVersion (integer)

상수 값: 1

기본값: 1

config object필수

Config for the stages op.

inputInput (string)필수

Input port key (a PipelineStep.inputs key)

pipeline object[]필수

Ordered stages (non-empty)

Possible values: >= 1

Array [

다음 중 하나(oneOf)

opOp (string)필수

상수 값: select

exprs object

하나 이상(anyOf)

object[]
null

Array [

nameName (string)필수

Output column name

exprExpr (string)필수

Polars-SQL expression, e.g. 'qty * unit_price'

]

columns object

하나 이상(anyOf)

string[]
null

Array [

string

]

versionVersion (integer)

상수 값: 1

기본값: 1

config object필수

Union configuration for two or more pipeline input ports. Inputs are aligned by column name, missing values become null, and duplicate rows are retained.

inputsstring[]필수

Input port keys to union (>= 2)

agent object

Workflow agent call spec (agent step)

하나 이상(anyOf)

AgentStepSpec
null

versionVersion (integer)

상수 값: 1

기본값: 1

agent_idAgent Id (string)필수

Agent resource ID (workflow mode)

timeout_seconds object

Per-call timeout. Capped at the agent control plane's fixed 300s forward timeout, above which the call 504s before this applies.

하나 이상(anyOf)

integer
null

integer

Possible values: > 0 and <= 300

inputs object

Input dataset mapping

하나 이상(anyOf)

object
null

object

outputs object

Output dataset mapping

하나 이상(anyOf)

object
null

object

dependencies object

List of dependent step names

하나 이상(anyOf)

string[]
null

Array [

string

]

options object

하나 이상(anyOf)

object
null

object

secrets object

Secret values to inject into contexts['secrets']

하나 이상(anyOf)

object[]
null

Array [

nameName (string)필수

contexts['secrets'] key the step reads

refRef (string)필수

Secret reference, e.g. secret://secret./token

]

PipelineStep
{
  "name": "string",
  "alias": "string",
  "description": "string",
  "script": "string",
  "transform": {
    "version": 1,
    "op": "stages",
    "config": {
      "input": "string",
      "pipeline": [
        {
          "op": "select",
          "exprs": [
            {
              "name": "string",
              "expr": "string"
            }
          ],
          "columns": [
            "string"
          ]
        },
        {
          "op": "with_columns",
          "exprs": [
            {
              "name": "string",
              "expr": "string"
            }
          ]
        },
        {
          "op": "filter",
          "expr": "string"
        },
        {
          "op": "drop",
          "columns": [
            "string"
          ]
        },
        {
          "op": "rename",
          "map": {}
        },
        {
          "op": "cast",
          "dtypes": {}
        },
        {
          "op": "sort",
          "by": [
            "string"
          ],
          "descending": false
        },
        {
          "op": "unique",
          "subset": [
            "string"
          ],
          "keep": "any"
        },
        {
          "op": "head",
          "n": 0
        },
        {
          "op": "tail",
          "n": 0
        },
        {
          "op": "slice",
          "offset": 0,
          "len": 0
        },
        {
          "op": "drop_nulls",
          "subset": [
            "string"
          ]
        },
        {
          "op": "fill_null",
          "expr": "string"
        },
        {
          "op": "with_row_index",
          "name": "string",
          "offset": 0
        },
        {
          "op": "group_by",
          "by": [
            "string"
          ],
          "agg": [
            {
              "name": "string",
              "expr": "string"
            }
          ]
        },
        {
          "op": "group_by_dynamic",
          "index_column": "string",
          "every": "string",
          "period": "string",
          "offset": "string",
          "closed": "left",
          "by": [
            "string"
          ],
          "agg": [
            {
              "name": "string",
              "expr": "string"
            }
          ]
        },
        {
          "op": "explode",
          "columns": [
            "string"
          ]
        },
        {
          "op": "unnest",
          "columns": [
            "string"
          ]
        },
        {
          "op": "unpivot",
          "on": [
            "string"
          ],
          "index": [
            "string"
          ]
        }
      ]
    }
  },
  "agent": {
    "version": 1,
    "agent_id": "string",
    "timeout_seconds": 0
  },
  "inputs": {},
  "outputs": {},
  "dependencies": [
    "string"
  ],
  "options": {},
  "secrets": [
    {
      "name": "string",
      "ref": "string"
    }
  ]
}