StagesTransform

The stages member of the PipelineTransform union.

versionVersion (integer)

Constant value: 1

Default value: 1

opOp (string)required

Constant value: stages

config objectrequired

Config for the stages op.

inputInput (string)required

Input port key (a PipelineStep.inputs key)

pipeline object[]required

Ordered stages (non-empty)

Possible values: >= 1

Array [

oneOf

• SelectStage
• WithColumnsStage
• FilterStage
• DropStage
• RenameStage
• CastStage
• SortStage
• UniqueStage
• HeadStage
• TailStage
• SliceStage
• DropNullsStage
• FillNullStage
• WithRowIndexStage
• GroupByStage
• GroupByDynamicStage
• ExplodeStage
• UnnestStage
• UnpivotStage

opOp (string)required

Constant value: select

exprs object

anyOf

• object[]
• null

Array [

nameName (string)required

Output column name

exprExpr (string)required

Polars-SQL expression, e.g. 'qty * unit_price'

]

columns object

anyOf

• string[]
• null

Array [

string

]

]

StagesTransform

{
  "version": 1,
  "op": "stages",
  "config": {
    "input": "string",
    "pipeline": [
      {
        "op": "select",
        "exprs": [
          {
            "name": "string",
            "expr": "string"
          }
        ],
        "columns": [
          "string"
        ]
      },
      {
        "op": "with_columns",
        "exprs": [
          {
            "name": "string",
            "expr": "string"
          }
        ]
      },
      {
        "op": "filter",
        "expr": "string"
      },
      {
        "op": "drop",
        "columns": [
          "string"
        ]
      },
      {
        "op": "rename",
        "map": {}
      },
      {
        "op": "cast",
        "dtypes": {}
      },
      {
        "op": "sort",
        "by": [
          "string"
        ],
        "descending": false
      },
      {
        "op": "unique",
        "subset": [
          "string"
        ],
        "keep": "any"
      },
      {
        "op": "head",
        "n": 0
      },
      {
        "op": "tail",
        "n": 0
      },
      {
        "op": "slice",
        "offset": 0,
        "len": 0
      },
      {
        "op": "drop_nulls",
        "subset": [
          "string"
        ]
      },
      {
        "op": "fill_null",
        "expr": "string"
      },
      {
        "op": "with_row_index",
        "name": "string",
        "offset": 0
      },
      {
        "op": "group_by",
        "by": [
          "string"
        ],
        "agg": [
          {
            "name": "string",
            "expr": "string"
          }
        ]
      },
      {
        "op": "group_by_dynamic",
        "index_column": "string",
        "every": "string",
        "period": "string",
        "offset": "string",
        "closed": "left",
        "by": [
          "string"
        ],
        "agg": [
          {
            "name": "string",
            "expr": "string"
          }
        ]
      },
      {
        "op": "explode",
        "columns": [
          "string"
        ]
      },
      {
        "op": "unnest",
        "columns": [
          "string"
        ]
      },
      {
        "op": "unpivot",
        "on": [
          "string"
        ],
        "index": [
          "string"
        ]
      }
    ]
  }
}