PType

Definition of PType.

Author: Wang, Cong(bigflow-opensource@baidu.com)

class bigflow.ptype.PType(node, pipeline)

基类：object

PType is a basic abstraction of data, which represents an immutable, partitioned collection of elements that can be operated on paraellel.

A PType can be either produced by:

• bigflow.pipeline.pipeline_base.PipelineBase.read()
• bigflow.pipeline.pipeline_base.PipelineBase.parallelize()
• bigflow.transforms of other PTypes

A PType can be consumed by:

• bigflow.pipeline.pipeline_base.PipelineBase.write()
• bigflow.pipeline.pipeline_base.PipelineBase.get()
• bigflow.transforms of itself

注解

End-users are not supposed to use its raw constructor.

参数:node (Node) -- LogicalPlan.Node

apply(transform, *args, **kargs)

Apply a transform on itself, p.apply(transform, *args) is equivalent to p.transform(*args)

参数:

• transform (function) -- transform to be applied
• *args -- variable length argument list

返回:

result of transform

返回类型:

PType

cache()

Persist this PType on storage.

get()

Get runtime value from this PType, equivalent to pipeline.get(self).

返回:runtime value 返回类型:object

node()

Return the internal node it holds.

注解

End-users are not supposed to use this function.

返回:The node. 返回类型:LogicalPlan.Node

pipeline()

Return the pipeline this PType belongs to

返回:pipeline 返回类型:Pipeline

serde()

Return the serde of the dataset

返回:The serde. 返回类型:LogicalPlan.Node