pyspark.streaming.StreamingContext.getOrCreate

classmethod StreamingContext.getOrCreate(checkpointPath, setupFunc)

Either recreate a StreamingContext from checkpoint data or create a new StreamingContext. If checkpoint data exists in the provided checkpointPath, then StreamingContext will be recreated from the checkpoint data. If the data does not exist, then the provided setupFunc will be used to create a new context.

Parameters

checkpointPathstr

Checkpoint directory used in an earlier streaming program

setupFuncfunction

Function to create a new context and setup DStreams

pyspark.streaming.StreamingContext.getActiveOrCreate pyspark.streaming.StreamingContext.remember