pyspark.SparkContext.parallelize

SparkContext.parallelize(c: Iterable[T], numSlices: Optional[int] = None) → pyspark.rdd.RDD[T]

Distribute a local Python collection to form an RDD. Using range is recommended if the input represents a range for performance.

Examples

>>> sc.parallelize([0, 2, 3, 4, 6], 5).glom().collect()
[[0], [2], [3], [4], [6]]
>>> sc.parallelize(range(0, 6, 2), 5).glom().collect()
[[], [0], [], [2], [4]]

pyspark.SparkContext.newAPIHadoopRDD pyspark.SparkContext.pickleFile