pyspark.sql.DataFrame.writeStream

property DataFrame.writeStream

Interface for saving the content of the streaming DataFrame out into external storage.

New in version 2.0.0.

Changed in version 3.5.0: Supports Spark Connect.

Returns

DataStreamWriter

Notes

This API is evolving.

Examples

>>> import time
>>> import tempfile
>>> df = spark.readStream.format("rate").load()
>>> type(df.writeStream)
<class '...streaming.readwriter.DataStreamWriter'>

>>> with tempfile.TemporaryDirectory(prefix="writeStream") as d:
...     # Create a table with Rate source.
...     query = df.writeStream.toTable(
...         "my_table", checkpointLocation=d)
...     time.sleep(3)
...     query.stop()