pyspark.sql.streaming.StreamingQuery.awaitTermination

StreamingQuery.awaitTermination(timeout: Optional[int] = None) → Optional[bool]

Waits for the termination of this query, either by query.stop() or by an exception. If the query has terminated with an exception, then the exception will be thrown. If timeout is set, it returns whether the query has terminated or not within the timeout seconds.

If the query has terminated, then all subsequent calls to this method will either return immediately (if the query was terminated by stop()), or throw the exception immediately (if the query has terminated with exception).

throws StreamingQueryException, if this query has terminated with an exception

New in version 2.0.0.

Parameters

timeoutint, optional

default None. The waiting time for specified streaming query to terminate.

Returns

bool, optional

The result whether specified streaming query has terminated or not within the timeout seconds if timeout is set. The StreamingQueryException will be thrown if the query has terminated with an exception.

Examples

>>> sdf = spark.readStream.format("rate").load()
>>> sq = sdf.writeStream.format('memory').queryName('query_awaitTermination').start()

Return whether the query has terminated or not within 5 seconds

>>> sq.awaitTermination(5)
False

>>> sq.stop()

Query Management pyspark.sql.streaming.StreamingQuery.exception