pyspark.InheritableThread

class pyspark.InheritableThread(target, *args, **kwargs)

Thread that is recommended to be used in PySpark instead of threading.Thread when the pinned thread mode is enabled. The usage of this class is exactly same as threading.Thread but correctly inherits the inheritable properties specific to JVM thread such as InheritableThreadLocal.

Also, note that pinned thread mode does not close the connection from Python to JVM when the thread is finished in the Python side. With this class, Python garbage-collects the Python thread instance and also closes the connection which finishes JVM thread correctly.

When the pinned thread mode is off, this works as threading.Thread.

New in version 3.1.0.

Notes

This API is experimental.

Methods

getName()
isDaemon()
is_alive()	Return whether the thread is alive.
join([timeout])	Wait until the thread terminates.
run()	Method representing the thread’s activity.
setDaemon(daemonic)
setName(name)
start(*args, **kwargs)	Start the thread’s activity.

Attributes

daemon	A boolean value indicating whether this thread is a daemon thread.
ident	Thread identifier of this thread or None if it has not been started.
name	A string used for identification purposes only.
native_id	Native integral thread ID of this thread, or None if it has not been started.

pyspark.BarrierTaskInfo pyspark.util.VersionUtils