pyspark.sql.functions.input_file_name¶

pyspark.sql.functions.input_file_name() → pyspark.sql.column.Column[source]¶

Creates a string column for the file name of the current Spark task.

New in version 1.6.0.

Changed in version 3.4.0: Supports Spark Connect.

Returns

Column: file names.

Examples

>>> import os
>>> path = os.path.abspath(__file__)
>>> df = spark.read.text(path)
>>> df.select(input_file_name()).first()
Row(input_file_name()='file:///...')

pyspark.sql.functions.coalesce

pyspark.sql.functions.isnan