pyspark.sql.functions.overlay#

pyspark.sql.functions.overlay(src, replace, pos, len=- 1)[source]#

Overlay the specified portion of src with replace, starting from byte position pos of src and proceeding for len bytes.

New in version 3.0.0.

Changed in version 3.4.0: Supports Spark Connect.

Parameters

srcColumn or str: column name or column containing the string that will be replaced
replaceColumn or str: column name or column containing the substitution string
posColumn or str or int: column name, column, or int containing the starting position in src
lenColumn or str or int, optional: column name, column, or int containing the number of bytes to replace in src string by ‘replace’ defaults to -1, which represents the length of the ‘replace’ string

Returns

Column: string with replaced values.

Examples

>>> df = spark.createDataFrame([("SPARK_SQL", "CORE")], ("x", "y"))
>>> df.select(overlay("x", "y", 7).alias("overlayed")).collect()
[Row(overlayed='SPARK_CORE')]
>>> df.select(overlay("x", "y", 7, 0).alias("overlayed")).collect()
[Row(overlayed='SPARK_CORESQL')]
>>> df.select(overlay("x", "y", 7, 2).alias("overlayed")).collect()
[Row(overlayed='SPARK_COREL')]