pyspark.sql.functions.get#
- pyspark.sql.functions.get(col, index)[source]#
Array function: Returns the element of an array at the given (0-based) index. If the index points outside of the array boundaries, then this function returns NULL.
New in version 3.4.0.
- Parameters
- Returns
Column
Value at the given position.
See also
Notes
The position is not 1-based, but 0-based index. Supports Spark Connect.
Examples
Example 1: Getting an element at a fixed position
>>> from pyspark.sql import functions as sf >>> df = spark.createDataFrame([(["a", "b", "c"],)], ['data']) >>> df.select(sf.get(df.data, 1)).show() +------------+ |get(data, 1)| +------------+ | b| +------------+
Example 2: Getting an element at a position outside the array boundaries
>>> from pyspark.sql import functions as sf >>> df = spark.createDataFrame([(["a", "b", "c"],)], ['data']) >>> df.select(sf.get(df.data, 3)).show() +------------+ |get(data, 3)| +------------+ | NULL| +------------+
Example 3: Getting an element at a position specified by another column
>>> from pyspark.sql import functions as sf >>> df = spark.createDataFrame([(["a", "b", "c"], 2)], ['data', 'index']) >>> df.select(sf.get(df.data, df.index)).show() +----------------+ |get(data, index)| +----------------+ | c| +----------------+
Example 4: Getting an element at a position calculated from another column
>>> from pyspark.sql import functions as sf >>> df = spark.createDataFrame([(["a", "b", "c"], 2)], ['data', 'index']) >>> df.select(sf.get(df.data, df.index - 1)).show() +----------------------+ |get(data, (index - 1))| +----------------------+ | b| +----------------------+
Example 5: Getting an element at a negative position
>>> from pyspark.sql import functions as sf >>> df = spark.createDataFrame([(["a", "b", "c"], )], ['data']) >>> df.select(sf.get(df.data, -1)).show() +-------------+ |get(data, -1)| +-------------+ | NULL| +-------------+