pyspark.sql.DataFrame.transform¶
-
DataFrame.transform(func)[source]¶ Returns a new
DataFrame. Concise syntax for chaining custom transformations.New in version 3.0.0.
- Parameters
- funcfunction
a function that takes and returns a
DataFrame.
Examples
>>> from pyspark.sql.functions import col >>> df = spark.createDataFrame([(1, 1.0), (2, 2.0)], ["int", "float"]) >>> def cast_all_to_int(input_df): ... return input_df.select([col(col_name).cast("int") for col_name in input_df.columns]) >>> def sort_columns_asc(input_df): ... return input_df.select(*sorted(input_df.columns)) >>> df.transform(cast_all_to_int).transform(sort_columns_asc).show() +-----+---+ |float|int| +-----+---+ | 1| 1| | 2| 2| +-----+---+