pyspark.sql.functions.
isnan
An expression that returns true iff the column is NaN.
New in version 1.6.0.
Examples
>>> df = spark.createDataFrame([(1.0, float('nan')), (float('nan'), 2.0)], ("a", "b")) >>> df.select(isnan("a").alias("r1"), isnan(df.a).alias("r2")).collect() [Row(r1=False, r2=False), Row(r1=True, r2=True)]