pyspark.pandas.notna#

pyspark.pandas.notna(obj)[source]#

Detect existing (non-missing) values.

Return a boolean same-sized object indicating if the values are not NA. Non-missing values get mapped to True. NA values, such as None or numpy.NaN, get mapped to False values.

Returns

bool or array-like of bool: Mask of bool values for each element that indicates whether an element is not an NA value.

See also

isna: Detect missing values for an array-like object.
Series.notna: Boolean inverse of Series.isna.
DataFrame.notnull: Boolean inverse of DataFrame.isnull.
Index.notna: Boolean inverse of Index.isna.
Index.notnull: Boolean inverse of Index.isnull.

Examples

Show which entries in a DataFrame are not NA.

>>> df = ps.DataFrame({'age': [5, 6, np.NaN],
...                    'born': [pd.NaT, pd.Timestamp('1939-05-27'),
...                             pd.Timestamp('1940-04-25')],
...                    'name': ['Alfred', 'Batman', ''],
...                    'toy': [None, 'Batmobile', 'Joker']})
>>> df
   age       born    name        toy
0  5.0        NaT  Alfred       None
1  6.0 1939-05-27  Batman  Batmobile
2  NaN 1940-04-25              Joker

>>> df.notnull()
     age   born  name    toy
0   True  False  True  False
1   True   True  True   True
2  False   True  True   True

Show which entries in a Series are not NA.

>>> ser = ps.Series([5, 6, np.NaN])
>>> ser
0    5.0
1    6.0
2    NaN
dtype: float64

>>> ps.notna(ser)
0     True
1     True
2    False
dtype: bool

>>> ps.notna(ser.index)
True