pyspark.pandas.window.ExponentialMoving.mean

ExponentialMoving.mean() → FrameLike

Calculate an online exponentially weighted mean.

Returns

Series or DataFrame

Returned object type is determined by the caller of the exponentially calculation.

See also

pyspark.pandas.Series.expanding

Calling object with Series data.

pyspark.pandas.DataFrame.expanding

Calling object with DataFrames.

pyspark.pandas.Series.mean

Equivalent method for Series.

pyspark.pandas.DataFrame.mean

Equivalent method for DataFrame.

Notes

There are behavior differences between pandas-on-Spark and pandas.

• the current implementation of this API uses Spark’s Window without specifying partition specification. This leads to move all data into single partition in single machine and could cause serious performance degradation. Avoid this method against very large dataset.

Examples

The below examples will show computing exponentially weighted moving average.

>>> df = ps.DataFrame({'s1': [.2, .0, .6, .2, .4, .5, .6], 's2': [2, 1, 3, 1, 0, 0, 0]})
>>> df.ewm(com=0.1).mean()
         s1        s2
0  0.200000  2.000000
1  0.016667  1.083333
2  0.547368  2.827068
3  0.231557  1.165984
4  0.384688  0.105992
5  0.489517  0.009636
6  0.589956  0.000876

>>> df.s2.ewm(halflife=1.5, min_periods=3).mean()
0         NaN
1         NaN
2    2.182572
3    1.663174
4    0.979949
5    0.593155
6    0.364668
Name: s2, dtype: float64

pyspark.pandas.window.Expanding.quantile GroupBy