Upgrading from PySpark 1.0-1.2 to 1.3ΒΆ

  • When using DataTypes in Python you will need to construct them (i.e. StringType()) instead of referencing a singleton.

Upgrading from PySpark 1.4 to 1.5 Migrating from Koalas to pandas API on Spark