API Reference

This page lists an overview of all public PySpark modules, classes, functions and methods.

• Spark SQL
  • Core Classes
  • Spark Session APIs
  • Configuration
  • Input and Output
  • DataFrame APIs
  • Column APIs
  • Data Types
  • Row
  • Functions
  • Window
  • Grouping
• Structured Streaming
  • Core Classes
  • Input and Output
  • Query Management
• MLlib (DataFrame-based)
  • Pipeline APIs
  • Parameters
  • Feature
  • Classification
  • Clustering
  • Functions
  • Vector and Matrix
  • Recommendation
  • Regression
  • Statistics
  • Tuning
  • Evaluation
  • Frequency Pattern Mining
  • Image
  • Utilities
• Spark Streaming
  • Core Classes
  • Streaming Management
  • Input and Output
  • Transformations and Actions
  • Kinesis
• MLlib (RDD-based)
  • Classification
  • Clustering
  • Evaluation
  • Feature
  • Frequency Pattern Mining
  • Vector and Matrix
  • Random
  • Recommendation
  • Regression
  • Statistics
  • Tree
  • Utilities
• Spark Core
  • Public Classes
  • Spark Context APIs
  • RDD APIs
  • Broadcast and Accumulator
  • Management
• Resource Management
  • Core Classes

Python Package Management Spark SQL