pyspark.sql.DataFrameWriter.jdbc

DataFrameWriter.jdbc(url: str, table: str, mode: Optional[str] = None, properties: Optional[Dict[str, str]] = None) → None[source]

Saves the content of the DataFrame to an external database table via JDBC.

New in version 1.4.0.

Changed in version 3.4.0: Supports Spark Connect.

Parameters
tablestr

Name of the table in the external database.

modestr, optional

specifies the behavior of the save operation when data already exists.

  • append: Append contents of this DataFrame to existing data.

  • overwrite: Overwrite existing data.

  • ignore: Silently ignore this operation if data already exists.

  • error or errorifexists (default case): Throw an exception if data already exists.

propertiesdict

a dictionary of JDBC database connection arguments. Normally at least properties “user” and “password” with their corresponding values. For example { ‘user’ : ‘SYSTEM’, ‘password’ : ‘mypassword’ }

Other Parameters
Extra options

For the extra options, refer to Data Source Option for the version you use.

Notes

Don’t create too many partitions in parallel on a large cluster; otherwise Spark might crash your external database systems.

pyspark.sql.DataFrameWriter.insertInto pyspark.sql.DataFrameWriter.json