repartition {SparkR} | R Documentation |
The following options for repartition are possible:
"Option 1" Return a new SparkDataFrame partitioned by the given columns into 'numPartitions'.
"Option 2" Return a new SparkDataFrame that has exactly 'numPartitions'.
"Option 3" Return a new SparkDataFrame partitioned by the given column(s), using 'spark.sql.shuffle.partitions' as number of partitions.
## S4 method for signature 'SparkDataFrame' repartition(x, numPartitions = NULL, col = NULL, ...)
x |
A SparkDataFrame |
numPartitions |
The number of partitions to use. |
col |
The column by which the partitioning will be performed. |
Other SparkDataFrame functions: SparkDataFrame-class
,
[[
, agg
,
arrange
, as.data.frame
,
attach
, cache
,
collect
, colnames
,
coltypes
, columns
,
count
, dapply
,
describe
, dim
,
distinct
, dropDuplicates
,
dropna
, drop
,
dtypes
, except
,
explain
, filter
,
first
, group_by
,
head
, histogram
,
insertInto
, intersect
,
isLocal
, join
,
limit
, merge
,
mutate
, ncol
,
persist
, printSchema
,
registerTempTable
, rename
,
sample
, saveAsTable
,
selectExpr
, select
,
showDF
, show
,
str
, take
,
unionAll
, unpersist
,
withColumn
, write.df
,
write.jdbc
, write.json
,
write.parquet
, write.text
## Not run:
##D sc <- sparkR.init()
##D sqlContext <- sparkRSQL.init(sc)
##D path <- "path/to/file.json"
##D df <- read.json(sqlContext, path)
##D newDF <- repartition(df, 2L)
##D newDF <- repartition(df, numPartitions = 2L)
##D newDF <- repartition(df, col = df$"col1", df$"col2")
##D newDF <- repartition(df, 3L, col = df$"col1", df$"col2")
## End(Not run)