SparkContext Task
In Spark’s architecture, the SparkContext object connects your driver program to a Spark cluster, manages jobs, and hands out tasks to worker nodes.
It’s the primary object in Spark Core that allows your application to talk to the Spark cluster.
SparkContext Task
Connecting to the cluster manager (Standalone, YARN, Mesos, Kubernetes)
Coordinating executors and task scheduling
Creating RDDs from data sources or collections
Managing configuration and job execution
Providing access to services like the Hadoop FileSystem, broadcast variables, and accumulators.
Top comments (0)