How to spark-submit to a remote Spark cluster via Intellij?
Architecture
- frontal node (accessible with a public IP)
- Spark cluster (only accessible from the frontal node)
Prepare
- The local Spark program setup
- A spark cluster deployed in the cluster
- Intellij with the plugin Big Data Tools
Chenghao Lyu
Ph.D. Candidate
My research interests include big data analytics systems, machine learning and multi-objective optimizations.