ClassNotFoundException is thrown when submitting a Spark job with --conf spark.executor.extraClassPath on Kubernetes

Description

ClassNotFoundException is thrown when submitting a Spark job when setting "spark.executor.extraClassPath" on Kubernetes.
E.g.
The below command will result a ClassNotFoundException in the executors when running on Kubernetes:
insightedge-submit ... --conf spark.executor.extraClassPath=<somepath>

 

As a workaround, you can add the following line to your Dockerfile:

ENV SPARK_EXTRA_CLASSPATH "${SPARK_EXTRA_CLASSPATH}:${INSIGHTEDGE_CLASSPATH}"

 

Workaround

None

Acceptance Test

N/A

Assignee

Yohana Khoury

Reporter

Yohana Khoury

Labels

Priority

Medium

SalesForce Case ID

None

Fix versions

Commitment Version/s

None

Due date

None

Product

InsightEdge

Edition

Enterprise

Platform

Java
Configure