JVM options not taken into consideration, spark-submit of java program -
using spark-submit i'm launching java program. i'm setting java arguments jvm not taken account. i'm trying specify max/min heap free ratio.
however, though arguments present (based on visualvm), memory graph shows don't affect running program.
here command i'm launching:
"/usr/jdk1.8.0_131/bin/java" "-xmx4096m" "-dspark.driver.extrajavaoptions=-xx:minheapfreeratio=30 -xx:maxheapfreeratio=30 -agentlib:jdwp=transport=dt_socket,server=y,suspend=n,address=5005" "-dspark.executor.extrajavaoptions=-xx:minheapfreeratio=30 -xx:maxheapfreeratio=30" "-xx:minheapfreeratio=30" "-xx:maxheapfreeratio=30"
as can see, i've tried setting arguments on driver, executor , java command.
to clear, i'm trying improve "unused" heap size.
when starting program these arguments (outside of spark), behaves correctly.
Comments
Post a Comment