I’d like to share the way to pass variable between spark and pyspark interpreter in Apache Zeppelin.
You can Zeppelin-Context in Apache Zeppelin.
ZeppelinContext extends map and it’s shared between the Apache Spark and Python environments. So you can put some objects using Scala (in an Apache Spark cell) and read it from Python, and vice versa as shown below.
From spark to pyspark interpreter
Put object from scala.
%spark
val myObject = "hello world"
z.put("objName", myObject)
Get object from python.
%pyspark
myObject = z.get("objName")
print(myObject)
From pyspark to spark interpreter
Put object from python.
%pyspark
myObject = "hello world"
z.put("objName", myObject)
Get object from scala.
%spark
val myObject = z.get("objName")
println(myObject)