How to pass variable between spark and pyspark interpreter in Apache Zeppelin

I’d like to share the way to pass variable between spark and pyspark interpreter in Apache Zeppelin.

You can Zeppelin-Context in Apache Zeppelin.
ZeppelinContext extends map and it’s shared between the Apache Spark and Python environments. So you can put some objects using Scala (in an Apache Spark cell) and read it from Python, and vice versa as shown below.

From spark to pyspark interpreter

Put object from scala.

%spark
val myObject = "hello world"
z.put("objName", myObject)

Get object from python.

%pyspark
myObject = z.get("objName")
print(myObject)

From pyspark to spark interpreter

Put object from python.

%pyspark
myObject = "hello world"
z.put("objName", myObject)

Get object from scala.

%spark
val myObject = z.get("objName")
println(myObject)

コメントを残す

メールアドレスが公開されることはありません。