How to run a Zeppelin notebook paragraph programatically using API on Pyspark/Spark interpreter

Apache Zeppelin provides several REST APIs for interaction and remote activation of zeppelin functionality.

Apache Zeppelin Notebook REST API

You can run the paragraph asynchronously by given note and paragraph id. And you can send JSON input.

I’d like to show about how to run paragraph using Apache Zeppelin API on Pyspark and Spark interpreter.

On Pyspark interpreter

Requests library is HTTP library for Python. It’s very easy to use.

Requests: HTTP for Humans™

%pyspark
import requests

url = "http://localhost:8080/api/notebook/job/2FEKW395V/paragraph_1596267288078_-1477477049"

params = { \
  "params" : {   
    "name" : "Tom"
  }  
}

requests.post(url, json = params)

Paragraph ID : paragraph_1596267288078_-1477477049

%spark
var a = z.textbox("name")

This is output.

On Spark interpreter

The Apache HttpComponents™ project is responsible for creating and maintaining a toolset of low level Java components focused on HTTP and associated protocols.

This project functions under the Apache Software Foundation (http://www.apache.org), and is part of a larger community of developers and users.

Apache HttpComponents

POST request is used by org.apche.http.client class. Gson is a Java library that can be used to convert Java Objects into their JSON representation. 

%spark
import org.apache.http.impl.client.HttpClientBuilder
import org.apache.http.client.methods.HttpPost
import org.apache.http.util.EntityUtils
import org.apache.http.entity.StringEntity
import com.google.gson.Gson

var url = "http://localhost:8080/api/notebook/job/2FEKW395V/paragraph_1596267288078_-1477477049"

case class Name(name: String)
case class Params(params: Name)
var name = Name("Tom")
var pa = Params(name)

var gson = new Gson
var json = gson.toJson(pa)
println(json)


var client = HttpClientBuilder.create().build()
var request = new HttpPost(url)
var params = new StringEntity(json)

request.setHeader("Content-type", "application/json; charset=UTF-8")
request.setEntity(params)

var response = client.execute(request)

System.out.println("status = " + response.getStatusLine().getStatusCode());
System.out.println(EntityUtils.toString(response.getEntity()));

Paragraph ID : paragraph_1596267288078_-1477477049

%spark
var a = z.textbox("name")

This is output.

コメントを残す

メールアドレスが公開されることはありません。