Hello Sail
This example's Spark SQL and DataFrame code runs unchanged on Sail. It starts a Sail server inside the Python process, connects a normal PySpark session to it over Spark Connect, and runs a SQL aggregation and a window function over generated data.
Run it
Run all commands below from the repository root (sail-examples/):
uv run quickstart/hello-sail/main.py
uv installs pysail and the PySpark client from the header of quickstart/hello-sail/main.py. Java is not needed.
Output
The newest capture appears first. Older output stays here when a result changes.
Current output · Sail 0.7.3 (revision 2)
Captured on Sail 0.7.3. Latest capture: 2026-11-07, Python 3.14.10.
+-----------+------+------+--------+----------+
|customer_id|orders| spend|last_day|spend_rank|
+-----------+------+------+--------+----------+
| 71| 40|2496.0| 22| 1|
| 142| 40|2492.0| 23| 2|
| 213| 40|2488.0| 24| 3|
| 34| 40|2484.0| 25| 4|
| 105| 40|2480.0| 26| 5|
| 176| 40|2476.0| 27| 6|
| 247| 40|2472.0| 28| 7|
| 68| 40|2468.0| 29| 8|
| 139| 40|2464.0| 30| 9|
| 210| 40|2460.0| 21| 10|
+-----------+------+------+--------+----------+
Earlier output · Sail 0.7.3 (revision 1)
Captured on Sail 0.7.3. Latest capture: 2026-11-05, Python 3.12.14.
+-----------+------+------------------+--------+----------+
|customer_id|orders| spend|last_day|spend_rank|
+-----------+------+------------------+--------+----------+
| 71| 40| 2496.0| 22| 1|
| 142| 40| 2492.0| 23| 2|
| 213| 40| 2488.0| 24| 3|
| 34| 40|2484.0000000000005| 25| 4|
| 105| 40| 2480.0| 26| 5|
| 176| 40| 2476.0| 27| 6|
| 247| 40| 2472.0| 28| 7|
| 68| 40| 2468.0| 29| 8|
| 139| 40| 2464.0| 30| 9|
| 210| 40| 2460.0| 21| 10|
+-----------+------+------------------+--------+----------+
Python code
main.py53 lines
# /// script
# requires-python = ">=3.12"
# dependencies = [
# "pysail==0.7.3",
# "pyspark-client==4.1.3",
# "pyarrow==26.0.0",
# "pandas==3.0.6",
# ]
# ///
"""Run Spark SQL and DataFrame code on Sail: no Java, no cluster, no setup."""
from pyspark.sql import DataFrame, SparkSession, Window
from pyspark.sql import functions as F
def run(spark: SparkSession) -> DataFrame:
# 10,000 synthetic orders, generated in the engine so the example needs no data files.
orders = spark.range(10_000).select(
(F.col("id") % 250).alias("customer_id"),
(F.col("id") % 30 + 1).alias("day"),
((F.col("id") * 7919) % 1000 / 10.0).alias("amount"),
)
orders.createOrReplaceTempView("orders")
# Plain Spark SQL.
per_customer = spark.sql(
"""
SELECT customer_id,
count(*) AS orders,
round(sum(amount), 2) AS spend,
max(day) AS last_day
FROM orders
GROUP BY customer_id
"""
)
# The DataFrame API and a window function, on the same data.
ranked = per_customer.withColumn(
"spend_rank", F.dense_rank().over(Window.orderBy(F.desc("spend"), "customer_id"))
)
return ranked.filter(F.col("spend_rank") <= 10).orderBy("spend_rank")
if __name__ == "__main__":
from pysail.spark import SparkConnectServer
server = SparkConnectServer()
server.start()
_, port = server.listening_address
spark = SparkSession.builder.remote(f"sc://localhost:{port}").getOrCreate()
run(spark).show()
spark.stop()
server.stop()
How it works
SparkConnectServer()starts Sail in the current process, on a free port.SparkSession.builder.remote(...)connects the standard PySpark client to that server.run(spark)is ordinary PySpark: the same function runs on Apache Spark, and this repository's tests check that both engines return the same rows.
To use a Sail server that is already running, skip the first two steps and connect with SparkSession.builder.remote("sc://localhost:50051").