Development with Apache Spark: Scala vs. PySpark

Spoiler alert: neither one wins. The contest was fierce, though - with many disciplines: ease of development, cost of maintenance, deployment, even popularity and reputation. Oh, and performance - it wouldn't count without talking about performance, would it?

• Allegro Tech Meeting 2021 (slajdy - PDF, PL, 2.4 MiB)
• PyWaw #100 (slides - PDF, EN, 2.1 MiB)
• PyCon PL 2022 (slides - PDF, EN, 2.1 MiB)

• Scala Spark vs Python PySpark: Which is better?
• PySpark and the JVM - introduction, part 1
• How does PySpark work? — step by step (with pictures)
• Stop using Pandas and start using Spark with Scala
• Spark performance for Scala vs Python
• Unit Testing with PySpark
• Spark by {Examples}