Sustaining Apache Arrow and Parquet: The Arctos Alliance with Alex Scammon

October 20th, 2026

48 mins 36 secs

Transcript

Link with Timestamp

Download MP3 (89 MB)

Your Host
Special Guest
Tags
open source oss arctos alliance apache arrow apache parquet open source sustainability open source funding alex scammon jay faulkner open source nonprofit ai pull requests artificial intelligence open source maintainers open source community columnar data format data engineering delta lake apache yunikorn etcd polars apache spark gr-oss out podcast tech podcast g-research

About this Episode

What happens when a critical open source software (OSS) community faces losing many of its core maintainers? In this reprise episode, host Jay Faulkner talks with Alex Scammon, co-CEO of G-Research Open Source Software, about how the Apache Arrow and Apache Parquet community responded, and why G-Research helped create the Arctos Alliance, a community-funded nonprofit that employs engineers to support the ecosystem. They also dig into how artificial intelligence (AI) generated pull requests (PRs) are changing trust between contributors and maintainers.

Guest
Alex Scammon, Co-CEO, G-Research Open Source Software (GR-OSS)

Host
Jay Faulkner, GR-OSS Out Podcast

Topics discussed

  • Why Apache Arrow (in-memory columnar data format) and Apache Parquet (on-disk columnar file format) underpin modern data science, machine learning (ML), and AI tooling
  • Why the Arctos Alliance is a community effort rather than one company’s project
  • How a small independent team can keep a project’s direction from being shaped by a single company
  • Halving Apache Arrow’s open issue count, from roughly 4,600 to about 2,200, and why issue tracker health matters to new contributors
  • Work in progress on tensor types and encryption
  • How AI-generated PRs are affecting maintainers, and why in-person community building matters more now, including a Delta Lake example
  • A candid story about contributions reverted in Apache YuniKorn, and lessons on communication in open source communities
  • Other projects G-Research has helped sustain: etcd, Apache Spark, Polars, and more

Links mentioned

G-Research is hiring in Dallas, TX and London, UK! Apply at https://gresearch.com/vacancies.

For a video version of this podcast, check out https://youtu.be/4sCJnYAfkXA.

The GR-OSS OUT Podcast is produced by Ben Wiley.