Reading the repo — stats and the shipped feed land in about ten seconds…
Apache Spark - A unified analytics engine for large-scale data processing
44.1k 29.4k 613 openLast shipped 1h ago
Apache Spark - A unified analytics engine for large-scale data processing
This week brought targeted fixes across streaming (global aggregation initialization, foreachBatch session handling, event-time preservation) and SQL execution (ASOF JOIN correctness, join sorting, DSv2 filter inference, and timestamp handling). Kubernetes support expanded with deployment pods…
Get this in your inbox every Monday →A deterministic 0–100 hygiene score — README, license, CI, tests, docs, and freshness.
Who ships this repo — author concentration and the bus factor across the last 300 mainline commits.
How welcoming this repo is to contributors — issue throughput, close time, responsiveness, and good-first-issue count.
What this project is built on — dependency count by ecosystem, the license mix, and anything worth a legal look before you adopt it.
Whether this project's CI can be trusted — pass rate, run times, flaky runs, and which workflow is the weak link.
Grounded in spark's README, structure, and recent commits — answers won't invent code they haven't seen.
A Monday email with what shipped, in plain English — no account needed.
Showing raw commit titles for the newest commits. Sign in to generate AI summaries.
[SPARK-60084][SQL] Fix PlanMerger losing a reused filter attribute before the aggregate
[MINOR][DOC] Fix broken HTML table in sql-error-conditions caused by placeholders inside code spans
[SPARK-59995][SQL] Drop sort orders that hold a partition transform from a V2 scan's output ordering
[MINOR][SQL] State the NULL STRUCT rule in the AsOfJoin.buildMatchExpressions Scaladoc
[SPARK-60012][CONNECT][SQL] Avoid copying Ivy caches when cloning ArtifactManager
[SPARK-59671][SQL] Validate a co-partitioned pair by its children's pairing
[SPARK-59962][SQL] Inline common expressions in a command instead of pre-evaluating them under it
[SPARK-33301][SQL] Split a large CASE WHEN into methods under whole-stage codegen
[SPARK-59919][SS] Fix OffsetSeqMetadataV2 missing shuffle partitions on rebinding
A floor, not a guess: counts only commits whose author, co-author trailer, or message explicitly credits an AI tool (Claude, Copilot, Cursor, aider, Codex…). Based on 30 mainline commits. Unattributed AI code isn't counted here — the full audit estimates that separately.
