Who Finishes the Job? A Study of Follow-Up Fixes and Commit Authorship on AI Coding Agent Pull Requests

The authors contribute a large-scale empirical study of follow-up-fix authorship after agent PR merges, human validation and an LLM judge for candidate fixes, an analysis of merge-time signals, and a released replication package.

Source: [11]

The authors recommend monitoring agent PRs beyond merge, noting that aggregate merge-time signals offer little separation between PRs that later receive fixes and those that do not; they also identify future research on monitoring frameworks and on how agent/model ownership affects code after a model is discontinued.

Source: [17], [26]

The agent cohort consists of 6,774 merged PRs from five agents in 891 repositories in AIDev-pop; the human cohort consists of merged PRs from AIDev’s curated human-PR table, restricted to the same repositories and merge-only conditions.

Source: [3]

The authors report that, among 4,505 agent merges with a fully observed 30-day window, 22.9% had a candidate fix and 4.5% had a verified fix.

Source: [20]

The authors report that half of the 30-day verified-fix incidence accumulated within the first week in both cohorts, and that the cumulative verified-fix incidence reached 4.5% for agent PRs and 2.6% for human PRs by day 30.

Source: [1]

Among 263 verified fixes on agent merges, 69.6% were opened by the same agent, 27.4% by a human, and 3.0% by another agent; among 110 verified fixes on human merges, 89.1% were opened by humans.

Source: [24]

For verified-fix PRs opened by agents other than Codex, commits between first push and merge were on average 87.4% agent-authored per PR, and 76.4% of those PRs had all commits classified as agent-authored; the authors exclude Codex from the commit measure because its commits lack agent markers.

Source: [13], [18]

In pooled comparisons, merge-time signal differences between agent PRs with and without a follow-up fix were negligible to small; in within-repository comparisons, a PR with ten times as many commits had 6.1 times the odds of a verified follow-up fix (p<0.001).

Source: [14], [21]

The authors scope their findings to five AIDev agents in repositories with at least 500 stars; other agents, less-active repositories, and behavior after the data window are outside the stated population.

Source: [8]

The authors state that commit-author classifications measure who committed or marked a change rather than who wrote it; because Codex commits carry no marker, they exclude Codex from commit decomposition and treat reported agent-authored shares as lower bounds.

Source: [6]

The study workflow extracts candidate follow-up-fix pairs, validates candidates with human annotation and an LLM judge, classifies fix-PR authorship, classifies commit authorship, and compares merge-time signals for agent PRs with and without verified fixes.

Source: [25]

The authors re-fetched commits, diffs, reviews, and comments for human PRs through the GitHub REST API because the original AIDev human-PR table contained PR-level columns only.

Source: [3]

The within-repository analysis used conditional logistic regression with repository controls and controlled for the number of non-boilerplate files shipped; the reported odds ratio compares merges ten times apart on a signal. The within-repository associations do not establish that a signal causes a later fix.

Source: [10], [12]

For the cross-cohort comparison, the authors use a shared observation-window cutoff of June 28, 2025; the comparison includes 2,012 agent PRs and 4,063 human PRs.

Source: [4], [19]

On binary Direct-fix versus non-fix labels, human annotators agreed on 45 of 50 sample pairs (Cohen’s κ=0.77); the LLM judge achieved κ=0.78 against human relabeling, and its Direct-fix precision was 90% on both cohorts (pooled 54/60, Wilson 95% CI 80–95%).

Source: [16], [23]

AIDev-pop applies a popular-repository threshold of more than 500 GitHub stars as of June 22, 2025; the agent PRs span December 24, 2024 to July 30, 2025, and the human PRs span January 1 to June 28, 2025.

Source: [3]