Effect of Pull Request Submission by listing individual OSS contributors on GitHub Sponsors

Key Points

The submission of pull requests experienced a temporary short-term increase, but eventually returned to pre-treatment levels, with a crowding-out effect emerging over time. Among contributors with a high level of community engagement (e.g., frequent contributions to others’ projects, large follower count), a negative impact on knowledge creation activities (i.e., the crowding-out effect) was observed. For contributors who set higher active funding goals, sponsorship appeared to have a positive effect on their knowledge creation activities.

Background

The world’s digital infrastructure is built on open-source software (OSS) developed on platforms like GitHub (GH), and supported by complementary knowledge-sharing platforms like Stack Overflow (SO). On these platforms, numerous programmers, maintainers, and researchers devote their efforts to building and maintaining code, as well as supporting users. Contributors play a crucial role not only in knowledge creation activities, such as contributing code and adding new features, but also in knowledge maintenance activities, such as reviewing and integrating community-submitted code.

However, it has been pointed out that OSS projects are increasingly burdened by the need to maintain existing code rather than create new code, and that many projects fail due to the lack of intrinsic motivation for maintenance work. Maintenance is often seen as “mundane but necessary,” and tends to lack intrinsic appeal. For example, the 2020 Linux Foundation OSS Contributor Survey highlighted that, although participants showed minimal interest in spending significant time on maintenance-related tasks, they emphasized that the most crucial support OSS projects need is for maintenance work—especially related to security improvements.

In this context, there is a growing trend to provide financial support to encourage and sustain volunteer contributors. In May 2019, GH launched GitHub Sponsors, a feature allowing individual OSS developers to receive donations from the community. Unlike traditional project- or activity-specific funding, GitHub Sponsors focuses on individual contributors, offering them the flexibility to reallocate their efforts not only within the host platform but also across other complementary platforms in which they participate. This study investigates the impact of such sponsorship-based funding on OSS contributors’ behavior—specifically, its effect on knowledge creation and maintenance activities on GH and the spillover effects on SO, a complementary knowledge-sharing platform.

Analytical Methods

Dataset

• Data Collection: The dataset is based on the behavior of contributors active on both GitHub (GH) and Stack Overflow (SO), obtained via APIs provided by both platforms. The panel data spans four years, from January 2018 to December 2021.
  • During the initial data collection period, GH Sponsors was available in 30 countries that supported STRIPE payments. The sample is therefore limited to GH contributors from these countries.
  • To select GH contributors, the following criteria were applied: owning at least 5 repositories, having more than 10 followers, and having joined GH before 2018. This resulted in a sample of 20,841 GH contributors.
  • SO profiles were identified using names, email addresses, and login IDs listed on GH, and only included if a GH URL matched on the SO profile. Ultimately, 5,910 GH contributors who had joined both platforms prior to the observation period were identified, of whom 1,467 were listed for sponsorship during the observation period.
  • Activity-related data were aggregated on a monthly basis.

Intervation / Explanatory Variable

• The intervention in this study is the contributor being listed on GitHub Sponsors (Sponsor listing).
• The primary explanatory variable of interest in this study is a binary indicator that equals 1 starting from the month the contributor is listed for sponsorship (Sponsor Listed).
• This variable is used to capture the effect starting from the point at which the contributor is listed on GitHub Sponsors.

Dependent Variables

• Knowledge Creation Activity on GitHub:
  • Number of Pull Requests (PRs) Submitted per month.
    • Proposals for changes to the codebase, including new features, bug fixes, and improvements.

Identification Strategy

• A Difference-in-Differences (DID) estimation was used to measure changes in pull request submission due to sponsorship.
• Coarsened Exact Matching (CEM) was applied to match contributors based on their activity levels prior to being listed for sponsorship.

Robustness Test

• The following supplemental analyses on PR submissions were conducted:
  • Use of alternative dependent variables for PR submissions.
  • Controlling for PR characteristics (e.g., size, number of files, number of commits, time to merge).
  • Considering self-merged PRs.
  • Considering PRs related to issues.

Results

• No significant overall effect of sponsorship on knowledge creation activity (PR submissions) on GitHub.。

  • A temporary increase is observed in the short term, but a crowding-out effect appears over time.
  • In the long term, the effect on PR submissions disappears due to a decline in intrinsic motivation.

• For contributors with high community engagement or those who set higher funding goals, a stronger negative effect (reduction in PR submissions) was observed.

• There is a tendency to see an increase in PR submissions to less popular projects.