If you have a monorepo where a big engineering team is regularly merging changes, the stability of the main branch degrades considerably. This happens because more engineers working on the same codebase introduce incompatible changes, causing builds to break even though their commits pass independently. Here’s an example: https://blog.mergequeue.com/managing-github-merges-for-high-....
Github has a setting to restrict branches to be up to date before merging, but turning on that setting ends up forcing a rebase-athon within the team. This results in wasted time, and all too often, a stressful scramble to figure out what changes broke the build.
We had this problem in our previous company where we looked for a solution to automate the process. We found this paper [1] published by Uber to manage monorepos at scale and we built a lightweight version of that internally. It immediately eliminated the overhead of keeping builds healthy. After that, we decided to build a public version to save others from re-inventing the wheel.
We spoke to engineers at Airbnb, Stripe, Uber, Shopify, Quora and other large companies who have internally built similar tools, but teams who need such tools the most often don’t have the bandwidth to dedicate developers to building and maintaining them.
MergeQueue (MQ) is a FIFO queue of Pull Requests (PRs) that works as a Github app. To use MQ as a developer, instead of merging manually, you just add a Github Label “mergequeue” to the PR. MQ then takes care of the rest: it sequentially updates or rebases the branch, monitors the configured CI checks and eventually merges the changes. If the checks fail, it will dequeue the PR, add comments describing the reason and move to the next one. For high output teams, MQ also offers batch mode to run CI in parallel batches. If you’d like to learn more, there’s a lot more here: https://mergequeue.com/documentation.
Currently, we are also piloting a way to manage “flaky” (i.e. unreliable) tests through MQ. This integrates with your CI provider (we currently support CircleCI), analyses test results and flags the tests that fail inconsistently. When flaky tests are identified, MQ reruns the test depending on the configuration set.
We charge by usage in an organization, so for instance if your organization has 100 developers but only 20 of them use MQ, only those 20 will be billed. You can sign up for a free 14 day trial without a credit card. We also support single tenant or on-prem deployments, have webhooks to connect to your other apps, offer multi-queue support, and are SOC2 certified!
We’d love for you to try MergeQueue and give us any feedback you have. If you've used something similar in the past, we're also curious to learn what problems you faced so we're better prepared for them ourselves :)
[1] https://eng.uber.com/research/keeping-master-green-at-scale/ - discussed at the time: Keeping master green at scale - https://news.ycombinator.com/item?id=19692820 - April 2019 (115 comments)
by Andrey
Azimov