Comprehensive Comparison of Continuous Integration Tools
Table of Contents
Continuous Integration has become a fundamental process in the software development lifecycle, and we’re all familiar with the popular CI tools in the market, but with over 50 Continuous Integration tools out there, how do you decide which one of them will work best for your organization?
Choosing which one to work with for the first time must surely feel like a daunting task: Open Source or proprietary? How Configurable is it? How much support and documentation are available? The questions go on and on. Some seasoned CI Server users will swear by Jenkins, others have deployed TFS and couldn’t be happier, but is it better than VSTS? And what about Travis – it’s supposed to be so easy to install! Or Atlassian – it integrates with Jira!
The unfortunate truth is that so far, no one tool has been able to check all the boxes. You’ll need to find the CI Tool that answers your organization’s specific needs. We’ve put together a list of the most popular CI tools with some of their pros and cons – to help you figure out which of their features make them the best fit for your organization.
#1: Jenkins
Java-based and open source – Jenkins is the one of the leading CI Servers out there. The project was forked from Hudson after a dispute between its principle creator Kohsuke Kawaguchi and Oracle, when Oracle bought Sun Microsystems. Jenkins then moved to CloudBees, a continuous delivery provider and strong supporter of open source software.
Jenkins is a cross-platform tool and it offers configuration both through GUI interface and console commands.
Pros: Being open source, Jenkins is extremely flexible thanks to a large developer community, a comprehensive plugin list and community support. Users also like the cross-platform build support.
Cons: The customization also has a downside: the quality and complexity of the plugins vary, and the overhead of testing them and their compatibility should be considered by new users.
#2: TeamCity
A Java-based CI server from JetBrains. It is commercial software and licensed under a proprietary license, but can be used for free if you use up to 20 build configurations and 3 build agents. Free license is granted to open-source projects as well.
Pros: Compatible with many build environments: supports many languages and tools for each of those languages. Another advantage that users point to is the easy installation process and intuitive interface.
Cons: Some users complain about TeamCity’s pricey rates. TeamCity’s support community is smaller, offering a smaller support base than Jenkins
#3: Bamboo
Atlassian’s CI Server. It supports builds in any programming language using any build tool, including Ant, Maven, make, and any command line tools. It offers both a cloud and a server version.
Pros: As an Atlassian solution, it integrates seamlessly with other Atlassian products, like JIRA, Confluence and Bitbucket. Some users also love the interface.
Cons: Some users complain there are not enough plug-ins for less-standard tasks, as well as citing the high price of the of the relatively few plugins that are offered.
#4: Travis CI
An open-source, hosted, distributed CI service used for GitHub projects. Initially designed for Rails apps, it now supports more than a dozen languages including PHP, JavaScript, C/C++, Objective-C, Python, and more. Configuration consists of simply adding a YAML to the root directory of your GitHub repository.
Pros: Considered by many to be a good CI server to start out with, because of the easy set-up as well as comprehensive documentation. One of the most popular hosted CI solutions.
Cons: Since support is limited to projects hosted on GitHub, and there is limited OSX and windows support, you need to consider your development environment before you make your choice.
#5: Microsoft TFS & VSTS
Team Foundation Server (TFS) provides source code management (either via Team Foundation Version Control or Git), and ALM capabilities. TFS can be used as a back end to numerous integrated development environments but is tailored for Microsoft Visual Studio and Eclipse (on Windows and non-Windows platforms).
Pros: Provides Support for free private repositories, easy to implement.
Cons: Some users found it challenging to get used to the interface. Others complained of slowness.
Visual Studio Team Services (VSTS) provides a set of cloud-powered collaboration tools that work with your existing IDE or editor.
Pros: Works with nearly every language and technology, includes project management tools like Kanban and scrum boards. Because it’s hosted on a cloud, it’s very easy to access from anywhere, and server maintenance is something users don’t need to worry about.
Cons: Does not support integration with SharePoint or Project Server.
A quick list of the pros and cons
