Report:

Magic Quadrant for DevOps Platforms

Top Vendor Products for DevOps Platforms in 2024

• Atlassian
• GitLab
• Harness
• Microsoft

DevOps Platforms Vendor Product Comparisons 2024

• Atlassian - Leader
  

  Pros:

  • Work management, service management and collaboration: Atlassian has effectively integrated Jira, Jira Service Management and Confluence to provide a platform that enables DevOps teams to collaboratively manage their work. Atlassian's customers find that these tools become a core part of their way of working. Atlassian's knowledge management tool, Confluence, has little competition and is widely used in software engineering organizations.
  • Platform ecosystem: The Atlassian Marketplace offers thousands of third-party applications and integrations that extend the functionality of the Atlassian platform. Atlassian has also built a worldwide network of partners that offer strong implementation, consulting and support capabilities.
  • Effective innovation: Atlassian has developed innovative new features, such as integrated DevSecOps planning, that build on its platform's strong agile work management capabilities. The platform uses a common data model to support innovation in areas like AI and analytics.

  Cons:

  • Sales model: Atlassian primarily sells its DevOps platform directly to customers via a fixed-price model. In Gartner client inquiries, some Atlassian customers have expressed frustration with this model. Atlassian's partner network provides a higher-touch sales approach and may offer discounts, and Atlassian offers enterprise customers the option of an enterprise license agreement and discounts on a case-by-case basis.
  • Lower adoption of CI/CD: Atlassian Bitbucket, which provides source code management and CI/CD, has on-par capabilities but low customer adoption relative to other Atlassian products. Customers will need integrations with external tools to get functionality such as feature flag management.
  • Limited regulatory compliance for cloud products: Atlassian has not yet obtained important regulatory compliance, such as FedRAMP, for its cloud products. At the same time, it ended support for its on-premises Server products in early 2024. As a result, customers in regulated industries may find that their only options are the on-premises Data Center products, which are more expensive than Server's.


• GitLab - Leader
  

  Pros:

  • Built-in security capabilities: GitLab continues to expand its security capabilities and has recently strengthened them via acquisitions such as Oxeye. This contributes to its Leader status in this Magic Quadrant and its improving position in our 2023 Magic Quadrant for Application Security Testing.
  • Market understanding: GitLab provides capabilities across the life cycle of software development and throughout the many roles and responsibilities that developers will operate. It provides a variety of tooling — from task management and planning, through to creation with code repositories, cloud development environments, securing of solutions and deployment. GitLab's mission of providing capabilities across these areas increases its value to enterprise developers.
  • Customer satisfaction: GitLab has one of the highest Net Promoter Scores on Gartner's Peer Insights for DevOps Platforms. At the time of evaluation, 92% of reviewers on Peer Insights would recommend GitLab. Specific areas of customer appreciation include the all-in-one platform and support of the DevSecOps approach.

  Cons:

  • Sales strategy: GitLab offers less aggressive discounts for enterprise customers than competitors within this research, especially for its premium product. This will impact clients that are negotiating for advantageous pricing.
  • One bundle for all: GitLab sells its DevOps platform as a single product, which may not be ideal for customers that only want to use a subset of its features. Customers cannot purchase specific capabilities individually. Recently, this has seen some change, with a new ability to add extra Enterprise Agile Planning seats to existing GitLab Ultimate licenses.
  • Sparse documentation areas: GitLab has rapidly expanded the functionality of its platform by integrating open-source software (OSS), but these capabilities vary in quality. Some of its new features provide no more functionality than what the core OSS provides, and some lack sufficient documentation. Also, GitLab has introduced some new OSS features into beta/early access without a clear timeline for general availability.


• Harness - Leader
  

  Pros:

  • Strong cloud-native capabilities: Harness provides platform engineering teams with a wide range of capabilities to support cloud-native DevOps. Recent acquisitions by Harness, such as Armory, have enabled the platform's capabilities to provide a more comprehensive solution that supports the entire product delivery life cycle.
  • Market responsiveness: Harness continues to effectively respond to market demands. It has added more DevOps-specific capabilities in the past year than any other vendor in this Magic Quadrant. For example: the release of Harness Code Repository, which removes the dependency on other tools; IaCM, for scaling the usage of Terraform with security and control; and Harness IDP, to better support platform engineering teams.
  • Feature management: The Harness platform natively supports feature management, which could displace a dedicated third-party feature management tool. It enables product teams to adopt progressive delivery by separating release from deployment, providing feature monitoring and reporting, and supporting the identification and removal of flags that are no longer needed.

  Cons:

  • Customer satisfaction: On Gartner Peer Insights, Harness' scores for integration and employment, service and support, and product capability lag those of other vendors within this research. Customer dislikes include lack of documentation and customer support.
  • Limited support for data residency: Harness has not yet expanded its cloud-hosting options beyond the U.S. Non-U.S. customers utilizing the Harness SaaS platform will struggle to meet their data residency compliance requirements.
  • Learning curve: As Harness continues to increase its support for the entire DevSecOps life cycle with new acquisitions, users have found it difficult to keep up with all of the new capabilities. Users have cited challenges with the lack of intuitive UI and the lack of detailed guidance on best practices or standards for using the platform.


• Microsoft - Leader
  

  Pros:

  • Strong developer community: Microsoft has a large developer presence and, at the time of evaluation, Gartner Peer Insights showed that 96% of reviewers would recommend GitHub — the highest percentage among all platforms in this Magic Quadrant. GitHub is the largest collection of OSS projects. Its strong developer community provides Microsoft with strong input on user needs, and it combines this with effective programs to gain feedback and test new capabilities.
  • Generative AI mind share: Microsoft has focused on delivering numerous innovative capabilities via GitHub Copilot to enable the use of GenAI in DevOps workflows. Interest in GitHub Copilot is often cited as a driver to shifting platform investments to GitHub.
  • Collaborative focus: GitHub incorporates many features that emphasize its social coding ethos. These include: Discussion for forum-like discussions; Actions, which enable CI and other workflows to be automated; strong support for code reviews and understanding dependencies; and the ability to carry out many actions from a mobile device.

  Cons:

  • Market confusion: GitHub and Azure DevOps are two independent platforms with duplicate features in some areas and unique parts in others. It is clear that GitHub is where innovation is driven, but for the large base of users on Azure DevOps, some of the messaging around the platform is unclear. The Azure DevOps roadmap shows a focus on feature integrations with GitHub.
  • Complexity: GitHub does not offer ease of use of other CI tools that have visual pipeline construction, thus demanding a longer learning curve if users are not familiar with YAML. While there are many integrations, setup and management of these integrations can be complex.
  • Developer and team focus: While GitHub is used by some very large teams and focused on the developer experience, it lacks the controls and capabilities sought by engineering and portfolio leaders, such as support for value stream metrics or enterprise agile. Customers may use Azure Boards in conjunction with GitHub to gain this functionality.


• Amazon Web Services - Visionary
  

  Pros:

  • Integration with AWS cloud services: Amazon continues to lead in market share for IaaS and PaaS, and it offers tools that keep users up to date with the latest capabilities. However, its approach has been somewhat fragmented. CodeCatalyst addresses this by providing a unified platform that integrates these capabilities, offering a streamlined path for effective building and deployment to AWS cloud services.
  • Pricing advantage: CodeCatalyst brings together a complete platform — from planning to delivery — and does it at a significantly lower price relative to other DevOps platforms in this Magic Quadrant, because the outcome will be users building AWS applications that drive the bottom line of AWS services.
  • Secure development environment: AWS has responded to customer demands for increased security and privacy. In the last year, this has meant the addition of single sign-on support and the use of Amazon Virtual Private Cloud to create isolated development and test environments. AWS PrivateLink enables a private, secure connection between AWS CodeBuild, Amazon Elastic Cloud Compute and on-premises applications. This capability enables customers to restrict all network traffic to the managed instance without the need for other access gateway technologies.

  Cons:

  • Still maturing: Amazon CodeCatalyst is just over a year old and, while it brings a broad set of functionality together, it still misses enterprise features such as support for DORA and other DevOps metrics. Also, there is no integrated wiki and just a few third-party CodeCatalyst workflows (though GitHub Actions can be used).
  • Limited third-party integrations: Amazon CodeCatalyst offers extensions to third-party developer tools and services, but few vendors are integrating with CodeCatalyst and other Amazon offerings at this point. This has historically been an issue with the Amazon DevOps components such as AWS CodePipeline.
  • Limited support for built-in application security capabilities: Amazon CodeCatalyst continues to lag the market in support for code and application security.


• Buildkite - Niche Player
  

  Pros:

  • Scalability and performance: Buildkite's strategy is to enable larger enterprises to run an unlimited number of build agents in parallel, reducing cycle times for building and testing software, and increasing developer efficiency. Some large Buildkite customers use more than 130,000 concurrent pipelines.
  • Intelligent workflows: Buildkite's platform embeds intelligent workflows into multiple aspects to tailor CI workflows to the needs of the product. This gives pipeline customers the flexibility to execute different steps during the build process based on runtime conditions, such as the branch, the environment or the results of previous steps.
  • Test optimization: Buildkite's test analytics tool enables customers to monitor the performance and reliability of their test suite. It provides tracking, reporting and analysis of the suite's efficiency to help software engineering teams identify long-running tests and flaky tests, and insights that enable teams to reorganize test suites to improve test cycle times.

  Cons:

  • Narrow focus: The Buildkite platform only provides solutions for parts of the CI/CD process. Deploying a comprehensive CI/CD pipeline will require integration with additional tools or DevOps platforms to fill the gaps not provided by the Buildkite platform.
  • Limited geographic support for cloud data centers: Buildkite is a SaaS platform that is hosted and serviced from cloud data centers in the U.S. Customers outside the U.S. cannot host Buildkite at a cloud data center in their region. Such customers should ensure that Buildkite's mechanisms for meeting data residency requirements are acceptable. Also, Buildkite does not offer the option to configure a backup or disaster recovery data center.
  • New hosted offering: Since the hosted offering is new, potential users should watch for feedback on performance and balance the ease of fully hosted versus the hybrid user-managed infrastructure, which offers control but requires significant operational overhead.


• CircleCI - Challenger
  

  Pros:

  • FedRAMP and SOC 2 Type II compliance: CircleCI differentiates itself from most of its competitors by providing the only FedRAMP-approved and SOC 2 Type II certification for its platform. CircleCI is an attractive option for customers in highly regulated industries.
  • Customer satisfaction: CircleCI scores a 4.7 on Gartner Peer Insights for its DevOps platform. Its evaluation, contracting and product capabilities are strengths, with scores reaching into the 4.8 range. Peer Insights ratings highlight the reliability of the platform and strong integration with other common DevOps tools.
  • Ease of use: On Gartner Peer Insights, CircleCI customers report high satisfaction with the platform's usability, including the ease of deployment, administration, configuration and debugging support. At the time of evaluation, 86% of Peer Insights reviewers would recommend using CircleCI. The platform has also effectively integrated AI by providing recommendations within the developer flow and enhancing developer experience within the IDE to help developers debug pipeline issues.

  Cons:

  • Narrow focus: The CircleCI platform provides solutions focused on improving developer experiences, such as build and test execution. It is not intended as a core tool for the Ops stage of the DevOps life cycle. Deploying a comprehensive CI/CD solution requires integration with other tools or DevOps platforms to implement capabilities not provided by the CircleCI platform.
  • Geographic strategy: Potential customers will have to navigate CircleCI's limited data residency support. CircleCI has no data center presence outside the U.S. and Canada. Customer options for cloud regions are limited to the U.S.
  • Pricing model: While CircleCI offers a free service, customers note that pricing can scale quickly as usage increases. Since the pricing model is based on credit consumption, referring to the full pricing list is advised as dependency on the platform increases.


• CloudBees - Challenger
  

  Pros:

  • Continuous compliance: CloudBees continues to excel at supporting customers in highly regulated industries. CloudBees Compliance provides automation to help customers comply with security and compliance standards (such as CIS, NIST, FedRAMP, PCI-DSS, HIPAA, CSA) and minimize exposure to security vulnerabilities.
  • Feature management: CloudBees remains one of few DevOps platforms to natively support feature management, with capabilities that match dedicated third-party feature management tools. We continue to see increased use of experimentation by DevOps platform users and the value of integration comes both from simplification and compliance, protecting customer data with strong regional support.
  • Customer engagement: CloudBees is strong from an operations perspective and in customer engagement. This comes from a long history of Jenkins at scale, CloudBees CI, CloudBees University and supporting a broad portfolio of enterprise clients. Customer reviews on Gartner Peer Insights support this statement.

  Cons:

  • Narrow focus: The CloudBees platform provides solutions focused on improving developer experiences by streamlining build and test execution and release orchestration. The platform relies on integrations with other tools or platforms to support the full DevOps life cycle needs, such as version control, planning, development and testing, and artifact management. Integration with other tooling is required for customers to achieve comprehensive DevOps platform capabilities.
  • Customer satisfaction: CloudBees' overall Peer Insights score — 4.3 out of 5 — is among the lowest of vendors within this research. Customers expressed dissatisfaction with such things as documentation and Kubernetes support.
  • Legacy perception: Users perceive Jenkins as a legacy CI platform and Gartner finds CloudBees' overall marketing strategy a weak point. When modernizing their developer tooling, organizations are seeking to reduce the level of effort to create and manage CI/CD for new cloud-native targets and start by looking elsewhere. This is driven not only by perception, but also the attraction of new platforms that provide a broader suite.


• JetBrains - Niche Player
  

  Pros:

  • Secure platform: JetBrains demonstrates a very strong approach to ensuring the security of its platform, including a comprehensive "red team" strategy. JetBrains maintained a record of zero security breaches throughout the past year.
  • Collaboration features: JetBrains' DevOps platform remains strong in its support of software engineering team collaboration and communication, which respondents to Gartner's Software Engineering Survey for 2024 tell us is the most valuable type of tool for increasing developer productivity.
  • Strong mind share: JetBrains has a strong presence in the IDE market and was one of the earliest vendors to offer a CI tool, which established its position in that market as well. This has given JetBrains a strong mind share among developers.

  Cons:

  • Limited innovation: JetBrains has delivered few innovative features for its DevOps tools due to its focus on enhancing its IDE products. JetBrains has not made any strategic acquisitions to gain innovative capabilities.
  • Questionable sales execution and pricing: JetBrains provided little evidence of effective direct sales execution and pricing compared with other vendors in this Magic Quadrant. Customers may choose to engage with the vendor's partners to negotiate and build a relationship.
  • Limited marketing strategy: JetBrains demonstrated a less sophisticated marketing strategy relative to its competitors in this Magic Quadrant. This may impact its ability to influence the market to align with its vision.


• JFrog - Challenger
  

  Pros:

  • Artifact management mind share: Artifactory continues to be the product for which JFrog is best known. It is a widely used and strong tool for managing and distributing binary artifacts. JFrog continues to enhance its capabilities (for example, by adding ML model management) and leverage its popularity to expand its DevOps platform.
  • Market responsiveness to security demand: With JFrog Xray, JFrog Curation and JFrog Advanced Security, the company has responded effectively to customer demand for greater security and auditability of software supply chains. JFrog has also responded to customer demand for simpler, consolidated toolchains with JFrog Pipelines.
  • Customer experience and support: JFrog demonstrates a high level of attention to customer satisfaction, service levels and technical support relative to other vendors in this Magic Quadrant, such as automatic director-level escalation of all customer support concerns.

  Cons:

  • Limited native CI/CD product capabilities: Given the company's focus on software supply chain management, JFrog Pipelines' CI/CD capabilities are not as strong as some of its competitors' offerings in this regard. All JFrog customers use JFrog Artifactory, but the majority of those customers choose to integrate third-party CI/CD solutions instead of using JFrog Pipelines.
  • Limited capabilities for collaboration and planning: JFrog lags competitors in areas such as collaboration and planning tools, choosing instead to focus its innovations on the capabilities of its Artifactory product, such as ML model management and advances in security.
  • Limited partner channel maturity: Though JFrog has partners worldwide, the partner channel accounts for only a small percentage of its sales.


• Red Hat - Challenger
  

  Pros:

  • Global availability: Red Hat's DevOps platform is available as a SaaS offering in all geographies with the exception of China.
  • Hybrid and multicloud: With the combined capabilities of Red Hat Ansible Automation Platform and Red Hat OpenShift, users have a package that enables applications to be delivered to all cloud platforms, with autoscaling and strong management tooling that spans from the edge to IBM mainframes.
  • Service level: Red Hat offers a service-level agreement that provides better response time and availability than other vendors in this research.

  Cons:

  • OpenShift-oriented: Red Hat's DevOps platform is oriented toward its OpenShift Container Platform. Users of other container management platforms may find it more difficult to implement Red Hat's offering than those of other vendors in this Magic Quadrant.
  • Limited reporting: Red Hat collects a variety of information across its various components, but doesn't have a cohesive reporting system. It is integrated with Prometheus, which can perform the data collection; this, in turn, can be used by Grafana to build dashboards.
  • Ansible integration with third-party tools: While it is possible to integrate many third-party tools with Red Hat Ansible, there are fewer directly supported integrations compared with other vendors' platforms in this Magic Quadrant.

Reference

• Gartner, Magic Quadrant for DevOps Platforms, 3 September 2024, ID G00796821

View Market Definition
View Vendor Movements