The Forrester Wave™: Integrated Software Delivery Platforms, Q2 2023

Integrated Software Delivery Platforms Vendor Product Comparisons 2023

Key features and pros and cons of top Integrated Software Delivery Platforms products:

GitLab - Leaders

Pros:

Balance between solid platform capabilities and overall strategy
End-to-end OOTB toolchains with high platform cohesion
Strong DevSecOps capabilities with OOTB SAST, DAST, and Secret Detection
One of few vendors with integrated test automation dashboard
Open core product with strong OSS community investment
Complete transparency in pricing with free tier available
Strong roadmap including enhanced supply chain security and pipeline security

Cons:

While platform capabilities are solid, enterprises still supplement with best-of-breed tools in some areas
On-par (not leading) scores for build automation, governance and compliance, and infrastructure as code

Amazon Web Services - Strong Performers

Pros:

Excellent administration and management capabilities with API-level user permissions
Top scores for governance and compliance with OOTB compliance templates
Strong support for all major authentication protocols including SCIM
Security is a first-class citizen
Platform cohesion and developer experience are on par

Cons:

Low scores for analytics and reports - lacking dev-related metrics
Release readiness scored low
Build automation missing test execution optimization
Vision focused mainly on CodeCatalyst beta rather than broader ISDP strategy

Atlassian - Strong Performers

Pros:

Top scores for platform cohesiveness and well-integrated product suite
Strong partner ecosystem
Confluence integration enhances developer experience and onboarding
Serves 80% of Fortune 500 with Jira agile planning tool
Data layer strategy to connect tools and provide VSM insights

Cons:

Pipeline security capabilities are lacking
Below-par OOTB security tools, preferring partner ecosystem approach
Deployment automation scored below par
Innovation confined to platform capabilities and data lake component

CircleCI - Strong Performers

Pros:

Outstanding build automation with optimized build processes
Top scores for infrastructure as code capabilities
Support for wide range of platforms including mainframe
Dedicated customer success engineers for each account
Strong OOTB metrics including DORA metrics
Parallelization of workflows and rerunning only failed parts

Cons:

Strategic vision limited to remaining best-of-breed CI/CD tool, not extending to broader ISDP
Low score for administration and management due to reliance on third-party VCS identities
Analytics dashboards not as extensive as higher-scoring vendors
Vision does not extend beyond CI/CD capabilities

CloudBees - Strong Performers

Pros:

Above-par deployment automation with blue/green, canary, and feature management
Top-notch release readiness, governance, and compliance
Complete integration with ITSM tools like ServiceNow
High scores for community with significant OSS contributions
Strong vision and large partner ecosystem
True partnership approach with customers

Cons:

Platform cohesion needs work - planned SaaS offering should address this
Build automation requires effort to manage fleets of build runners
Pipeline security needs to go beyond branch security
Innovation scored on par, not leading

Codefresh - Strong Performers

Pros:

Above-par build automation and CI capabilities
Unique pipeline debugging - step in/out of containerized build steps
Strong support for GitOps methodology powered by Argo CD
Active OSS community participation as Argo project maintainer
Strong IaC support for Kubernetes configurations
OOTB DORA metrics included

Cons:

Below-par governance and compliance with limited ITSM integration
Below-par support for functional test automation
Smaller-than-average partner ecosystem
Innovation confined to Argo and GitOps improvements
Limited support for legacy platforms beyond Kubernetes

Harness - Strong Performers

Pros:

Top scores for platform cohesion with great OOTB experience
Top scores for build automation including intelligent test automation
Supports multiple deployment strategies OOTB including blue/green and canary
Top-scoring analytics with dev, process, and operational metrics
Can pinpoint code causing pipeline errors for faster remediation
Willingness to engage directly with development teams

Cons:

Smaller-than-average community program
Below-par UI customization capabilities with limited scope
Limited number of third-party integrations compared to others
Nascent footprint in user and open source community

Microsoft - Strong Performers

Pros:

Top innovation scores - GitHub Copilot sets new industry standards
Top scores for developer experience with GitHub Actions, Codespaces, and Copilot
Top scores for OOTB security tools like Dependabot
Extensive partner ecosystem
Large online community
Top scoring for APIs and third-party tool support

Cons:

Governance and compliance need attention - missing key enterprise governance features
Below-par deployment automation lacking blue/green and canary scenarios
Below-par data and analytics - mostly operational metrics
Vision and roadmap unclear on Azure DevOps and GitHub unification
Administration missing granular permissions of higher-scoring vendors

Flexagon - Contenders

Pros:

Perfect scores from reference customers for customer support
Above-par test automation with native functional test capabilities
High scores for version control supporting largest number of VCSes
Strong environment management capabilities
Good platform cohesion with end-to-end offering
Supports traditional and business platforms (Salesforce, Oracle, SAP)

Cons:

Developer experience was not a focus area and scored low
Below-par innovation with lacking investment in leading-edge areas
Small partner ecosystem compared to others
Complexity in setting up and operating fleets of build runners
Pricing transparency could be improved

JFrog - Contenders

Pros:

Superior version control due to strength of Artifactory
Top-notch pipeline security with signed artifacts and signed pipelines
Strong supply chain security measures
Good platform cohesion requiring few setup steps
Strong user community and OSS participation
Partner mentality in customer relationships

Cons:

Below-par vision - overly focused on security at expense of other ISDP capabilities
Below-par roadmap with limited scope
Needs investment in release readiness, data, and analytics
Limited support for test automation
Policy as code needs expansion to all ISDP aspects

OpenText - Contenders

Pros:

Top-rated platform support including mainframe
Strongest governance and compliance with comprehensive change management
Superior test automation with autoclassification of test failures
Above-par version control supporting virtually any VCS
DORA and flow metrics included
Strong ties between business and development

Cons:

Pipeline security lags other vendors
Low scores for infrastructure as code
Vision focused more on business concerns than CI/CD capabilities
Opaque pricing compared to competitors
Innovation focused on AI/ML in testing but limited elsewhere

VMware - Contenders

Pros:

One of the best pipeline security with signed pipelines and commits
Good platform cohesion with minimal setup required
Built-in knowledge management system for developer onboarding
On-par governance and compliance with OOTB policies
Deployment automation includes blue/green and canary strategies
Vendor always willing to accept and act on feedback

Cons:

Below-par administration and management missing features of higher scorers
Light on data and analytics capabilities
Platform support seems limited to Kubernetes workloads
Vision broad but missing clarity on long-term direction
Limited support for non-Kubernetes environments

IBM - Challengers

Pros:

Solid build automation, deployment automation, and release readiness
Strong analytics and graphics through UrbanCode Velocity
Support for large variety of target platforms
Good bridge between legacy and modern infrastructure
Solid development community through IBM Garage
On-par partner ecosystem with top systems integrators

Cons:

Platform cohesion suffers from complicated installation process
Administration and management offer limited capabilities (e.g., only LDAP)
Bare bones developer experience
Roadmap emphasizes maintenance versus new features
Innovation proof points are lacking
Pipelines don't offer same capability level as modern tools
Declined to participate in full Forrester Wave evaluation

FAQs

Q: Which vendors are Leaders in 2023?

A: GitLab

Q: What does "Leader" mean in Forrester Wave terms?

A: Leaders in the Forrester Wave have the highest combined scores for current offering and strategy. GitLab leads with a balance between solid platform capabilities and overall strategy, offering end-to-end OOTB toolchains, strong DevSecOps capabilities, excellent developer experience, and transparent pricing. Leaders demonstrate strong vision for delivering excellent developer experience without sacrificing security or compliance, invest heavily in innovation including areas like AI/ML engineering, actively support the open source community, and have clear roadmaps addressing enterprise needs such as enhanced supply chain security, pipeline security, and granular security controls.

Q: How do I interpret a Forrester Wave?

A: https://www.forrester.com/policies/forrester-wave-methodology/

Report:

Top Vendor Products for Integrated Software Delivery Platforms in 2023

Integrated Software Delivery Platforms Vendor Product Comparisons 2023

Pros:

Cons:

Pros:

Cons:

Pros:

Cons:

Pros:

Cons:

Pros:

Cons:

Pros:

Cons:

Pros:

Cons:

Pros:

Cons:

Pros:

Cons:

Pros:

Cons:

Pros:

Cons:

Pros:

Cons:

Pros:

Cons:

FAQs

Q: Which vendors are Leaders in 2023?

Q: What does "Leader" mean in Forrester Wave terms?

Q: How do I interpret a Forrester Wave?

Reference