Report:

Magic Quadrant for Digital Experience Monitoring

Top Vendor Products for Digital Experience Monitoring in 2024

• Catchpoint
• Datadog
• Dynatrace
• New Relic

Digital Experience Monitoring Vendor Product Comparisons 2024

• Catchpoint - Leader
  

  Pros:

  • Product: Catchpoint's Internet Performance Monitoring allows organizations to simulate and monitor user interactions with websites, applications and APIs from nearly 3,000 global vantage points in 300 cities across more than 90 countries. This monitoring also extends to last-mile network perspectives. Catchpoint also monitors internet outages globally that may impact customer business.
  • Global reach and scalability: Catchpoint's network of monitoring nodes is considerably larger than competitors and is located in backbone, ISP, cloud, last-mile, BGP and mobile environments across the globe. This extensive reach allows for comprehensive monitoring of digital experiences from various geographic locations and network conditions.
  • Vision: Catchpoint's innovation and market responsiveness demonstrate a clear understanding of client needs, particularly around modern distributed cloud-based applications.

  Cons:

  • Geographic strategy: Monitoring data is stored in data centers in the U.S., which may raise concerns for customers about compliance with local laws and regulations, privacy, legal and financial risks, and operational challenges.
  • Functionality: Catchpoint lacks session replay and customer journey mapping, which may be important for some clients. Clients would need to install additional products from separate vendors to cover these monitoring types.
  • Pricing information: Unlike other Leaders in this research, Catchpoint does not publish pricing information on its website. This lack of transparency is not in line with general trends in this area and hampers potential customers when estimating costs.


• Datadog - Leader
  

  Pros:

  • Product features: Datadog Mobile RUM helps monitor the quality of experience (QOE) and provides insights into the end-user experience for applications running on smartphones and tablets, as well as smart TVs powered by Apple tvOS, Android TV, and Roku — a differentiated offering in this area. Gartner clients who develop or support applications on media streaming devices will find the native support of these operating systems important.
  • Protect sensitive data: Datadog's Sensitive Data Scanner can automatically discover, classify and redact sensitive data across logs, application performance monitoring (APM) spans and RUM events at ingestion. This, coupled with client-side controls, protects organizations against data privacy risks.
  • Real mobile device testing: Datadog's Mobile Application Testing replays tests on real mobile devices rather than using emulation. Organizations desiring synthetic monitoring that most closely represents the end-user perspective may find using real hardware a compelling capability.

  Cons:

  • SaaS-only solution: Datadog is only available as a SaaS solution. While this is the preferred mode for most clients, some organizations might require a self-managed or self-hosted solution to comply with regulatory, compliance or data residency policies.
  • Usage-based pricing: The vendor's usage-based pricing requires customers to manage their DEM tool expenses by adjusting sampling rates for RUM and the frequency, number of platforms, locations and retries for STM tests. Cost-related reporting and alerting are available; however, customers may incur unexpected costs without manual oversight.
  • No automated IPM capability: IPM can be achieved only by manually analyzing RUM performance or STM traceroute data. Organizations wanting the ease of AI and analytics to surface internet impacts on application performance will find Datadog lacking in this area.


• Dynatrace - Leader
  

  Pros:

  • Unified observability platform: The Dynatrace platform extends beyond digital experience to include full-stack observability. This enables application developers and site reliability engineering (SRE) teams to combine experience data with application and infrastructure telemetry to troubleshoot and resolve issues down to the code level.
  • Use of AI: Dynatrace's predictive AI capabilities extend to user experience and business-oriented use cases, including probabilistic forecasting of business key performance indicators (KPIs), which can facilitate targeting customers who experience poor performance with promotional campaigns.
  • Customer-centric: Dynatrace demonstrates a strong understanding of client needs, particularly regarding the transition of applications to modern architectures. This understanding has led to innovations such as their AI and Grail, the Dynatrace data lakehouse, which facilitates improved digital experience monitoring.

  Cons:

  • Platform breadth: For clients who are concerned only with end-user experience and not engaged in developing their own applications, Dynatrace's offering has a steeper learning curve and may be more expensive than a stand-alone solution.
  • Suitability for SMBs: Dynatrace's target audience is large enterprises with use cases encompassing multiple personas. While alternative channels, such as public cloud marketplaces, make it more accessible, small and midsize businesses may find that cost justification limits Dynatrace's use to monitoring only the most business-critical systems.
  • Pricing caution: Using the Dynatrace Platform Subscription (DPS) pricing model, clients draw down against a committed annual minimum spend, greatly benefiting from usage flexibility. Clients may need to implement internal controls based on Dynatrace's trending, forecasting and alerting capabilities to avoid overspend against their original estimate, which can be particularly easy to do with synthetics and real user monitoring.


• New Relic - Leader
  

  Pros:

  • Integrated solution: New Relic DEM capabilities complement the overall observability platform; a comprehensive, extensible solution that provides a deep understanding of how application performance and infrastructure health impact user experience.
  • Ease of use: New Relic's UI provides a modern, user-friendly interface with a range of reporting options, as well as ease of configuration.
  • Vendor support: Gartner clients report high levels of satisfaction with the vendor account and support team, noting that they are responsive and open to implementing feedback and enhancement requests.

  Cons:

  • Ease of use: Gartner clients reported challenges in mastering the vendor's proprietary query language, New Relic Query Language (NRQL, based on SQL). New Relic has introduced a natural language interface that automatically creates NRQL queries, allowing nontechnical and business users access to observability insights.
  • Geographic footprint: New Relic's service delivery platforms are located in the U.S. and EU (Germany). Organizations outside of their core regions, particularly those in APAC, may need to confirm that the company's offering meets any region-specific performance or sovereignty requirements. Note that synthetic tests can be executed from private locations globally.
  • Pricing: New Relic's observability platform pricing model is based on the number of users and the volume of telemetry ingested, but pricing for synthetics does not follow this model. Users obtain a set amount of synthetic tests based on the type of license (pro or enterprise), and this does not increase with the number of users. Exceeding the allocated quantity requires customers to purchase extra synthetic tests, creating cost management complexity.


• Cisco - Challenger
  

  Pros:

  • Deployment options: AppDynamics can be either SaaS or self-managed, allowing clients flexibility around their choice of deployment. Some industries and regions may require an on-premises offering.
  • CI/CD integration: AppDynamics Synthetic Transaction Monitoring (STM) can be integrated with the continuous integration/continuous delivery (CI/CD) pipeline to be invoked during application rollout for testing at the time of release to determine and automate the next action, such as continuing the release or rollback.
  • Market presence: Both Cisco and Splunk are brands with significant market presence and are frequently mentioned in Gartner client inquiries. Both have an extensive, diverse customer base that relies on these products to monitor their critical infrastructure and workloads.

  Cons:

  • Portfolio complexity: After Cisco's acquisition of Splunk in March 2024, the DEM portfolio lacks coherence, with the overall solution consisting of multiple components with integration being a work in progress. This has caused confusion among clients who are unsure which components from Cisco best serve their needs.
  • Functionality: To gain visibility into IPM, AppDynamics Real User Monitoring (RUM) requires a token to be enabled through an out-of-the-box integration with Cisco ThousandEyes, a separate paid product that was not included as part of their DEM solution for this research.
  • Cost control: AppDynamics DEM uses a token-based system for RUM licensing and metered durations for synthetic monitoring, which can add complexity and potentially lead to unexpected costs if not carefully managed.


• IBM - Visionary
  

  Pros:

  • Market presence: IBM's global reach, strong customer base and extensive product portfolio make it a strong choice for larger organizations seeking a single vendor to cover their broader observability needs.
  • Deployment models: Instana provides a range of deployment modes, including the capability to be deployed as a managed SaaS offering on various cloud providers or as a self-hosted, on-premises solution. This flexibility provides clients with a deeper level of control over their data sovereignty to align with regulatory or national requirements.
  • Use of AI: The additions of anomaly detection using IBM Watson technology and the 'Smart Alert' function to the product functionality are rated highly by clients, as they reduce operator toil and speeds time to problem notification.

  Cons:

  • Features: IBM Instana lacks capabilities that are offered by Leaders in this area. For example, it lacks capabilities for session replay, and network path analysis is only available through other IBM products, such as SevOne. However, integration with these products remains a challenge.
  • Pricing: Instana's core observability platform has a price-per-host metric; however, if a customer wants to execute synthetic tests from Istana-managed points of presence (PoPs), they will need to pay separately for it. This may challenge the ability of enterprises to perform accurate cost estimates.
  • Support and responsiveness: Some customers have reported issues with the responsiveness of IBM's customer support and a slow escalation process, which can delay issue resolution.


• ITRS - Visionary
  

  Pros:

  • Ease of use: ITRS Uptrends' console and interface are coherent and intuitive, allowing for quick adoption of the product. This will speed time to value with minimal impact on IT operators' time.
  • Full trial: Uptrends also provides potential clients the ability to explore full product functionality for 30 days without commitment. This allows users to experiment with the product and build their own proof of concept to determine suitability.
  • Portfolio: ITRS has a wide portfolio of monitoring solutions that integrate with Uptrends, offering the ability to extend observability across the enterprise while remaining functional as a stand-alone offering.

  Cons:

  • Limited functionality: ITRS' solution lacks key capabilities, such as session replay, and does not enable customer journey mapping. This limits its ability to cover common DEM use cases for IT operations teams.
  • Market visibility: General awareness and adoption of ITRS among Gartner clients are relatively low.
  • Pricing model: ITRS Uptrends pricing is based on a number of different metrics, including credits, minutes and an overall license fee. This makes estimation and comparison of pricing difficult for clients budgeting their costs.


• Riverbed - Visionary
  

  Pros:

  • Ease of use: Customers report positive sentiment regarding the intuitive interface, which gives an integrated view across devices, networks and applications, reducing operator load for onboarding and management.
  • Portfolio: Riverbed's Riverbed IQ product is an interesting differentiator for Riverbed. It ingests telemetry from the Aternity products and allows for cross-domain analysis and correlation, which may help improve mean time to resolution.
  • Product: Riverbed Aternity's network path analysis functionality allows for easy visualization of the performance of the underlying network infrastructure, enabling IT operations teams to quickly identify performance issues at different layers in the ecosystem.

  Cons:

  • Pricing transparency: Unlike leaders in this area, Riverbed's pricing is not available on the vendor's website. This makes it challenging for customers wishing to compare the cost of the solution with that of other DEM vendors.
  • No session replay: Not having session replay may result in reduced user insight and problem diagnosis, hindering the potential for optimization opportunities.
  • Dependencies: The Aternity User Journey Intelligence tool is an OEM rebadging of Blue Triangle (another vendor in this research). OEM software may be subject to changes based on contractual relationships between vendors, which can cause uncertainty long term for clients.


• Blue Triangle - Niche Player
  

  Pros:

  • Customer scale: Blue Triangle clients include large enterprises, such as retailers, travel and healthcare companies, that have very high performance and availability demands on their customer-facing websites. Blue Triangle provides an easy-to-deploy, scalable and cost-effective solution that is able to meet these high-scale demands.
  • Software development: Comparison and analysis of performance for different periods help DevOps/app developers check the impact of changes pre- and postdeployments, increasing agility and responsiveness in release cycles.
  • Hybrid visibility: Blue Triangle's customer journey analysis provides insight into the common blind spot of hybrid apps that include both native apps and web apps, allowing IT operations teams visibility across these modern applications.

  Cons:

  • Network insight: Network path analysis is limited in functionality, making proactive trace analysis a challenge.
  • User interface: Compared with leading vendors in this market, the product's UI appears dated. Customers have expressed challenges with areas such as visualization, data access and analytics.
  • Strategy: While Blue Triangle performs well with its existing functionality, its strategy and roadmap are limited in comparison with other vendors. Clients making a strategic purchase for their DEM need to evaluate the roadmap against their future needs.


• ip-label - Niche Player
  

  Pros:

  • Thick client monitoring: ip-label was the only solution assessed that included thick client application monitoring. Many legacy applications do not have a fully functioning web interface, which can limit the usefulness of other digital experience monitoring solutions in this scenario.
  • No-code STM: Ekara Studio is a true no-code development tool for creating STM scripts, which Ekara calls 'user journeys.' STM replay of user journeys is constructed using graphical blocks for simple and complex actions. ip-label uses mobile devices for synthetic user journeys using real customers' mobile applications.
  • Pop-up management: A common problem for STM replay is unexpected window pop-ups on web applications that trigger script failures, leading to false positives. Ekara includes the option for its Magic Wand feature during replay, which leverages AI to manage unexpected pop-ups, allowing normal script replay to continue.

  Cons:

  • User interface: The Ekara interface appears dated compared to its competitors and may hamper adoption and hinder time to value during implementation.
  • Functionality: Ekara lacks the breadth of functionality of many of its competitors, which may fail to attract customers that require a more comprehensive solution. Organizations should confirm that Ekara provides the required functionality.
  • Documentation: Compared with other vendors in the market, the product documentation and the Ekara website are limited and difficult to navigate.


• ManageEngine - Niche Player
  

  Pros:

  • Global presence: ManageEngine can support path analysis based on multiple global locations. Customers have the option of choosing multiple global or private locations for setting up website monitoring.
  • Market presence: ManageEngine has a significant presence in the market with its other IT operations management (ITOM) tools, and also boosts its brand awareness with multiple free tools and on-demand product training for implementations and use cases.
  • Pricing: ManageEngine's pricing model is very competitive compared to the competition, especially when combined with the broader offerings from ManageEngine, including IT service management (ITSM) and security products.

  Cons:

  • Visualization: The volume of raw data made available as-is for visualization in some scenarios limits its usefulness.
  • Scale issues: ManageEngine customers report complexity in configurations and management at scale, including degraded performance. They also find integrations with third-party tools challenging.
  • Functionality: The product lacks customer journey mapping and session replay, hampering business owners' ability to make decisions based on insights.


• SolarWinds - Niche Player
  

  Pros:

  • Detailed analysis: SolarWinds' products enable a proactive approach to network path analysis by using existing probes or creating new ones, providing a hop-by-hop analysis between the probe and the application.
  • Dashboards: Customers have expressed positive sentiments regarding SolarWinds' dashboarding capabilities, highlighting the simple onboarding experience and fast time to value.
  • Partners: SolarWinds has a diverse partner program that helps with the promotion, distribution and support of the vendor's products across different regions to a wider audience.

  Cons:

  • Portfolio complexity: SolarWinds' extensive range of products supports various observability use cases, including APM, database and infrastructure; however, this also contributes to the complexity of the monitoring tool landscape, which can be confusing for users.
  • Integration: Although SolarWinds has a wide array of integrations, some customers frequently cite challenges in integrating with third-party tools, which limits the potential use cases and extensibility.
  • No mobile monitoring: Mobile app monitoring, an optional capability for this research, is a gap in SolarWinds' latest version; however, it is on the vendor's roadmap.

Reference

• Gartner, Magic Quadrant for Digital Experience Monitoring, 21 October 2024, ID G00808790

View Market Definition
View Vendor Movements