Datadog's State of Cloud Costs 2024 Report Finds Spending on GPU Instances Growing 40% as Organizations Experiment with AI
Datadog's State of Cloud Costs 2024 report reveals a 40% increase in spending on GPU instances as organizations explore AI and large language models (LLMs). GPUs, over 200% faster than CPUs, are critical for these AI workloads. Despite increased spending on cloud compute, 83% of container costs are wasted due to idle resources. Moreover, 83% of organizations still use outdated technologies, spending 17% of their EC2 budgets on them. Only 67% are utilizing cloud service discounts, down from 72% last year. Conversely, green technology adoption is up, with Arm-based instances now accounting for 18% of EC2 budgets, offering better performance and lower energy consumption.
- GPU spend increased by 40%, indicating strong investment in AI and LLMs.
- GPUs are over 200% faster than CPUs for parallel processing, benefiting AI workloads.
- Arm-based instances spend doubled to 18% of EC2 budgets, highlighting a shift towards green technology.
- Arm-based instances use up to 60% less energy, offering better performance at a lower cost.
- 83% of container costs are wasted due to idle resources.
- 17% of EC2 budgets are spent on outdated technologies, which are less cost-effective.
- Participation in cloud service discounts dropped to 67%, down from 72% last year.
- 54% of wasted container spend is due to overprovisioning cluster infrastructure.
Insights
The 40% increase in spending on GPU instances is indicative of a growing trend among organizations to leverage AI and large language models (LLMs). This increase suggests a significant shift towards AI-driven initiatives, which could imply higher future costs as these projects move from experimentation to full production. For investors, this trend is important as it reflects potential for future revenue growth within Datadog’s cloud monitoring and security services. However, the substantial idle costs associated with containers (83%) could be a red flag, indicating inefficiencies in resource allocation. This might suggest room for improvement in cloud cost management, an area where Datadog could enhance its service offerings and potentially increase its market share. The report also highlights a decrease in the use of discount programs, which might signal a lack of cost optimization strategies among organizations. Investors should watch if Datadog can capitalize on these inefficiencies by pushing its cost management solutions more aggressively.
The emphasis on GPU instances underscores their critical role in the AI landscape due to their superior parallel processing capabilities, which can be more than 200% faster than traditional CPUs. This speed is essential for training LLMs and other AI workloads. The report's finding that the most widely used GPU-based instances are the least expensive suggests that many organizations are still in the trial phase of AI adoption. This could mean a future increase in demand for higher-end, more expensive GPU instances as these AI projects scale up. Additionally, the shift towards Arm-based instances, which are not only more energy-efficient but also offer better performance at a lower cost, is a trend worth noting. This reflects a broader industry move towards sustainable and cost-effective computing solutions. Investors should be aware of these technology trends as they could influence future cloud infrastructure investments and the adoption rates of newer, more efficient technologies.
The report reveals that outdated technologies continue to consume a significant portion of cloud budgets, with 83% of organizations spending an average of 17% on previous-generation technologies. This persistence can represent a barrier to cost optimization and innovation. Market trends such as the growing allocation towards containers (up to 35%) despite high idle resource costs suggest that organizations are increasingly containerizing their applications but might lack the expertise to do so efficiently. The decline in participation in discount programs (only 67% this year) could indicate either a lack of awareness or an inability to commit to long-term contracts, which might reflect on organizations' budgeting strategies. These insights can help investors understand potential areas of growth for Datadog's services, especially in providing solutions that address these inefficiencies.
GPUs can be more than
"Today, the most widely used type of GPU-based instance is also the least expensive. This suggests that many customers are still in the experimentation phase with AI and applying the GPU instance to their early efforts in adaptive AI, machine learning inference and small-scale training," said Yrieix Garnier, VP of Product at Datadog. "We expect that as organizations expand their AI activities and move them into production, they will be spending a larger proportion of their cloud compute budget as they use more expensive types of GPU-based instances."
In addition to more companies spending compute on AI projects, the report found that containers were a common theme of wasted spend among organizations. In fact,
Other key findings from the report include:
- Outdated Technologies Are Widely Used: AWS's current infrastructure offerings commonly both outperform their previous-generation versions and cost less, but
83% of organizations still spend an average of17% of their EC2 budgets on previous-generation technologies. - Fewer Organizations Are Taking Advantage of Discounts: Cloud service providers offer commitment-based discounts on many of their services—for example, AWS has discount programs for Amazon EC2, Amazon RDS, Amazon SageMaker and others—but only
67% of organizations are participating in these discounts, down from72% last year. - Green Technology Is on the Rise for Better Performance and Cost: On average, organizations that use Arm-based instances spend
18% of their EC2 compute budget on them—twice as much as they did a year ago. Instance types based on the Arm processor use up to60% less energy than similar EC2s and often provide better performance at a lower cost.
For the report, Datadog analyzed AWS cloud cost data from hundreds of organizations and explored how their use of emerging and previous-generation technologies, patterns of cloud resource usage, and participation in AWS discount programs all contributed to their cloud costs.
Datadog's State of Cloud Costs 2024 is available now. For the full results, please visit: https://www.datadoghq.com/state-of-cloud-costs/. To learn how Datadog helps companies optimize their cloud costs, visit: https://www.datadoghq.com/product/cloud-cost-management/.
About Datadog
Datadog is the observability and security platform for cloud applications. Our SaaS platform integrates and automates infrastructure monitoring, application performance monitoring, log management, user experience monitoring, cloud security and many other capabilities to provide unified, real-time observability and security for our customers' entire technology stack. Datadog is used by organizations of all sizes and across a wide range of industries to enable digital transformation and cloud migration, drive collaboration among development, operations, security and business teams, accelerate time to market for applications, reduce time to problem resolution, secure applications and infrastructure, understand user behavior and track key business metrics.
Forward-Looking Statements
This press release may include certain "forward-looking statements" within the meaning of Section 27A of the Securities Act of 1933, as amended, or the Securities Act, and Section 21E of the Securities Exchange Act of 1934, as amended including statements on the benefits of new products and features. These forward-looking statements reflect our current views about our plans, intentions, expectations, strategies and prospects, which are based on the information currently available to us and on assumptions we have made. Actual results may differ materially from those described in the forward-looking statements and are subject to a variety of assumptions, uncertainties, risks and factors that are beyond our control, including those risks detailed under the caption "Risk Factors" and elsewhere in our Securities and Exchange Commission filings and reports, including the Quarterly Report on Form 10-Q filed with the Securities and Exchange Commission on November 7, 2023, as well as future filings and reports by us. Except as required by law, we undertake no duty or obligation to update any forward-looking statements contained in this release as a result of new information, future events, changes in expectations or otherwise.
Contact
Dan Haggerty
press@datadoghq.com
View original content to download multimedia:https://www.prnewswire.com/news-releases/datadogs-state-of-cloud-costs-2024-report-finds-spending-on-gpu-instances-growing-40-as-organizations-experiment-with-ai-302172281.html
SOURCE Datadog, Inc.
FAQ
What did Datadog's State of Cloud Costs 2024 report find about GPU spending?
Why are GPUs important for AI and LLM workloads according to Datadog's report?
What percentage of container costs are wasted according to Datadog's report?
How much of EC2 budgets are spent on outdated technologies?
What is the trend in cloud service discount participation?
What does the report say about the use of Arm-based instances?
