Cloudflare Enhances AI Inference Platform with Powerful GPU Upgrade, Faster Inference, Larger Models, Observability, and Upgraded Vector Database

Rhea-AI Summary

Cloudflare has announced significant enhancements to its Workers AI platform, designed to help developers build faster and more powerful AI applications. Key improvements include:

• Upgraded GPUs in over 180 cities worldwide for faster inference and support for larger models like Llama 3.1 70B
• New persistent logs in AI Gateway for better monitoring and optimization
• Enhanced Vectorize database, now supporting up to 5 million vectors per index with reduced query latency

These upgrades aim to minimize network latency and improve AI application performance globally. Cloudflare's CEO, Matthew Prince, emphasized the importance of network speeds and regional availability as AI becomes more integrated into daily life.

Positive

• Expanded GPU network to over 180 cities worldwide, enhancing global AI accessibility
• Support for larger AI models, including Llama 3.1 70B and Llama 3.2 series
• Introduction of persistent logs in AI Gateway for improved monitoring and optimization
• Vectorize database capacity increased from 200,000 to 5 million vectors per index
• Reduced median query latency in Vectorize from 549ms to 31ms

Negative

• None.

Insights

AI Technology Analyst

Cloudflare's announcement of enhanced AI capabilities is a significant development in the AI infrastructure space. The upgrade to more powerful GPUs and support for larger models like Llama 3.1 70B demonstrates a substantial leap in processing power. This allows for more complex AI tasks and improved user experiences, potentially opening new markets for AI-driven applications.

The expansion of GPU availability to over 180 cities globally is a game-changer for AI inference. By reducing latency and bringing processing closer to end-users, Cloudflare is addressing a critical bottleneck in AI deployment. This could accelerate the adoption of AI in everyday applications, particularly in regions previously underserved by centralized AI infrastructure.

The improvements to Vectorize, with increased vector capacity and reduced query latency, are particularly noteworthy. These enhancements can significantly improve the performance and cost-effectiveness of AI applications, especially in search and recommendation systems. The 94% reduction in median query latency to 31ms is impressive and could lead to more responsive AI-powered services.

Financial Analyst

From a financial perspective, Cloudflare's AI enhancements position the company strongly in the rapidly growing AI infrastructure market. The expanded capabilities could drive increased adoption of Cloudflare's services, potentially leading to higher revenue streams from AI-related offerings.

The focus on efficiency and cost-effectiveness, particularly with the improvements in Vectorize, could attract cost-conscious enterprises looking to implement AI solutions. This may lead to increased customer acquisition and retention, positively impacting Cloudflare's financial performance.

However, investors should consider the capital expenditure required for the global GPU expansion and ongoing R&D investments. While these investments are likely necessary to maintain competitiveness, they may impact short-term profitability. The long-term potential for market share growth and revenue expansion in the AI sector could outweigh these near-term costs, making this a strategically sound move for Cloudflare's future growth prospects.

Workers AI is the easiest place to build and scale AI applications; can now deploy larger models and handle more complex AI tasks

SAN FRANCISCO--(BUSINESS WIRE)-- Cloudflare, Inc. (NYSE: NET), the leading connectivity cloud company, today announced powerful new capabilities for Workers AI, the serverless AI platform, and its suite of AI application building blocks, to help developers build faster, more powerful and more performant AI applications. Applications built on Workers AI can now benefit from faster inference, bigger models, improved performance analytics, and more. Workers AI is the easiest platform to build global AI applications and run AI inference close to the user, no matter where in the world they are.

As large language models (LLMs) become smaller and more performant, network speeds will become the bottleneck to customer adoption and seamless AI interactions. Cloudflare’s globally distributed network helps to minimize network latency, setting it apart from other networks that are typically made up of concentrated resources in limited data centers. Cloudflare’s serverless inference platform, Workers AI, now has GPUs in more than 180 cities around the world, built for global accessibility to provide low latency times for end users all over the world. With this network of GPUs, Workers AI has one of the largest global footprints of any AI platform, and has been designed to run AI inference locally as close to the user as possible and help keep customer data closer to home.

“As AI took off last year, no one was thinking about network speeds as a reason for AI latency, because it was still a novel, experimental interaction. But as we get closer to AI becoming a part of our daily lives, the network, and milliseconds, will matter,” said Matthew Prince, co-founder and CEO, Cloudflare. “As AI workloads shift from training to inference, performance and regional availability are going to be critical to supporting the next phase of AI. Cloudflare is the most global AI platform on the market, and having GPUs in cities around the world is going to be what takes AI from a novel toy to a part of our everyday life, just like faster Internet did for smartphones.”

Cloudflare is also introducing new capabilities that make it the easiest platform to build AI applications with:

• Upgraded performance and support for larger models: Now, Cloudflare is enhancing their global network with more powerful GPUs for Workers AI to upgrade AI inference performance and run inference on significantly larger models like Llama 3.1 70B, as well as the collection of Llama 3.2 models with 1B, 3B, 11B (and 90B soon). By supporting larger models, faster response times, and larger context windows, AI applications built on Cloudflare’s Workers AI can handle more complex tasks with greater efficiency – thus creating natural, seamless end-user experiences.
• Improved monitoring and optimizing of AI usage with persistent logs: New persistent logs in AI Gateway, available in open beta, allow developers to store users’ prompts and model responses for extended periods to better analyze and understand how their application performs. With persistent logs, developers can gain more detailed insights from users’ experiences, including cost and duration of requests, to help refine their application. Over two billion requests have traveled through AI Gateway since launch last year.
• Faster and more affordable queries: Vector databases make it easier for models to remember previous inputs, allowing machine learning to be used to power search, recommendations, and text generation use-cases. Cloudflare’s vector database, Vectorize, is now generally available, and as of August 2024 now supports indexes of up to five million vectors each, up from 200,000 previously. Median query latency is now down to 31 milliseconds (ms), compared to 549 ms. These improvements allow AI applications to find relevant information quickly with less data processing, which also means more affordable AI applications.

To learn more, please check out the resources below:

• Blog: Cloudflare’s Bigger, Better, Faster AI platform
• Blog: Making Workers AI faster and more efficient: Performance optimization with KV cache compression and speculative decoding
• Join us online for demos, product announcements, and more at our first Builder Day Live Stream today, September 26 at 11am PT. Register at https://builderday.pages.dev.

About Cloudflare

Cloudflare, Inc. (NYSE: NET) is the leading connectivity cloud company on a mission to help build a better Internet. It empowers organizations to make their employees, applications and networks faster and more secure everywhere, while reducing complexity and cost. Cloudflare’s connectivity cloud delivers the most full-featured, unified platform of cloud-native products and developer tools, so any organization can gain the control they need to work, develop, and accelerate their business.

Powered by one of the world’s largest and most interconnected networks, Cloudflare blocks billions of threats online for its customers every day. It is trusted by millions of organizations – from the largest brands to entrepreneurs and small businesses to nonprofits, humanitarian groups, and governments across the globe.

Learn more about Cloudflare’s connectivity cloud at cloudflare.com/connectivity-cloud. Learn more about the latest Internet trends and insights at https://radar.cloudflare.com.

Follow us: Blog | X | LinkedIn | Facebook | Instagram

Forward-Looking Statements

This press release contains forward-looking statements within the meaning of Section 27A of the Securities Act of 1933, as amended, and Section 21E of the Securities Exchange Act of 1934, as amended, which statements involve substantial risks and uncertainties. In some cases, you can identify forward-looking statements because they contain words such as “may,” “will,” “should,” “expect,” “explore,” “plan,” “anticipate,” “could,” “intend,” “target,” “project,” “contemplate,” “believe,” “estimate,” “predict,” “potential,” or “continue,” or the negative of these words, or other similar terms or expressions that concern Cloudflare’s expectations, strategy, plans, or intentions. However, not all forward-looking statements contain these identifying words. Forward-looking statements expressed or implied in this press release include, but are not limited to, statements regarding the capabilities and effectiveness of Workers AI, AI Gateway, Vectorize, R2, and Cloudflare’s other products and technology, the benefits to Cloudflare’s customers from using Workers AI, AI Gateway, Vectorize, R2, and Cloudflare’s other products and technology, the timing of when Workers AI, AI Gateway, Vectorize, R2, or any of its related features will be generally available to all current and potential Cloudflare customers, Cloudflare’s technological development, future operations, growth, initiatives, or strategies, and comments made by Cloudflare’s CEO and others. Actual results could differ materially from those stated or implied in forward-looking statements due to a number of factors, including but not limited to, risks detailed in Cloudflare’s filings with the Securities and Exchange Commission (SEC), including Cloudflare’s Quarterly Report on Form 10-Q filed on August 1, 2024, as well as other filings that Cloudflare may make from time to time with the SEC.

The forward-looking statements made in this press release relate only to events as of the date on which the statements are made. Cloudflare undertakes no obligation to update any forward-looking statements made in this press release to reflect events or circumstances after the date of this press release or to reflect new information or the occurrence of unanticipated events, except as required by law. Cloudflare may not actually achieve the plans, intentions, or expectations disclosed in Cloudflare’s forward-looking statements, and you should not place undue reliance on Cloudflare’s forward-looking statements.

© 2024 Cloudflare, Inc. All rights reserved. Cloudflare, the Cloudflare logo, and other Cloudflare marks are trademarks and/or registered trademarks of Cloudflare, Inc. in the U.S. and other jurisdictions. All other marks and names referenced herein may be trademarks of their respective owners.

View source version on businesswire.com: https://www.businesswire.com/news/home/20240926193441/en/

Cloudflare, Inc.

Daniella Vallurupalli

Vice President, Head of Global Communications

press@cloudflare.com

Source: Cloudflare, Inc.

FAQ

What new capabilities has Cloudflare (NET) added to its Workers AI platform?

Cloudflare has added more powerful GPUs for faster inference, support for larger models like Llama 3.1 70B, persistent logs in AI Gateway for better monitoring, and improved Vectorize database with increased capacity and reduced latency.

How many cities does Cloudflare's (NET) GPU network now cover for AI inference?

Cloudflare's GPU network for AI inference now covers more than 180 cities around the world.

What improvements have been made to Cloudflare's (NET) Vectorize database?

Cloudflare's Vectorize database now supports up to 5 million vectors per index, up from 200,000, and has reduced median query latency from 549ms to 31ms.

When will Cloudflare (NET) host its first Builder Day Live Stream?

Cloudflare will host its first Builder Day Live Stream on September 26 at 11am PT.