DigitalOcean Launches Enhanced AI Models with DeepSeek V3.2, MiniMax-M2.5, and Qwen 3.5 397B
DigitalOcean has announced the general availability of its latest AI models—DeepSeek V3.2, MiniMax-M2.5, and Qwen 3.5 397B—on its Serverless Inference platform. This release promises significant performance improvements, boasting the fastest output speed among major providers as tested by Artificial Analysis. Specifically, DeepSeek V3.2 achieves an impressive output of 230 tokens per second and a Time-to-First-Token (TTFT) of under one second for inputs of 10,000 tokens.
The Shift Towards Inference Efficiency
The landscape of artificial intelligence development is undergoing a crucial transformation, moving from model training to optimizing inference efficiency. This change is largely driven by the increasing demand for real-time applications such as conversational agents and interactive systems that require low-latency responses to remain engaging for users. A delay exceeding one second in TTFT can lead to user frustration and abandonment of the application.
As modern AI workflows become more complex, the need for rapid inference becomes even more critical. Tasks often involve multiple sequential model calls where even slight delays in Time-Per-Output-Token (TPOT) can accumulate into noticeable latency for users. Fast inference not only enhances user experience but also offers businesses reliable performance at lower costs, making it essential for scaling AI applications effectively.
Benchmarking Performance Across New Models
The newly released models have demonstrated remarkable performance metrics during testing. For DeepSeek V3.2 with 10,000 input tokens, the benchmarks reveal:
- Output speed: 230 tokens per second (3.9 times faster than AWS Bedrock’s 59 tokens per second)
- TTFT: 0.96 seconds (only Google Vertex outperformed this among twelve tested providers)
- Balanced performance across latency and output speed: DigitalOcean ranks favorably in the Artificial Analysis Latency vs. Output Speed chart.
This level of performance was not achieved solely through advanced hardware; it required a comprehensive optimization strategy across all layers of the technology stack.
Hardware Innovations: NVIDIA Blackwell Ultra
The cornerstone of DigitalOcean’s performance breakthrough lies in its use of NVIDIA’s HGX B300 GPU featuring the Blackwell Ultra architecture. This hardware boasts a substantial increase in capacity with 288GB of HBM3e memory—50% more than its predecessor—and offers 1.5 times greater NVFP4 compute power. Such advancements are vital for managing the high throughput demands of DeepSeek and Qwen at scale.
Initial deployments in virtualized environments encountered a performance hit; however, close collaboration with NVIDIA allowed DigitalOcean to unlock the full potential of this cutting-edge architecture.
Model Optimization Techniques
The implementation of NVFP4 quantization has played a significant role in enhancing model efficiency by utilizing a specialized four-bit floating-point format that reduces memory usage significantly while boosting inference throughput. This approach leverages the unique capabilities of the Blackwell Ultra architecture to achieve substantial performance gains without compromising model accuracy.
A highly customized software stack was also essential to translate this hardware power into exceptional inference speeds:
- Tensor Parallelism: Distributing large model layers across multiple GPUs enables running models that exceed single GPU memory capacity.
- Kernel Fusion: This technique merges multiple operations into one GPU kernel to minimize overhead and accelerate processing.
- Programmatic Dependent Launch: Overlapping kernels helps mitigate launch overheads and improves performance for low-batch-size workloads by approximately 10%.
- Speculative Decoding and Multi-Token Prediction (MTP): These features enhance token generation speeds while maintaining output quality through advanced predictive techniques.
A Proven Impact on Businesses
The effectiveness of these optimizations is already evident in production environments. Workato, which automates over one trillion workflows using agentic AI, reported a staggering reduction in TTFT by 77%, alongside a 79% decrease in end-to-end latency and a 67% cut in inference costs after migrating to DigitalOcean’s platform.
This feedback underscores how DigitalOcean’s commitment to optimizing AI infrastructure can significantly accelerate business processes and enhance overall operational efficiency.
What This Means
The launch of DeepSeek V3.2, MiniMax-M2.5, and Qwen 3.5 397B on DigitalOcean’s Serverless Inference platform marks a pivotal moment in AI development focused on efficiency and speed. As businesses increasingly rely on real-time AI applications, these advancements offer crucial tools for maintaining competitive advantage through enhanced user experiences and reduced operational costs.
For more information, read the original report here.
