DigitalOcean Inference Engine Launches New Server-Side Tools

DigitalOcean Launches Server-Side Tools for Enhanced AI Inference

DigitalOcean has introduced Server-Side Tools, now in Public Preview for its Inference Engine, allowing AI models to access a range of web-based functionalities directly within inference requests. This innovative feature enables models to search the web, fetch data, and interact with various systems seamlessly without the need for additional infrastructure or credentials. The update aims to enhance the capabilities of AI applications by providing real-time access to information and resources.

Overview of Server-Side Tools

Server-Side Tools integrate web search, web fetch, DigitalOcean Knowledge Bases, Model Context Protocol (MCP) servers, and support for Anthropic and OpenAI tools into the inference process. This integration allows developers to streamline their workflows by enabling models to pull in current information and interact with proprietary data sources effortlessly. Users can activate these tools using their existing DigitalOcean Model Access Key.

Key Features of Server-Side Tools

Web Search: Real-Time Information Retrieval

The Web Search feature empowers models to retrieve up-to-date information directly from the internet during inference. This capability is particularly beneficial for applications that require current news, documentation, or live data. By leveraging Web Search, developers can enhance research workflows and support experiences that depend on recent events or dynamic information not included in a model’s training data.

Web Fetch: Content Extraction from URLs

Web Fetch allows models to pull content from specific URLs or documents such as PDFs during inference. This functionality is essential for summarizing web pages, extracting structured data from various documents, or obtaining reference material on demand. Both Web Search and Web Fetch are powered by Exa, with usage-based pricing available on DigitalOcean’s pricing page.

Web Mode: Simplified Integration

For agent frameworks that limit configuration options to model names only, DigitalOcean offers Web Mode. This feature automatically enables both Web Search and Web Fetch through the model field without requiring explicit tool definitions. Consequently, it simplifies integration with frameworks that do not allow detailed tool configurations.

Connecting Models to Proprietary Data

In addition to accessing web-based information, Server-Side Tools facilitate interaction with proprietary systems through two main options:

• DigitalOcean Knowledge Bases: Models can automatically retrieve relevant content from indexed data without needing a separate retrieval pipeline. By attaching a knowledge base and sending a request, the model can ground its responses in user-specific content.
• MCP Servers: The Model Context Protocol connects models to internal systems and services by exposing APIs and databases. This allows models to retrieve information and perform actions like updating systems or triggering workflows directly within inference requests.

Compatibility with Existing Toolsets

For developers already utilizing Anthropic or OpenAI tool conventions, DigitalOcean ensures compatibility with existing tool definitions. There is no requirement to rewrite application logic or adapt to a new interface when integrating with the Inference Engine.

• Anthropic tools include: Web fetch, Tool search, Bash commands, Text editing capabilities, Computer use functions.
• OpenAI tools include: Function calling features, Computer use functions, Tool search options, Apply Patch commands, Local shell access.

The usage of these tools incurs token costs based on consumption; comprehensive details regarding supported tools can be found in DigitalOcean’s documentation.

The Impact on Developer Workflows

The introduction of Server-Side Tools significantly enhances coding agents and developer workflows. Coding assistants such as Claude Code and Codex rely on capabilities like web search and content fetching to gather context and complete tasks effectively. By integrating these functionalities directly within inference requests, DigitalOcean simplifies the operation of coding agents without necessitating additional tool management.

The Server-Side Tools are currently available in Public Preview through existing DigitalOcean Model Access Keys. No new credentials or account modifications are required for users looking to leverage this new functionality. Developers can specify tools as part of their inference requests or enable Web Mode via the model URL for immediate access.

What This Means for Developers

The launch of Server-Side Tools marks a significant advancement in how developers can build AI applications using DigitalOcean’s infrastructure. With enhanced capabilities for real-time data access and seamless integration with proprietary systems, developers are empowered to create more responsive and intelligent applications. As DigitalOcean continues to expand its offerings in this area, users can expect even more robust features that will streamline development processes while reducing complexity.

For more information, read the original report here.