Hands-on safety with Hoverfly Cloud
In our previous discussion, we explored the critical "why" behind API simulation for Agentic AI: the inherent risks of autonomous agents interacting with the world via APIs and the undeniable need for a safe testing environment. Now, let’s dive into the "how." How can development and QA teams practically leverage API simulation to build robust, secure, and compliant AI agents?
The answer lies in the versatility of advanced API simulation tools like Hoverfly Cloud, which offers a powerful suite of modes designed to give you unparalleled control over your AI's testing environment. This complements our focus on Compliant and Secure Application Testing by extending its principles to the unique demands of AI.
Moving beyond basic mocking: a deeper dive into Hoverfly Cloud’s modes
Traditional API simulation (often referred to as mocking) provides static responses, which is useful for basic unit testing. However, the dynamic and often unpredictable nature of Agentic AI demands a more sophisticated approach. Hoverfly Cloud's core modes provide this depth:
1. Capture Mode: grounding AI in reality (safely)
What Capture Mode does:
Capture Mode allows you to record actual API interactions between your AI agent (or the application it's being integrated into) and real external services. It acts like a proxy, passively observing and saving every request and response. These recordings are then stored as "simulations" which can be replayed.
Why it's crucial for AI:
- Realistic baselines: Before an AI agent is fully autonomous, you'll want to ensure it understands and processes information from real-world APIs accurately. Capture Mode helps you build a library of real API traffic for future reference
- Understanding "normal" behaviour: By capturing successful interactions, you create a benchmark for expected behaviour. This helps in training the AI to parse responses correctly and identify deviations when things go wrong
- Reduced dependency on live systems (post-capture): Once captured, these interactions can be replayed repeatedly without ever touching the actual external API, protecting live systems from excessive load and ensuring consistent test data
AI agent use case:
Imagine an AI agent designed to retrieve product information from an e-commerce API. You can run a few queries against the live API in Capture Mode to build a representative set of responses (e.g., successful product lookups, products not found, out-of-stock items). This captured data then forms a realistic foundation for subsequent testing of the AI's data parsing and decision-making logic.
2. Simulate Mode: the controlled sandbox for decision-making
What Simulate Mode does:
Simulate Mode is where Hoverfly Cloud intercepts requests and serves pre-defined or captured responses. While Capture Mode creates simulations, Simulate Mode uses them. This is the heart of controlled testing, allowing you to replay previously captured traffic or serve custom-defined API responses.
Why it's crucial for AI:
- Targeted scenario testing: This is invaluable for testing specific AI decisions. You can replay captured success scenarios, or create custom simulations for various error codes (400s, 500s), network latency, malformed data, or even specific business logic responses (e.g., "customer account suspended")
- Testing failure recovery: How does your AI agent react when an API endpoint is unreachable or returns an unexpected error? Simulate Mode allows you to precisely inject these failures by replaying specific error simulations to test the AI's resilience and error-handling mechanisms. This is vital for preventing AI-driven "runaway" processes
- Exploring edge cases: It's often impractical or impossible to generate all edge cases in a live environment. By creating custom simulations, you can craft unique API responses (e.g., an extremely long string, a specific numeric overflow, an empty array) to see how your AI agent processes them
- Securing sensitive data flows: As discussed in our previous article, you can use synthetic data within your simulations for sensitive APIs (payment gateways, PII lookups), ensuring no real data is ever compromised during testing
AI agent use case:
An AI agent that approves or rejects loan applications based on external credit score APIs. In Simulate Mode, you can:
- Replay captured scenarios of varying credit scores (excellent, good, poor)
- Serve custom simulations for API timeouts or service unavailability to test the agent's fallback logic
- Provide a custom simulation where an API returns an unexpected data format to check for parsing robustness
- Provide a custom simulation for a "fraudulent activity detected" response to test the agent's escalation path
3. Spy Mode: scaling up for robustness and adaptive testing
What Spy Mode does:
Spy Mode primarily simulates the API but automatically passes requests through to the real API if it doesn't recognise the interaction. This ensures that your application remains functional, even if a simulation is incomplete. It's a hybrid approach combining the benefits of simulation with the safety net of real API interaction.
Why it's crucial for AI:
- Testing AI learning and generalisation: If your AI agent is designed to adapt to varying API responses or to explore new API interactions, Spy Mode allows it to learn from real interactions when a simulation isn't available, while still providing controlled responses for known scenarios
- Realism at scale with fallback: For APIs that return large, complex datasets, Spy Mode can provide simulated data for known patterns and seamlessly pass through to the real API for novel or evolving interactions, ensuring your AI can process vast amounts of information efficiently and correctly
- Gradual simulation adoption: As your AI agent evolves and its API interactions change, Spy Mode allows for a phased approach to simulation. You can start with a basic set of simulations and let Spy Mode handle the "unknowns" by forwarding them to the real API, gradually building out a comprehensive simulation suite
AI agent use case:
An AI agent aggregating market data from various financial APIs. With Spy Mode, you can:
- Provide simulations for common stock price lookups, and let the agent interact with the live API for less frequent or new market data queries
- Test an AI's ability to handle new API versions or endpoints gracefully, as Spy Mode would automatically forward requests for unsimulated endpoints to the real API
- Ensure continuity of testing even if not all API responses have been fully simulated, as the agent can fall back to the real API
4. Passthrough Mode: uncovering unintended consequences (or simply observing)
What Passthrough Mode does:
Passthrough Mode simply forwards all requests directly to the real service without attempting to match or simulate them. It acts as a conduit, allowing requests to flow seamlessly to the real API and back without any intervention. This is ideal for scenarios where you want to interact with the real API without modifying the environment or incurring additional overhead.
AI agent use case:
An AI agent managing customer support queries by integrating with a CRM API.
You can use Passthrough Mode to ensure:
- The AI agent is making the correct calls to the live CRM API endpoints it's authorised to use (by flagging calls to unauthorised endpoints)
- The data it's sending to the CRM API continues to match the expected, compliant format and content in a live environment, even after an AI model update
- Any new AI model deployment doesn't inadvertently introduce new, unapproved API calls or change existing ones in a way that violates compliance or business rules, by directly observing its live behaviour
The Hoverfly Cloud advantage: a holistic approach to AI safety
The combination of these powerful Hoverfly modes provides a holistic framework for testing Agentic AI. It's not just about isolated API tests; it's about creating a controlled ecosystem where your AI agents can learn, operate, fail gracefully, and ultimately succeed, all within a secure and risk-free environment. This approach is fundamental to accelerating AI development, ensuring compliance, and building trust in the autonomous systems that will define our future.
Put these powerful capabilities to work for your AI initiatives. Contact us for a personalised demo of Hoverfly Cloud and discover how our API simulation suite can become your indispensable partner in developing safe and secure Agentic AI.
