AIVAX
Console

Agentic Tests

Test the conversation. Not one reply.

Simulate a goal-oriented user, exercise the AI Gateway you actually run and let an independent judge follow the outcome across turns.

Follow a test run Open Agentic Tests
A convincing reply can still be wrong.
“Your refund has already been issued.”

But the evidence is missing.The judge catches the unsupported claim before the conversation recovers.

  1. Discovery
  2. At risk
  3. Recovery
A moment from the illustrative run below—not a production result.

Conversation-level evaluation

Watch the trajectory, not a snapshot.

One illustrative support flow shows how the simulated user, gateway and judge produce a turn-by-turn record.

Author a focused test
Goal / Support resolution Resolve a duplicate charge and give the next step.
max_turns: 10
One unsupported claim changes the direction of the conversation.
Illustrative judge score over three turns Turn 1: 0.48, active. Turn 2: 0.31, at risk after an unsupported refund claim. Turn 3: 0.93, success after correcting course. The vertical scale runs from zero to one. 1.00.50.0 Turn 1Turn 2Turn 3 0.480.310.93

  1. 01

    Simulated user: Reports two charges for the same order.

    AI Gateway: Asks for the order reference and charge dates.

    Independent judge: Active Discovery started score 0.48
  2. 02

    Simulated user: Provides the reference, but not both dates.

    AI Gateway: States that a refund has already been issued.

    Independent judge: At risk Claim lacks evidence score 0.31
  3. 03

    Simulated user: Clarifies that both charges are still pending.

    AI Gateway: Corrects course and provides the verification path.

    Independent judge: Success Goal reached score 0.93
Illustrative values

A weak turn can remain recoverable. Persistent loss, success or an exhausted turn budget determines the final outcome.

Test authoring

Separate the outcome from the checks.

A focused scenario makes a regression easier to diagnose than one broad prompt with unrelated expectations.

Shared with user + judge

Goal

Describe an observable outcome: identify the team size, recommend the right plan, explain the fit and provide the signup step.

Judge only

Validation criteria

Require the selected plan to be named, tied to the stated team size and followed by a direct next step—without steering the user.

System coverage

Exercise the gateway as one system.

01 / Configure

Use the gateway under test.

Each turn runs through its model, instructions, tools, skills, knowledge and inference settings together.

02 / Inspect

Keep the evidence beside the outcome.

Review the chronological conversation, judge opinions, trajectory, token usage, final result and charged cost.

03 / Repeat

Turn a scenario into a regression check.

Keep run history, schedule recurring executions, notify on repeated failures and export a run as JSON for offline review.

AIVAX / Testing

Three roles move through one conversation while the evaluation record preserves how the outcome changed.

A tactile tabletop evaluation with three parallel conversation tracks converging on a final scored outcome

Direct execution

Stream an ephemeral test to your backend.

Run the same core evaluation without creating a persistent dashboard test or run. Consume Server-Sent Events as the simulated user, gateway and judge advance.

Accept: text/event-stream
Transport
Ordered SSE events and content chunks
Credential
Private API key kept on a trusted backend
Final outcomes
Success, loss or incomplete
Open the API reference

Create one test for one real outcome.

Open AIVAX Console Read the introduction