Use this page to understand how Simulations are used in AI Studio.
Overview
Simulations are used to evaluate agent quality through simulated conversations.
Use Simulations before broad rollout to test whether AI Studio configuration behaves as expected across Specialists, Skills, Playbooks, Guardrails, Knowledge Base, Copilots, and Automations.
This is feature is currently being tested and will be available on bank.demo soon.
User flow
Simulations are mainly used by administrators, not day-to-day users.
Admins use Simulations to test whether AI configuration behaves as expected before users rely on it.
|
Admin action |
What happens |
|---|---|
|
Admin creates or selects a simulated conversation |
The AI configuration is tested against that scenario. |
|
Simulation runs |
The system produces an output using current configuration. |
|
Admin reviews the output |
Gaps or unexpected behavior can be identified. |
|
Admin adjusts configuration |
The scenario can be rerun after changes. |
How Simulations work
Where to find Simulations
To open Simulations:
-
Go to Admin.
-
Select AI Studio.
-
Under Evaluate, select Simulations.
Current screen state
The current Simulations screen shows an empty-state message:
Run simulated conversations to evaluate agent quality.
This indicates Simulations are intended to test AI behavior before users depend on it.
What Simulations may support
Simulations may support:
-
Testing Specialist behavior.
-
Testing Skills and output formats.
-
Testing Playbook logic.
-
Testing Guardrail behavior.
-
Testing Knowledge Base grounding.
-
Reviewing expected responses against sample prompts.
-
Comparing outputs before activation or rollout.
Admin configuration flow
When to use Simulations
Use Simulations before enabling or updating:
-
New Specialists.
-
High-impact Skills.
-
Playbooks used for executive reporting.
-
Guardrails that constrain user behavior.
-
Automations that run without manual prompting.
-
Copilots used in user-facing workflows.
-
Knowledge Base changes that affect cited answers.
Configuration guidance
Simulation prompts should reflect real user behavior.
Include:
-
Clear prompts.
-
Ambiguous prompts.
-
Edge cases.
-
Restricted requests.
-
Prompts that should use a specific Specialist, Skill, or Playbook.
-
Prompts that should not trigger a specific workflow.
Governance notes
Use Simulations as a quality gate before rollout. Do not rely only on configuration review. Test the actual behavior users are likely to see.
Next step
Related articles