Workflows

Workflows define sequences of tool calls that represent realistic usage patterns. They help test multi-step interactions and verify that tools work together correctly.

What Are Workflows?

A workflow is a YAML definition that describes:

• Steps: Sequential tool calls
• Argument Mapping: Data flow between steps using JSONPath
• Assertions: Verifications after each step
• State Tracking: Optional monitoring of state changes

Basic Structure

# my-workflow.workflow.yaml
id: file_lifecycle
name: File Lifecycle
description: Create, read, modify, and delete a file
expectedOutcome: File is created, modified, read, and deleted successfully
steps:
  - tool: write_file
    description: Create a new file
    args:
      path: /tmp/test-file.txt
      content: "Hello, World!"
    assertions:
      - path: "$.success"
        condition: truthy

  - tool: read_file
    description: Verify file contents
    args:
      path: /tmp/test-file.txt
    assertions:
      - path: "$.content"
        condition: equals
        value: "Hello, World!"

  - tool: delete_file
    description: Clean up
    args:
      path: /tmp/test-file.txt
    optional: true

Running Workflows

Workflows are configured in bellwether.yaml and can run in both check and explore modes.

From a YAML File

1. Create bellwether-workflows.yaml (or any filename).
2. Point config to it:

# bellwether.yaml
workflows:
  path: "./bellwether-workflows.yaml"

Then run:

bellwether check

Auto-Generate (Check Mode)

Check mode can auto-generate workflows from tool patterns and save them to output.dir:

# bellwether.yaml
workflows:
  autoGenerate: true

bellwether check

LLM-Based Discovery (Explore Mode)

Explore mode can ask the LLM to discover workflows:

# bellwether.yaml
workflows:
  discover: true

bellwether explore

State Tracking

Enable state tracking between steps:

# bellwether.yaml
workflows:
  trackState: true

Argument Mapping

Pass data between steps using JSONPath:

steps:
  - tool: create_user
    description: Create a new user
    args:
      name: "Test User"

  - tool: get_user
    description: Fetch the created user
    argMapping:
      id: "$steps[0].result.user_id"

JSONPath Expressions

Expression	Description
$steps[0].result	Result of first step
$steps[0].result.id	id field from first step
$steps[1].result.items[0]	First item from second step

Assertions

Verify step results:

assertions:
  - path: "$.status"
    condition: equals
    value: "active"
    message: "User should be active"

  - path: "$.items"
    condition: type
    value: array

  - path: "$.count"
    condition: truthy

Assertion Conditions

Condition	Description
exists	Value is present
truthy	Value is truthy
equals	Exact match
contains	Contains substring
type	Type check (string, array, object, number)

Example Workflows

Authentication Flow

id: auth_flow
name: Authentication Flow
steps:
  - tool: login
    args:
      username: testuser
      password: ${TEST_PASSWORD}
    assertions:
      - path: "$.session_token"
        condition: exists

  - tool: get_profile
    argMapping:
      token: "$steps[0].result.session_token"
    assertions:
      - path: "$.email"
        condition: exists

  - tool: logout
    argMapping:
      token: "$steps[0].result.session_token"

CRUD Operations

id: resource_crud
name: Resource CRUD
steps:
  - tool: create_resource
    args:
      name: "Test Resource"
    assertions:
      - path: "$.id"
        condition: exists

  - tool: update_resource
    argMapping:
      id: "$steps[0].result.id"
    args:
      name: "Updated Resource"

  - tool: get_resource
    argMapping:
      id: "$steps[0].result.id"
    assertions:
      - path: "$.name"
        condition: equals
        value: "Updated Resource"

  - tool: delete_resource
    argMapping:
      id: "$steps[0].result.id"
    optional: true

Optional Steps

Mark steps that can fail without failing the workflow:

steps:
  - tool: required_operation
    description: Must succeed

  - tool: cleanup
    description: Nice to have
    optional: true  # Workflow continues if this fails

Environment Variables

Workflow arguments are treated as literal values. Keep secrets in bellwether.yaml under server.env instead of embedding them in workflow files.

Best Practices

1. Name steps descriptively

   description: Create test user for order workflow

2. Use meaningful assertions

   assertions:
     - path: "$.order.status"
       condition: equals
       value: "confirmed"
       message: Order should be confirmed after payment

3. Clean up after workflows

   - tool: delete_test_data
     optional: true

4. Keep workflows focused - One workflow per user journey

See Also

• Workflow Authoring Guide - Advanced workflow patterns
• Drift Detection - Track workflow behavior changes
• check - Running workflows