📚 Spex Framework - Technical Reference

Following the "5 Types of Documentation" framework for comprehensive coverage

🎯 1. TUTORIALS (Learning-Oriented)

Getting Started with Spex

Your First Spex in 5 Minutes:

Add to your project:

# mix.exs
{:sexy_spex, path: "../spex", only: [:test, :dev]}

Create your first spex:

# test/spex/calculator_spex.exs
defmodule Calculator.BasicSpex do
use SexySpex

spex "calculator can add numbers" do
 scenario "adding two positive numbers" do
   given_ "two numbers", context do
     {:ok, Map.merge(context, %{a: 5, b: 3})}
   end

   when_ "we add them", context do
     {:ok, Map.put(context, :result, Calculator.add(context.a, context.b))}
   end

   then_ "we get the sum", context do
     assert context.result == 8
     {:ok, context}
   end
 end
end
end

Run it:
```
mix spex test/spex/calculator_spex.exs
```

Tutorial: GUI Testing with Scenic

Step-by-step guide for testing Scenic applications... (See README.md for complete GUI testing tutorial)

🛠 2. HOW-TO GUIDES (Problem-Oriented)

How to Test File Operations

spex "file save functionality" do
  scenario "save to new file" do
    given_ "unsaved content", context do
      ScenicMCP.send_text("My document content")
      {:ok, context}
    end

    when_ "user saves with Ctrl+S", context do
      ScenicMCP.send_key("s", [:ctrl])
      # Handle save dialog...
      {:ok, context}
    end

    then_ "file is saved successfully", context do
      # Verification logic...
      {:ok, context}
    end
  end
end

How to Test Visual Changes

spex "UI updates correctly" do
  scenario "theme change" do
    given_ "light theme is active", context do
      {:ok, before} = ScenicMCP.take_screenshot("light_theme")
      {:ok, Map.put(context, :before, before)}
    end

    when_ "user switches to dark theme", context do
      ScenicMCP.send_key("t", ["ctrl", "shift"])  # Toggle theme
      {:ok, context}
    end

    then_ "UI changes to dark theme", context do
      {:ok, dark} = ScenicMCP.take_screenshot("dark_theme")
      # Could add image comparison here
      {:ok, Map.put(context, :after, dark)}
    end
  end
end

How to Create Custom Adapters

defmodule MyApp.CustomAdapter do
  @behaviour SexySpex.Adapter
  
  def setup do
    # Initialize your testing environment
  end
  
  def take_screenshot(filename) do
    # Your screenshot implementation
  end
  
  # Implement other required callbacks...
end

📖 3. EXPLANATION (Understanding-Oriented)

Why Spex Exists

The Problem: Traditional testing often creates a gap between requirements, tests, and documentation. Tests become stale, documentation gets outdated, and requirements are lost in translation.

The Solution: Spex provides executable specifications - tests that are written in business language and serve as living documentation. They:

Express requirements in readable Given-When-Then format
Execute as actual tests to validate functionality
Generate visual evidence through screenshots
Remain synchronized with the codebase by necessity

AI-Driven Development: Spex is optimized for AI systems that can:

Understand requirements written in natural language
Generate executable specifications automatically
Run tests against live applications
Analyze results and iterate improvements

Architecture Philosophy

Adapter Pattern: Different testing environments need different approaches:

Default Adapter: Basic testing without external dependencies
ScenicMCP Adapter: GUI testing with visual feedback
Custom Adapters: Extensible for any testing scenario

Reporter System: Clean separation between test execution and output formatting allows:

Consistent visual output across different test types
Easy customization of reporting format
Integration with external reporting systems

DSL Design: The macro-based DSL provides:

Compile-time validation of test structure
Runtime flexibility for dynamic scenarios
Clean integration with ExUnit for familiar testing patterns

When to Use Spex vs ExUnit

Use Spex When:

Writing acceptance tests or integration tests
Need visual evidence (screenshots)
Testing GUI applications
Requirements need to be readable by non-developers
AI is involved in test generation or execution

Use ExUnit When:

Writing unit tests for pure functions
Testing internal implementation details
Performance is critical (minimal overhead needed)
No need for business-readable format

How the Spex Tag System Works

When you use SexySpex in a test module, it automatically adds @moduletag spex: true to the entire module. This leverages ExUnit's built-in tag filtering system:

defmodule MyApp.IntegrationSpex do
  use SexySpex  # This adds @moduletag spex: true automatically
  
  spex "user workflow" do
    # This creates a regular ExUnit test tagged with :spex
  end
end

Running Spex Tests:

mix test - Runs all tests EXCEPT those tagged with :spex (ExUnit excludes them by default)
mix test --include spex - Includes tests tagged with :spex in addition to regular tests
mix test --only spex - Runs ONLY tests tagged with :spex
mix spex - Custom task that starts the application and runs spex tests

This design allows spex tests to be excluded by default (since they may require special setup like GUI applications) while still being easily runnable when needed.

📋 4. REFERENCE (Information-Oriented)

Complete API Reference

Core Modules

`Spex`

Main entry point for the framework

__using__/1 - Macro to set up spex environment
setup/1 - Initialize adapters and configuration

`SexySpex.DSL`

Domain-specific language macros

spex/2 - Define a specification
- Parameters: name (string), opts (keyword list)
- Options: :description, :tags, :context, :fail_on_error_logs
scenario/2 - Define a test scenario within a spex
given_/1 - Execute a registered given by atom (see Reusable Givens below)
given_/3 - Inline precondition: given_ "desc", context do … end
when_/3 - Action: when_ "desc", context do … end
then_/3 - Outcome: then_ "desc", context do … end
and_/3 - Additional step: and_ "desc", context do … end
register_given/3 - Register a reusable given by name

Step return contract (all forms): every step block must return {:ok, context}. Bare :ok is not allowed. There is no map-merge or pass-through magic — the value returned becomes the next step's context. If a step doesn't change context, return {:ok, context} explicitly.

Reusable Givens

Registering givens:

defmodule MySpex do
  use SexySpex

  register_given :logged_in_user, context do
    user = %{id: 1, name: "Test"}
    {:ok, Map.put(context, :user, user)}
  end

  register_given :with_admin_role, context do
    {:ok, Map.put(context, :role, :admin)}
  end

  register_given :reset_database, context do
    MyApp.Repo.delete_all(MyApp.User)
    {:ok, context}
  end
end

register_given compiles each given to a public function def name(context), so they can be called from the same module or any module that imports it.

Using registered givens:

scenario "example" do
  given_ :logged_in_user
  given_ :with_admin_role
  given_ :reset_database
end

`SexySpex.Givens`

Module for creating shared given libraries

Use SexySpex.Givens for modules that only define reusable givens (no spex/scenario macros pulled in). Consume them via plain Elixir import:

defmodule MyApp.SharedGivens do
  use SexySpex.Givens

  register_given :test_user, context do
    {:ok, Map.put(context, :user, %{id: 1})}
  end
end

defmodule MyApp.SomeSpex do
  use SexySpex
  import MyApp.SharedGivens

  spex "..." do
    scenario "..." do
      given_ :test_user
    end
  end
end

Lookup order: Local definitions shadow imports — same as any Elixir function call.

Note on Step Types: given_, when_, then_, and and_ are functionally identical — they only differ in the label written by the reporter. Each macro:

Reports the step type and description to SexySpex.Reporter
Executes the block (via SexySpex.StepExecutor for manual/timed modes)
Validates the return value via SexySpex.Runtime.process_step_result/2
Threads the returned context into the next step

Manual Mode and Step Control

Important: Manual mode pauses between DSL blocks, not between individual lines of code within each block.

Execution Flow:

scenario "example flow" do
  given_ "setup", context do
    # All code here executes without pause
    line1()
    line2()
    line3()
    {:ok, context}
  end
  # PAUSE HAPPENS HERE in manual mode

  when_ "action", context do
    # All code here executes without pause
    action1()
    action2()
    {:ok, context}
  end
  # PAUSE HAPPENS HERE in manual mode

  then_ "verification", context do
    # All code here executes without pause
    assert1()
    assert2()
    {:ok, context}
  end
end

For Fine-Grained Control: Break actions into smaller DSL blocks:

# Instead of:
when_ "complex user interaction", context do
  send_text("Hello")      # No pause
  send_key("backspace")   # No pause
  send_text(" World")     # No pause
  {:ok, context}
end

# Use multiple blocks:
when_ "user types Hello", context do
  send_text("Hello")
  {:ok, context}
end
# Pause here

and_ "user corrects text", context do
  send_key("backspace")
  {:ok, context}
end
# Pause here

and_ "user completes with World", context do
  send_text(" World")
  {:ok, context}
end

`SexySpex.Reporter`

Output formatting and progress tracking

start_spex/2 - Begin reporting for a specification
spex_passed/1 - Report successful completion
spex_failed/2 - Report failure with error details
start_scenario/1 - Begin scenario reporting
scenario_passed/1 - Report scenario success
scenario_failed/2 - Report scenario failure
step/2 - Report individual Given-When-Then steps

Adapters

Adapter Architecture

You must explicitly specify an adapter - there is no default adapter. This ensures clear intent about your testing environment.

Required Adapter Functions:

defaults/0 - Returns default configuration map
setup/1 - Initialize adapter with configuration

`SexySpex.Adapters.ScenicMCP`

Scenic GUI testing adapter

setup/0 - Verify MCP server connection
app_running?/1 - Check TCP connection to MCP server
wait_for_app/2 - Wait for MCP server to be ready
execute_command/2 - Send commands via MCP protocol
send_text/1 - Send text input to application
send_key/2 - Send keyboard input with modifiers
take_screenshot/1 - Capture application screenshots
inspect_viewport/0 - Get application state information

Mix Tasks

`Mix.Tasks.Spex`

Command-line interface for running spex

Usage: mix spex [options] [files]

Options:

--only-spex - Run only spex tests (skip ExUnit)
--pattern PATTERN - File pattern to match (default: test/spex/*/_spex.exs)
--verbose - Show detailed output
--timeout MS - Test timeout in milliseconds
--help - Show help message

Examples:

mix spex                                    # Run all spex
mix spex test/spex/user_login_spex.exs     # Run specific file
mix spex --pattern "**/integration_*.exs" # Pattern matching
mix spex --verbose --timeout 120000       # Verbose with 2min timeout

Configuration Reference

Application Configuration:

config :sexy_spex,
  adapter: SexySpex.Adapters.ScenicMCP,   # Default: SexySpex.Adapters.Default
  screenshot_dir: "test/screenshots", # Default: "."
  port: 9999                          # Default: 9999 (for ScenicMCP)

Runtime Configuration:

# In test setup
Application.put_env(:sexy_spex, :adapter, SexySpex.Adapters.ScenicMCP)
Application.put_env(:sexy_spex, :screenshot_dir, "tmp/screenshots")

Error Reference

Common Errors and Solutions:

could not load spex.ex. Reason: enoent
- Cause: Wrong path in Code.require_file/2
- Solution: Check relative paths, use {:sexy_spex, path: ".."} in deps
No Scenic MCP server detected on port 9999
- Cause: Scenic application not running with MCP enabled
- Solution: Start app with iex -S mix, verify scenic_mcp in deps
Spex failed: module MySpex is not loaded
- Cause: Compilation errors in spex file
- Solution: Check syntax, ensure all modules are available

⚡ 5. TROUBLESHOOTING (Problem-Solving)

Common Issues

Spex Files Won't Load

# Error: could not load test/spex/my_spex.exs

Debugging Steps:

Check file syntax: elixir -c test/spex/my_spex.exs
Verify spex dependency: mix deps.get && mix deps.compile
Check file paths in Code.require_file/2
Ensure all required modules are available

ScenicMCP Connection Fails

# Error: No Scenic MCP server detected

Debugging Steps:

Verify app is running: ps aux | grep beam
Check port is open: lsof -i :9999
Test connection manually: telnet localhost 9999
Check scenic_mcp dependency in target app
Verify MCP server starts with app

Screenshots Not Generated

# Error: Screenshot file does not exist

Debugging Steps:

Check screenshot directory exists and is writable
Verify adapter configuration
Test with absolute paths
Check disk space availability
Review adapter implementation

Tests Pass but Features Don't Work

# Spex passes but feature is broken

Debugging Steps:

Add more granular assertions
Take screenshots at each step
Add viewport inspection calls
Test manually to verify expected behavior
Add visual validation (OCR, image comparison)

Performance Issues

Slow Test Execution

Reduce screenshot frequency
Optimize sleep/wait times
Run spex in parallel where possible
Use mocking for expensive operations

Memory Usage

Clean up screenshot files after tests
Avoid keeping large objects in test state
Use streaming for large data sets

Integration Issues

CI/CD Pipeline Integration

# Example GitHub Actions
- name: Run Spex Tests
  run: |
    mix deps.get
    mix spex --only-spex
    
- name: Archive Screenshots
  uses: actions/upload-artifact@v2
  with:
    name: spex-screenshots
    path: test/screenshots/

IDE Integration

Configure test runner to recognize .exs files in test/spex/
Set up screenshot viewer for test artifacts
Configure syntax highlighting for spex DSL

📚 Additional Resources

Hex Documentation: https://hexdocs.pm/spex
GitHub Repository: (Your repo URL)
Examples: See test/spex/ directory for working examples
Support: Create issues on GitHub for questions/bugs

This technical reference covers all aspects of the Spex framework from learning to troubleshooting, ensuring developers can effectively use spex for AI-driven development workflows.

← Previous Page Spex by Example

Next Page → Boundary Enforcement