Troubleshooting Guide¶

Common issues and solutions for OSA community onboarding and operations.

Quick Diagnostic Checklist¶

Before diving into specific errors:

1. Validate your config:

   uv run osa validate src/assistants/your-community/config.yaml
   

2. Check environment variables:

   macOSLinuxWindows

   echo $ANTHROPIC_API_KEY_YOUR_COMMUNITY
   # Should print your API key, not empty
   

   echo $ANTHROPIC_API_KEY_YOUR_COMMUNITY
   # Should print your API key, not empty
   

   echo $env:ANTHROPIC_API_KEY_YOUR_COMMUNITY
   # Should print your API key, not empty
   

   If your community funds itself through OpenRouter instead, the variable is named OPENROUTER_API_KEY_YOUR_COMMUNITY; everything below works the same way, against OpenRouter rather than Anthropic.

3. Test API key:

   uv run osa validate src/assistants/your-community/config.yaml --test-api-key
   

4. Check server status:

   uv run osa health
   

Configuration Validation Errors¶

Error: YAML Syntax Error¶

Symptom:

YAML syntax error at line 15, column 3: mapping values are not allowed here

Causes: - Incorrect indentation (must use spaces, not tabs) - Missing colon after key - Improper list formatting - Special characters not quoted

Solutions:

Check indentation:

# Wrong - using tabs
documentation:
    - title: My Doc

# Correct - using spaces
documentation:
  - title: My Doc

Quote special characters:

# Wrong - colon in unquoted string
description: HED: Event annotation

# Correct - quoted
description: "HED: Event annotation"

List formatting:

# Wrong - missing dash
cors_origins:
  https://example.com

# Correct - with dash
cors_origins:
  - https://example.com

Validation:

# Use a YAML validator
python -c "import yaml; yaml.safe_load(open('config.yaml'))"

Error: Community ID must be kebab-case¶

Symptom:

Community ID must be kebab-case (lowercase, hyphens): MyProject

Cause: Community ID contains uppercase letters, underscores, or invalid characters.

Solution:

# Wrong
id: MyProject
id: my_project
id: my.project
id: -myproject   # Leading hyphen

# Correct
id: myproject
id: my-project
id: my-project-2024

Rules: - Lowercase letters only - Numbers allowed - Hyphens allowed (not leading/trailing) - No underscores, dots, or spaces

Error: Invalid CORS origin¶

Symptom:

Invalid CORS origin 'example.com'. Must be a valid origin (e.g., 'https://example.org')

Cause: CORS origin missing scheme or improperly formatted.

Solution:

# Wrong - missing scheme
cors_origins:
  - example.com
  - www.example.com

# Wrong - includes path
cors_origins:
  - https://example.com/docs

# Correct
cors_origins:
  - https://example.com
  - https://www.example.com
  - https://*.example.com

Common Mistakes: - Forgetting https:// prefix - Including path (/docs) - Including query string (?foo=bar) - Using * alone (not allowed) - A wildcard on a shared hosting domain, such as https://*.pages.dev, which authorizes every site on that platform

Error: Preload requires source_url¶

Symptom:

DocSource 'My Doc' has preload=True but no source_url

Cause: Document configured with preload: true but missing source_url field.

Solution:

# Wrong - preload without source_url
documentation:
  - title: Core Docs
    url: https://example.com/docs
    preload: true

# Correct - source_url provided
documentation:
  - title: Core Docs
    url: https://example.com/docs
    source_url: https://raw.githubusercontent.com/org/repo/main/docs.md
    preload: true

Why: Preloaded documents are fetched and embedded in the system prompt. The source_url must point to the raw markdown or text content.

Error: Repository must be in 'org/repo' format¶

Symptom:

Repository must be in 'org/repo' format, got: hed-specification

Cause: GitHub repository missing organization prefix.

Solution:

# Wrong - missing org
github:
  repos:
    - hed-specification

# Correct - org/repo format
github:
  repos:
    - hed-standard/hed-specification

Error: Invalid DOI format¶

Symptom:

Invalid DOI format (expected '10.xxxx/yyyy'): doi.org/10.1234/example

Cause: DOI includes URL prefix or doesn't match expected format.

Solution:

# Wrong - includes prefix
citations:
  dois:
    - https://doi.org/10.1234/example
    - doi.org/10.1234/example

# Correct - DOI only
citations:
  dois:
    - 10.1234/example

Valid DOI Format: - Must start with 10. - Format: 10.xxxx/yyyy - No URL prefixes

API Key Issues¶

Warning: API key env var not set¶

Symptom:

ANTHROPIC_API_KEY_MYPROJECT not set
Validation passed with warnings

Impact: - Assistant will fall back to platform API key - Costs billed to platform, not your community - Shared rate limits apply

Solution:

For local testing:

# Add to shell profile
echo 'export ANTHROPIC_API_KEY_MYPROJECT="sk-ant-..."' >> ~/.zshrc
source ~/.zshrc

# Verify
echo $ANTHROPIC_API_KEY_MYPROJECT

# Add to shell profile
echo 'export ANTHROPIC_API_KEY_MYPROJECT="sk-ant-..."' >> ~/.bashrc
source ~/.bashrc

# Verify
echo $ANTHROPIC_API_KEY_MYPROJECT

# Set for current session
$env:ANTHROPIC_API_KEY_MYPROJECT = "sk-ant-..."

# Set permanently (persists across sessions)
[Environment]::SetEnvironmentVariable("ANTHROPIC_API_KEY_MYPROJECT", "sk-ant-...", "User")

# Verify
echo $env:ANTHROPIC_API_KEY_MYPROJECT

For production (server):

# Add to .env file
echo 'ANTHROPIC_API_KEY_MYPROJECT="sk-ant-..."' >> .env

Verify:

uv run osa validate src/assistants/myproject/config.yaml
# Should show: "✓ ANTHROPIC_API_KEY_MYPROJECT is set (Anthropic)"

The env var name must match ANTHROPIC_API_KEY_[A-Z0-9_]+, so that a config cannot point the assistant at an unrelated secret. A community whose config sets openrouter_api_key_env_var instead is reported the same way, as (OpenRouter), and its variable must match OPENROUTER_API_KEY_[A-Z0-9_]+. If a config sets both, the Anthropic key is the one that pays, and the one osa validate reports.

Error: API key test failed (401 Unauthorized)¶

Symptom:

Invalid API key (401 Unauthorized)

Causes: - API key is invalid or expired - Wrong key format - Key revoked in the Anthropic Console

Solution:

1. Verify key format:

   macOSLinuxWindows

   echo $ANTHROPIC_API_KEY_MYPROJECT
   # Should start with: sk-ant-
   

   echo $ANTHROPIC_API_KEY_MYPROJECT
   # Should start with: sk-ant-
   

   echo $env:ANTHROPIC_API_KEY_MYPROJECT
   # Should start with: sk-ant-
   

2. Check the key in the Anthropic Console:

3. Visit https://platform.claude.com/settings/keys
4. Verify key exists and is active

5. Check that the workspace it belongs to has not hit a spend limit

6. Generate new key if needed:

7. Go to https://platform.claude.com/settings/keys
8. Create new API key

9. Update environment variable

10. Test directly:

    curl https://api.anthropic.com/v1/models \
      -H "x-api-key: $ANTHROPIC_API_KEY_MYPROJECT" \
      -H "anthropic-version: 2023-06-01"
    # Should return 200 OK with model list
    

    A community key is used against Anthropic's own API rather than the platform's endpoint on AWS, because it is not authorized on the platform's workspace. That is the endpoint --test-api-key probes too, so the curl above and the validator agree.

    For an OpenRouter community key, the equivalent is:

    curl https://openrouter.ai/api/v1/models \
      -H "Authorization: Bearer $OPENROUTER_API_KEY_MYPROJECT"
    

Error: API key test failed (403 Forbidden)¶

Symptom:

API key lacks permissions (403 Forbidden)

Cause: API key doesn't have necessary permissions, or the account has no funds left.

Solution:

1. Check the account balance:
2. Visit https://platform.claude.com/settings/billing
3. Ensure the account has credit available

4. For an OpenRouter community key, check https://openrouter.ai/credits instead

5. Check key permissions:

6. A key scoped to one workspace cannot spend from another

7. Verify the key has access to the models you want to use

8. Add funds:

9. Top up the account, or raise the workspace spend limit
10. Test again afterwards

Runtime Errors¶

Error: CORS policy blocked¶

Browser Console:

Access to fetch at 'https://api.osc.earth/osa/...' from origin 'https://mysite.com'
has been blocked by CORS policy

Cause: Your website origin not listed in cors_origins config.

Solution:

1. Add your origin to config:

   cors_origins:
     - https://mysite.com
     - https://www.mysite.com  # Don't forget www variant
   

2. Redeploy assistant (config changes require restart)

3. Verify origin exactly matches:

   // In browser console
   console.log(window.location.origin)
   // Must match exactly (including https://)
   

Common Issues: - Forgot www subdomain variant - http:// vs https:// mismatch - Port number missing (e.g., :3000) - Trailing slash in config (remove it)

Error: Widget not loading¶

Symptom: Widget icon doesn't appear or widget doesn't open.

Causes: 1. Script not loaded 2. Wrong community ID 3. API endpoint unreachable 4. JavaScript errors

Diagnosis:

1. Check browser console (F12 -> Console):

   Look for errors like:
   - Failed to load osa-chat-widget.js
   - OSAChatWidget is not defined (see Script placement below)
   - Community 'xxx' not found
   

2. Verify script loads:

   <!-- Check this in your HTML -->
   <script src="https://demo.osc.earth/osa-chat-widget.js"></script>
   

3. Check network tab:

4. osa-chat-widget.js should load (200 OK)
5. API requests should succeed

Solutions:

Wrong community ID:

<!-- Wrong - ID doesn't match config -->
<script src="https://demo.osc.earth/osa-chat-widget.js"></script>
<script>
    OSAChatWidget.setConfig({
        communityId: 'wrong-id'  // Check this matches config.yaml
    });
</script>

<!-- Correct -->
<script src="https://demo.osc.earth/osa-chat-widget.js"></script>
<script>
    OSAChatWidget.setConfig({
        communityId: 'hed'  // Must match config.yaml id field
    });
</script>

There is no OSAWidget global and init() takes no arguments; all configuration goes through setConfig().

Script placement:

<!-- Wrong - setConfig() runs before the widget script defines OSAChatWidget -->
<head>
    <script>
        OSAChatWidget.setConfig({ communityId: 'hed' });
    </script>
    <script src="https://demo.osc.earth/osa-chat-widget.js"></script>
</head>

<!-- Correct - load the widget script first -->
<body>
    <script src="https://demo.osc.earth/osa-chat-widget.js"></script>
    <script>
        OSAChatWidget.setConfig({ communityId: 'hed' });
    </script>
</body>

The <script src="...osa-chat-widget.js"> tag must come before the inline setConfig() call, and must never carry defer or async unless paired with data-no-auto-init and a manual OSAChatWidget.init() call afterward; see Loading the Script Asynchronously.

API endpoint:

// Check if the widget's backend is reachable (production)
fetch('https://widget.osc.earth/osa/health')
    .then(r => r.json())
    .then(console.log);
// Should show: {status: "healthy"}

Error: Messages not getting responses¶

Symptom: Widget accepts input but shows loading spinner indefinitely.

Causes: 1. API key not configured 2. Network errors 3. Model timeout 4. Backend server down

Diagnosis:

1. Check browser network tab:
2. Look for failed API requests
3. Check response status codes

4. Look for timeout errors

5. Check API health: The widget's own traffic goes through https://widget.osc.earth/osa (the Cloudflare Worker proxy in front of the backend), so check that host first, not the backend it sits in front of:

   curl https://widget.osc.earth/osa/health
   

   https://api.osc.earth/osa/health reaches the FastAPI backend behind the worker directly, which is useful too, but a widget-specific issue (CORS, rate limiting, Turnstile) shows up at widget.osc.earth, not at api.osc.earth.

6. Check backend logs (if you have access):

   docker logs osa-prod
   

Solutions:

API key missing: - Verify anthropic_api_key_env_var is set on server, or openrouter_api_key_env_var for a community funded through OpenRouter - Check env var exists: echo $ANTHROPIC_API_KEY_XXX - Restart server after adding env var

Network errors: - Check firewall rules - Verify API endpoint accessible - Check DNS resolution

Timeouts: - May indicate model is slow or overloaded - Try claude-haiku-4-5, which is the faster of the two Claude models, or lower reasoning_effort (see below) - Check status.anthropic.com, or OpenRouter's status page for an OpenRouter community

Deployment Issues¶

Error: Config file not found¶

Symptom:

Error: Config file not found: src/assistants/myproject/config.yaml

Cause: Config file not in expected location or path incorrect.

Solution:

1. Check file exists:

   ls src/assistants/myproject/config.yaml
   

2. Verify directory structure:

   src/assistants/
   └── myproject/
       └── config.yaml
   

3. Check file name:

4. Must be exactly config.yaml
5. Lowercase, not Config.yaml

Error: Assistant not discovered¶

Symptom: After deployment, osa myproject ask "test" returns:

Error: Unknown command 'myproject'

Cause: Assistant not registered in registry.

Diagnosis:

1. Check discovery:

   uv run python -c "from src.assistants import registry; print([a.id for a in registry.list_all()])"
   

2. Check directory structure:

   ls src/assistants/
   # Should show: myproject/
   

Solution:

1. Ensure config.yaml exists:

   ls src/assistants/myproject/config.yaml
   

2. Restart server:

   # Discovery happens at startup
   uv run uvicorn src.api.main:app --reload
   

3. Check for validation errors:

   uv run osa validate src/assistants/myproject/config.yaml
   

Testing Issues¶

Error: Tests fail with import errors¶

Symptom:

ImportError: cannot import name 'validate' from 'src.cli.validate'

Cause: Module not in Python path or not installed.

Solution:

1. Sync dependencies:

   uv sync
   

2. Run tests from repo root:

   cd /path/to/osa
   uv run pytest tests/
   

3. Check Python path:

   uv run python -c "import sys; print(sys.path)"
   # Should include current directory
   

Error: Tests fail with fixture errors¶

Symptom:

fixture 'tmp_path' not found

Cause: Using old pytest version or fixture not available.

Solution:

1. Update pytest:

   uv sync --upgrade
   

2. Verify pytest version:

   uv run pytest --version
   # Should be >= 7.0
   

Performance Issues¶

Issue: Widget loads slowly¶

Symptoms: - Widget takes 3-5+ seconds to appear - First message slow to respond

Causes: 1. Large system prompt (too many preloaded docs) 2. Slow model 3. Network latency 4. Cold start (server sleeping)

Solutions:

Reduce preloaded docs:

# Before - 5 preloaded docs = slow
documentation:
  - title: Doc 1
    preload: true
  - title: Doc 2
    preload: true
  # ... 3 more preloaded

# After - 1-2 critical docs only
documentation:
  - title: Core Spec
    preload: true
  - title: API Ref
    preload: false  # Fetch on-demand

Use the faster model:

# Before - Sonnet 5.5 is the more capable of the two Claude models, and the slower
default_model: claude-sonnet-5-5

# After - Haiku 4.5, the platform default
default_model: claude-haiku-4-5

Think less:

# Every model with reasoning levels runs at high unless told otherwise
reasoning_effort: medium    # or low

Thinking costs latency before the first token, and every offered model runs at high by default. Lowering reasoning_effort is often a better first step than changing the model: on one NWB question GPT-6 Luna's median time to first text was about 5 seconds at high, 11 at xhigh and 48 at max. Haiku's level is a thinking budget (1,024, 2,048 or 4,096 tokens) and Sonnet decides per request, so a thinking-heavy question is slower on either than a lookup is. See reasoning_effort.

Check network:

# Test latency of the host the widget actually talks to
time curl https://widget.osc.earth/osa/health
# Should be < 1 second; https://api.osc.earth/osa is the FastAPI
# backend behind this worker, useful for isolating worker vs. backend latency

Issue: High API costs¶

Symptoms: - AWS Marketplace bill higher than expected - Usage exceeded budget

Causes: 1. claude-sonnet-5-5 where claude-haiku-4-5 would do 2. A high reasoning_effort (thinking tokens are billed as output, and every model runs at high unless set) 3. Long conversations (context accumulation) 4. Many users 5. Preloaded docs increasing prompt size 6. Prompt caching disabled, so the same system prompt is billed at full rate on every turn

Solutions:

Use the cheaper model:

# Sonnet 5.5: twice Haiku's rate, worth it for genuinely hard questions
default_model: claude-sonnet-5-5

# Haiku 4.5: the default, and enough for documentation Q&A
default_model: claude-haiku-4-5

# GPT-6 Luna on Amazon Bedrock: about a ninth of Haiku's input rate
default_model: openai.gpt-6-luna

Think less:

reasoning_effort: medium    # every model runs at high unless set

Reduce preloaded docs: - Each preloaded doc adds to every request - Move to on-demand retrieval - Keep preloaded docs minimal

Keep prompt caching on: - Cache reads bill at a tenth of the input rate, and the system prompt is the largest repeated part of a request - A cache write costs more than an uncached read once, then pays for itself on the second turn

Monitor usage: - GET /{community_id}/metrics reports total_estimated_cost from the same pricing table the platform bills against (admin or community key required) - Set up budget alerts in AWS - Track usage by community ID

Cost Comparison: | Model | Input (per 1M tokens) | Output (per 1M tokens) | Use Case | |-------|----------------------|------------------------|----------| | claude-haiku-4-5 | $1.00 | $5.00 | General Q&A, FAQ generation | | claude-sonnet-5-5 | $2.00 | $10.00 | Complex reasoning, long threads | | openai.gpt-6-luna | $0.11 | $0.55 | Documentation Q&A (Amazon Bedrock) | | qwen.qwen3-next-80b-a3b | $0.14 | $1.20 | Documentation Q&A (Amazon Bedrock) | | openai.gpt-oss-120b | $0.15 | $0.60 | Documentation Q&A (Amazon Bedrock) |

Opus is not offered here. Every offered model is priced at or below Sonnet 5.5, and the three Bedrock models at or below Haiku 4.5, so a community's cost is predictable and the fallback is always the cheaper one.

Widget Integration Issues¶

Issue: Widget appears but can't read page content¶

Symptom: Assistant says "I cannot access the current page content" when asked about page.

Cause: enable_page_context disabled or tool not available.

Solution:

# Enable page context tool
enable_page_context: true

Verify:

# Check config
grep enable_page_context src/assistants/myproject/config.yaml
# Should show: enable_page_context: true

Issue: Widget positioning problems¶

Symptom: Widget icon overlaps with page content or appears in wrong location.

Cause: CSS conflicts with your site's styles.

Solution:

Add custom CSS:

<style>
  /* Adjust widget position */
  .osa-chat-widget {
    bottom: 20px !important;
    right: 20px !important;
    z-index: 9999 !important;
  }
</style>

Check for conflicts:

// In browser console
console.log(getComputedStyle(document.querySelector('.osa-chat-widget')));

Documentation Sync Issues¶

Error: Documentation fetch fails¶

Symptom: Assistant says "I couldn't retrieve that documentation."

Causes: 1. source_url unreachable 2. GitHub rate limit 3. URL changed/moved

Diagnosis:

# Test URL directly
curl -I https://raw.githubusercontent.com/org/repo/main/docs.md
# Should return 200 OK

Solutions:

URL moved:

# Update to new URL
documentation:
  - title: My Doc
    url: https://newsite.org/docs
    source_url: https://raw.githubusercontent.com/org/repo/main/docs/newpath.md

GitHub rate limit: - Wait an hour for limit reset - Use GitHub token for higher limits - Move docs to CDN

HTTPS required:

# Wrong - HTTP not secure
source_url: http://example.com/docs.md

# Correct - HTTPS
source_url: https://example.com/docs.md

Getting Help¶

If you're still stuck after trying these solutions:

1. Check logs:

   # Local development
   uv run uvicorn src.api.main:app --reload
   # Watch for errors in output
   
   # Production
   docker logs osa-prod
   

2. Run health check:

   uv run osa health
   

3. Validate config again:

   uv run osa validate src/assistants/your-community/config.yaml --test-api-key
   

4. File an issue:

5. GitHub: https://github.com/OpenScience-Collective/osa/issues

6. Include:

   • Error message (full text)
   • Config file (sanitized, remove API keys!)
   • Steps to reproduce
   • Environment (OS, Python version)

7. Ask in discussions:

8. https://github.com/OpenScience-Collective/osa/discussions

Prevention Checklist¶

Before deploying to production:

• Config validated locally (osa validate)
• API key tested (--test-api-key)
• CORS origins match production domains
• Documentation URLs verified (returns 200 OK)
• Widget tested on actual website
• Costs estimated based on expected usage
• Monitoring/alerts configured
• Team trained on troubleshooting basics

Common Error Messages Reference¶

Quick lookup table: