Agent Parameters

Overview

Agent parameters control how your AI model generates responses. Understanding these settings helps you balance creativity, accuracy, and cost for your specific use case.

Core Parameters

Temperature

Temperature controls the randomness of the model's output.

Example Configuration:

jsonCopy
{
  "temperature": 0.3,
  "use_case": "customer_support"
}

Max Tokens

Maximum tokens limits the response length. One token is approximately 4 characters or 0.75 words.

Considerations:

• Higher limits increase response time and cost
• Set based on expected response needs
• Leave headroom for complete answers

Top P (Nucleus Sampling)

Top P controls diversity by limiting token selection to a probability mass.

Best Practice: Use either temperature OR top_p, not both. If using top_p, set temperature to 1.0.

Frequency Penalty

Reduces repetition by penalizing tokens based on how often they appear.

Presence Penalty

Encourages discussing new topics by penalizing tokens that have appeared at all.

Model-Specific Parameters

OpenAI (GPT-4, GPT-3.5)

jsonCopy
{
  "model": "gpt-4-turbo",
  "temperature": 0.7,
  "max_tokens": 1024,
  "top_p": 1.0,
  "frequency_penalty": 0.0,
  "presence_penalty": 0.0,
  "stop": ["\n\n", "END"]
}

Anthropic (Claude)

jsonCopy
{
  "model": "claude-3-5-sonnet",
  "temperature": 0.7,
  "max_tokens": 1024,
  "top_p": 0.9,
  "top_k": 40
}

Claude-specific:

• top_k: Limits to top K most likely tokens (default: 40)

Google (Gemini)

jsonCopy
{
  "model": "gemini-pro",
  "temperature": 0.7,
  "max_output_tokens": 1024,
  "top_p": 0.9,
  "top_k": 40
}

Local Models (Ollama)

jsonCopy
{
  "model": "llama3:70b",
  "temperature": 0.7,
  "num_predict": 1024,
  "top_p": 0.9,
  "top_k": 40,
  "repeat_penalty": 1.1
}

Ollama-specific:

• num_predict: Equivalent to max_tokens
• repeat_penalty: Controls repetition (default: 1.1)

Parameter Presets

Arcanflows provides presets for common use cases:

Precise (Factual/Code)

jsonCopy
{
  "temperature": 0.1,
  "max_tokens": 2048,
  "top_p": 0.95,
  "frequency_penalty": 0.0,
  "presence_penalty": 0.0
}

Balanced (General)

jsonCopy
{
  "temperature": 0.5,
  "max_tokens": 1024,
  "top_p": 1.0,
  "frequency_penalty": 0.3,
  "presence_penalty": 0.3
}

Creative (Writing/Brainstorming)

jsonCopy
{
  "temperature": 0.9,
  "max_tokens": 2048,
  "top_p": 0.95,
  "frequency_penalty": 0.5,
  "presence_penalty": 0.5
}

Use Case Recommendations

Configuring Parameters in Arcanflows

Via UI

1. Go to Agents → Select your agent
2. Click Settings → Model Configuration
3. Adjust parameters using sliders or input fields
4. Click Save

Via API

bashCopy
curl -X PATCH "https://api.arcanflows.com/api/v1/agents/{agent_id}" \
  -H "X-API-Key: your_api_key" \
  -H "Content-Type: application/json" \
  -d '{
    "model_config": {
      "temperature": 0.5,
      "max_tokens": 1024,
      "top_p": 1.0
    }
  }'

Testing Parameters

A/B Testing

Compare different parameter configurations:

1. Create agent variants with different settings
2. Route traffic to each variant
3. Measure response quality and user satisfaction
4. Select optimal configuration

Evaluation Metrics

• Response quality: Manual review or automated scoring
• Response time: Latency measurements
• Cost: Token usage tracking
• User satisfaction: Feedback and ratings

Best Practices

1. Start conservative: Begin with low temperature (0.3-0.5), adjust up if needed
2. Match use case: Factual tasks need low temperature, creative tasks need higher
3. Monitor costs: Higher max_tokens increases API costs
4. Test thoroughly: Different prompts may need different settings
5. Document choices: Record why specific settings were chosen
6. Review regularly: Revisit settings as use cases evolve

Troubleshooting

Responses too repetitive

• Increase frequency_penalty (0.3-0.7)
• Increase presence_penalty (0.3-0.7)
• Slightly increase temperature

Responses too random/off-topic

• Decrease temperature (0.1-0.3)
• Decrease top_p (0.8-0.95)
• Improve system prompt with constraints

Responses cut off

• Increase max_tokens
• Instruct model to be concise in system prompt

Slow responses

• Decrease max_tokens
• Use faster model variant
• Optimize system prompt length