Signals

Signals are custom metrics and feedback tied to your traces. Use them to track user satisfaction, quality scores, and any custom metrics relevant to your AI application.

What Are Signals?

Signals let you record:

User feedback - Thumbs up/down, star ratings
Quality metrics - Relevance, accuracy, helpfulness scores
Custom metrics - Any numeric, boolean, or categorical value

Signals are linked to traces, allowing you to correlate feedback with specific AI interactions.

Signal Types

Type	Description	Example Values
`feedback`	Direct user feedback	thumbs up, thumbs down
`rating`	Numeric ratings	1-5 stars, 1-10 score
`sentiment`	Sentiment analysis	positive, negative, neutral
`quality`	Quality assessment	0.0-1.0 score
`completion`	Goal completion	true/false
`custom`	Any custom metric	varies

Signal Properties

Property	Type	Required	Description
`signalName`	string	Yes	Name of the signal
`value`	any	Yes	Signal value
`traceId`	string	No	Links to a trace
`spanId`	string	No	Links to specific span
`agentName`	string	No	Agent that generated signal
`signalType`	string	No	Type categorization
`source`	string	No	`user`, `system`, or `llm`
`confidence`	number	No	Confidence score (0-1)
`reasoning`	string	No	Explanation for the value
`metadata`	object	No	Additional context

Recording Signals

await foil.trace(async (ctx) => {
  // Do AI work...

  // Record user feedback
  await ctx.recordFeedback(true); // thumbs up

  // Record star rating
  await ctx.recordRating(4); // 4 out of 5 stars

  // Record custom signal
  await ctx.recordSignal('response_quality', 0.85, {
    signalType: 'quality',
    source: 'llm',
    confidence: 0.92
  });
});

Signal Sources

Source	Description
`user`	Direct user feedback
`system`	Automated system metrics
`llm`	LLM-based evaluation

Batch recording

Record multiple signals at once to reduce API calls:

await foil.recordSignalBatch([
  { signalName: 'relevance', value: 0.9, traceId: 'trace-123', signalType: 'quality' },
  { signalName: 'helpfulness', value: 0.85, traceId: 'trace-123', signalType: 'quality' },
  { signalName: 'goal_completed', value: true, traceId: 'trace-123', signalType: 'completion' }
]);

Full example: customer-support-agent — records feedback signals (thumbs up/down) on every conversation trace.

Best Practices

Use consistent signal names

Standardize names across your application for better aggregation:

user_rating not rating, stars, score
response_quality not quality, resp_qual

Include confidence for LLM signals

When using LLMs to evaluate, always include a confidence score.

Link signals to traces

Always include traceId to correlate signals with the AI interaction that generated them.

Getting Started

SDKs

Concepts

Features

Signals

Signals

What Are Signals?

Signal Types

Signal Properties

Recording Signals

Signal Sources

Best Practices

Next Steps

Agents

Analytics

Getting Started

SDKs

Concepts

Features

​Signals

​What Are Signals?

​Signal Types

​Signal Properties

​Recording Signals

​Signal Sources

​Best Practices

​Next Steps

Agents

Analytics

Signals

What Are Signals?

Signal Types

Signal Properties

Recording Signals

Signal Sources

Best Practices

Next Steps