Traces Guide
Correlate capitan signal pairs into OTEL spans.
Basic Correlation
A span is created by matching start and end signals via a correlation key:
requestStarted := capitan.NewSignal("request.started", "Request started")
requestCompleted := capitan.NewSignal("request.completed", "Request completed")
requestID := capitan.NewStringKey("request_id")
schema := aperture.Schema{
Traces: []aperture.TraceSchema{
{
Start: "request.started",
End: "request.completed",
CorrelationKey: "request_id",
SpanName: "http_request",
},
},
}
ap, _ := aperture.New(cap, logProvider, meterProvider, traceProvider)
ap.Apply(schema)
// Start span
cap.Emit(ctx, requestStarted, requestID.Field("REQ-123"))
// ... request processing ...
// End span
cap.Emit(ctx, requestCompleted, requestID.Field("REQ-123"))
The correlation key value REQ-123 links the start and end events.
Note: The correlation key must reference a StringKey. String values provide reliable correlation matching.
Span Attributes
Fields from both start and end events become span attributes:
methodKey := capitan.NewStringKey("method")
statusKey := capitan.NewIntKey("status")
durationKey := capitan.NewDurationKey("duration")
// Start event fields
cap.Emit(ctx, requestStarted,
requestID.Field("REQ-123"),
methodKey.Field("GET"),
)
// End event fields
cap.Emit(ctx, requestCompleted,
requestID.Field("REQ-123"),
statusKey.Field(200),
durationKey.Field(150*time.Millisecond),
)
// Span includes: method="GET", status=200, duration=150000000
Span Timeout
Configure maximum time to wait for an end event:
schema := aperture.Schema{
Traces: []aperture.TraceSchema{
{
Start: "request.started",
End: "request.completed",
CorrelationKey: "request_id",
SpanName: "http_request",
SpanTimeout: "5m", // Default: 5m
},
},
}
If end doesn't arrive within timeout:
- Span is ended with timeout status
- Memory for pending span is released
Timeout values use Go duration syntax: 5m, 30s, 1h, 500ms.
Concurrent Spans
Multiple spans can be in-flight simultaneously:
// Multiple requests overlapping
cap.Emit(ctx, reqStarted, requestID.Field("REQ-001"))
cap.Emit(ctx, reqStarted, requestID.Field("REQ-002"))
cap.Emit(ctx, reqStarted, requestID.Field("REQ-003"))
// End in any order
cap.Emit(ctx, reqCompleted, requestID.Field("REQ-002")) // REQ-002 span ends
cap.Emit(ctx, reqCompleted, requestID.Field("REQ-001")) // REQ-001 span ends
cap.Emit(ctx, reqCompleted, requestID.Field("REQ-003")) // REQ-003 span ends
Each span tracks independently via correlation value.
Out-of-Order Events
Aperture handles out-of-order event delivery gracefully.
Why This Happens
Capitan processes observers per-signal, not globally. Each signal has its own execution queue, so observers for request.completed may execute before observers for request.started—even if the start was emitted first. This is a feature, not a bug: it prevents slow observers on one signal from blocking others.
Why It Doesn't Matter
Capitan events capture their timestamp at emission time, not delivery time. Aperture uses these emission timestamps when creating spans, so delivery order is irrelevant to span accuracy.
How It Works
If the end event arrives before the start:
- End event data is stored (correlation ID, timestamp, context)
- When start arrives, both timestamps are used to create the span
- Span duration is calculated correctly from start to end
// End delivered before start (different signal queues)
cap.Emit(ctx, reqCompleted, requestID.Field("REQ-123")) // Stored, waiting for start
cap.Emit(ctx, reqStarted, requestID.Field("REQ-123")) // Span created with correct timestamps
// Result: span with accurate start time, end time, and duration
This design ensures trace accuracy regardless of observer execution order.
Missing Correlation Key
If an event lacks the correlation key:
- Event is logged (if log config allows)
- Trace correlation is skipped
- Warning logged about missing key
// Missing requestID field - no span correlation
cap.Emit(ctx, reqStarted) // Logged, but no span started
Multiple Trace Configurations
Different signal pairs can create different spans:
schema := aperture.Schema{
Traces: []aperture.TraceSchema{
{
Start: "http.request.started",
End: "http.request.done",
CorrelationKey: "request_id",
SpanName: "http_request",
},
{
Start: "db.query.started",
End: "db.query.done",
CorrelationKey: "query_id",
SpanName: "db_query",
},
{
Start: "cache.op.started",
End: "cache.op.done",
CorrelationKey: "cache_key",
SpanName: "cache_op",
},
},
}
Span Context Propagation
Spans inherit trace context from the event context:
// Parent span (created via direct OTEL)
ctx, parentSpan := ap.Tracer("orders").Start(ctx, "process-order")
defer parentSpan.End()
// Child span (created via signal correlation)
cap.Emit(ctx, paymentStarted, paymentID.Field("PAY-123"))
// ... payment processing ...
cap.Emit(ctx, paymentCompleted, paymentID.Field("PAY-123"))
// ^ This span is a child of process-order
Context Extraction for Traces
Add context values as span attributes:
type ctxKey string
const userIDKey ctxKey = "user_id"
ap.RegisterContextKey("user_id", userIDKey)
schema := aperture.Schema{
Traces: []aperture.TraceSchema{
{
Start: "request.started",
End: "request.completed",
CorrelationKey: "request_id",
SpanName: "request",
},
},
Context: &aperture.ContextSchema{
Traces: []string{"user_id"},
},
}
ap.Apply(schema)
ctx = context.WithValue(ctx, userIDKey, "user-456")
cap.Emit(ctx, reqStarted, requestID.Field("REQ-789"))
// ^ Span includes user_id="user-456" attribute
Using Tracer Directly
Access the underlying OTEL tracer for manual spans:
tracer := ap.Tracer("orders")
// Manual span creation
ctx, span := tracer.Start(ctx, "custom-operation")
defer span.End()
// Span attributes
span.SetAttributes(attribute.String("order_id", "ORD-123"))
// Events within span
span.AddEvent("payment-processed")
Signal correlation and direct tracer use work together.
Schema Configuration
Via YAML:
traces:
- start: request.started
end: request.completed
correlation_key: request_id
span_name: http-request
span_timeout: 5m
- start: db.query.started
end: db.query.completed
correlation_key: query_id
span_name: db-query
span_timeout: 30s
Load and apply:
configBytes, _ := os.ReadFile("config.yaml")
schema, _ := aperture.LoadSchemaFromYAML(configBytes)
ap.Apply(schema)