Technical correlation and business correlation

Both phrases get used to mean "tie these calls together", and they are treated as interchangeable more or less everywhere. They are not. They answer different questions, they cost different amounts, and they fail in different places. Choosing the wrong one is how an investigation ends with a chain that looks complete and is not.

The question decides which one you need

Almost every incident starts as a sentence about a thing, not about a request. Order 4821 was charged twice. This customer's export came back empty. The payment settled but the notification never went out. Nobody opens a ticket about trace 4bf92f3577b34da6a3ce929d0e0e4736.

A distributed trace answers a narrower question: what did this one request do, from the moment it entered the system. That is a genuinely useful question and often the right first one. It is just not the question that was asked, and the distance between the two is where most of the time goes.

Technical correlation

A technical correlation identifier exists for no reason other than to link calls. W3C traceparent, B3, x-request-id, x-correlation-id: an opaque token minted at the edge and passed along by each hop, carried in a header, a metadata entry, a record header or a query comment.

What it costs to consume is nothing, which is why it feels free. What it costs to produce is instrumentation on every hop that has to appear in the chain, and that bill is paid by whoever owns each service. The chain is exactly as complete as the coverage, and no more.

Where it stops, in practice:

• Protocols with nowhere to put it. Redis RESP has no header slot and no instrumentation puts context there. The Redis hop is simply absent from the trace, which reads as "the cache was not involved" rather than "the cache was not visible".
• Systems you do not own. The vendor appliance, the payment provider, the mainframe adapter, the binary whose build nobody can reproduce. Adding an SDK is not on the table.
• Asynchronous and deferred work. Context propagation across a queue is possible and frequently not done. A value written on Monday and picked up by a batch on Thursday has no live trace to belong to.
• Sampling. Tracing backends drop most traces on purpose. You discover you needed a particular one after it was discarded.
• Meaning, over time. A trace id is a fact about a request and nothing else. Six weeks later, in an audit or a customer dispute, nobody has one.

Business correlation

A business correlation identifier is one your organisation already uses: an order reference, a customer id, an IBAN, a policy number, a site code. It is not added to the traffic - it is already in the traffic, because the services are exchanging it in order to do their job. It sits in an HTTP body, a protobuf field, a Redis key, a SQL parameter, a Kafka record.

Getting it out is extraction, not instrumentation. You describe where the value lives on each protocol, once, in configuration, and the tool pulls it off the wire from then on. That is real work and it is worth being honest about it: somebody has to know that the order id is the orders.id column here and message.order.id there.

In exchange:

• It reaches uninstrumented hops. A hop only has to carry the value, not to have been changed. The datastore, the legacy service and the third party all correlate.
• It crosses time. The same value in a request today and in a batch job on Thursday is the same value. There is no live context to lose.
• It is the identifier the question used. The ticket, the support call, the regulator and the customer all speak in order references.
• It aggregates. "Every call that touched this customer this week" is a query. "Every trace belonging to this customer" generally is not.

It has one real limit, and it is the mirror of the technical one: a hop that carries only an internal surrogate key, and never the business value, cannot be linked on that value. The difference is that this failure is visible - there is no tag on that communication - rather than quietly producing a shorter chain that looks whole.

They slice the system in different directions

This is the part that is usually missed, and it is not a matter of coverage or effort. Even with perfect instrumentation on every hop, the two identifiers cut the same traffic along different axes, and one axis cannot be derived from the other.

A trace is a slice of time through the system. It groups whatever happened to be causally downstream of one entry point, within one request's lifetime. A business identifier is a slice through an object's life. It groups everything that ever touched one order, across requests that have nothing technically to do with each other and may be days apart.

The bulk call

Take the sharpest case. A nightly synchronisation pulls five hundred orders in a single call, or one Kafka batch carries two hundred events. That is one request, so it carries one trace id. On a trace you can see that the sync ran, how long it took and what it called next - but every one of those five hundred orders is folded into the same span. The trace cannot tell you what happened to yours, because at that hop your order stopped being a thing and became a row.

Filter on the order reference instead and the bulk call is right there in the chain, alongside the four hundred and ninety-nine others it also carried, because the identifier is in the payload it moved. Adding a per-object trace id to that call is not an option either: there is no such thing. One request cannot carry five hundred trace contexts.

Background work and events

The same thing happens every time work leaves the request that triggered it. An order is created by a web request, amended by a support agent an hour later, settled by a payment webhook the next morning, picked up by a reconciliation job that night, and archived by a retention sweep a month after that. Five entry points, five traces, no technical relationship between any of them - and one order, which is what the question was about.

Trace context can be propagated across an asynchronous boundary and sometimes is. What it cannot do is survive the object being stored and re-read: once the row is written, the next thing to touch it arrives with its own context, or with none. The business identifier is written into the row, so it is still there when the batch job reads it back.

This is why "we already have distributed tracing" does not close the question. Tracing answers what a request did. It does not answer what happened to a thing, and most of what an organisation needs to reconstruct - a disputed charge, a duplicated shipment, a record that drifted - is about a thing.

Side by side

Why not simply instrument everything

Because instrumentation is a change to production code, and extraction is a change to a configuration file. That difference decides who has to be persuaded, how long it takes, and whether it is possible at all on the hop that matters. The systems that most often sit in the middle of a hard investigation are precisely the ones nobody is willing to rebuild.

There is a tempting middle path - inject the order id as a header on every hop and treat it as trace context - and it is the worst of both. You take on the full instrumentation burden to obtain something extraction already gives you, and you still do not cover the hop you could not change in the first place.

How Spider Analyzer does both

Spider has one mechanism underneath, a tag: a value lifted out of a decoded communication and stored with it, searchable and aggregatable whatever protocol carried it. Technical and business correlation are the same mechanism pointed at different places.

Technical correlation ships pre-written. A fresh install extracts W3C traceparent, x-request-id and x-correlation-id on HTTP, gRPC, Kafka record headers and sqlcommenter PostgreSQL comments, with no rules written. It captures the stable 32-hex trace id out of a traceparent rather than the whole string, which is the detail that decides whether a chain links at all. Redis is not correlated this way and will not be.

Business correlation is the original design and reaches all five decoded protocols, Redis included, because the value is read from the payload rather than from a header somebody added. Both kinds of tag get the same gesture in the product: one click on a communication opens the whole chain across protocols, and which tag names count as a chain identifier is a display setting - so pointing it at a business identifier lights up traffic captured months ago, retroactively, with no re-capture.

Because Spider captures the packets and decrypts them with keys lifted from process memory rather than reading plaintext out of a TLS library, correlation happens over the full record: the handshake, the certificate and the connection that failed before any application data moved are all still there next to the requests. How the decryption works →

What we recommend

1. Turn on technical correlation and leave it on. It costs nothing, and on day one it proves which services actually talk to each other rather than which ones the diagram says do.
2. Pick the two or three identifiers your organisation argues about in tickets. Not ten - the ones that appear in the sentence when something goes wrong.
3. Write the extraction rules for those, per protocol. This is the ten minutes that turns a request tracer into something that answers the question the business asked.
4. Read the gap between the two chains. Where the business chain is longer, that is a hop your tracing never saw, and it is usually worth knowing about.

See it on live traffic

The Spider demo runs a synthetic shop where both identifiers ride along on the same orders: a traceparent injected by real OpenTelemetry instrumentation, and the order UUID pulled out of wherever it already lives on each hop. The first reaches four protocols with nothing configured. The second reaches all five, because somebody spent ten minutes describing where the order id sits. The Redis hop is the difference, and it is left visible on purpose. Open the live demo →

Related

• Reconstructing an incident days after it happened
• How Spider compares to Kubeshark, Pixie, Hubble and DeepFlow
• Capturing and replaying traffic between services
• Real incidents Spider diagnosed in production