Instrumenting license verification with OpenTelemetry
You instrument every database call and outbound HTTP request in your app, but the one dependency that decides whether the software is allowed to run — the license check — is usually invisible until a customer opens a ticket. This how-to wires a Keyright verification into OpenTelemetry the way you would any critical dependency: a Meter with a verify counter, a duration histogram and an expiry gauge; an ActivitySource span around the refresh path that records whether the answer came online, from cache, or from the grace window; the low-cardinality tags that make a dashboard and the high-cardinality ones (the key, the licensee, the machine id) that must never become label values; and the OTLP wiring and the four alerts worth paging on.
You have an ActivitySource around your outbound HTTP calls, a histogram on your database round-trips, and a dashboard that lights up the moment your payment provider gets slow. Then look at the one call that decides whether your software is allowed to run tonight — the license verification — and it is almost always a blind spot. It fails, it slows down, it quietly starts serving every customer from a cached lease because your Keyright instance is briefly unreachable, and the first anyone hears of it is a support ticket.
A license check is a dependency call. It deserves the same treatment as any other: a metric you can alert on and a span you can read when one activation is mysteriously slow. This post wires a Keyright verification into OpenTelemetry end to end — and, just as importantly, keeps the one secret involved out of your telemetry backend.
Two signals, two jobs
Metrics and traces answer different questions, and you want both.
- Metrics answer “is the fleet healthy right now?” They are cheap, pre-aggregated counters and histograms: how many verifications succeeded, how many were denied, how long they took, how close the active license is to expiry. This is what a dashboard and an alert rule read.
- Traces answer “why was this one slow or wrong?” A span around a single verification carries its timing, its outcome, and whether the answer came from the service or from a cached lease — the detail you need the one time a customer in a particular region reports a stall.
The temptation is to lean on logs for all of it. Logs are the wrong tool for the first question: you cannot cheaply ask “what fraction of verifications in the last five minutes were denied?” of a log aggregator, but a counter answers it for free.
A Meter and its instruments
Everything starts with one Meter and the instruments hung off it. Declare them once, statically, so there is a single place that owns the names and units.
using System.Diagnostics;
using System.Diagnostics.Metrics;
using Keyright; // LicenseInfo, LicenseStatus, RefreshStatus
public static class LicenseTelemetry
{
public const string MeterName = "Delta1.Licensing";
public const string ActivitySourceName = "Delta1.Licensing";
private static readonly Meter Meter = new(MeterName, "1.0.0");
internal static readonly ActivitySource Activity = new(ActivitySourceName, "1.0.0");
// How many verifications, tagged by status/outcome/source.
private static readonly Counter<long> VerifyCount =
Meter.CreateCounter<long>("keyright.verify.count",
unit: "{verification}",
description: "License verifications, tagged by status and source.");
// How long each verification took.
private static readonly Histogram<double> VerifyDuration =
Meter.CreateHistogram<double>("keyright.verify.duration",
unit: "ms",
description: "Wall time spent verifying a license.");
// How close the active license is to expiry (an async gauge, polled by the SDK).
private static volatile int _daysRemaining = -1;
static LicenseTelemetry() =>
Meter.CreateObservableGauge("keyright.license.days_remaining",
() => _daysRemaining,
unit: "d",
description: "Days until the active license expires; -1 when unknown.");
public static void SetDaysRemaining(int days) => _daysRemaining = days;
}
Three instruments, three questions. The Counter<long> is the rate of verifications and, once tagged, the rate of denials. The Histogram<double> is latency — and because a verification may hit the network, its tail is the thing you watch. The ObservableGauge is a pull-based reading: OpenTelemetry calls the callback at collection time, so you never emit a point unless a scrape asks for one, which is exactly right for a slow-moving value like “days until expiry.”
Recording a verification
Keyright’s Validate() returns a LicenseInfo whose Status is a bounded enum (Valid, Expired, Revoked, SignatureInvalid, MachineMismatch, ClockTampered, NoLicense) and whose DaysRemaining / IsExpiringSoon you already lean on elsewhere. The refresh path, through RefreshAsync, tells you where the answer came from — the service, a 304-not-modified cache hit, or a cached lease served because the service was unreachable. That provenance is the single most useful tag you can attach.
public static void RecordVerify(LicenseInfo info, double elapsedMs, string source)
{
// Keep this list SHORT and BOUNDED — see the cardinality rule below.
var tags = new TagList
{
{ "status", info.Status.ToString() }, // 7 values
{ "outcome", info.Status == LicenseStatus.Valid ? "allow" : "deny" }, // 2 values
{ "source", source }, // online | cache | grace
{ "expiring_soon", info.IsExpiringSoon } // true | false
};
VerifyCount.Add(1, tags);
VerifyDuration.Record(elapsedMs, tags);
SetDaysRemaining(info.DaysRemaining);
}
The tags you keep, and the ones that will cost you
A metric’s cost is not the number of points — it is the number of distinct tag combinations, because each one is a separate time series your backend stores and indexes. Four bounded tags like the ones above multiply out to a few dozen series: fine. Add one unbounded tag and the series count explodes with your customer base. This is called a cardinality blowout, and a license check is a minefield for it, because the most obviously relevant fields are exactly the ones you must not use as labels.
- The license key is a bearer secret. It must never be a tag value — that ships the key straight into your metrics store and your traces backend, the same mistake the logging guidance warns about, just in a different pipe.
- The licensee name, the machine fingerprint, and any key fingerprint are unbounded. One series per customer or per device defeats the point of a pre-aggregated metric and quietly runs up your observability bill.
The rule is simple: tags are for things with a small, fixed set of values. Identity goes on a span, where it is a one-off attribute on a single trace you sampled, not a label multiplied across every point forever.
Tracing the refresh path
Metrics tell you that denials rose; a span tells you why one verification behaved the way it did. Wrap the verification in an activity, time it off a monotonic Stopwatch (the wall clock can jump mid-measurement), and record the provenance as span attributes and events.
public async Task<LicenseInfo> VerifyAsync(string licenseKey, CancellationToken ct)
{
using Activity? span = LicenseTelemetry.Activity.StartActivity(
"license.verify", ActivityKind.Client);
var sw = Stopwatch.StartNew();
string source = "online";
// Pull the freshest lease we can; fall through to cache/grace on an outage.
RefreshResult refresh = await _client.RefreshAsync(ct);
source = refresh.Status switch
{
RefreshStatus.Refreshed => "online",
RefreshStatus.NotModified => "cache", // 304 — nothing changed
RefreshStatus.ServiceUnreachable => "grace", // served from a signed cached lease
_ => "online"
};
LicenseInfo info = _client.Validate(licenseKey);
sw.Stop();
// Identity and detail live on the span — one trace, not a metric label.
span?.SetTag("license.status", info.Status.ToString());
span?.SetTag("license.source", source);
span?.SetTag("license.days_remaining", info.DaysRemaining);
span?.SetTag("license.key_id", info.KeyId); // non-secret handle, if your payload carries one
span?.SetStatus(info.Status == LicenseStatus.Valid
? ActivityStatusCode.Ok
: ActivityStatusCode.Error);
if (source == "grace")
span?.AddEvent(new ActivityEvent("served-from-grace-window"));
LicenseTelemetry.RecordVerify(info, sw.Elapsed.TotalMilliseconds, source);
return info;
}
Note the division of labour. license.key_id — a non-secret handle, never the raw key — goes on the span, where it is a single attribute on a sampled trace, not a label stamped on a time series forever. The source == "grace" event is the breadcrumb that, months later, explains why a verification stayed green for an hour while the service was down.
Wiring the exporter
Register the meter and the source with the OpenTelemetry SDK and point them at your collector. The license instruments ride the same OTLP pipeline as the rest of your app; there is nothing license-specific about the transport.
builder.Services.AddOpenTelemetry()
.ConfigureResource(r => r.AddService("acme-designer"))
.WithMetrics(m => m
.AddMeter(LicenseTelemetry.MeterName)
.AddView("keyright.verify.duration", new ExplicitBucketHistogramConfiguration
{
// Buckets sized for a call that is usually local but sometimes hits the network.
Boundaries = new double[] { 1, 5, 10, 25, 50, 100, 250, 500, 1000 }
})
.AddOtlpExporter())
.WithTracing(t => t
.AddSource(LicenseTelemetry.ActivitySourceName)
.AddOtlpExporter());
The explicit histogram boundaries matter: the default buckets are tuned for sub-millisecond work, and they will bunch every network-bound verification into one overflow bucket where you can no longer see the tail. Size the buckets for the call you are actually making.
The four things worth alerting on
With the counter, histogram and gauge flowing, four rules cover the failure modes that actually page someone. Expressed loosely, backend-agnostic:
- Deny rate.
rate(keyright.verify.count{outcome="deny"})climbing is either a revocation that propagated, a bad clock on a fleet of machines, or a signing-key mismatch after a rotation. A sudden step is the signal. - Grace saturation. A nonzero and rising share of
source="grace"means your instance has been unreachable long enough that clients are living off cached leases. It is not customer-visible yet — which is exactly why you want the alert beforeMaxOfflineDurationruns out and it becomes an outage. - Verify latency tail. The p99 of
keyright.verify.durationis your early warning that the refresh round-trip is degrading, well before it turns into timeouts. - Days remaining.
keyright.license.days_remainingdropping below a threshold (say 7) is the fleet-wide version of the renewal nudge — one alert instead of one surprised customer per expiry.
The payoff
None of this changes what the license check decides — the gate still fails closed on a revoked or expired key. What changes is that the decision is no longer invisible. A verification is now a first-class dependency on your dashboard: you can see its rate, its latency, and the moment it quietly shifts from answering online to answering from grace. And because the design draws a hard line between bounded tags and identity, you get all of that observability without a single license key, licensee name, or machine id ever landing in your telemetry backend.
Try Nebula.NET
Harden your .NET code in minutes — start with the free edition.