Medium DevOps

Structured Logging and Request IDs Across Services: JSON Logs, Laravel Context and Middleware

Stop grepping interleaved text logs. Emit JSON events, stamp every line with one request ID, and carry it through queued jobs and service calls with Laravel Context.

17 Nov, 2026 9 min Updated 23 Oct, 2026 10 Code blocks
Diagram
graph LR A[Client] --> B[Load balancer] B -->|X-Request-Id| C[Laravel middleware] C --> D[Context request_id] D --> E[JSON logs] D --> F[Queued job] F --> E C -->|X-Request-Id header| G[Payments service] G --> E E --> H[Search by request_id]

A customer reports that checkout failed at 09:00. You open the logs and find two hundred "Payment failed" lines in that minute, interleaved with cart loads, webhook calls and queue workers from six containers. Which line is hers? Which "Gateway timeout" warning belongs to which failure? Did the confirmation job that ran three seconds later come from the same request? With plain-text logs, you grep, guess and give up.

Two habits fix this. Structured logs write every entry as a JSON object with named fields, so a log platform can filter on order_id or level instead of matching substrings. Correlation IDs stamp every entry caused by one incoming request with the same request_id, across the HTTP request, the jobs it queues and the services it calls. Together they turn "grep and guess" into one query.

From Text Lines to Events

This is what most Laravel apps log by default:

[2026-11-17 09:00:01] production.ERROR: Payment failed {"order":991}
[2026-11-17 09:00:01] production.INFO: Cart loaded
[2026-11-17 09:00:01] production.ERROR: Payment failed {"order":992}
[2026-11-17 09:00:02] production.WARNING: Gateway timeout

The order ID is buried in the message, the timeout has no link to either failure, and nothing tells you which request produced what. The same events as structured records look like this (one line per event, pretty-printing is for humans only):

{"message":"Payment failed","context":{"order_id":991,"gateway":"stripe","duration_ms":8012},"level_name":"ERROR","channel":"production","datetime":"2026-11-17T09:00:01.412+00:00","extra":{"request_id":"01JD2Z8Q4M7X","user_id":7,"route":"checkout.pay"}}

Now context holds facts about this specific event, and extra holds ambient facts about the request it happened in: the request ID, the user, the route. Every field is indexable. "All errors for user 7 in the last hour" and "every line for request 01JD2Z8Q4M7X" become trivial queries.

How the ID Flows

The ID is created once, at the edge, and then only copied:

graph LR A[Client] --> B[Load balancer] B -->|X-Request-Id| C[Laravel middleware] C --> D[Context request_id] D --> E[JSON logs] D --> F[Queued job] F --> E C -->|X-Request-Id header| G[Payments service] G --> E E --> H[Search by request_id]

If your load balancer or ingress already generates an ID (many do, often as X-Request-Id), reuse it so its access logs line up with your app logs. If not, the app generates one. Every outgoing call forwards it, and every service logs it under the same field name.

Emitting JSON Logs in Laravel

Monolog ships a JsonFormatter. Point a channel at stderr with it; in containers, stdout and stderr are what the platform collects:

// config/logging.php (excerpt)
use Monolog\Formatter\JsonFormatter;
use Monolog\Handler\StreamHandler;

return [
    'default' => env('LOG_CHANNEL', 'stack'),

    'channels' => [
        'stack' => [
            'driver' => 'stack',
            'channels' => explode(',', env('LOG_STACK', 'json')),
            'ignore_exceptions' => false,
        ],

        // One JSON object per line on stderr: the container runtime ships it,
        // the log platform (Loki, ELK, CloudWatch, Datadog) indexes the fields.
        'json' => [
            'driver' => 'monolog',
            'level' => env('LOG_LEVEL', 'info'),
            'handler' => StreamHandler::class,
            'handler_with' => ['stream' => 'php://stderr'],
            'formatter' => JsonFormatter::class,
        ],
    ],
];

Writing to stderr instead of storage/logs/laravel.log also removes a classic production problem: log files filling the container's disk because nothing rotates them.

Assigning the Request ID With Middleware and Context

Laravel 11 introduced the Context facade, and it is the right tool here. Anything you Context::add() is appended to every log record written during the request, and it is serialized with queued jobs so the worker's logs carry the same values. Log::withContext() still exists, but it only affects one log channel for the current request and does not travel to jobs.

namespace App\Http\Middleware;

use Closure;
use Illuminate\Http\Request;
use Illuminate\Support\Facades\Context;
use Illuminate\Support\Str;
use Symfony\Component\HttpFoundation\Response;

final class AssignRequestId
{
    public const HEADER = 'X-Request-Id';

    public function handle(Request $request, Closure $next): Response
    {
        $requestId = $this->resolve($request->headers->get(self::HEADER));

        // Context data is attached to every log record and travels into queued jobs.
        Context::add('request_id', $requestId);
        Context::add('route', $request->route()?->getName());
        Context::add('method', $request->method());

        $response = $next($request);
        $response->headers->set(self::HEADER, $requestId);

        return $response;
    }

    private function resolve(?string $incoming): string
    {
        $incoming = trim((string) $incoming);

        // Accept an upstream ID only if it is short and boring; never log arbitrary input.
        return preg_match('/^[A-Za-z0-9._-]{8,64}$/', $incoming) === 1
            ? $incoming
            : (string) Str::ulid();
    }
}

Register it first in the stack, so that even an authentication failure or a rate-limit rejection is logged with an ID:

// bootstrap/app.php
use App\Http\Middleware\AssignRequestId;
use Illuminate\Foundation\Application;
use Illuminate\Foundation\Configuration\Middleware;

return Application::configure(basePath: dirname(__DIR__))
    ->withRouting(web: __DIR__.'/../routes/web.php', api: __DIR__.'/../routes/api.php')
    ->withMiddleware(function (Middleware $middleware): void {
        // Prepend so even early failures (auth, throttling) are logged with the ID.
        $middleware->prepend(AssignRequestId::class);
    })
    ->create();

Validating the incoming header is not paranoia. The header comes from the client. Without the pattern check, someone can send a 10 KB value, or one with newlines and JSON fragments designed to forge extra log entries in line-based formats. If the value is not a short, boring token, replace it.

Propagating Across Jobs and Services

Inside the app, add facts once and log events with their own fields:

use App\Jobs\SendOrderConfirmation;
use Illuminate\Support\Facades\Context;
use Illuminate\Support\Facades\Log;

// In a controller or action, after authentication:
Context::add('user_id', $request->user()?->id);

// Per-call context: the facts about THIS event, as fields, not baked into the message.
Log::info('Order placed', [
    'order_id' => $order->id,
    'total_cents' => $order->total_cents,
    'items' => $order->items->count(),
]);

// The job is serialized with the current Context. When a worker runs it,
// its logs carry the same request_id as the HTTP request that queued it.
SendOrderConfirmation::dispatch($order->id);

// Context for this log channel only, for the rest of the request:
Log::withContext(['tenant_id' => $tenant->id]);

For calls to other services, forward the ID automatically instead of remembering it at each call site. Http::globalRequestMiddleware() applies to every request made through Laravel's HTTP client:

namespace App\Providers;

use Illuminate\Support\Facades\Context;
use Illuminate\Support\Facades\Http;
use Illuminate\Support\ServiceProvider;
use Psr\Http\Message\RequestInterface;

final class ObservabilityServiceProvider extends ServiceProvider
{
    public function boot(): void
    {
        // Every outgoing call made with the Http client forwards the current ID,
        // so the next service can log under the same request_id.
        Http::globalRequestMiddleware(function (RequestInterface $request): RequestInterface {
            $id = Context::get('request_id');

            return $id === null ? $request : $request->withHeader('X-Request-Id', $id);
        });
    }
}

The receiving service does the same dance in its own stack. In Node.js, AsyncLocalStorage plays the role of Laravel's Context: it keeps per-request data available across await boundaries without passing it through every function:

// payments-service/src/request-context.js (Node 22, Express 5, pino)
const { AsyncLocalStorage } = require('node:async_hooks')
const { randomUUID } = require('node:crypto')
const pino = require('pino')

const storage = new AsyncLocalStorage()
const baseLogger = pino({ base: { service: 'payments' } })

function requestContext(req, res, next) {
  const incoming = req.get('x-request-id') ?? ''
  const requestId = /^[A-Za-z0-9._-]{8,64}$/.test(incoming) ? incoming : randomUUID()

  res.set('x-request-id', requestId)
  storage.run({ logger: baseLogger.child({ request_id: requestId }) }, next)
}

const log = () => storage.getStore()?.logger ?? baseLogger

module.exports = { requestContext, log }

When an incident happens, one query pulls the whole story, in order, from every service:

# Loki / LogQL: everything that happened for one request, across services
{env="production"} | json | request_id="01JD2Z8Q4M7X"

# CloudWatch Logs Insights
fields @timestamp, service, level_name, message
| filter extra.request_id = "01JD2Z8Q4M7X"
| sort @timestamp asc

If you already run OpenTelemetry, use its trace ID (from the W3C traceparent header) as your correlation ID and log it as trace_id, so logs link straight to traces. The pattern is identical; only the header and field name change.

Testing the Request ID Middleware

The middleware has two jobs worth testing: reuse a valid upstream ID, and reject anything else. Pest feature tests cover both, including that the ID inside Context matches the one returned to the client:

use Illuminate\Support\Facades\Context;
use Illuminate\Support\Facades\Route;

beforeEach(function () {
    Route::get('/_test/ping', fn () => ['request_id' => Context::get('request_id')]);
});

it('reuses a well-formed incoming request id', function () {
    $this->getJson('/_test/ping', ['X-Request-Id' => 'edge-7f3a9c21'])
        ->assertHeader('X-Request-Id', 'edge-7f3a9c21')
        ->assertJsonPath('request_id', 'edge-7f3a9c21');
});

it('replaces a malicious incoming id with a fresh one', function () {
    $response = $this->getJson('/_test/ping', ['X-Request-Id' => "x\n{\"level\":\"EMERGENCY\"}"]);

    $id = $response->headers->get('X-Request-Id');

    expect($id)->toMatch('/^[0-9A-HJKMNP-TV-Z]{26}$/')
        ->and($response->json('request_id'))->toBe($id);
});

It is also worth one test per critical log line that asserts the fields you alert on actually exist. Dashboards and alerts break silently when someone renames order_id to orderId.

Common Pitfalls

  • Putting data in the message. "Payment failed for order 991" cannot be grouped or filtered. Keep the message constant and put variables in context fields.
  • Logging secrets and personal data. Request bodies, Authorization headers, tokens and full card data do not belong in logs. Log IDs, not payloads, and redact known sensitive keys in a Monolog processor as a safety net.
  • Generating a new ID in each service. Then nothing correlates. Only the edge generates; everyone else forwards.
  • Inconsistent field names. request_id, requestId and req_id across three services means three queries. Agree on names once and document them.
  • Forgetting background work. Scheduled commands and queue jobs started without a request have no ID. Add one at the start of each command (Context::add('request_id', (string) Str::ulid())) so their logs still group.
  • High-cardinality labels. In Loki-style systems, request_id must be a parsed field, not a stream label; using it as a label creates one stream per request and hurts the log store.

When to Use It, and When Not To

Use structured JSON logs and request IDs for anything that runs in more than one process: multiple web containers, queue workers, or more than one service. The setup is one middleware, one channel config and one HTTP client hook, and it pays for itself in the first incident.

A single small app on one server, read by one person with tail -f, can live with plain text for a while; JSON is harder to read raw. Even then, adding the request ID to the line format costs nothing. And do not mistake correlation IDs for tracing: they tell you which lines belong together, not how long each step took. When latency across services is the question, add OpenTelemetry tracing and keep the logs linked to it by ID.

Interactive challenge

Try the challenge

Practice what you just learned. Write your solution, reveal hints if you get stuck.

Instructions

Implement two pieces of a structured logger. First, resolve_request_id(incoming, generate): trim the incoming header value (which may be null/None) and return it if it matches ^[A-Za-z0-9._-]{8,64}$ in full; otherwise return generate(). Second, a StructuredLogger class whose constructor takes a clock function returning the timestamp string. withContext(array $context) (Python: with_context(dict)) merges into a shared context; later values win and earlier key order is kept. log(level, message, context = []) returns ONE JSON line (no newline characters). Allowed levels are debug, info, warning, error; anything else throws InvalidArgumentException (Python: ValueError). The JSON object starts with ts (clock value), level and message, followed by the shared context merged with the call context (call context wins for duplicate keys; a key keeps its first position). Context keys named ts, level or message are ignored, never overwriting the reserved fields. Values of keys named password, token, authorization or cookie, compared case-insensitively, are replaced by the string [REDACTED] (the original key spelling is kept).

Starter code

php
function resolve_request_id(?string $incoming, Closure $generate): string
{
    // TODO: trim $incoming; keep it if it matches ^[A-Za-z0-9._-]{8,64}$, otherwise return $generate()
    return '';
}

final class StructuredLogger
{
    private const LEVELS = ['debug', 'info', 'warning', 'error'];
    private const SENSITIVE = ['password', 'token', 'authorization', 'cookie'];

    /** @var array<string, mixed> */
    private array $shared = [];

    public function __construct(private Closure $clock) {}

    /** @param array<string, mixed> $context */
    public function withContext(array $context): void
    {
        // TODO: merge $context into the shared context (later values win)
    }

    /** @param array<string, mixed> $context */
    public function log(string $level, string $message, array $context = []): string
    {
        // TODO: reject unknown levels with InvalidArgumentException
        // TODO: build ts, level, message, then shared + call context (call wins, reserved keys cannot be overridden)
        // TODO: redact sensitive keys (case-insensitive) and return one JSON line
        return '{}';
    }
}
python
import json
import re
from typing import Any, Callable

LEVELS = {"debug", "info", "warning", "error"}
SENSITIVE = {"password", "token", "authorization", "cookie"}


def resolve_request_id(incoming: str | None, generate: Callable[[], str]) -> str:
    # TODO: strip incoming; keep it if it matches [A-Za-z0-9._-]{8,64}, else return generate()
    raise NotImplementedError


class StructuredLogger:
    def __init__(self, clock: Callable[[], str]) -> None:
        self.clock = clock
        self.shared: dict[str, Any] = {}

    def with_context(self, context: dict[str, Any]) -> None:
        # TODO: merge context into self.shared (later values win)
        raise NotImplementedError

    def log(self, level: str, message: str, context: dict[str, Any] | None = None) -> str:
        # TODO: ValueError for unknown levels; ts, level, message first; then shared + context
        #       (context wins, reserved keys cannot be overridden); redact sensitive keys; one JSON line
        raise NotImplementedError

Your solution

Hints

All hints revealed
Hint 1

The regex must match the whole string, so use anchors in PHP and re.fullmatch in Python

Hint 2

Build the record with the reserved fields first, then loop over the merged context and skip keys already present

Hint 3

PHP spread [...$a, ...$b] and Python {**a, **b} both keep the first position of a key while taking the later value

Solution · php
function resolve_request_id(?string $incoming, Closure $generate): string
{
    $candidate = trim((string) $incoming);

    if (preg_match('/^[A-Za-z0-9._-]{8,64}$/', $candidate) === 1) {
        return $candidate;
    }

    return $generate();
}

final class StructuredLogger
{
    private const LEVELS = ['debug', 'info', 'warning', 'error'];
    private const SENSITIVE = ['password', 'token', 'authorization', 'cookie'];

    /** @var array<string, mixed> */
    private array $shared = [];

    public function __construct(private Closure $clock) {}

    /** @param array<string, mixed> $context */
    public function withContext(array $context): void
    {
        $this->shared = [...$this->shared, ...$context];
    }

    /** @param array<string, mixed> $context */
    public function log(string $level, string $message, array $context = []): string
    {
        if (! in_array($level, self::LEVELS, true)) {
            throw new InvalidArgumentException("Unknown level {$level}");
        }

        $record = ['ts' => ($this->clock)(), 'level' => $level, 'message' => $message];

        foreach ([...$this->shared, ...$context] as $key => $value) {
            if (array_key_exists($key, $record)) {
                continue;
            }

            $record[$key] = in_array(strtolower((string) $key), self::SENSITIVE, true) ? '[REDACTED]' : $value;
        }

        return json_encode($record, JSON_UNESCAPED_SLASHES | JSON_UNESCAPED_UNICODE | JSON_THROW_ON_ERROR);
    }
}
Solution · python
import json
import re
from typing import Any, Callable

LEVELS = {"debug", "info", "warning", "error"}
SENSITIVE = {"password", "token", "authorization", "cookie"}


def resolve_request_id(incoming: str | None, generate: Callable[[], str]) -> str:
    candidate = (incoming or "").strip()
    if re.fullmatch(r"[A-Za-z0-9._-]{8,64}", candidate):
        return candidate
    return generate()


class StructuredLogger:
    def __init__(self, clock: Callable[[], str]) -> None:
        self.clock = clock
        self.shared: dict[str, Any] = {}

    def with_context(self, context: dict[str, Any]) -> None:
        self.shared = {**self.shared, **context}

    def log(self, level: str, message: str, context: dict[str, Any] | None = None) -> str:
        if level not in LEVELS:
            raise ValueError(f"Unknown level {level}")
        record: dict[str, Any] = {"ts": self.clock(), "level": level, "message": message}
        for key, value in {**self.shared, **(context or {})}.items():
            if key in record:
                continue
            record[key] = "[REDACTED]" if key.lower() in SENSITIVE else value
        return json.dumps(record, separators=(",", ":"), ensure_ascii=False)

More from DevOps