Loki & LogQL

New here? Start with the Quickstart to launch Icelake + Grafana locally and wire up the Loki datasource in 5 minutes.

Overview

Icelake implements the Loki push API for log ingestion and supports LogQL for querying. This makes it a drop-in replacement for Grafana Loki with S3-native storage.

Getting Started

1. Configure Log Shipping

Send logs to Icelake using the Loki push API endpoint. Authentication is HTTP Basic auth — username is your client ID, password is your API key (ilk_…):

# Loki Push API endpoint
POST https://api.icelake.eu/loki/api/v1/push

# Authentication header (Basic base64("client_id:api_key"))
Authorization: Basic <base64>

2. Fluent Bit Configuration

Configure Fluent Bit to ship logs to Icelake:

fluent-bit.conf

[OUTPUT]
    Name        loki
    Match       *
    Host        api.icelake.eu
    Port        443
    URI         /loki/api/v1/push
    Labels      job=fluent-bit
    HTTP_User   your-client-id
    HTTP_Passwd ilk_your-api-key-here
    TLS         on

3. Grafana Logs Drilldown

Add Icelake as a Loki datasource in Grafana:

Grafana Datasource

Name: Icelake Logs
Type: Loki
URL: https://api.icelake.eu/loki
Authentication: Basic auth
  User:     your-client-id
  Password: ilk_your-api-key-here

LogQL Query Reference

Stream Selectors

Filter logs by label matchers:

# Exact match
{service="api-gateway"}

# Regex match
{service=~"api-.*"}

# Not equal
{service!="debug"}

# Multiple labels
{service="api-gateway", environment="production"}

Line Filters

Further filter log lines by content:

# Contains string
{service="api-gateway"} |= "error"

# Does not contain
{service="api-gateway"} != "debug"

# Regex match on line
{service="api-gateway"} |~ "status=[45]\\d{2}"

Aggregations

Aggregate log data for dashboards:

# Count logs per service over 5 minutes
count_over_time({job="fluent-bit"}[5m])

# Rate of error logs
rate({service="api-gateway"} |= "error" [1m])

# Top 10 services by log volume
topk(10, sum by(service) (rate({job="fluent-bit"}[5m])))

Log Format

Logs are stored as Parquet files on S3:

S3 Storage Layout

s3://icelake-data/
└── logs/
    └── tenant=default/
        ├── 2026/02/20/
        │   ├── logs_001.parquet
        │   └── compacted_001.parquet
        └── 2026/02/21/
            └── logs_001.parquet

Each Parquet file contains:

Column	Type	Description
timestamp	INT64	Unix nanoseconds
message	STRING	Log line content
labels	MAP	Key-value label pairs
tenant_id	STRING	Tenant identifier

Loki Push API Format

Send logs in the Loki JSON format:

Push Request Body

{
  "streams": [
    {
      "stream": {
        "service": "api-gateway",
        "environment": "production"
      },
      "values": [
        ["1708000000000000000", "GET /api/health 200 12ms"],
        ["1708000001000000000", "POST /api/data 201 45ms"]
      ]
    }
  ]
}

Query API Endpoints

Endpoint	Method	Description
/loki/api/v1/push	POST	Ingest log streams
/loki/api/v1/query	GET	Instant LogQL query
/loki/api/v1/query_range	GET	Range LogQL query
/loki/api/v1/labels	GET	List label names
/loki/api/v1/label/<name>/values	GET	List label values

Best Practices

Performance Tip: Use structured logging with consistent label schemas for optimal query performance.

• Keep label cardinality low (avoid unique IDs as labels)
• Use line filters to narrow down results before aggregations
• Set appropriate retention policies per tenant
• Configure compaction for log-heavy workloads

Related docs

• Quickstart — end-to-end Icelake + Grafana setup in 5 minutes
• OpenTelemetry — alternative path if you’re standardising on OTLP
• Query Interfaces — Grafana, pgwire SQL, REST, and the Loki Query API