Architecture

Architecture overview

Icelake is a fully managed European data research and observability platform. Under the hood it runs as a single Rust service that handles the full pipeline from ingestion through query — we operate it for you, you consume it as a SaaS.

Key components

Ingestion layer

Multi-protocol HTTP endpoints for Prometheus, Loki, and OpenTelemetry data, plus a native MQTT broker for IoT telemetry.

Storage engine

ClickHouse — a columnar, open-source analytics engine — with compressed tables on S3-backed object storage in Europe. Your data is isolated per tenant and exportable anytime via SQL or CSV.

Query engine

ClickHouse SQL via pgwire, plus LogQL and PromQL for Grafana compatibility over the same data. User queries run under a read-only database role.

Rollups & merges

Materialized 10-minute, hourly and daily rollups are maintained as data arrives, and ClickHouse merges data parts in the background so queries stay fast. Runs transparently on our side — nothing to configure.

Data flow

Every signal follows the same pipeline:

1. Ingestion — Data arrives via Prometheus remote write, Loki push API, OTLP HTTP, or MQTT
2. Parsing — Protocol-specific parsers normalise the payload into a common internal format
3. Storage — Normalised data is batched into ClickHouse tables (metrics, logs, geo), every row tagged with your tenant
4. Rollups — Materialized views keep 10-minute, hourly and daily aggregates up to date at insert time
5. Query — ClickHouse answers SQL, PromQL, and LogQL queries; long-range metric queries are routed to the rollups automatically

Multi-tenant isolation

Every Icelake account is a tenant. All data, API keys, and queries are scoped to the tenant boundary:

• Tenant-scoped storage — every row carries its tenant, and every query is rewritten to your tenant before it runs
• API key authentication — keys are hashed with SHA-256 and scoped to a tenant
• Query isolation — you can only query data your tenant owns; user SQL runs under a read-only database role that cannot write, change schema, or reach external storage
• Rate limiting — per-tenant ingestion rate limits applied at the edge

Security note: API keys are hashed before storage. The plaintext ilk_… secret is shown once at creation and never persisted on our side.

pgwire SQL interface

Icelake exposes a PostgreSQL-compatible interface on sql.icelake.eu:5432 so any PostgreSQL client can query your data directly. Username is your API key’s client ID, password is the ilk_… secret.

-- Connect with any PostgreSQL client
psql "postgresql://your-client-id:ilk_your-api-key-here@sql.icelake.eu:5432/icelake?sslmode=require"

-- Query metrics
SELECT timestamp, value, labels
FROM metrics
WHERE metric_name = 'http_requests_total'
  AND timestamp > now() - INTERVAL 1 HOUR
ORDER BY timestamp DESC;

-- Query logs (labels are JSON — read them with JSONExtractString)
SELECT timestamp, message, labels
FROM logs
WHERE JSONExtractString(labels, 'service') = 'api-gateway'
  AND timestamp > now() - INTERVAL 30 MINUTE;

For a notebook-first experience, see Jupyter Notebooks.

Rollups and background merges

Two things keep queries fast as your data grows, both on our side:

• Rollups — materialized views maintain 10-minute, hourly and daily aggregates (metrics_10m, metrics_1h, metrics_1d, and the matching log rollups) at insert time. Long-range chart queries are routed to them automatically.
• Merges — ClickHouse continuously merges freshly inserted data parts into larger, sorted ones in the background, concurrently with ingestion and queries.

You don’t configure or operate either. It just happens.

What you control

• API keys — create, rotate, and revoke per-purpose keys from the admin UI
• Data sources — MQTT brokers, uploaded CSVs and Sheets, and the endpoints you point Prometheus/Loki/OTEL at
• Dashboards and visualisations — composed in the admin UI or in your own Grafana
• Retention — configurable per tenant (contact us if you need a longer window)

What we handle

• ClickHouse storage, rollups, merges, and backups on S3-backed storage
• pgwire SQL engine, PromQL, LogQL, OTLP parsing
• High availability, rate limiting, and DDoS protection
• Keycloak-based authentication and PKCE flow for the admin UI