---
sidebar_label: Data lake
toc_max_heading_level: 3
doc_id: b7e4a1f3-6c82-4d9a-a5f1-3e8b72c0d4e6
description: >-
  Query a SQL-accessible replica of your nullplatform organization's data through the Data Lake API.
keywords:
  - data lake
  - replica
  - audit
  - query
  - SQL
  - reporting
  - governance
---

import Tabs from '@theme/Tabs';
import TabItem from '@theme/TabItem';

# Data lake <span className="heading-release-pill">New</span>

:::info 🧪 Early preview
The **data lake is in early preview**. Table schemas and query behavior may change as the product evolves.
:::

The **data lake** gives you SQL access (using the [ClickHouse SQL dialect](https://clickhouse.com/docs/en/sql-reference)) to a queryable replica of your nullplatform organization's data. This includes audit events plus the current state of your applications, scopes, deployments, parameters, and other resources, all through a single endpoint.

Use it to build custom reports, investigate incidents, or feed data into external dashboards and BI tools.


## Prerequisites

To use the Data Lake API, you need an **API key** with one of these roles: **Admin**, **SecOps**, **Dev**, **DevOps**, or **Insight Viewer**. All roles are scoped by [NRN](/docs/NRN), so query results are limited to the resources your token can access. See [API keys](/docs/authorization/api-keys) to create one and [Roles](/docs/authorization/roles) for role details.

## Execute a query

Send a [POST request](/docs/api/query-execute) to `https://api.nullplatform.com/data/lake/query` with your SQL statement.

You can send the SQL in three ways:

<Tabs>
<TabItem value="json" label="JSON body" default>

```bash
curl -X POST 'https://api.nullplatform.com/data/lake/query' \
  -H 'Content-Type: application/json' \
  -H 'Authorization: Bearer <token>' \
  -d '{ "query": "SELECT * FROM audit_events LIMIT 10" }'
```

</TabItem>
<TabItem value="text" label="Plain text body">

```bash
curl -X POST 'https://api.nullplatform.com/data/lake/query' \
  -H 'Content-Type: text/plain' \
  -H 'Authorization: Bearer <token>' \
  -d 'SELECT * FROM audit_events LIMIT 10'
```

</TabItem>
<TabItem value="query" label="Query string parameter">

```bash
curl -X POST 'https://api.nullplatform.com/data/lake/query?query=SELECT%20*%20FROM%20audit_events%20LIMIT%2010' \
  -H 'Authorization: Bearer <token>'
```

</TabItem>
</Tabs>

The **response** streams directly from the data lake. By default, results come back in tab-separated format. To get JSON, add a `FORMAT JSON` clause to your SQL:

```sql
SELECT * FROM audit_events LIMIT 10 FORMAT JSON
```

### Parametrized queries

You can use ClickHouse parametrized queries by passing `param_` prefixed query string parameters:

```bash
curl -X POST 'https://api.nullplatform.com/data/lake/query?param_user_id=42' \
  -H 'Content-Type: text/plain' \
  -H 'Authorization: Bearer <token>' \
  -d 'SELECT * FROM audit_events WHERE user_id = {user_id:Int64} FORMAT JSON'
```

Parameters are passed directly to ClickHouse, so you can use any supported [ClickHouse type](https://clickhouse.com/docs/en/sql-reference/data-types) in the placeholder.

## SQL restrictions

The API enforces strict validation to keep queries safe:

| Rule | Detail |
|---|---|
| **Read-only** | Only `SELECT`, `WITH`, `SHOW`, `DESCRIBE`, `DESC`, and `EXPLAIN` statements are allowed |
| **Single statement** | Each request must contain exactly one SQL statement |
| **Max length** | Queries can't exceed 50,000 characters |
| **Query timeout** | Queries must complete within 30 seconds |

Queries that violate the read-only, single-statement, or length rules return a `400 Invalid SQL` error. Queries that exceed the timeout are aborted and return a `409` query execution error.

:::note
The API also blocks SQL functions that reach external data sources, such as `remote`, `url`, `file`, `s3`, `mysql`, and `postgresql`. Queries that use any of these return a `403 Forbidden SQL` error.
:::

## NRN-based access control

Your query results are automatically scoped to the resources your token has access to. The API resolves your token's [NRN](/docs/NRN) grants and passes them as a session filter to the data lake.

For example, if your token grants access to `organization=1:account=2`, you'll only see audit events for that account and everything below it (namespaces, applications, scopes).

The filtering is hierarchical: a grant at the organization level includes all descendant resources. Redundant grants (like `organization=1` and `organization=1:account=2`) are automatically deduplicated, keeping only the broadest scope.

## Rate limiting

The API enforces a rate limit of **60 requests per minute** per authentication token. If you exceed this limit, you'll receive a `429 Too Many Requests` response with retry timing headers.

There is no hard cap on result size. In practice, queries are bounded by the 30-second timeout and the rate limit above.

## Error responses

| Status | Code | Description |
|---|---|---|
| 400 | `FST_ERR_INVALID_SQL` | SQL validation failed (not read-only, multiple statements, or exceeds max length) |
| 401 | `FST_ERR_AUTHENTICATION` | Missing or invalid bearer token |
| 403 | `FST_ERR_AUTHORIZATION` | Token has no granted resources for `governance:lake:read` |
| 403 | `FST_ERR_FORBIDDEN_SQL` | Query matches an AST restriction pattern |
| 409 | `FST_ERR_CLICKHOUSE_QUERY` | ClickHouse returned an error while executing the query |
| 429 | `Too Many Requests` | Rate limit exceeded |

All errors return a plain text body in ClickHouse format:

```
Code: 0. DB::Exception: Invalid SQL query: Multiple SQL statements are not allowed
```

## Next steps

- [Table reference](/docs/data-lake/tables): browse all tables organized by domain, with column schemas and relationships.
- [Connect with a database client](/docs/data-lake/connect): set up a connection from DBeaver, DataGrip, or other JDBC-compatible tools.