Observability

Observability settings

Logs

Logs settings

Disable health logs

Disable /health endpoint requests logging. Default value is false

ID	observability.disable-health-logs
Type	Bool
Default value	(no default)
Environment variables	BOX_OBSERVABILITY_DISABLE_HEALTH_LOGS , BOX_LOGGING_DISABLE__HEALTH__LOGS
Sensitive	false — can be set via Ul and environment variable
Hot reload	true — can be changed at runtime

Log file path

If provided, enables mode to pipe logs as JSON into the file by specified path.

If ElasticSearch URL is provided then the file is used as a fallback in case if ElasticSearch is not available.

ID	observability.log-file.path
Type	String
Default value	(no default)
Environment variables	BOX_OBSERVABILITY_LOG_FILE_PATH , AIDBOX_LOGS
Sensitive	false — can be set via Ul and environment variable
Hot reload	false — requires Aidbox restart

Logs max lines

Sets the limit of log records to push into the file When the limit is reached, the current log file is renamed with .old postfix and a new log file is created

ID	observability.log-file.max-lines
Type	Int
Default value	10000
Environment variables	BOX_OBSERVABILITY_LOG_FILE_MAX_LINES , AIDBOX_LOGS_MAX_LINES
Sensitive	false — can be set via Ul and environment variable
Hot reload	false — requires Aidbox restart

Logging SQL min duration

Threshold for logging only long queries. Analogous from PostgreSQL. Log only requests whose execution time exceeds the specified number of milliseconds.

ID	observability.sql.min-duration
Type	Int
Default value	-1
Environment variables	BOX_OBSERVABILITY_SQL_MIN_DURATION , BOX_LOGGING_SQL_MIN__DURATION
Sensitive	false — can be set via Ul and environment variable
Hot reload	false — requires Aidbox restart

Logging SQL max length

Max length of a query to be logged.

ID	observability.sql.max-length
Type	Int
Default value	500
Environment variables	BOX_OBSERVABILITY_SQL_MAX_LENGTH , BOX_LOGGING_SQL_MAX__LENGTH
Sensitive	false — can be set via Ul and environment variable
Hot reload	false — requires Aidbox restart

Metrics

Metrics settings

Metrics server port

Port on which Aidbox will expose metrics. To disable metrics server, leave this setting empty

ID	observability.metrics.server-port
Type	Int
Default value	(no default)
Environment variables	BOX_METRICS_PORT
Sensitive	false — can be set via Ul and environment variable
Hot reload	false — requires Aidbox restart

Enable Postgres metrics

Controls whether to provide metrics related to PostgreSQL

ID	observability.metrics.enable-postgres-metrics
Type	Bool
Default value	true
Environment variables	BOX_OBSERVABILITY_METRICS_ENABLE_POSTGRES_METRICS , BOX_METRICS_POSTGRES_ON
Sensitive	false — can be set via Ul and environment variable
Hot reload	false — requires Aidbox restart

Stdout

Stdout settings

Stdout log level

Controls stdout with specified logs level. Possible values: off, fatal, error, warn, info, debug, trace, all, true By setting one of these levels you would also get all the levels to the left. e.g. if you set log level to warn you would also get log events with fatal and error levels (off is excluded).

ID	observability.stdout.log-level
Type	Enum
Values	— — — — — — — — —
Default value	(no default)
Environment variables	BOX_OBSERVABILITY_STDOUT_LOG_LEVEL , AIDBOX_STDOUT_JSON
Sensitive	false — can be set via Ul and environment variable
Hot reload	false — requires Aidbox restart

Pretty print log level

Controls pretty print of logs to stdout with specified level. Possible values: off, fatal, error, warn, info, debug, trace, all, true By setting one of these levels you would also get all the levels to the left. e.g. if you set log level to warn you would also get log events with fatal and error levels (off is excluded).

ID	observability.stdout.pretty-log-level
Type	Enum
Values	— — — — — — — — —
Default value	warn
Environment variables	BOX_OBSERVABILITY_STDOUT_PRETTY_LOG_LEVEL , AIDBOX_STDOUT_PRETTY
Sensitive	false — can be set via Ul and environment variable
Hot reload	false — requires Aidbox restart

Google log level

Produces in Google Logging format with specified log level. Possible values: off, fatal, error, warn, info, debug, trace, all, true By setting one of these levels you would also get all the levels to the left. e.g. if you set log level to warn you would also get log events with fatal and error levels (off is excluded).

ID	observability.stdout.google-log-level
Type	Enum
Values	— — — — — — — — —
Default value	(no default)
Environment variables	BOX_OBSERVABILITY_STDOUT_GOOGLE_LOG_LEVEL , AIDBOX_STDOUT_GOOGLE_JSON
Sensitive	false — can be set via Ul and environment variable
Hot reload	false — requires Aidbox restart

Loki

Grafana Loki settings

Loki URL

Loki URL to enable Aidbox logs uploading into Loki

ID	observability.loki-url
Type	String
Default value	(no default)
Environment variables	BOX_OBSERVABILITY_LOKI_URL , AIDBOX_LK_URL
Sensitive	false — can be set via Ul and environment variable
Hot reload	false — requires Aidbox restart

Loki batch size

Loki batch size for log uploading. Aidbox uploads logs when either at least specific observability.loki.batch-size collected or time passed from previous log uploading exceeds observability.loki.batch-timeout.

ID	observability.loki.batch-size
Type	String
Default value	(no default)
Environment variables	BOX_OBSERVABILITY_LOKI_BATCH_SIZE , AIDBOX_LK_BATCH_SIZE
Sensitive	false — can be set via Ul and environment variable
Hot reload	false — requires Aidbox restart

Loki batch timeout

How long to wait before uploading Aidbox uploads logs when either at least observability.loki.batch-size entries collected or time passed from previous log uploading exceeds observability.loki.batch-timeout.

ID	observability.loki.batch-timeout
Type	Int
Default value	(no default)
Environment variables	BOX_OBSERVABILITY_LOKI_BATCH_TIMEOUT , AIDBOX_LK_BATCH_TIMEOUT
Sensitive	false — can be set via Ul and environment variable
Hot reload	false — requires Aidbox restart

Loki stream

Stream refers to the labels or metadata associated with a log stream Is defined by a unique set of labels, which serve as the stream key. For example: {"box": "aidbox"}

ID	observability.loki.stream
Type	String
Default value	(no default)
Environment variables	BOX_OBSERVABILITY_LOKI_STREAM , AIDBOX_LK_STREAM
Sensitive	false — can be set via Ul and environment variable
Hot reload	false — requires Aidbox restart

Datadog

Datadog settings

Datadog API Key

Datadog API Key.

ID	observability.datadog.api-key
Type	String
Default value	(no default)
Environment variables	BOX_OBSERVABILITY_DATADOG_API_KEY , BOX_DD_API_KEY
Sensitive	true — can be set only via environment variable
Hot reload	false — requires Aidbox restart

Datadog regional site

The regional site for a Datadog customer.

ID	observability.datadog.site
Type	Enum
Values	— — — — — —
Default value	(no default)
Environment variables	BOX_OBSERVABILITY_DATADOG_SITE , BOX_DD_SITE
Sensitive	false — can be set via Ul and environment variable
Hot reload	false — requires Aidbox restart

Datadog log tags

Tags associated with your logs. Convenient for transferring the name of the environment. For example env:staging.

ID	observability.datadog.tags
Type	String
Default value	(no default)
Environment variables	BOX_OBSERVABILITY_DATADOG_TAGS , BOX_DD_TAGS
Sensitive	false — can be set via Ul and environment variable
Hot reload	false — requires Aidbox restart

Datadog log file path

Fallback file to write logs in if uploading to Datadog fails

ID	observability.datadog.logs
Type	String
Default value	(no default)
Environment variables	BOX_OBSERVABILITY_DATADOG_LOGS , AIDBOX_DD_LOGS
Sensitive	false — can be set via Ul and environment variable
Hot reload	false — requires Aidbox restart

Datadog log batch size

How many log entries to collect before uploading. Aidbox uploads logs when either at least specific observability.datadog.batch-size collected or time passed from previous log uploading exceeds observability.datadog.batch-timeout.

ID	observability.datadog.batch-size
Type	Int
Default value	(no default)
Environment variables	BOX_OBSERVABILITY_DATADOG_BATCH_SIZE , BOX_DD_BATCH_SIZE
Sensitive	false — can be set via Ul and environment variable
Hot reload	false — requires Aidbox restart

Datadog log batch timeout

How long to wait before uploading Aidbox uploads logs when either at least specific observability.datadog.batch-size collected or time passed from previous log uploading exceeds observability.datadog.batch-timeout.

ID	observability.datadog.batch-timeout
Type	Int
Default value	(no default)
Environment variables	BOX_OBSERVABILITY_DATADOG_BATCH_TIMEOUT , BOX_DD_BATCH_TIMEOUT
Sensitive	false — can be set via Ul and environment variable
Hot reload	false — requires Aidbox restart

Elastic

Elastic settings

Elastic Search auth

Format: : Basic auth credentials for ElasticSearch. API key is not supported.

ID	observability.elastic.search-auth
Type	String
Default value	(no default)
Environment variables	BOX_OBSERVABILITY_ELASTIC_SEARCH_AUTH , AIDBOX_ES_AUTH
Sensitive	true — can be set only via environment variable
Hot reload	false — requires Aidbox restart

Elastic Search URL

If provided, enables mode to push logs to ElasticSearch

ID	observability.elastic.search-url
Type	String
Default value	(no default)
Environment variables	BOX_OBSERVABILITY_ELASTIC_SEARCH_URL , AIDBOX_ES_URL
Sensitive	false — can be set via Ul and environment variable
Hot reload	false — requires Aidbox restart

Elastic Search batch size

Log batch size used to optimize log shipping performance. The default value is 200

ID	observability.elastic.batch-size
Type	Int
Default value	(no default)
Environment variables	BOX_OBSERVABILITY_ELASTIC_BATCH_SIZE , AIDBOX_ES_BATCH_SIZE
Sensitive	false — can be set via Ul and environment variable
Hot reload	false — requires Aidbox restart

Elastic Search batch timeout

Timeout to post a batch to ElasticSearch. If there is not enough records to reach full batch size

ID	observability.elastic.batch-timeout
Type	Int
Default value	60000
Environment variables	BOX_OBSERVABILITY_ELASTIC_BATCH_TIMEOUT , AIDBOX_ES_BATCH_TIMEOUT
Sensitive	false — can be set via Ul and environment variable
Hot reload	false — requires Aidbox restart

Elastic Search log format

Custom index format string. The default value is 'aidbox-logs'-yyyy-MM-dd.

ID	observability.elastic.index-pattern
Type	String
Default value	'aidbox-logs'-yyyy-MM-dd
Environment variables	BOX_OBSERVABILITY_ELASTIC_INDEX_PATTERN , AIDBOX_ES_INDEX_PAT
Sensitive	false — can be set via Ul and environment variable
Hot reload	false — requires Aidbox restart

Grafana

Grafana settings

Grafana URL for metrics dashboards

Grafana URL to update the metrics dashboards.

ID	observability.grafana.url
Type	String
Default value	(no default)
Environment variables	BOX_OBSERVABILITY_GRAFANA_URL , BOX_METRICS_GRAFANA_URL
Sensitive	false — can be set via Ul and environment variable
Hot reload	false — requires Aidbox restart

Grafana user for metrics dashboards

Grafana user to update the metrics dashboards.

ID	observability.grafana.user
Type	String
Default value	(no default)
Environment variables	BOX_OBSERVABILITY_GRAFANA_USER , BOX_METRICS_GRAFANA_USER
Sensitive	false — can be set via Ul and environment variable
Hot reload	false — requires Aidbox restart

Grafana password for metrics dashboards

Grafana password to update the metrics dashboards.

ID	observability.grafana.password
Type	String
Default value	(no default)
Environment variables	BOX_OBSERVABILITY_GRAFANA_PASSWORD , BOX_METRICS_GRAFANA_PASSWORD
Sensitive	true — can be set only via environment variable
Hot reload	false — requires Aidbox restart

OTEL

OpenTelemetry settings

OTEL auth token

The default bearer token for authentication.

ID	observability.otel.default-auth-token
Type	String
Default value	(no default)
Environment variables	BOX_OBSERVABILITY_OTEL_DEFAULT_AUTH_TOKEN , BOX_METRICS_OTEL_DEFAULT_AUTH_TOKEN
Sensitive	true — can be set only via environment variable
Hot reload	false — requires Aidbox restart

OTEL metrics interval

The default time interval (in seconds) for sending metrics to OTEL.

ID	observability.otel.default-interval
Type	Int
Default value	5
Environment variables	BOX_OBSERVABILITY_OTEL_DEFAULT_INTERVAL , BOX_METRICS_OTEL_DEFAULT_INTERVAL
Sensitive	false — can be set via Ul and environment variable
Hot reload	false — requires Aidbox restart

OTEL request headers

The default headers for OTEL requests, formatted as HeaderName:HeaderValue.

ID	observability.otel.default-request-headers
Type	String
Default value	(no default)
Environment variables	BOX_OBSERVABILITY_OTEL_DEFAULT_REQUEST_HEADERS , BOX_METRICS_OTEL_DEFAULT_REQUEST_HEADERS
Sensitive	false — can be set via Ul and environment variable
Hot reload	false — requires Aidbox restart