Bourse.RateLimiter.Shaping (bourse v0.9.0)

Copy Markdown View Source

Shapes endpoint rate-limit descriptors into Bourse.RateLimiter checks and updates rate-limit state from response headers.

Authored buckets execute as token buckets: max_size is capacity and refill_per_sec is the drain rate. A request is admitted when the bucket holds its cost. There is no fixed 60s window and no skip-record exemption.

Endpoints whose authored cost outruns the wait bound

Because no cost is exempt, an endpoint accrues cost / refill_per_sec seconds before it is admitted. Where that accrual exceeds the per-call wait budget (default Bourse.Defaults.rate_limit_max_wait_ms/0) the call returns {:error, %Bourse.Error{type: :rate_limit_exceeded}} naming the venue and the required wait, instead of a skip-record pass-through that let the request go out unlimited and collect the venue's own 429.

Callers that accept a longer wait pass :rate_limit_max_wait_ms on the request (or :max_wait_ms on maybe_rate_limit/4). The budget is per call, so concurrent consumers do not race on process-wide config. Exceeding the budget never bypasses accounting. Long-period endpoints (okx's published one-per-month POST asset/monthly-statement, binance-family POST papi/margin/repay-debt) are refused immediately at the default bound; the error's retry_after is the accrual, not a multi-hour sleep.

Summary

Types

Full rate key including bucket axis

Token-bucket limit passed to RateLimiter.check_rates/1

Credential slice of a rate key: API key string or :public

One RateLimiter.check_rates/1 triple

Base rate key without bucket axis

Functions

Builds RateLimiter check triples from an endpoint rate-limit descriptor.

Checks rate limit if enabled — blocks until capacity is available.

Parses rate limit headers from a response and updates the ETS state store.

Builds the base rate-limiter key from an exchange: {exchange_id, api_key | :public}.

Types

axis_key()

@type axis_key() :: {String.t(), credential_key(), String.t()}

Full rate key including bucket axis

bucket_limit()

@type bucket_limit() :: %{capacity: number(), refill_per_sec: number()}

Token-bucket limit passed to RateLimiter.check_rates/1

credential_key()

@type credential_key() :: String.t() | :public

Credential slice of a rate key: API key string or :public

rate_check()

@type rate_check() :: {axis_key(), bucket_limit(), number()}

One RateLimiter.check_rates/1 triple

rate_key()

@type rate_key() :: {String.t(), credential_key()}

Base rate key without bucket axis

Functions

build_rate_limit_checks(rate_key, exchange, endpoint_rate_limit)

@spec build_rate_limit_checks(rate_key(), Bourse.Exchange.t(), term()) :: [
  rate_check()
]

Builds RateLimiter check triples from an endpoint rate-limit descriptor.

Accepts a numeric weight, a map with :cost/:axes/:max_size/ :refill_per_sec, a list of those maps, or falls back to weight 1 on the default "request" axis. Venue-level config["rate_limit_bucket"] fills max_size / refill_per_sec gaps; rate_limit_ms is only a refill fallback (1000 / rate_limit_ms per second, capacity 1).

maybe_rate_limit(rate_key, exchange, endpoint_rate_limit, opts \\ [])

@spec maybe_rate_limit(rate_key(), Bourse.Exchange.t(), term(), keyword()) ::
  :ok | {:error, Bourse.Error.t()}

Checks rate limit if enabled — blocks until capacity is available.

Returns {:error, %Bourse.Error{}} when the pre-request wait would exceed the per-call budget (:max_wait_ms, default Bourse.Defaults.rate_limit_max_wait_ms/0), naming the venue and the wait.

maybe_update_state(exchange, resp_headers)

@spec maybe_update_state(Bourse.Exchange.t(), map()) :: :ok

Parses rate limit headers from a response and updates the ETS state store.

Returns :ok when the exchange doesn't send rate limit headers (OKX, Kraken, etc.) — the normal case for most exchanges, not an error.

rate_key(exchange)

@spec rate_key(Bourse.Exchange.t()) :: rate_key()

Builds the base rate-limiter key from an exchange: {exchange_id, api_key | :public}.