Back to overview
Degraded

JSON & Grammar mode producing wrong results on some serverless models

Nov 22 at 02:00am UTC
Affected services
Llama 3 70B Instruct chat completion API
Llama 3 8B Instruct chat completion API

Resolved
Nov 22 at 06:30pm UTC

The issues with grammar and json modes were resolved

Created
Nov 22 at 02:00am UTC

Some of the requests utilizing JSON or Grammar mode may return wrong results not satisfying the grammar. The issue affects a subset of serverless models and is non-deterministic.

Requests without JSON or Grammar aren't affected.