All content below is read from files recorded during the run (see outputs/). Nothing is simulated.
GET https://integrate.api.nvidia.com/v1/models (worked without any key). Examples: 01-ai/yi-large, adept/fuyu-8b, ai21labs/jamba-1.5-large-instruct, openai/gpt-oss-20b, moonshotai/kimi-k3, nvidia/nemotron-3-ultra-550b-a55bHeader of type `authorization` was missingThe model 'meta/llama-3.1-8b-instruct' has reached its end of life on 2026-08-26T09:00:00Z and is no longer available.NVIDIA_API_KEY on this machine; creating the key is the user's action. No authenticated NVIDIA chat answer exists, so none is shown.qwen/qwen3.8-27b:free)This is what the NVIDIA side should be compared against once a key exists. The same script (outputs/run_matrix.py) takes only BASE_URL / key / model.
| task | passed | median latency | mean completion tokens | first recorded answer |
|---|---|---|---|---|
| a_summary | 3/3 | 11.3s | 1735 | Marlow’s shrinking catch forces a choice between a cold-storage warehouse and a tourist promenade. After debate, a compromise proposes a smaller north-dock warehouse and modest south-dock promenade, funded by a grant and yacht fees. The mayor promises a spring vote and transparent assumptions, while fishermen form a cooperative to improve prices. Residents agree change is needed, decided openly. |
| b_json | 3/3 | 1.3s | 135 | {"name":"Dana Levi","city":"Haifa","amount":1250} |
| c_code | 3/3 | 24.3s | 2109 | ```python
def is_prime(n: int) -> bool:
"""
Return True if n is a prime number, otherwise False.
A prime is an integer greater than 1 with no divisors other than 1 and
itself. This checks divisibility up to the square root of n.
"""
if n < 2:
return False
if n in (2, 3):
return True
if n % 2 == 0 or n % 3 == 0:
return False
# Check candidates of the form 6k + 1 and 6k - 1.
i = 5
while i * i <= n:
if n % i == 0 or n % (i + 2) == 0:
return False
i += 6
return True
``` |
| d_hebrew | 3/3 | 2.4s | 104 | בירת ישראל היא ירושלים. |
| e_tool | 3/3 | 0.9s | 58 | [{"type": "function", "index": 0, "id": "ff5491bd-0c89-98f2-af2a-b4714218b9ef_0", "function": {"name": "get_weather", "arguments": "{\"city\":\"Paris\"}"}}] |