Agentic real-world work tasks, (Elo-500)/2000
Legal agentic work, criterion pass rate
Kubernetes incident root-cause analysis
Measured by Output Speed (tokens per second)
Measured by Time (seconds) to First Token
Seconds to output 500 tokens, calculated based on time to first token, 'thinking' time for reasoning models, and output speed
Source: Hacker News — This article was automatically imported from the source. Read full article at original source →
