Large Language Models News
ScarfBench shows coding agents still struggle with Java migrations
IBM Research's ScarfBench tests whether AI coding agents can preserve behavior while migrating Java applications across enterprise frameworks.
By The AI Feed Desk
Hugging Face and Every Eval Ever make model-card scores more inspectable
Community Evals and Every Eval Ever now connect model-page benchmark scores to structured provenance records.
By The AI Feed Desk
OpenAI's GeneBench-Pro makes biology benchmarks about judgment
GeneBench-Pro tests whether AI agents can handle ambiguous computational-biology analysis, not just clean benchmark questions.
By The AI Feed Desk
Claude Sonnet 5 turns Anthropic's default model into an agent model
Anthropic made Sonnet 5 the default for Free and Pro users while positioning it as a lower-cost agentic model close to Opus 4.8.
By The AI Feed Desk
TraceLab turns real Codex and Claude Code sessions into serving data
The University of Washington TraceLab release studies coding agents as production workloads, with public traces across sessions, tool calls, tokens, cache behavior, and latency.
By The AI Feed Desk
Allen AI's DiScoFormer tests one transformer for density and score
The Hugging Face writeup frames DiScoFormer as a reusable estimator for density and score, with stronger high-dimensional results than kernel density estimation.
By The AI Feed Desk
Cognition's FrontierCode asks whether AI code would survive review
FrontierCode evaluates coding agents on mergeability, code quality, scope, tests, and maintainer judgment instead of only functional correctness.
By The AI Feed Desk
Hugging Face makes vLLM serving a one-command Jobs workflow
HF Jobs can now spin up a private OpenAI-compatible vLLM endpoint for tests, evals, and batch generation without provisioning servers or managing Kubernetes.
By The AI Feed Desk
AA-Briefcase tests agents on messy business work
Artificial Analysis' AA-Briefcase benchmark evaluates models on multi-week knowledge-work projects with documents, email, Slack data, deliverables, and graded analysis quality.
By The AI Feed Desk
Mistral OCR 4 adds structured document extraction for enterprise RAG
Mistral OCR 4 returns bounding boxes, block types, confidence scores, and 170-language coverage so document AI can feed retrieval, citations, and agent workflows.
By The AI Feed Desk
Hugging Face measures whether tools are agent-friendly
Hugging Face's agent-focused benchmark tests whether software changes help coding agents finish tasks with fewer errors, tokens, and detours.
By The AI Feed Desk
Sakana Fugu turns model orchestration into one API
Sakana AI is positioning Fugu as a single API that dynamically coordinates expert models for coding, reasoning, and other complex multi-step tasks.
By The AI Feed Desk
OpenAI brings ChatGPT and Codex to Samsung Electronics employees
OpenAI says Samsung Electronics is deploying ChatGPT Enterprise and Codex to all employees in Korea and all Device eXperience employees worldwide.
By The AI Feed Desk
OpenAI puts ChatGPT Enterprise spend into the admin console
OpenAI is adding credit usage analytics and updated spend controls for ChatGPT Enterprise, including ChatGPT and Codex usage by user, product, and model.
By The AI Feed Desk
MosaicLeaks makes research-agent privacy a query-log problem
ServiceNow researchers introduce MosaicLeaks, a benchmark showing how deep research agents can leak private enterprise facts through ordinary-looking web queries.
By The AI Feed Desk
DeepSWE makes coding-agent rankings a cost question
DeepSWE's June 20 leaderboard update separates frontier coding agents by pass rate, cost, output tokens, and agent steps across long-horizon software tasks.
By The AI Feed Desk
MLPerf Mobile v6.0 gives on-device LLMs a real test surface
MLCommons added standardized Android LLM tests to MLPerf Mobile v6.0, including Llama 3.2 1B, Llama 3.2 3B, and Llama 3.1 8B Instruct workloads.
By The AI Feed Desk