Windows Agent Arena (WAA) 🪟 is a scalable OS platform for testing and benchmarking of multi-modal AI agents.
-
Updated
Apr 13, 2026 - Python
#
ai-benchmark
Here are 93 public repositories matching this topic...
An agent benchmark with tasks in a simulated software company.
-
Updated
Nov 17, 2025 - Python
Frontier Models playing the board game Diplomacy.
-
Updated
Jun 1, 2026 - Python
The definitive benchmark for AI agents on OpenClaw. 45 tasks across 4 tiers. Powered by MyClaw.ai
-
Updated
Mar 9, 2026 - Python
Ranking LLMs on agentic tasks
-
Updated
May 21, 2026 - Jupyter Notebook
llmBench is a high-depth benchmarking tool designed to measure the raw performance of local LLM runtimes (Ollama, llama.cpp) while providing deep hardware intelligence.
-
Updated
Mar 15, 2026 - Python
🤖 A curated list of resources for testing AI agents - frameworks, methodologies, benchmarks, tools, and best practices for ensuring reliable, safe, and effective autonomous AI systems
testing qa benchmark machine-learning evaluation chaos artificial-intelligence chaos-monkey testing-tools awesome-list quality-assurance ai-safety ai-agents chaos-engineering llm llm-evaluation agentic-ai ai-benchmark agent-evaluation
-
Updated
May 28, 2025
📊 Daily auto-updated snapshots of all Arena AI (LMSYS Chatbot Arena) leaderboards — LLM, Vision, Code, Video, Image & more. Structured JSON with historical tracking.
benchmark machine-learning ai leaderboard text-to-video llm lmsys ai-benchmark chatbot-arena arena-ai
-
Updated
Jun 25, 2026 - Python
MindTrial: Evaluate and compare AI language models (LLMs) on text-based tasks with optional file/image attachments and tool use. Supports multiple providers (OpenAI, Google, Anthropic, DeepSeek, Mistral AI, xAI, Alibaba, Moonshot AI, OpenRouter), custom tasks in YAML, and HTML/CSV/JSON reports.
opensource openai xai artificial-intelligence-projects anthropic ai-tool openrouter qwen deepseek mistral-ai llm-evaluation-framework google-gemini-ai llm-benchmarking moonshot-ai language-models-ai llm-comparison ai-benchmark ai-evaluation-tools grok-ai ai-model-comparison
-
Updated
Jun 25, 2026 - Go
Precision-aware retrieval benchmark for LLM memory systems.
benchmark information-retrieval ai memory evaluation memory-benchmark ai-agents long-term-memory rag llm ai-memory ai-benchmark memory-evaluation
-
Updated
Jun 26, 2026 - TypeScript
Benchmark for evaluating AI epistemic reliability - testing how well LLMs handle uncertainty, avoid hallucinations, and acknowledge what they don't know.
python nlp benchmark machine-learning ai evaluation collaborate ai-safety llm llm-evaluation hallucination-detection ai-benchmark llm-benchmark
-
Updated
Mar 22, 2026 - Python
Benchmark abierto en español de 141 LLMs (89 con 13K+ runs reales y juez Phi-4 independiente). Quality, costo, velocidad, long-context y fuga de credenciales como dimensiones separadas. Alternativas a Claude, GPT y Gemini para agentes n8n/OpenClaw. Calculadora interactiva con tus propios pesos.
ai-agents startup-tools n8n ai-models emprendedores openrouter ollama llm-evaluation llm-comparison ai-benchmark llm-benchmark spanish-ai openclaw hermes-agent claude-alternatives gpt-alternatives phi4-judge benchmark-en-espanol emprendedores-ia
-
Updated
Jun 26, 2026 - Python
A curated list of evaluation tools, benchmark datasets, leaderboards, frameworks, and resources for assessing model performance.
-
Updated
May 11, 2026 - Python
GDB: GraphicDesignBench - A real-world benchmark for evaluating AI on graphic design tasks
svg design benchmark layout evaluation typography graphic-design lottie vlm multimodal vision-language-model ai-benchmark
-
Updated
May 5, 2026 - Python
GTA (Guess The Algorithm) Benchmark - A tool for testing AI reasoning capabilities
python docker flask benchmark machine-learning puzzle reverse-engineering educational pattern-recognition ctf binary-analysis algorithm-analysis computational-thinking algorithmic-reasoning ai-benchmark
-
Updated
Jan 12, 2025 - Python
This repository contains the results and code for the MLPerf™ Storage v2.0 benchmark.
-
Updated
Aug 4, 2025 - Python
Benchmark local LLM models: speed, quality, and hardware fitness scoring. CLI, MCP server, and IDE plugins.
-
Updated
Jun 2, 2026 - TypeScript
The first comprehensive benchmark for evaluating AI coding agents on Salesforce development tasks. Tests Apex, LWC, Flows, and more.
-
Updated
Jan 27, 2026 - Python
RetardBench is an open, no-censorship benchmark that ranks large language models purely on how retarded they are.
jailbreak large-language-models llm prompt-injection red-teaming-tools ollama llm-evaluation uncensored-llm open-llm-leaderboard llm-jailbreaks prompt-injection-llm-security ai-benchmark ai-red-teaming llm-benchmark
-
Updated
Mar 2, 2026 - TypeScript
The first open evaluation framework for AI continuity. 250 narrative tests, 1835 verification questions, 10 checkpoints. Benchmark for AI memory systems, stateful agents, and long-term context persistence. No LLM in the evaluation loop.
persistent-memory evaluation-framework memory-benchmark ai-agents continuity long-term-memory temporal-reasoning llm-evaluation ai-memory ai-benchmark context-engineering stateful-agents narrative-testing
-
Updated
Apr 21, 2026 - Python
Improve this page
Add a description, image, and links to the ai-benchmark topic page so that developers can more easily learn about it.
Add this topic to your repo
To associate your repository with the ai-benchmark topic, visit your repo's landing page and select "manage topics."