Search — OpenCompass 0.5.4 documentation

GitHub
Homepage
Leaderboard
LLM
VLM

Table of Contents

Get Started

Installation and Environment Setup
Quick Start

Basic Tutorials

OpenCompass Workflow and Core Concepts
Running a Complete Evaluation with a Configuration File
MMEngine-Based Configuration Syntax
Model Configuration Overview
Dataset Selection and Configuration
Dataset Download and Caching
Understanding Outputs and Result Summaries

Prompts and Input Construction

Prompt Templates
Model-side Conversation Template Protocol
Prompt Preview and Debugging

Execution, Parallelism, and Task Management

Task Partitioning, Runners, and Task Types
Cross-Task Concurrent Inference and Evaluation Watching
Command-Line Reference

Evaluation Methods

Answer Extraction, Postprocessing, and Evaluator Selection
LLM as Judge Evaluation
Mathematical Capability Evaluation
Cascade Evaluation
Code Evaluation
Subjective Evaluation Guidance
Repeated Evaluation, Repeated Sampling, and Stability
Multimodal Evaluation Overview

Extending OpenCompass

Extension Mechanisms Overview
Add a dataset
Dataset Quick Evaluation Tutorial
Add a Model
Adding Postprocessors, Evaluators, and Summarizers

Tools and Recovery

Tools Overview
Discovering and Comparing Configurations
Inspecting Predictions and Analyzing Error Cases
Analyzing Repetition and Response Length
Testing APIs and Message Formats
Notifications and Task Monitoring

Other Documentation

Contributing to OpenCompass
Frequently Asked Questions
CircularEval
Data Contamination Assessment
Evaluation with LMDeploy
Long Context Evaluation Guidance
Needle In A Haystack Evaluation
Prompt Attack

Docs >
Search
以中文阅读

Shortcuts

© Copyright 2023, OpenCompass.

Built with Sphinx using a theme provided by Read the Docs.

@沪ICP备2021009351号-23 OpenCompass Open Platform Service Agreement

GitHub