Model-side Conversation Template Protocol¶

meta_template is defined in a model configuration. After the dataset template constructs the input, it can add model-wide instructions or adapt the conversation to the model’s required format. It supports both the standard RawPromptTemplate and the legacy PromptTemplate, but uses a different structure for each:

Dataset-side template	`meta_template` structure	Primary purpose
`RawPromptTemplate`	`role`/`content` list	Insert messages or prepend instructions to existing messages
`PromptTemplate`	Dictionary containing `round`	Map legacy roles or wrap model-specific special tokens

The two structures cannot be combined. Dataset-specific task instructions should normally remain in the dataset template. Use model-side meta_template only for content that should be applied consistently to the same model across evaluations.

Appending Content to a Standard Template¶

A standard prompt template directly produces a list of system, user, and assistant messages. In this case, define the model’s meta_template as a role/content list as well:

from opencompass.models import OpenAISDK

models = [
    dict(
        type=OpenAISDK,
        abbr='my-api-model',
        path='served-model-name',
        key='ENV',
        openai_api_base='https://example.com/v1',
        meta_template=[
            dict(
                role='system',
                content='Answer concisely and put the final answer last.\n',
            ),
        ],
        max_seq_len=32768,
        max_out_len=4096,
        batch_size=8,
    )
]

Suppose the dataset template produces these messages:

[
    {'role': 'system', 'content': 'Solve the following problem.'},
    {'role': 'user', 'content': 'What is 1 + 1?'},
]

After processing by the model-side template, the input is:

[
    {
        'role': 'system',
        'content': (
            'Answer concisely and put the final answer last.\n'
            'Solve the following problem.'
        ),
    },
    {'role': 'user', 'content': 'What is 1 + 1?'},
]

The processing rules are:

when a model-side message and the dataset message at the corresponding position have the same role, the model-side content is prepended to the dataset message;
when no subsequent dataset message has that role, the model-side message is inserted as a new message;
model-side list entries are processed in declaration order, and role must be system, user, or assistant.

This list form performs no legacy role mapping and does not use round, api_role, begin, end, or generate. It is intended for model types that directly process standard message lists, including OpenAI-compatible API models. Support in other model types depends on their template-parsing implementation.

Adapting Model Formats for Legacy Templates¶

The legacy PromptTemplate produces an intermediate structure with roles such as SYSTEM, HUMAN, and BOT. In this case, meta_template uses a dictionary to convert those roles into API messages or wrap them in strings required by a local language model.

API and Chat-template Models¶

For API models and models that construct input through a tokenizer chat template, only role mapping is required:

from opencompass.models import OpenAISDK

api_meta_template = dict(
    round=[
        dict(role='HUMAN', api_role='user'),
        dict(role='BOT', api_role='assistant', generate=True),
    ],
    reserved_roles=[
        dict(role='SYSTEM', api_role='system'),
    ],
)

models = [
    dict(
        type=OpenAISDK,
        abbr='my-api-model',
        path='served-model-name',
        key='ENV',
        openai_api_base='https://example.com/v1',
        meta_template=api_meta_template,
        max_seq_len=32768,
        max_out_len=4096,
        batch_size=8,
    )
]

role must match a role name in the legacy dataset template.
api_role specifies the converted message role.
generate=True marks the role whose content is generated by the model. In generative inference, the final corresponding BOT content is not sent as input; PPL inference retains the complete candidate content.
reserved_roles declares roles that may occur in begin or end rather than in every round and is commonly used for SYSTEM.

If SYSTEM in the legacy dataset template specifies fallback_role='HUMAN' but the model-side template does not define SYSTEM, that message is processed as HUMAN.

Local Language Models in One-stop Evaluation¶

For local language models that accept strings directly, role-level begin and end fields add the markers required by the model:

lm_meta_template = dict(
    begin='Meta instruction: You are a helpful assistant.\n',
    round=[
        dict(
            role='HUMAN',
            begin='<HUMAN>: ',
            end='<eoh>\n',
        ),
        dict(
            role='BOT',
            begin='<BOT>: ',
            end='<eob>\n',
            generate=True,
        ),
    ],
    reserved_roles=[
        dict(
            role='SYSTEM',
            begin='<SYSTEM>: ',
            end='<eosys>\n',
        ),
    ],
)

Here, begin and end are part of the model protocol rather than task instructions. OpenCompass applies these markers according to the role order in the legacy dataset template. In generative inference, input stops after the begin of the role marked generate=True, and the model continues from that point.

Models that use a tokenizer chat template normally need only the role mapping from the preceding section. Do not also write the same special tokens manually, as this may apply the template twice.

Fields Supported for Legacy Template Scenarios¶

The dictionary form of meta_template supports these primary fields:

Field	Purpose
`round`	Required. Defines the role order in one conversation round and the conversion rule for each role.
`reserved_roles`	Optional. Declares roles that may occur outside the regular rounds.
`begin`, `end`	Optional. Adds a global prefix or suffix to the complete input, primarily for models that accept strings directly.
`eos_token_id`	Optional. Specifies a generation stop token for certain local models; support depends on the model type.

Role entries in round and reserved_roles support:

Field	Purpose
`role`	Matches a role name in the legacy `PromptTemplate`.
`api_role`	Maps the role to one used by an API or chat template.
`begin`, `end`	Wraps this role’s content in direct text input.
`prompt`	Optional default content, overridden when the dataset template supplies content for the same role.
`generate`	Marks a role generated by the model; normally set to `True` only for `BOT`.

Each role may be declared only once across round and reserved_roles. Use api_role for API or chat-template configurations and role-level begin and end for direct text input. Do not combine both forms in the same role definition.

Behavior Without `meta_template`¶

Standard messages produced by RawPromptTemplate remain unchanged.
The role-based structure produced by the legacy PromptTemplate loses its role wrappers, and the prompt values are normally concatenated in order.

Accordingly, prefer RawPromptTemplate for API and chat-template evaluation. Role mapping is required only when supporting legacy dataset configurations. When directly loading a local model that requires a specialized conversation format, ensure that its model configuration provides the correct meta_template.