prompt_strategies.llama2_chat

prompt_strategies.llama2_chat

Prompt Strategy for finetuning Llama2 chat models see also https://github.com/facebookresearch/llama/blob/6c7fe276574e78057f917549435a2554000a876d/llama/generation.py#L213 for ma reference implementation.

This implementation is based on the Vicuna PR and the fastchat repo, see also: https://github.com/lm-sys/FastChat/blob/cdd7730686cb1bf9ae2b768ee171bdf7d1ff04f3/fastchat/conversation.py#L847

Use dataset type: “llama2_chat” in conig.yml to use this prompt style.

E.g. in the config.yml:

datasets:
  - path: llama_finetune_train.jsonl
    type: llama2_chat

The dataset itself should look like this:

{'conversations':[{"from": "human", "value": "Who are you?"}, {"from": "gpt", "value": "I am Vicuna"},...]}

in a jsonl file. The first message should be from the human, the second from gpt. For a custom system message, the first “from” can be “system” (followed by alternating “human” and “gpt” turns).

Important: Don’t use “special_tokens:” in your config.yml if you are not sure what you are doing!

Classes

Name	Description
LLama2ChatTokenizingStrategy	Tokenizing strategy for Llama2 prompts.
Llama2ChatConversation	A class that manages prompt templates and keeps all conversation history.
Llama2ChatPrompter	A prompter that generates prompts for Llama2 models.

LLama2ChatTokenizingStrategy

prompt_strategies.llama2_chat.LLama2ChatTokenizingStrategy(
    self,
    *args,
    **kwargs,
)

Tokenizing strategy for Llama2 prompts. adapted from https://github.com/lm-sys/FastChat/blob/main/fastchat/train/train.py

Llama2ChatConversation

prompt_strategies.llama2_chat.Llama2ChatConversation(
    self,
    name='llama2',
    system="[INST] <<SYS>>\nYou are a helpful, respectful and honest assistant. Always answer as helpfully as possible, while being safe. Your answers should not include any harmful, unethical, racist, sexist, toxic, dangerous, or illegal content. Please ensure that your responses are socially unbiased and positive in nature.\n\nIf a question does not make any sense, or is not factually coherent, explain why instead of answering something not correct. If you don't know the answer to a question, please don't share false information.\n<</SYS>>\n\n",
    roles=('[INST]', '[/INST]'),
    messages=list(),
    offset=0,
)

A class that manages prompt templates and keeps all conversation history. copied from https://github.com/lm-sys/FastChat/blob/main/fastchat/conversation.py

Methods

Name	Description
append_message	Append a new message.
get_prompt	Get the prompt for generation.

append_message

prompt_strategies.llama2_chat.Llama2ChatConversation.append_message(
    role,
    message,
)

Append a new message.

get_prompt

prompt_strategies.llama2_chat.Llama2ChatConversation.get_prompt()

Get the prompt for generation.

Llama2ChatPrompter

prompt_strategies.llama2_chat.Llama2ChatPrompter()

A prompter that generates prompts for Llama2 models.