原文链接：GitHub - togethercomputer/OpenChatKit

OpenChatKit是一个由前OpenAI研究员共同打造的开源聊天机器人平台。它包含了训练好的大型语言模型、定制配方和可扩展的检索系统，可以帮助用户快速构建高精度、多功能的聊天机器人应用。

其中，最核心的组件是一个经过微调的、具有200亿参数的语言模型——GPT-NeoXT-Chat-base-20B。这个模型基于EleutherAI的GPT-NeoX模型，重点调整了多轮对话、问答、分类、提取和摘要等多项任务，并使用了4300万条高质量指令进行训练。这使得OpenChatKit在处理聊天对话时可以提供高精度、流畅的回答。

除此之外，OpenChatKit还提供了定制配方的功能，可以帮助用户根据自己的数据集微调模型，以提高模型在特定任务上的表现。另外，该平台还提供了可扩展的检索系统，可以从文档存储库、API或实时更新信息源等多个来源中检索信息，以提供更全面的回答。

 

OpenChatKit provides a powerful, open-source base to create both specialized and general purpose chatbots for various applications. The kit includes an instruction-tuned language models, a moderation model, and an extensible retrieval system for including up-to-date responses from custom repositories. OpenChatKit models were trained on the OIG-43M training dataset, which was a collaboration between Together, LAION, and Ontocord.ai.

In this repo, you'll find code for:

• Training GPT-NeoXT-Chat-base-20B, a 20B parameter chat model (see docs/GPT-NeoXT-Chat-base-20B.md)
• Training Pythia-Chat-base-7B, a 7B parameter chat model
• Testing inference using either of the chat models
• Augmenting the model with additional context from a retrieval index

• Getting Started
  • Requirements
  • Chatting with Pythia-Chat-base-7B
• Reproducing Pythia-Chat-base-7B
  • Downloading training data and the base model
  • (Optional) 8bit Adam
  • Training the model
  • Converting weights to Huggingface format
  • Testing the new model
• Monitoring
  • Loguru
  • Weights & Biases
• Experimental: Retrieval-Augmented Models
• See Also
• License
• Citing OpenChatKit
• Acknowledgements

In this tutorial, you will download Pythia-Chat-base-7B, an instruction-tuned language model, and run some some inference requests against it using a command-line tool.

Pythia-Chat-base-7B is a 7B-parameter fine-tuned variant of Pythia-6.9B-deduped from Eleuther AI. Pre-trained weights for this model are available on Huggingface as togethercomputer/Pythia-Chat-base-7B under an Apache 2.0 license.

More details can be found on the model card for Pythia-Chat-base-7B on Huggingface.

Before you begin, you need to install PyTorch and other dependencies.

1. Install Miniconda from their website.

2. Install Git LFS from their website.

3. Install the  hooks.

git lfs install

1. Install mamba in the  environment so it's available in all environments.

conda install mamba -n base -c conda-forge

1. Create an environment called OpenChatKit using the  file at the root of this repo.

Note Use  to create the environment. It's much faster than using .

mamba env create -f environment.yml 

1. Activate the new conda environment.

conda activate OpenChatKit

To help you try the model, inference/bot.py is a simple command-line test harness that provides a shell inferface enabling you to chat with the model. Simply enter text at the prompt and the model replies. The test harness also maintains conversation history to provide the model with context.

Start the bot by calling  from the root for the repo.

python inference/bot.py --model togethercomputer/Pythia-Chat-base-7B

Loading the model can take some time, but once it's loaded, you are greeted with a prompt. Say hello.

$ python inference/bot.py 
Loading /home/csris/src/github.com/togethercomputer/OpenChatKit/inference/https://blog.csdn.net/qq_41771998/article/huggingface_models/GPT-NeoXT-Chat-base-20B to cuda:1...
Welcome to OpenChatKit shell.   Type /help or /? to list commands.

>>> Hello.
Hello human.

>>> 

Enter additional queries at the prompt, and the model replies. Under the covers, the shell is forming a prompt with all previous queries and passes that to the model to generate more text.

The shell also supports additional commands to inspect hyperparamters, the full prompt, and more. Commands are prefixed with a .

Note The  command exits the shell.

Please see the inference README for more details about arguments, running on multiple/specific GPUs, and running on consumer hardware.

This tutorial walks through reproducing the Pythia-Chat-base-7B model by fine-tuning Eleuther AI's Pythia-6.9B-deduped model using the OIG dataset.

The chat model was trained on the OIG dataset built by LAION, Together, and Ontocord.ai. To download the dataset from Huggingface run the command below from the root of the repo.

python data/OIG/prepare.py

Note You can help make this chat model better by contributing data! See the OpenDataHub repo for more details.

once the command completes, the data will be in the  directory.

Pythia-Chat-base-7B is a fine-tuned variant of Pythia-6.9B-deduped from Eleuther AI. To download the model and prepare it for fine tuning, run this command from the root of the repo.

python pretrained/Pythia-6.9B-deduped/prepare.py

The weights for this model will be in the  directory.

To use 8bit-adam during training, install the  package.

pip install bitsandbytes # optional, to use 8bit-adam

The  script configures and runs the training loop. After downloading the dataset and the base model, run:

bash training/finetune_Pythia-Chat-base-7B.sh

As the training loop runs, checkpoints are saved to the  directory at the root of the repo.

Please see the training README for more details about customizing the training run.

Before you can use this model to perform inference, it must be converted to the Huggingface format. Run this command from the root of the repo to do so.

mkdir huggingface_models 
  && python tools/convert_to_hf_gptneox.py 
       --config-name EleutherAI/pythia-6.9b-deduped 
       --ckpt-path model_ckpts/Pythia-Chat-base-7B/checkpoint_100 
       --save-path huggingface_models/Pythia-Chat-base-7B 
       --n-stages 4 
       --n-layer-per-stage 8 
       --fp16

where the  flag will load and store models in fp16.

Make sure to replace  with the latest checkpoint in the  directory.

You can use the OpenChatKit Shell test harness to chat with the new model. From the root of the repo, run

python inference/bot.py

By default the script will load the model named Pythia-Chat-base-7B under the  directory, but you can override that behavior by specifying .

python inference/bot.py --model https://blog.csdn.net/qq_41771998/article/details/huggingface_models/GPT-NeoXT-Chat-base-20B

once the model has loaded, enter text at the prompt and the model will reply.

$ python inference/bot.py 
Loading /home/csris/src/github.com/togethercomputer/OpenChatKit/inference/https://blog.csdn.net/qq_41771998/article/huggingface_models/GPT-NeoXT-Chat-base-20B to cuda:1...
Welcome to OpenChatKit shell.   Type /help or /? to list commands.

>>> Hello.
Hello human.

>>> 

The shell also supports additional commands to inspect hyperparamters, the full prompt, and more. Commands are prefixed with a .

Note The  command exits the shell.

Please see the inference README for more details about arguments, running on multiple/specific GPUs, and running on consumer hardware.

By default, the training script simply prints the loss as training proceeds, but it can also output metrics to a file using loguru or report them to Weights & Biases.

Add the flag  to your training script to log to 

To use Weights & Biases, first login with your Weights & Biases token.

wandb login

And set  in the training script to enable logging to Weights & Biases.

Warning Retrieval support is experimental.

The code in  implements a python package for querying a Faiss index of Wikipedia. The following steps explain how to use this index to augment queries in the test harness with context from the retriever.

1. Download the Wikipedia index.

python data/wikipedia-3sentence-level-retrieval-index/prepare.py

1. Run the bot with the  flag.

python inference/bot.py --retrieval

After starting, the bot will load both the chat model and the retrieval index, which takes a long time. once the model and the index are loaded, all queries will be augmented with extra context.

$ python inference/bot.py --retrieval
Loading /OpenChatKit/inference/https://blog.csdn.net/qq_41771998/article/huggingface_models/GPT-NeoXT-Chat-base-20B to cuda:0...
Loading retrieval index...
Welcome to OpenChatKit shell.   Type /help or /? to list commands.

>>> Where is Zurich?
Where is Zurich?
Zurich is located in Switzerland.

>>>

• docs/GPT-NeoXT-Chat-base-20B.md. OpenChatKit also provides a larger, 20B parameter chat model that was trained on GPT-NeoXT-Chat-base-20B from Eleuther AI.

All code in this repository was developed by Together Computer except where otherwise noted. Copyright (c) 2023, Together Computer. All rights reserved. The code is licensed under the Apache 2.0 license.

This repository also contains code written by a number of other authors. Such contributions are marked and the relevant licensing is included where appropriate.