SimpleDao
搭建自己的Chat GPT
2023-03-22, 访问数: 1201

创建虚拟环境

使用python 3.11.2

  1. python -m venv .env
  2. source .env/bin/activate

安装python包

  1. # pytorch <3
  2. # numpy <3
  3. # for huggingface transformers <3 (to load GPT-2 checkpoints)
  4. pip install transformers
  5. # for huggingface datasets <3 (if you want to download + preprocess OpenWebText)
  6. pip install datasets
  7. # for OpenAI's fast BPE code <3
  8. pip install tiktoken
  9. # for optional logging <3
  10. pip install wandb
  11. pip install tqdm
  12. pip install torch

运行nanoGPT

下载

  1. git clone https://github.com/karpathy/nanoGPT.git
  2. cd nanoGPT

运行

  1. python sample.py --init_from=gpt2 --device=cpu --num_samples=2 --max_new_tokens=100 --start="What is computer?"

结果

  1. Overriding: init_from = gpt2
  2. Overriding: device = cpu
  3. Overriding: num_samples = 2
  4. Overriding: max_new_tokens = 100
  5. Overriding: start = What is computer?
  6. loading weights from pretrained gpt: gpt2
  7. forcing vocab_size=50257, block_size=1024, bias=True
  8. overriding dropout rate to 0.0
  9. number of parameters: 123.65M
  10. Downloading (…)lve/main/config.json: 100%
  11. Downloading pytorch_model.bin: 100% 548M/548M
  12. Downloading (…)neration_config.json: 100%
  13. No meta.pkl found, assuming GPT-2 encodings...
  14. What is computer?
  15. Computer is a tool that makes it easier to create in Java. It is a program that can be used both as a database and as an editor. It can be used to create 3D models and data objects from files. The program also allows you to edit and modify data in your program directory.
  16. What is data?
  17. Data refers to the type of data that is stored in a computer. Sometimes, a file has a specific name or was assigned a name. The file
  18. ---------------
  19. What is computer?
  20. Computer is the biggest, most important technical technology of humankind. Computer science is the art of creating and applying it in a digital environment. The computer is just one part of each of our lives. Many of our lives are about computer inputs and outputs. While the majority of our lives are about computers, this does not mean that your computer is controlled by a computer. You are not at all controlled by a computer in the way you would interact with other people. Your computer is just a computer
  21. ---------------
  • num_samples=2 表示输出两个结果
  • init_from=gpt2 表示用gpt2这个预训练模型,可以用别的模型
    • gpt2:124M参数量
    • gpt2-medium:350M参数量
    • gpt2-large:774M参数量
    • gpt2-xl:1558M参数量(15亿)

网上开源的预训练模型最多也就15亿,远远比不上Chat GPT3的1750亿参数量,所以效果相差很大。

集成到web

参考上面的simple.py脚本,添加代码到web接口即可。不过使用的模型越大,越消耗内存和cpu。

参考资料