山大软院创新实训之大模型篇（二）——Llama Factory微调Qwen实践

山大软院创新实训之大模型篇（二）——Llama Factory微调Qwen实践

LLaMA Factory 是一个用于微调和部署 LLaMA (Large Language Model Applications) 模型的框架。它旨在简化大语言模型的使用和管理，提供了强大的工具来支持从模型训练、微调到部署的整个流程。

虽然基于LLaMA，但LLaMA Factory也提供了许多其他领域大模型的微调功能。

我本想基于ChatGLM进行微调，但微调过程中有部分老是报错，所以我改为基于Qwen进行微调。

首先，我安装并运行了llama factory。

git clone https://github.com/hiyouga/LLaMA-Factory.git

cd LLaMA-Factory
pip install -r requirements.txt
pip install transformers_stream_generator bitsandbytes tiktoken auto-gptq optimum autoawq
pip install --upgrade tensorflow
pip uninstall flash-attn -y

CUDA_VISIBLE_DEVICES=0 USE_MODELSCOPE_HUB=1 python src/webui.py

根据格式构造数据集，数据集部分内容如下。llama factory微调Qwen时，要求的数据集格式如下：

[
    {
        "instruction": "用户指令（必填）",
        "input": "用户输入（选填）",
        "output": "模型回答（必填）",
        "system": "系统提示词（选填）",
        "history": [
            ["第一轮指令（选填）", "第一轮回答（选填）"],
            ["第二轮指令（选填）", "第二轮回答（选填）"]
        ]
    }
]

参考Huatuo-GPT微调ChatGLM的数据集，我制作数据集格式如下，样本如下：

[
      {
            "instruction": "If you are a doctor, please answer the medical questions based on the patient's description.",
            "input": "Doctor, I have been experiencing sudden and frequent panic attacks. I don't know what to do.",
            "output": "Well, based on what you're telling me, it sounds like you may be suffering from panic disorder. The best course of action is to start with psychotherapy and mental health counseling. Additionally, we should conduct an electrocardiogram to make sure that there are no physical issues causing your panic attacks. We will also need to perform a depression screen and a toxicology screen to rule out any other underlying causes. Finally, I would recommend a comprehensive psychological and psychiatric evaluation and therapy to help manage your symptoms."
      },
      {
            "instruction": "If you are a doctor, please answer the medical questions based on the patient's description.",
            "input": "Doctor, I have been experiencing a hoarse voice for a few weeks now and it's not getting any better despite taking medication. What could be the problem?",
            "output": "It's possible that you have a vocal cord polyp. To confirm this, we need to conduct tracheoscopy and laryngoscopy with a biopsy. We may also need to conduct an occupational therapy assessment such as speech therapy, other diagnostic procedures like an interview or consultation, physical therapy exercises, as well as diagnostic procedures on your nose, mouth, and pharynx to rule out any other underlying conditions. Additionally, we may need to evaluate your eyes and ears for any related conditions, such as ophthalmologic and otologic diagnosis and treatment."
      },
      {
            "instruction": "If you are a doctor, please answer the medical questions based on the patient's description.",
            "input": "Hi, Doctor. I think I might have Turner syndrome. ",
            "output": "Okay, in that case, you will need to undergo a complete physical skin exam, an ultrasonography, and some other diagnostic procedures such as an interview, evaluation, and consultation. Additionally, we'll need to perform an echocardiography, examine your breast, and check your eyes and ears for any issues. We'll also do a depression screening to assess your mental health."
      },
      {
            "instruction": "If you are a doctor, please answer the medical questions based on the patient's description.",
            "input": "Doctor, I've been experiencing a condition called cryptorchidism. My testicles have not descended properly into the scrotum. What medical tests do I need to take?",
            "output": "You will need to undergo a complete physical skin exam to see the extent of the undescended testicles. Then, we need to conduct Ultrasonography (Ultrasound) to find out the exact location of the testicles. After that, a pelvis exam and rectal examination will be done to determine if the testicles have descended into the pelvic region. If not, then other OR therapeutic procedures related to male genital or nervous system procedures may be required. We will also do an occupational therapy assessment to assess your speech therapy."
      },
      ...
]

我的预计工作流程如下：先准备好训练和验证数据集，进行必要的数据预处理和格式转换。然后选择合适的预训练模型，配置训练参数和超参数。使用 LLaMA Factory 的训练脚本和工具进行模型训练和微调，监控训练过程中的性能指标。对训练好的模型进行评估和验证，确保模型在验证集上的表现符合预期。最后将微调好的模型部署到实际应用中，进行推理和预测。

需要现在llama factory中计算数据集存放路径的sha值并放入data文件中，计算sha值的python程序如下。通过逐块读取文件内容，计算文件的SHA-1哈希值，并在文件不存在时处理异常。它适用于需要验证文件完整性或唯一性的场景。

import hashlib
def calculate_sha1(file_path):
    sha1 = hashlib.sha1()
    try:
        with open(file_path, 'rb') as file:
            while True:
                data = file.read(8192)  # Read in chunks to handle large files
                if not data:
                    break
                sha1.update(data)
        return sha1.hexdigest()
    except FileNotFoundError:
        return "File not found."
 
file_path = './Desktop/self_cognition.json'
sha1_hash = calculate_sha1(file_path)
print("SHA-1 Hash:", sha1_hash)

进行微调，设置微调参数。我的主要微调参数设置如下：参数设置

使用llama-factory直接chat，测试微调结果。可以看出，我所微调过的模型对于医疗问答特定任务，效果比原Qwen效果好。

参考博客：https://blog.csdn.net/weixin_44480960/article/details/137092717

总结