Llama 3 基于知识库应用实践（一）

一、概述

Llama 3 是Meta最新推出的开源大语言模型，其8B和13B参数的模型的性能与之前的Llama 2相比实现了质的飞跃。以下是官方给出的模型性能评测对比结果（引自：https://ai.meta.com/blog/meta-llama-3/），如Llama 3 的8B模型在这个级别上获得了新的SOTA：

在架构方面，Llama 3 采用了一个相对标准的decoder-only的transformer架构，相较于Llama 2，Llama 3做了以下几个方面的改进：

       -Llama 3 使用了基于128K tokens词库的tokenizer，能更有效地对语言进行编码

       -采用分组查询注意力机制（GQA）来改进模型推理效率

       -基于8192 tokens的上下文长度来训练模型，并且使用掩码（mask）来确保自注意力不跨越文档边界

二、Llama 3 模型加载

Ollama是一个免费开源的应用，通过它可以方便地在本地以相对较低的硬件需求来运行各种大语言模型，它也提供了package manager来让用户以命令行的方式快速有效地下载和运行LLMs。

首先运行以下命令加载Llama 3：

这里加载的模型是Meta-Llama-3-8B-Instruct：

Ollama加载Llama 3完成后打印这行信息，表示可以正常调用模型了：

[GIN] 2024/04/28 - 02:53:44 | 200 | 13.435240965s |       127.0.0.1 | POST     "/api/generate"

三、Llama 3模型调用测试

参照Ollama提供的API，执行以下命令：

curl -X POST http://localhost:11434/api/generate -d '{"model": "llama3","prompt":"Why is the sky blue?"}'

打印信息如下：

可以看出以上命令执行的是流式处理，即按行依次输出Llama 3 针对输入prompt生成的每个单词，也可以改用下面的命令来一次性输出Llama 3生成的response：

curl -X POST http://localhost:11434/api/generate -d '{"model": "llama3","prompt":"Why is the sky blue?", "stream":false}'

生成的response如下：

{"model":"llama3","created_at":"2024-04-28T03:12:47.652533109Z","response":"What a great question!\n\nThe short answer: The sky appears blue because of the way that light from the sun interacts with the Earth's atmosphere.\n\nHere's a more detailed explanation:\n\n1. **Sunlight**: When the sun emits light, it contains all the colors of the visible spectrum (red, orange, yellow, green, blue, indigo, and violet).\n2. **Atmosphere**: The Earth's atmosphere is made up of different gases, including nitrogen (N2) and oxygen (O2). These gases scatter shorter wavelengths of light more than longer wavelengths.\n3. **Scattering**: When sunlight enters the Earth's atmosphere, it encounters these gas molecules and scatters in all directions. The shorter blue wavelengths are scattered more than the longer red wavelengths because they have a higher energy and interact more with the smaller gas molecules.\n4. **Blue dominance**: As a result of this scattering, the blue light is distributed throughout the sky, making it appear blue to our eyes. This is known as Rayleigh scattering, named after the British physicist Lord Rayleigh, who first described the phenomenon in the late 19th century.\n\n**Other factors that influence the color of the sky:**\n\n1. **Dust and water vapor**: Tiny particles in the air, like dust, smoke, and water vapor, can also scatter light and change its apparent color.\n2. **Clouds**: Clouds can reflect or absorb certain wavelengths of light, making the sky appear whiter, grayer, or even darker.\n3. **Time of day**: The angle of the sun in relation to the observer's position on Earth affects the color of the sky. During sunrise and sunset, the sun's rays have to travel longer distances through the atmosphere, scattering blue light more and making the sky appear more red or orange.\n\nSo, to summarize: the sky appears blue because of the combined effects of sunlight scattering by the Earth's atmosphere and other factors that influence the color of the sky."

下面测试一下当使用中文提问时，Llama 3如何给出回答，输入命令如下：

curl -X POST http://localhost:11434/api/generate -d '{"model": "llama3","prompt":"为何天空是蓝色的？", "stream":false}'

生成的response如下：

{"model":"llama3","created_at":"2024-04-28T04:32:20.650993108Z","response":"A question that has puzzled humans for centuries! 🌌🔵\n\nThe short answer is: because of the way light interacts with our atmosphere.\n\nHere's a more detailed explanation:\n\n1. **Scattering**: When sunlight enters Earth's atmosphere, it encounters tiny molecules of gases like nitrogen (N2) and oxygen (O2). These molecules scatter the shorter, blue wavelengths of light more than the longer, red wavelengths. This is known as Rayleigh scattering.\n2. **Mie scattering**: As sunlight travels through the atmosphere, it also encounters larger particles like dust, water vapor, and pollutants. These particles scatter light in a way that favors shorter wavelengths, again favoring blue and violet colors. This is called Mie scattering.\n3. **Atmospheric composition**: The Earth's atmosphere is mostly composed of nitrogen (78%) and oxygen (21%), with small amounts of other gases like argon, carbon dioxide, and water vapor. The presence of these gases affects the way light interacts with our atmosphere.\n4. **Sunlight**: The color of sunlight itself plays a role in why we see blue skies. Sunlight is white, but it contains more blue and violet wavelengths than red or orange wavelengths.\n\nNow, when you combine all these factors, you get the perfect conditions for blue skies:\n\n* Shorter wavelengths (blue) are scattered more by atmospheric molecules and particles.\n* Longer wavelengths (red, orange) are transmitted more directly to our eyes without being scattered as much.\n* The dominant colors in sunlight (blue, violet) are enhanced due to scattering.\n\nVoilà! You get the beautiful blue color of a clear sky. 🌊💙\n\n(Note: This is a simplified explanation. If you want to dive deeper into the physics behind it, I'd be happy to provide more details!) 😊"

虽然使用了中文提问，但是Llama 3依然是使用英文进行回答，那么修改一下prompt，要求使用中文回答：

curl -X POST http://localhost:11434/api/generate -d '{"model": "llama3","prompt":"为何天空是蓝色的？请使用中文回答", "stream":false}'

这次给出的response基本上是中文，但是其中也包括了一些英文单词：

{"model":"llama3","created_at":"2024-04-28T04:33:53.558010562Z","response":"😊\n\n天空为什么是蓝色的是一个长期被人们探索的问题。科学家们发现，这是由于太阳光线的特性。\n\n我们知道，太阳是一颗白色的星球，它 emits 一些不同波长的光线，如红、橙、黄、绿、蓝、 indigo 和紫等色。这些光线在进入地球大气层时，会遇到不同的吸收和散射作用。\n\n在蓝色的部分，这些光线遇到了地球大气层中的某种物质，它们被吸收或散射，使得我们所见到的只是蓝色的光线。这是因为碳酸盐（carbonate）和硫酸盐（sulfate）等物质在大气层中存在，能够吸收红、橙、黄色的光线，而蓝色的光线则被反射回我们的眼睛中。\n\n因此，我们看到的天空是蓝色的，是由于太阳光线经过地球大气层后，只剩下了蓝色的部分被反射回来的结果。这个过程也决定了我们所见到的天空的颜色，具体来说，它们受到大气层中的各种物质和温度等因素的影响。\n\n总之，天空是蓝色的，是太阳光线经过地球大气层后剩下的蓝色的部分被反射回来的结果。🌌"

总结