windows10笔记本安装和测试deepseek-janus多模态模型

1、安装 anconda 软件：下载地址：Download Anaconda Distribution | Anaconda

2、克隆（下载代码）

git clone https://github.com/deepseek-ai/Janus.git

3、安装 python 3.8环境

conda create -n janus python=3.8

4、进入环境：

conda activate janus

5、安装依赖：

pip install -r requirements.txt

6、编写测试代码：

import torch
from transformers import AutoModelForCausalLM

from janus.models import MultiModalityCausalLM, VLChatProcessor
from janus.utils.io import load_pil_images

# specify the path to the model
model_path = "deepseek-ai/Janus-1.3B"
vl_chat_processor: VLChatProcessor = VLChatProcessor.from_pretrained(model_path)
tokenizer = vl_chat_processor.tokenizer

vl_gpt: MultiModalityCausalLM = AutoModelForCausalLM.from_pretrained( model_path, trust_remote_code=True)
vl_gpt = vl_gpt.to(torch.bfloat16).cuda().eval()

conversation = [
    {
        "role": "User",
        "content": "<image_placeholder>\n 图片上是什么？",
        "images": ["images/test.png"],
    },
    {"role": "Assistant", "content": ""},
]

# load images and prepare for inputs
pil_images = load_pil_images(conversation)
prepare_inputs = vl_chat_processor(  conversations=conversation, images=pil_images, force_batchify=True).to(vl_gpt.device)

# # run image encoder to get the image embeddings
inputs_embeds = vl_gpt.prepare_inputs_embeds(**prepare_inputs)

# # run the model to get the response
outputs = vl_gpt.language_model.generate(
    inputs_embeds=inputs_embeds,  attention_mask=prepare_inputs.attention_mask,
    pad_token_id=tokenizer.eos_token_id, bos_token_id=tokenizer.bos_token_id,
    eos_token_id=tokenizer.eos_token_id,  max_new_tokens=512, do_sample=False, use_cache=True,
)

answer = tokenizer.decode(outputs[0].cpu().tolist(), skip_special_tokens=True)
print(f"{prepare_inputs['sft_format'][0]}", answer)

我的笔记本配置是 3060， 先测试下 1.3B模型

7、测试代码：

python test.py

原始图片：（test.png）

输出内容：

User: <image_placeholder>
图片上是什么？

Assistant: 图片上是一只小鸟站在一个金属结构上。
背景是模糊的绿色，可能是草地或树木。
小鸟的羽毛主要是灰色和白色，头部有黑色和白色的斑纹。

占用显存大概 5G；结果非常精准，解析速度大概 3秒，简直难以置信；

8、报错及解决：

8.1 报错：
causal_mask = torch.triu(causal_mask, diagonal=1) 
RuntimeError: "triu_tril_cuda_template" not implemented for 'BFloat16'

解决方案：
pip install torch==2.2.2
pip install transformers==4.44.0

8.2 报错：ImportError: FlashAttention2 has been toggled on, 
but it cannot be used due to the following error:
the package flash_attn seems to be not installed. 

解决方案：
pip install flash-attn --no-build-isolation

9、多显卡支持方案：

修改代码：

# 指定使用的 GPU 设备
device = torch.device("cuda:1")
vl_gpt = vl_gpt.to(torch.bfloat16).to(device).eval()

deepseek 推理模型、多模态模型都非常棒，完全可以独立部署及商用，感兴趣的朋友可以加我粉丝；