mieki256's diary

_segment-anythingを使って画像を単純化してみる|とりにく
_ゼロショットのセグメンテーション(Segment Anything Model,Python,PyTorch を使用)(Windows 上)
_Shriinivas/gimpsegany: Gimp plugin for integration with Meta Segment Anything

今回は、X:\aiwork\segment-anything\ 以下に、Pythonの仮想環境を作ってインストールしてみた。

また、自分の環境はCUDA11.8がインストールしてあるので、それを前提にしてインストール作業を進めた。

git clone https://github.com/facebookresearch/segment-anything.git
cd segment-anything
python -m venv venv
.\venv\Scripts\activate
python -m pip install pip -U
python -m pip install -U torch torchvision torchaudio numpy --index-url https://download.pytorch.org/whl/cu118
python -m pip install -U opencv-python

python -m pip install -e .
python -m pip install -U opencv-python pycocotools matplotlib onnxruntime onnx gdown

以下を打てば処理にGPUが使えるかどうか確認できるらしい。True が出てきたら利用できる。

> python -c "import torch; print(torch.__version__, torch.cuda.is_available())"
2.5.1+cu118 True

モデルデータをダウンロードする。それぞれ、358MB、1.2GB、2.4GBのサイズ。

mkdir pretrained_checkpoint
cd pretrained_checkpoint
curl -L -O https://dl.fbaipublicfiles.com/segment_anything/sam_vit_b_01ec64.pth
curl -L -O https://dl.fbaipublicfiles.com/segment_anything/sam_vit_l_0b3195.pth
curl -L -O https://dl.fbaipublicfiles.com/segment_anything/sam_vit_h_4b8939.pth

上記の実行結果。

(venv) X:\aiwork\segment-anything\segment-anything> mkdir pretrained_checkpoint
(venv) X:\aiwork\segment-anything\segment-anything> cd pretrained_checkpoint
(venv) X:\aiwork\segment-anything\segment-anything\pretrained_checkpoint> curl -L -O https://dl.fbaipublicfiles.com/segment_anything/sam_vit_b_01ec64.pth
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  357M  100  357M    0     0  10.3M      0  0:00:34  0:00:34 --:--:-- 8915k

(venv) X:\aiwork\segment-anything\segment-anything\pretrained_checkpoint> curl -L -O https://dl.fbaipublicfiles.com/segment_anything/sam_vit_l_0b3195.pth
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 1191M  100 1191M    0     0  10.2M      0  0:01:56  0:01:56 --:--:-- 10.9M

(venv) X:\aiwork\segment-anything\segment-anything\pretrained_checkpoint> curl -L -O https://dl.fbaipublicfiles.com/segment_anything/sam_vit_h_4b8939.pth
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 2445M  100 2445M    0     0  10.1M      0  0:04:00  0:04:00 --:--:-- 10.5M

デモを動かす。

cd ..
python -m pip install -U timm
python scripts/amg.py --checkpoint ./pretrained_checkpoint/sam_vit_l_0b3195.pth --model-type vit_l --input ./notebooks/images --output ./results

結果は以下。

(venv) X:\aiwork\segment-anything\segment-anything> python scripts/amg.py --checkpoint ./pretrained_checkpoint/sam_vit_l_0b3195.pth --model-type vit_l --input ./notebooks/images --output ./results
Loading model...
X:\aiwork\segment-anything\segment-anything\segment_anything\build_sam.py:105: FutureWarning: You are using `torch.load` with `weights_only=False` (the current default value), which uses the default pickle module implicitly. It is possible to construct malicious pickle data which will execute arbitrary code during unpickling (See https://github.com/pytorch/pytorch/blob/main/SECURITY.md#untrusted-models for more details). In a future release, the default value for `weights_only` will be flipped to `True`. This limits the functions that could be executed during unpickling. Arbitrary objects will no longer be allowed to be loaded via this mode unless they are explicitly allowlisted by the user via `torch.serialization.add_safe_globals`. We recommend you start setting `weights_only=True` for any use case where you don't have full control of the loaded file. Please open an issue on GitHub for any issues related to this experimental feature.
  state_dict = torch.load(f)
Processing './notebooks/images\dog.jpg'...
[ WARN:0@13.716] global loadsave.cpp:848 cv::imwrite_ Unsupported depth image for selected encoder is fallbacked to CV_8U.
Processing './notebooks/images\groceries.jpg'...
Processing './notebooks/images\truck.jpg'...
Done!

./notebooks/images/*.jpg の画像ファイル群に対して処理が行われて、./results/ 以下に結果画像が保存される。領域毎にpng画像が作成されていた。

Pythonの仮想環境上であれば、segment-anything が利用できそうだと分かった。

後は GIMPのプラグインから呼び出せれば…。しかし、GIMPを起動した際に、件の仮想環境を利用できる状態になっていないと…。そのあたりどうしたらいいのか…。