MMDetection
Tutorials
Google Colaboratory - Image and Video with Faster R-CNN on Version 3.3.0
https://colab.research.google.com/drive/11pbaSUnA8eWopz2-OHh9XkCiTQdbMl8t
Google Colaboratory - MS-COCO Training with Faster R-CNN on Version 3.3.0
https://colab.research.google.com/drive/1tYssvTbCa0zuICyeVShUx42PGMeJxWlW
Google Colaboratory - Image and Video with Faster R-CNN/YOLOX/DETR/Mask R-CNN/Panoptic FPN/Mask2Former on Version 2.28.2
https://colab.research.google.com/drive/1hbOLBwOBeCVOELl3-I2yXD0QTmWKVylZ
Getting Started
https://mmdetection.readthedocs.io/en/latest/get_started.html
https://qiita.com/apiss/items/48475abc20abf26c0d27
https://qiita.com/fam_taro/items/7f028dfeae2a79a10fe1
Easier Preparation on Google Colab
The official approach is using OpenMIM and the latest repositories on GitHub but the following steps simply work on Google Colab:
!pip install mmengine !pip install mmcv==2.1.0 !pip install mmdet
Tricks and Traps for Version 3
## 01. MMEngine is recommended to be installed mim install mmengine
## 02. show_result_pyplot and model.show_result() have been removed (see below for alternative approaches) #from mmdet.apis import show_result_pyplot
## 03. Register all modules from mmdet.utils import register_all_modules register_all_modules()
## 04. Some configuration file names have been changed #mim download mmdet --config faster_rcnn_r50_fpn_1x_coco --dest $config_dir mim download mmdet --config faster-rcnn_r50_fpn_1x_coco --dest $config_dir
Troubleshooting - MMCV Compatibility on Google Colab
https://github.com/open-mmlab/mmcv/issues/3059
## Downgrading PyTorch on Google Colab (2.3.0+cu121) for MMCV Compatibility (Confirmed 2024/07/04) !pip install torch==2.1.0 torchvision==0.16.0 torchaudio==2.1.0 --index-url https://download.pytorch.org/whl/cu121 !pip install --upgrade openmim !mim install 'mmcv==2.1.0'
Troubleshooting - Conflict with Jedi (on Google Colab)
## Resolving the dependency conflicts for openmim: ipython 7.9.0 requires jedi>=0.10 !pip install --upgrade jedi
Troubleshooting - model.show_result() was removed after Version 2
https://github.com/open-mmlab/mmdetection/issues/10380
- MMEngine Visualizer is recommended to be used instead of show_result()
- Option A is the official approach, but font sizes might be too small if on a large image
- Option B is a tailor-made approach for visualization with annotations
## Loading Image import mmcv image = mmcv.imread(img, channel_order='rgb') #image = mmcv.imconvert(image, 'bgr', 'rgb')
## Model and Result model = init_detector(config_file, checkpoint_file, device=device) result = inference_detector(model, img)
## Option A from mmdet.registry import VISUALIZERS visualizer = VISUALIZERS.build(model.cfg.visualizer) visualizer.dataset_meta = model.dataset_meta visualizer.add_datasample('result', image, data_sample=result, draw_gt=False, show=True, wait_time=0, pred_score_thr=0.70, out_file='result.jpg') visualizer.get_image() visualizer.show()
## Option B import torch import cv2 from mmengine.visualization import Visualizer pred_score_thr = 0.70 scores = result.pred_instances.scores[torch.where(result.pred_instances.scores>=pred_score_thr)] labels = result.pred_instances.labels[torch.where(result.pred_instances.scores>=pred_score_thr)] bboxes = result.pred_instances.bboxes[torch.where(result.pred_instances.scores>=pred_score_thr)] labels_classes = list(map(lambda x: model.dataset_meta['classes'][x], labels.tolist())) labels_palette = list(map(lambda x: model.dataset_meta['palette'][x], labels.tolist())) labels_print = list(map(lambda x, y: f'{x}: {(y * 100.0):.1f}', labels_classes, scores.tolist())) bboxes_origins = list(map(lambda x: [x[0], x[1]], bboxes.tolist())) visualizer = Visualizer(image=image) visualizer.draw_bboxes(bboxes, edge_colors=labels_palette, line_widths=3) visualizer.draw_texts(labels_print, torch.tensor(bboxes_origins), font_sizes=30, colors='white', bboxes=dict(facecolor='red', edgecolor='black', linewidth=0, alpha=0.8)) visualizer.get_image() visualizer.show() cv2.imwrite('result.jpg', cv2.cvtColor(visualizer.get_image(), cv2.COLOR_RGB2BGR))
Training with Custom Dataset
https://mmdetection.readthedocs.io/en/latest/user_guides/dataset_prepare.html
https://mmdetection.readthedocs.io/en/latest/user_guides/train.html
Information - Loss Functions
Information - Optimizers
https://pytorch.org/docs/stable/optim.html
optimizer=dict(_delete_=True) removes originally-defined parameters in dict()
Information - Freeze Parameters
https://github.com/open-mmlab/mmdetection/blob/main/mmdet/models/backbones/resnet.py
- frozen_stages = -1 : not freezing any parameters
- frozen_stages = 0 : freezing the stem part only
- frozen_stages = n [n >= 1] : freezing the stem part and first n stages
Results
Faster R-CNN
Mask R-CNN
Mask2Former
MS-COCO Training - to be continued...
Results (Video)
References
https://github.com/open-mmlab/mmdetection/blob/master/docs/en/get_started.md/
https://mmdetection.readthedocs.io/en/latest/get_started.html
https://mmdetection.readthedocs.io/en/latest/1_exist_data_model.html
https://dev.classmethod.jp/articles/mmdetection-detect-samples/
https://github.com/open-mmlab/mmcv/issues/3059
https://github.com/open-mmlab/mmdetection/issues/10380
https://qiita.com/apiss/items/48475abc20abf26c0d27
https://qiita.com/fam_taro/items/7f028dfeae2a79a10fe1
https://qiita.com/saliton/items/24b69d1fa274b21c489a
https://qiita.com/sister_DB/items/42560f551b65dd976217
Acknowledgments
Daiphys is a professional-service company for research and development of leading-edge technologies in physics and engineering.
Get started accelerating your business through our deep expertise in R&D with AI, quantum computing, and space development; please get in touch with Daiphys today!
Daiphys Technologies LLC - https://www.daiphys.com/