## Motivation

Make MMSeginferencer easier to be used

## Modification

1. Add `_load_weights_to_model` to MMSeginferencer, it is for get
`dataset_meta` from ckpt
2. Modify and remove some parameters of `__call__`, `visualization` and
`postprocess`
3. Add function of save seg mask, remove dump pkl.
4. Refine docstring of MMSeginferencer and SegLocalVisualizer
5. Add the user documentation of MMSeginferencer

## BC-breaking (Optional)

yes, remove some parameters, we need to discuss whether keep them with
deprecated waring or just remove them as the MMSeginferencer just merged
in mmseg a few days.

Co-authored-by: xiexinch <xiexinch@outlook.com>

## Motivation

Support `MMSegInferencer` for providing an easy and clean interface for
single or multiple images inferencing.

Ref: https://github.com/open-mmlab/mmengine/pull/773
https://github.com/open-mmlab/mmocr/pull/1608

## Modification

- mmseg/apis/mmseg_inferencer.py
- mmseg/visualization/local_visualizer.py
- demo/image_demo_with_inferencer.py

## Use cases (Optional)

Based on https://github.com/open-mmlab/mmengine/tree/inference



Add a new image inference demo with `MMSegInferencer`

- demo/image_demo_with_inferencer.py

```shell
python demo/image_demo_with_inferencer.py demo/demo.png fcn_r50-d8_4xb2-40k_cityscapes-512x1024
```

---------

Co-authored-by: MeowZheng <meowzheng@outlook.com>

## Motivation

The registry now supports auto-import modules from the given location.

register_all_modules before running is no longer needed. The modules
will be lazy-imported during building.

- [x] This PR can be merged after
https://github.com/open-mmlab/mmengine/pull/643. The MMEngine version
should be updated.

Ref: https://github.com/open-mmlab/mmdetection/pull/9143

## Motivation

In MMEngine >= 0.2.0, it might directly determine what the backend is by
using the `data_root` path.

## Modification

Set all default `backend_args` values are `None`.

## Motivation 
Add REFUGE datasets
Old PR: https://github.com/open-mmlab/mmsegmentation/pull/2420



---------

Co-authored-by: MengzhangLI <mcmong@pku.edu.cn>

## Motivation

The DETR-related modules have been refactored in
open-mmlab/mmdetection#8763, which causes breakings of MaskFormer and
Mask2Former in both MMDetection (has been fixed in
open-mmlab/mmdetection#9515) and MMSegmentation. This pr fix the bugs in
MMSegmentation.

### TO-DO List

- [x] update configs
- [x] check and modify data flow
- [x] fix unit test
- [x] aligning inference
- [x] write a ckpt converter
- [x] write ckpt update script
- [x] update model zoo
- [x] update model link in readme
- [x] update
[faq.md](https://github.com/open-mmlab/mmsegmentation/blob/dev-1.x/docs/en/notes/faq.md#installation

)

## Tips of Fixing other implementations based on MaskXFormer of mmseg

1. The Transformer modules should be built directly. The original
building with register manner has been refactored.
2. The config requires to be modified. Delete `type` and modify several
keys, according to the modifications in this pr.
3. The `batch_first` is set `True` uniformly in the new implementations.
Hence the data flow requires to be transposed and config of
`batch_first` needs to be modified.
4. The checkpoint trained on the old implementation should be converted
to be used in the new one.

### Convert script

```Python
import argparse
from copy import deepcopy
from collections import OrderedDict

import torch

from mmengine.config import Config
from mmseg.models import build_segmentor
from mmseg.utils import register_all_modules
register_all_modules(init_default_scope=True)


def parse_args():
    parser = argparse.ArgumentParser(
        description='MMSeg convert MaskXFormer model, by Li-Qingyun')
    parser.add_argument('Mask_what_former', type=int,
                        help='Mask what former, can be a `1` or `2`',
                        choices=[1, 2])
    parser.add_argument('CFG_FILE', help='config file path')
    parser.add_argument('OLD_CKPT_FILEPATH', help='old ckpt file path')
    parser.add_argument('NEW_CKPT_FILEPATH', help='new ckpt file path')
    args = parser.parse_args()
    return args


args = parse_args()

def get_new_name(old_name: str):
    new_name = old_name

    if 'encoder.layers' in new_name:
        new_name = new_name.replace('attentions.0', 'self_attn')

    new_name = new_name.replace('ffns.0', 'ffn')

    if 'decoder.layers' in new_name:

        if args.Mask_what_former == 2:
            # for Mask2Former
            new_name = new_name.replace('attentions.0', 'cross_attn')
            new_name = new_name.replace('attentions.1', 'self_attn')
        else:
            # for Mask2Former
            new_name = new_name.replace('attentions.0', 'self_attn')
            new_name = new_name.replace('attentions.1', 'cross_attn')

    return new_name
    
def cvt_sd(old_sd: OrderedDict):
    new_sd = OrderedDict()
    for name, param in old_sd.items():
        new_name = get_new_name(name)
        assert new_name not in new_sd
        new_sd[new_name] = param
    assert len(new_sd) == len(old_sd)
    return new_sd
    
if __name__ == '__main__':
    cfg = Config.fromfile(args.CFG_FILE)
    model_cfg = cfg.model

    segmentor = build_segmentor(model_cfg)

    refer_sd = segmentor.state_dict()
    old_ckpt = torch.load(args.OLD_CKPT_FILEPATH)
    old_sd = old_ckpt['state_dict']

    new_sd = cvt_sd(old_sd)
    print(segmentor.load_state_dict(new_sd))

    new_ckpt = deepcopy(old_ckpt)
    new_ckpt['state_dict'] = new_sd
    torch.save(new_ckpt, args.NEW_CKPT_FILEPATH)
    print(f'{args.NEW_CKPT_FILEPATH} has been saved!')
```

Usage:
```bash
# for example
python ckpt4pr2532.py 1 configs/maskformer/maskformer_r50-d32_8xb2-160k_ade20k-512x512.py original_ckpts/maskformer_r50-d32_8xb2-160k_ade20k-512x512_20221030_182724-cbd39cc1.pth cvt_outputs/maskformer_r50-d32_8xb2-160k_ade20k-512x512_20221030_182724.pth
python ckpt4pr2532.py 2 configs/mask2former/mask2former_r50_8xb2-160k_ade20k-512x512.py original_ckpts/mask2former_r50_8xb2-160k_ade20k-512x512_20221204_000055-4c62652d.pth cvt_outputs/mask2former_r50_8xb2-160k_ade20k-512x512_20221204_000055.pth
```

---------

Co-authored-by: MeowZheng <meowzheng@outlook.com>

## Motivation

Use the new fileio from mmengine
https://github.com/open-mmlab/mmengine/pull/533

## Modification

1. Use `mmengine.fileio` to repalce FileClient  in mmseg/datasets
2. Use `mmengine.fileio` to repalce FileClient in
mmseg/datasets/transforms
3. Use `mmengine.fileio` to repalce FileClient in mmseg/visualization

## BC-breaking (Optional)

we modify all the dataset configurations, so please use the latest config file.

This is an almost exact duplicate of #2500 (that was made to the
`master` branch) now applied to the `1.x` branch.

---

## Motivation

I want to fix a bug through this PR. The bug occurs when two options --
`reduce_zero_label=True`, and custom classes are used.
`reduce_zero_label` remaps the GT seg labels by remapping the zero-class
to 255 which is ignored. Conceptually, this should occur *before* the
`label_map` is applied, which maps *already reduced labels*. However,
currently, the `label_map` is applied before the zero label is reduced.

## Modification

The modification is simple:
- I've just interchanged the order of the two operations by moving a few
lines from bottom to top.
- I've added a test that passes when the fix is introduced, and fails on
the original `master` branch.

## BC-breaking (Optional)

I do not anticipate this change braking any backward-compatibility.

## Checklist

- [x] Pre-commit or other linting tools are used to fix the potential
lint issues.
  - _I've fixed all linting/pre-commit errors._
- [x] The modification is covered by complete unit tests. If not, please
add more unit test to ensure the correctness.
  - _I've added a unit test._ 
- [x] If the modification has potential influence on downstream
projects, this PR should be tested with downstream projects, like MMDet
or MMDet3D.
  - _I don't think this change affects MMDet or MMDet3D._
- [x] The documentation has been modified accordingly, like docstring or
example tutorials.
- _This change fixes an existing bug and doesn't require modifying any
documentation/docstring._

## Motivation

Support for biomedical 3d images augmentation.

## Modification

Add BioMedical3DRandomFlip in mmseg/datasets/transforms/transforms.py.

Co-authored-by: MeowZheng <meowzheng@outlook.com>

## Motivation

The motivation of this PR is to add `gt_edge_map` field to support
boundary loss.

## Modification

- GenerateEdge
Modify `gt_edge` field to `gt_edge_map`.

- PackSegInputs
Add `gt_edge_map` to data_sample.

- stack_batch
Pad `gt_edge_map` to max_shape.

## BC-breaking (Optional)

No

## Use cases (Optional)

Reference `GenerateEdge`.

## Motivation

Fix incorrect `img_shape` value assignment.

## Modification

- mmseg/datasets/transforms/transforms.py

## Motivation

Add Synapse dataset in MMSegmentation.
Old PR: https://github.com/open-mmlab/mmsegmentation/pull/2372.

## Motivation
Add the 3d pad transform for biomedical images, which follows the design
of the nnUNet.

Add the random gamma correction transform for biomedical images, which
follows the design of the nnUNet.

## Motivation

implement nnUNet-style Gaussian Noise and Blur

* tta init

* use mmcv transform

* test city

* add multiscale

* fix merge

* add softmax to post process

* add ut

* add tta pipeline to other datasets

* remove softmax

* add encoder_decoder_tta ut

* add encoder_decoder_tta ut

* rename

* rename file

* rename config

* rm aug_test

* move flip to post process

* fix channel

* [Feature] Add Biomedical 3D array random crop transform

* fix lint

* fix gen crop bbox

* fix gen crop bbox

* docstring

* typo

Co-authored-by: MeowZheng <meowzheng@outlook.com>

* init commits

* fix crop size

* add seg_data2instance_data method

* add ut and update requirement

* update configs and readme

* add model-indel

* update optional requirements

* fix results

* fix lint error

* update results

* update results

* remove mmdet from requirements/optional.txt

* use try import and update README

* add docstring to overwrtied method

* minor change

Co-authored-by: MengzhangLI <mcmong@pku.edu.cn>

* [Feature] Support MaskFormer(NeurIPS'2021) in MMSeg 1.x

* add mmdet try except logic

* refactor config files

* add readme

* fix config

* update models & logs

* add MMDET installation and fix info

* fix comments

* fix

* fix config norm optimizer setting

* update models & logs & unittest

* add docstring of MaskFormerHead

* wait for mmdet 3.0.0rc4

* replace seg_mask with seg_logits & add docstring for batch_input_shape

* use mmdet3.0.0rc4

* fix readme and modify config comments

* add mmdet installation in pr_stage_test.yml

* update mmcv version in pr_stage_test.yml

* add mmdet in build_cpu of pr_stage_test.yml

* modify mmdet& mmcv installation in merge_stage_test.yml

* fix typo

* update test.yml

* update test.yml

* add img_padding_size

* minor change

* add pad_shape to data_samples

* [WIP] LIP dataset

* wip

* keep473

* lip dataset prepare

* add ut and test data

* [Feature]Add Decathlon dataset

* fix test data

* add file

* remove order

* revise default value for prefix

* modify example

* revise based on  comments

* add comments for ut

* [WIP]Generate Edge for dataset

* add ut

* add repr

* add inti

* [Feature] 2.0 PoolFormer

* fix mmcls version

* fix ut error

* fix ut

* fix ut

* [WIP] Add BioMedical data loading

* add depends nibabel

* fix bug

* fix ut

* fix

* add test data

* xyz2zyx zyx2xyz

* format

* remove ignore empty

* remove ignore empty

* remove with seg in LoadBiomedicalAnnotation

* float32

* docstring

* toxyz

* docstring

* introduction

* add evaluator doc and fix typo

* fix ut

* add test dataflow

* add runner reference

* add dataloader config

* fix typo

* minor change

* add out_channels

* fix forward

* add decode_head ut

* add segmentor ut

* refine postprocess

* fix

* [Enhancement]Add `out_file` in add_datasample to for save vis image directly

* comments

* ut

* add pyupgrade hook

* run pyupgrade precommit hook

* [Fix] Config name in ut

* lint

* [WIP] Refactor data flow

* model return

* [WIP] Refactor data flow

* support data_samples is optional

* fix benchmark

* fix base

* minors

* rebase

* fix api

* ut

* fix api inference

* comments

* docstring

* docstring

* docstring

* fix bug of slide inference

* add assert c > 1

* [WIP] mmcv-clean

* [WIP]Remove modules from mmcv.runner and mmcv.utils

* wip

* fix import mmengine

* remove ut

* loadcheckpoint in mae