* [Enhance] New-style CPU training and inference.

* assert mmcv version

* SyncBN to BN in training and testing

* SyncBN to BN in training and testing

* upload untracked files to this branch

* delete gpu_ids

* fix bugs

* assert args.gpu_id in train.py

* use cfg.gpu_ids = [args.gpu_id]

* use cfg.gpu_ids = [args.gpu_id]

* fix typo

* fix typo

* fix typos

* Fix bug in non-distributed training

* Fix bug in non-distributed testing

* delete uncomment lines

* add args.gpus

* [Feature] benchmark can add work_dir and repeat times

* change the parameter's name

* change the name of the log file

* add skp road

* add default work dir

* make it optional

* Update tools/benchmark.py

Co-authored-by: Miao Zheng <76149310+MeowZheng@users.noreply.github.com>

* Update tools/benchmark.py

Co-authored-by: Miao Zheng <76149310+MeowZheng@users.noreply.github.com>

* fix typo

* modify json name

Co-authored-by: Miao Zheng <76149310+MeowZheng@users.noreply.github.com>

* [Feature] Change option to cfg-option

* add expire date and fix the docs

* modify docstring

* [Feature] Add tool to show origin or augmented train data

* [Feature] Support eval concate dataset

* Add docstring and modify evaluate of concate dataset

Signed-off-by: FreyWang <wangwxyz@qq.com>

* format concat dataset in subfolder of imgfile_prefix

Signed-off-by: FreyWang <wangwxyz@qq.com>

* add unittest of concate dataset

Signed-off-by: FreyWang <wangwxyz@qq.com>

* update unittest for eval dataset with CLASSES is None

Signed-off-by: FreyWang <wangwxyz@qq.com>

* [FIX] bug of generator,  which lead metric to nan when pre_eval=False

Signed-off-by: FreyWang <wangwxyz@qq.com>

* format code

Signed-off-by: FreyWang <wangwxyz@qq.com>

* add more unittest

* add more unittest

* optim concat dataset builder

* Modify default work dir when training.

* Refactor gather_models.py.

* Add train and test matching list.

* Regression benchmark list.

* lower readme name to upper readme name.

* Add url check tool and model inference test tool.

* Modify tool name.

* Support duplicate mode of log json url check.

* Add regression benchmark evaluation automatic tool.

* Add train script generator.

* Only Support script running.

* Add evaluation results gather.

* Add exec Authority.

* Automatically make checkpoint root folder.

* Modify gather results save path.

* Coarse-grained train results gather tool.

* Complete benchmark train script.

* Make some little modifications.

* Fix checkpoint urls.

* Fix unet checkpoint urls.

* Fix fast scnn & fcn checkpoint url.

* Fix fast scnn checkpoint urls.

* Fix fast scnn url.

* Add differential results calculation.

* Add differential results of regression benchmark train results.

* Add an extra argument to select model.

* Update nonlocal_net & hrnet checkpoint url.

* Fix checkpoint url of hrnet and Fix some tta evaluation results and modify gather models tool.

* Modify fast scnn checkpoint url.

* Resolve new comments.

* Fix url check status code bug.

* Resolve some comments.

* Modify train scripts generator.

* Modify work_dir of regression benchmark results.

* model gather tool modification.

* Support progressive test with fewer memory cost.

* Temp code

* Using processor to refactor evaluation workflow.

* refactor eval hook.

* Fix process bar.

* Fix middle save argument.

* Modify some variable name of dataset evaluate api.

* Modify some viriable name of eval hook.

* Fix some priority bugs of eval hook.

* Depreciated efficient_test.

* Fix training progress blocked by eval hook.

* Depreciated old test api.

* Fix test api error.

* Modify outer api.

* Build a sampler test api.

* TODO: Refactor format_results.

* Modify variable names.

* Fix num_classes bug.

* Fix sampler index bug.

* Fix grammaly bug.

* Support batch sampler.

* More readable test api.

* Remove some command arg and fix eval hook bug.

* Support format-only arg.

* Modify format_results of datasets.

* Modify tool which use test apis.

* support cityscapes eval

* fixed cityscapes

* 1. Add comments for batch_sampler;

2. Keep eval hook api same and add deprecated warning;

3. Add doc string for dataset.pre_eval;

* Add efficient_test doc string.

* Modify test tool to compat old version.

* Modify eval hook to compat with old version.

* Modify test api to compat old version api.

* Sampler explanation.

* update warning

* Modify deploy_test.py

* compatible with old output, add efficient test back

* clear logic of exclusive

* Warning about efficient_test.

* Modify format_results save folder.

* Fix bugs of format_results.

* Modify deploy_test.py.

* Update doc

* Fix deploy test bugs.

* Fix custom dataset unit tests.

* Fix dataset unit tests.

* Fix eval hook unit tests.

* Fix some imcompatible.

* Add pre_eval argument for eval hooks.

* Update eval hook doc string.

* Make pre_eval false in default.

* Add unit tests for dataset format_results.

* Fix some comments and bc-breaking bug.

* Fix pre_eval set cfg field.

* Remove redundant codes.

Co-authored-by: Jiarui XU <xvjiarui0826@gmail.com>

* Add header for files

* Delete header in config files

* Add  save_best option in eval_hook.

* Update meta to fix best model can not test bug

* refactor with _do_evaluate

* remove redundent

* add meta

Co-authored-by: Jiarui XU <xvjiarui0826@gmail.com>

* fixed fp16

* update fps

* [Improvement] Move train_cfg/test_cfg inside model

* fixed config dict

* fixed config dict

* memory efficient test

* implement efficient test

* merge

* Add document and docstring

* fix unit test

* add memory usage report