kwcoco.cli.coco_eval module¶
Wraps the logic in kwcoco/coco_evaluator.py with a command line script
- class kwcoco.cli.coco_eval.CocoEvalCLIConfig(data=None, default=None, cmdline=False)[source]¶
Bases:
Config
Evaluate and score predicted versus truth detections / classifications in a COCO dataset
- default = {'ap_method': <Value(None: 'pycocotools')>, 'area_range': <Value(None: ['all'])>, 'assign_workers': <Value(None: 8)>, 'classes_of_interest': <Value(<class 'list'>: None)>, 'compat': <Value(None: 'mutex')>, 'draw': <Value(None: True)>, 'expt_title': <Value(<class 'str'>: '')>, 'force_pycocoutils': <Value(None: False)>, 'fp_cutoff': <Value(None: inf)>, 'ignore_classes': <Value(<class 'list'>: None)>, 'implicit_ignore_classes': <Value(None: ['ignore'])>, 'implicit_negative_classes': <Value(None: ['background'])>, 'iou_bias': <Value(None: 1)>, 'iou_thresh': <Value(None: 0.5)>, 'load_workers': <Value(None: 0)>, 'max_dets': <Value(None: inf)>, 'monotonic_ppv': <Value(None: True)>, 'out_dpath': <Value(<class 'str'>: './coco_metrics')>, 'ovthresh': <Value(None: None)>, 'pred_dataset': <Value(<class 'str'>: None)>, 'true_dataset': <Value(<class 'str'>: None)>, 'use_area_attr': <Value(None: 'try')>, 'use_image_names': <Value(None: False)>}¶
- class kwcoco.cli.coco_eval.CocoEvalCLI[source]¶
Bases:
object
- name = 'eval'¶
- CLIConfig¶
alias of
CocoEvalCLIConfig
- classmethod main(cmdline=True, **kw)[source]¶
Example
>>> # xdoctest: +REQUIRES(module:kwplot) >>> from kwcoco.cli.coco_eval import * # NOQA >>> import ubelt as ub >>> from kwcoco.cli.coco_eval import * # NOQA >>> from os.path import join >>> import kwcoco >>> dpath = ub.Path.appdir('kwcoco/tests/eval').ensuredir() >>> true_dset = kwcoco.CocoDataset.demo('shapes8') >>> from kwcoco.demo.perterb import perterb_coco >>> kwargs = { >>> 'box_noise': 0.5, >>> 'n_fp': (0, 10), >>> 'n_fn': (0, 10), >>> } >>> pred_dset = perterb_coco(true_dset, **kwargs) >>> true_dset.fpath = join(dpath, 'true.mscoco.json') >>> pred_dset.fpath = join(dpath, 'pred.mscoco.json') >>> true_dset.dump(true_dset.fpath) >>> pred_dset.dump(pred_dset.fpath) >>> draw = False # set to false for faster tests >>> CocoEvalCLI.main( >>> true_dataset=true_dset.fpath, >>> pred_dataset=pred_dset.fpath, >>> draw=draw, out_dpath=dpath)
- kwcoco.cli.coco_eval.main(cmdline=True, **kw)[source]¶
Todo
[X] should live in kwcoco.cli.coco_eval
CommandLine
# Generate test data xdoctest -m kwcoco.cli.coco_eval CocoEvalCLI.main kwcoco eval \ --true_dataset=$HOME/.cache/kwcoco/tests/eval/true.mscoco.json \ --pred_dataset=$HOME/.cache/kwcoco/tests/eval/pred.mscoco.json \ --out_dpath=$HOME/.cache/kwcoco/tests/eval/out \ --force_pycocoutils=False \ --area_range=all,0-4096,4096-inf nautilus $HOME/.cache/kwcoco/tests/eval/out