kwcoco.cli.coco_eval module

Wraps the logic in kwcoco/coco_evaluator.py with a command line script

class kwcoco.cli.coco_eval.CocoEvalCLIConfig(*args, **kwargs)

Bases: CocoEvalConfig

Evaluate and score predicted versus truth detections / classifications in a COCO dataset

Valid options: []

Parameters:

• *args – positional arguments for this data config

• **kwargs – keyword arguments for this data config

default = {'ap_method': <Value('pycocotools')>, 'area_range': <Value(['all'])>, 'assign_workers': <Value(8)>, 'classes_of_interest': <Value(None)>, 'compat': <Value('mutex')>, 'draw': <Value(True)>, 'expt_title': <Value('')>, 'force_pycocoutils': <Value(False)>, 'fp_cutoff': <Value(inf)>, 'ignore_classes': <Value(None)>, 'implicit_ignore_classes': <Value(['ignore'])>, 'implicit_negative_classes': <Value(['background'])>, 'iou_bias': <Value(1)>, 'iou_thresh': <Value(0.5)>, 'load_workers': <Value(0)>, 'max_dets': <Value(inf)>, 'monotonic_ppv': <Value(True)>, 'out_dpath': <Value('./coco_metrics')>, 'pred_dataset': <Value(None)>, 'true_dataset': <Value(None)>, 'use_area_attr': <Value('try')>, 'use_image_names': <Value(False)>}

class kwcoco.cli.coco_eval.CocoEvalCLI

Bases: object

name = 'eval'

CLIConfig

alias of CocoEvalCLIConfig

classmethod main(cmdline=True, **kw)

Example

>>> # xdoctest: +REQUIRES(module:kwplot)
>>> from kwcoco.cli.coco_eval import *  # NOQA
>>> import ubelt as ub
>>> from kwcoco.cli.coco_eval import *  # NOQA
>>> from os.path import join
>>> import kwcoco
>>> dpath = ub.Path.appdir('kwcoco/tests/eval').ensuredir()
>>> true_dset = kwcoco.CocoDataset.demo('shapes8')
>>> from kwcoco.demo.perterb import perterb_coco
>>> kwargs = {
>>>     'box_noise': 0.5,
>>>     'n_fp': (0, 10),
>>>     'n_fn': (0, 10),
>>> }
>>> pred_dset = perterb_coco(true_dset, **kwargs)
>>> true_dset.fpath = join(dpath, 'true.mscoco.json')
>>> pred_dset.fpath = join(dpath, 'pred.mscoco.json')
>>> true_dset.dump(true_dset.fpath)
>>> pred_dset.dump(pred_dset.fpath)
>>> draw = False  # set to false for faster tests
>>> CocoEvalCLI.main(cmdline=False,
>>>     true_dataset=true_dset.fpath,
>>>     pred_dataset=pred_dset.fpath,
>>>     draw=draw, out_dpath=dpath)

kwcoco.cli.coco_eval.main(cmdline=True, **kw)

Todo

• [X] should live in kwcoco.cli.coco_eval

CommandLine

# Generate test data
xdoctest -m kwcoco.cli.coco_eval CocoEvalCLI.main

kwcoco eval \
    --true_dataset=$HOME/.cache/kwcoco/tests/eval/true.mscoco.json \
    --pred_dataset=$HOME/.cache/kwcoco/tests/eval/pred.mscoco.json \
    --out_dpath=$HOME/.cache/kwcoco/tests/eval/out \
    --force_pycocoutils=False \
    --area_range=all,0-4096,4096-inf

nautilus $HOME/.cache/kwcoco/tests/eval/out

kwcoco.cli.coco_eval._CLI

alias of CocoEvalCLI