:py:mod:`kwcoco.cli.coco_eval`
==============================

.. py:module:: kwcoco.cli.coco_eval

.. autoapi-nested-parse::

   Wraps the logic in kwcoco/coco_evaluator.py with a command line script



Module Contents
---------------

Classes
~~~~~~~

.. autoapisummary::

   kwcoco.cli.coco_eval.CocoEvalCLIConfig
   kwcoco.cli.coco_eval.CocoEvalCLI



Functions
~~~~~~~~~

.. autoapisummary::

   kwcoco.cli.coco_eval.main



Attributes
~~~~~~~~~~

.. autoapisummary::

   kwcoco.cli.coco_eval._CLI


.. py:class:: CocoEvalCLIConfig(data=None, default=None, cmdline=False)

   Bases: :py:obj:`scriptconfig.Config`

   Base class for custom configuration objects

   A configuration that can be specified by commandline args, a yaml config
   file, and / or a in-code dictionary. To use, define a class variable named
   "default" and assing it to a dict of default values. You can also use
   special `Value` classes to denote types. You can also define a method
   `normalize`, to postprocess the arguments after this class receives them.

   Usage:
       Create a class that herits from this class.

       Assign the "default" class-level variable as a dictionary of options

       The keys of this dictionary must be command line friendly strings.

       The values of the "defaults dictionary" can be literal values or
       instances of the :class:`scriptconfig.Value` class, which allows
       for specification of default values, type information, help strings,
       and aliases.

       You may also implement normalize (function with that takes no args and
       has no return) to postprocess your results after initialization.

       When creating an instance of the class the defaults variable is used
       to make a dictionary-like object. You can override defaults by
       specifying the ``data`` keyword argument to either a file path or
       another dictionary. You can also specify ``cmdline=True`` to allow
       the contents of ``sys.argv`` to influence the values of the new
       object.

       An instance of the config class behaves like a dictinary, except that
       you cannot set keys that do not already exist (as specified in the
       defaults dict).

       Key Methods:

           * dump - dump a json representation to a file

           * dumps - dump a json representation to a string

           * argparse - create the argparse object associated with this config

           * argparse - create an :class:`argparse.ArgumentParser` object that
               is defined by the defaults of this config.

           * load - rewrite the values based on a filepath, dictionary, or
               command line contents.

   :ivar _data: this protected variable holds the raw state of the config
                object and is accessed by the dict-like

   :ivar _default: this protected variable maintains the default values for
                   this config.


   .. rubric:: Example

   >>> # Inherit from `Config` and assign `default`
   >>> import scriptconfig as scfg
   >>> class MyConfig(scfg.Config):
   >>>     default = {
   >>>         'option1': scfg.Value((1, 2, 3), tuple),
   >>>         'option2': 'bar',
   >>>         'option3': None,
   >>>     }
   >>> # You can now make instances of this class
   >>> config1 = MyConfig()
   >>> config2 = MyConfig(default=dict(option1='baz'))

   .. py:attribute:: __doc__
      

      

   .. py:attribute:: default
      

      


.. py:class:: CocoEvalCLI

   .. py:attribute:: name
      :annotation: = eval

      

   .. py:attribute:: CLIConfig
      

      

   .. py:method:: main(cls, cmdline=True, **kw)
      :classmethod:

      .. rubric:: Example

      >>> # xdoctest: +REQUIRES(module:kwplot)
      >>> import ubelt as ub
      >>> from kwcoco.cli.coco_eval import *  # NOQA
      >>> from os.path import join
      >>> import kwcoco
      >>> dpath = ub.ensure_app_cache_dir('kwcoco/tests/eval')
      >>> true_dset = kwcoco.CocoDataset.demo('shapes8')
      >>> from kwcoco.demo.perterb import perterb_coco
      >>> kwargs = {
      >>>     'box_noise': 0.5,
      >>>     'n_fp': (0, 10),
      >>>     'n_fn': (0, 10),
      >>> }
      >>> pred_dset = perterb_coco(true_dset, **kwargs)
      >>> true_dset.fpath = join(dpath, 'true.mscoco.json')
      >>> pred_dset.fpath = join(dpath, 'pred.mscoco.json')
      >>> true_dset.dump(true_dset.fpath)
      >>> pred_dset.dump(pred_dset.fpath)
      >>> draw = False  # set to false for faster tests
      >>> CocoEvalCLI.main(
      >>>     true_dataset=true_dset.fpath,
      >>>     pred_dataset=pred_dset.fpath,
      >>>     draw=draw)



.. py:function:: main(cmdline=True, **kw)

   .. todo:: - [X] should live in kwcoco.cli.coco_eval

   .. rubric:: CommandLine

   .. code-block:: bash


       # Generate test data
       xdoctest -m kwcoco.cli.coco_eval CocoEvalCLI.main

       kwcoco eval \
           --true_dataset=$HOME/.cache/kwcoco/tests/eval/true.mscoco.json \
           --pred_dataset=$HOME/.cache/kwcoco/tests/eval/pred.mscoco.json \
           --out_dpath=$HOME/.cache/kwcoco/tests/eval/out \
           --force_pycocoutils=False \
           --area_range=all,0-4096,4096-inf

       nautilus $HOME/.cache/kwcoco/tests/eval/out


.. py:data:: _CLI