`kwcoco.metrics.drawing`¶

Module Contents¶

Functions¶

`draw_perclass_roc`(cx_to_info, classes=None, prefix='', fnum=1, fp_axis='count', **kw)	Parameters cx_to_info (PerClass_Measures \| Dict)
`demo_format_options`()
`concice_si_display`(val, eps=1e-08, precision=2, si_thresh=4)	Display numbers in scientific notation if above a threshold
`_realpos_label_suffix`(info)	Creates a label suffix that indicates the number of real positive cases
`draw_perclass_prcurve`(cx_to_info, classes=None, prefix='', fnum=1, **kw)	Parameters cx_to_info (PerClass_Measures \| Dict)
`draw_perclass_thresholds`(cx_to_info, key='mcc', classes=None, prefix='', fnum=1, **kw)	Parameters cx_to_info (PerClass_Measures \| Dict)
`draw_roc`(info, prefix='', fnum=1, **kw)	Parameters info (Measures \| Dict)
`draw_prcurve`(info, prefix='', fnum=1, **kw)	Draws a single pr curve.
`draw_threshold_curves`(info, keys=None, prefix='', fnum=1, **kw)	Parameters info (Measures \| Dict)

kwcoco.metrics.drawing.draw_perclass_roc(cx_to_info, classes=None, prefix='', fnum=1, fp_axis='count', **kw)[source]¶

Parameters

cx_to_info (PerClass_Measures | Dict)
fp_axis (str) – can be count or rate

kwcoco.metrics.drawing.demo_format_options()[source]¶

kwcoco.metrics.drawing.concice_si_display(val, eps=1e-08, precision=2, si_thresh=4)[source]¶

Display numbers in scientific notation if above a threshold

Parameters

eps (float) – threshold to be formated as an integer if other integer conditions hold.
precision (int) – maximum significant digits (might print less)
si_thresh (int) – If the number is less than 10^{si_thresh}, then it will be printed as an integer if it is within eps of an integer.

References

https://docs.python.org/2/library/stdtypes.html#string-formatting-operations

Example

>>> grid = {
>>>     'sign': [1, -1],
>>>     'exp': [1, -1],
>>>     'big_part': [0, 32132e3, 4000000032],
>>>     'med_part': [0, 0.5, 0.9432, 0.000043, 0.01, 1, 2],
>>>     'small_part': [0, 1321e-3, 43242e-11],
>>> }
>>> for kw in ub.named_product(grid):
>>>     sign = kw.pop('sign')
>>>     exp = kw.pop('exp')
>>>     raw = (sum(map(float, kw.values())))
>>>     val = sign * raw ** exp if raw != 0 else sign * raw
>>>     print('{:>20} - {}'.format(concice_si_display(val), val))
>>> from kwcoco.metrics.drawing import *  # NOQA
>>> print(concice_si_display(40000000432432))
>>> print(concice_si_display(473243280432890))
>>> print(concice_si_display(473243284289))
>>> print(concice_si_display(473243289))
>>> print(concice_si_display(4739))
>>> print(concice_si_display(473))
>>> print(concice_si_display(0.432432))
>>> print(concice_si_display(0.132432))
>>> print(concice_si_display(1.0000043))
>>> print(concice_si_display(01.00000000000000000000000000043))

kwcoco.metrics.drawing._realpos_label_suffix(info)[source]¶

Creates a label suffix that indicates the number of real positive cases versus the total amount of cases considered for an evaluation curve.

Parameters: info (Dict) – with keys, nsuppert, realpos_total

Example

>>> from kwcoco.metrics.drawing import *  # NOQA
>>> info = {'nsupport': 10, 'realpos_total': 10}
>>> _realpos_label_suffix(info)
10/10
>>> info = {'nsupport': 10.0, 'realpos_total': 10.0}
>>> _realpos_label_suffix(info)
10/10
>>> info = {'nsupport': 10.3333, 'realpos_total': 10.22222}
>>> _realpos_label_suffix(info)
10.22/10.33
>>> info = {'nsupport': 10.000000001, 'realpos_total': None}
>>> _realpos_label_suffix(info)
10
>>> info = {'nsupport': 10.009}
>>> _realpos_label_suffix(info)
10.01
>>> info = {'nsupport': 3331033334342.432, 'realpos_total': 1033334342.432}
>>> _realpos_label_suffix(info)
1e9/3.3e12
>>> info = {'nsupport': 0.007, 'realpos_total': 0.0000893}
>>> _realpos_label_suffix(info)
8.9e-5/0.007

kwcoco.metrics.drawing.draw_perclass_prcurve(cx_to_info, classes=None, prefix='', fnum=1, **kw)[source]¶

Parameters: cx_to_info (PerClass_Measures | Dict)

Example

>>> # xdoctest: +REQUIRES(module:kwplot)
>>> from kwcoco.metrics.drawing import *  # NOQA
>>> from kwcoco.metrics import DetectionMetrics
>>> dmet = DetectionMetrics.demo(
>>>     nimgs=3, nboxes=(0, 10), n_fp=(0, 3), n_fn=(0, 2), classes=3, score_noise=0.1, box_noise=0.1, with_probs=False)
>>> cfsn_vecs = dmet.confusion_vectors()
>>> print(cfsn_vecs.data.pandas())
>>> classes = cfsn_vecs.classes
>>> cx_to_info = cfsn_vecs.binarize_ovr().measures()['perclass']
>>> print('cx_to_info = {}'.format(ub.repr2(cx_to_info, nl=1)))
>>> import kwplot
>>> kwplot.autompl()
>>> draw_perclass_prcurve(cx_to_info, classes)
>>> # xdoctest: +REQUIRES(--show)
>>> kwplot.show_if_requested()

kwcoco.metrics.drawing.draw_perclass_thresholds(cx_to_info, key='mcc', classes=None, prefix='', fnum=1, **kw)[source]¶

Parameters: cx_to_info (PerClass_Measures | Dict)

Note

Each category is inspected independently of one another, there is no notion of confusion.

Example

>>> # xdoctest: +REQUIRES(module:kwplot)
>>> from kwcoco.metrics.drawing import *  # NOQA
>>> from kwcoco.metrics import ConfusionVectors
>>> cfsn_vecs = ConfusionVectors.demo()
>>> classes = cfsn_vecs.classes
>>> ovr_cfsn = cfsn_vecs.binarize_ovr(keyby='name')
>>> cx_to_info = ovr_cfsn.measures()['perclass']
>>> import kwplot
>>> kwplot.autompl()
>>> key = 'mcc'
>>> draw_perclass_thresholds(cx_to_info, key, classes)
>>> # xdoctest: +REQUIRES(--show)
>>> kwplot.show_if_requested()

kwcoco.metrics.drawing.draw_roc(info, prefix='', fnum=1, **kw)[source]¶

Parameters: info (Measures | Dict)

Note

There needs to be enough negative examples for using ROC to make any sense!

Example

>>> # xdoctest: +REQUIRES(module:kwplot, module:seaborn)
>>> from kwcoco.metrics.drawing import *  # NOQA
>>> from kwcoco.metrics import DetectionMetrics
>>> dmet = DetectionMetrics.demo(nimgs=30, null_pred=1, classes=3,
>>>                              nboxes=10, n_fp=10, box_noise=0.3,
>>>                              with_probs=False)
>>> dmet.true_detections(0).data
>>> cfsn_vecs = dmet.confusion_vectors(compat='mutex', prioritize='iou', bias=0)
>>> print(cfsn_vecs.data._pandas().sort_values('score'))
>>> classes = cfsn_vecs.classes
>>> info = ub.peek(cfsn_vecs.binarize_ovr().measures()['perclass'].values())
>>> # xdoctest: +REQUIRES(--show)
>>> import kwplot
>>> kwplot.autompl()
>>> draw_roc(info)
>>> kwplot.show_if_requested()

kwcoco.metrics.drawing.draw_prcurve(info, prefix='', fnum=1, **kw)[source]¶

Draws a single pr curve.

Parameters: info (Measures | Dict)

Example

>>> # xdoctest: +REQUIRES(module:kwplot)
>>> from kwcoco.metrics import DetectionMetrics
>>> dmet = DetectionMetrics.demo(
>>>     nimgs=10, nboxes=(0, 10), n_fp=(0, 1), classes=3)
>>> cfsn_vecs = dmet.confusion_vectors()

>>> classes = cfsn_vecs.classes
>>> info = cfsn_vecs.binarize_classless().measures()
>>> import kwplot
>>> kwplot.autompl()
>>> draw_prcurve(info)
>>> # xdoctest: +REQUIRES(--show)
>>> kwplot.show_if_requested()

kwcoco.metrics.drawing.draw_threshold_curves(info, keys=None, prefix='', fnum=1, **kw)[source]¶

Parameters: info (Measures | Dict)

Example

>>> # xdoctest: +REQUIRES(module:kwplot)
>>> import sys, ubelt
>>> sys.path.append(ubelt.expandpath('~/code/kwcoco'))
>>> from kwcoco.metrics.drawing import *  # NOQA
>>> from kwcoco.metrics import DetectionMetrics
>>> dmet = DetectionMetrics.demo(
>>>     nimgs=10, nboxes=(0, 10), n_fp=(0, 1), classes=3)
>>> cfsn_vecs = dmet.confusion_vectors()
>>> info = cfsn_vecs.binarize_classless().measures()
>>> keys = None
>>> import kwplot
>>> kwplot.autompl()
>>> draw_threshold_curves(info, keys)
>>> # xdoctest: +REQUIRES(--show)
>>> kwplot.show_if_requested()

kwcoco.metrics.drawing¶

Module Contents¶

Functions¶

`kwcoco.metrics.drawing`¶