kwcoco.util.util_json module

Deprecated: use kwutil.Json instead

kwcoco.util.util_json.ensure_json_serializable(dict_, normalize_containers=False, verbose=0)

Attempt to convert common types (e.g. numpy) into something json compliant

Convert numpy and tuples into lists

Parameters:

normalize_containers (bool) – if True, normalizes dict containers to be standard python structures. Defaults to False.

Note

This function is now in kwutil. Might be best to use that variant. See kwutil.util_json.ensure_json_serializable().

Example

>>> data = ub.ddict(lambda: int)
>>> data['foo'] = ub.ddict(lambda: int)
>>> data['bar'] = np.array([1, 2, 3])
>>> data['foo']['a'] = 1
>>> data['foo']['b'] = (1, np.array([1, 2, 3]), {3: np.int32(3), 4: np.float16(1.0)})
>>> dict_ = data
>>> print(ub.urepr(data, nl=-1))
>>> assert list(find_json_unserializable(data))
>>> result = ensure_json_serializable(data, normalize_containers=True)
>>> print(ub.urepr(result, nl=-1))
>>> assert not list(find_json_unserializable(result))
>>> assert type(result) is dict

kwcoco.util.util_json.find_json_unserializable(data, quickcheck=False)

Recurse through json datastructure and find any component that causes a serialization error. Record the location of these errors in the datastructure as we recurse through the call tree.

Note

This function is now in kwutil. Might be best to use that variant. See kwutil.util_json.find_json_unserializable().

Parameters:

• data (object) – data that should be json serializable

• quickcheck (bool) – if True, check the entire datastructure assuming its ok before doing the python-based recursive logic.

Returns:

list of “bad part” dictionaries containing items

’value’ - the value that caused the serialization error

’loc’ - which contains a list of key/indexes that can be used to lookup the location of the unserializable value. If the “loc” is a list, then it indicates a rare case where a key in a dictionary is causing the serialization error.

Return type:

List[Dict]

Example

>>> from kwcoco.util.util_json import *  # NOQA
>>> part = ub.ddict(lambda: int)
>>> part['foo'] = ub.ddict(lambda: int)
>>> part['bar'] = np.array([1, 2, 3])
>>> part['foo']['a'] = 1
>>> # Create a dictionary with two unserializable parts
>>> data = [1, 2, {'nest1': [2, part]}, {frozenset({'badkey'}): 3, 2: 4}]
>>> parts = list(find_json_unserializable(data))
>>> print('parts = {}'.format(ub.urepr(parts, nl=1)))
>>> # Check expected structure of bad parts
>>> assert len(parts) == 2
>>> part = parts[1]
>>> assert list(part['loc']) == [2, 'nest1', 1, 'bar']
>>> # We can use the "loc" to find the bad value
>>> for part in parts:
>>>     # "loc" is a list of directions containing which keys/indexes
>>>     # to traverse at each descent into the data structure.
>>>     directions = part['loc']
>>>     curr = data
>>>     special_flag = False
>>>     for key in directions:
>>>         if isinstance(key, list):
>>>             # special case for bad keys
>>>             special_flag = True
>>>             break
>>>         else:
>>>             # normal case for bad values
>>>             curr = curr[key]
>>>     if special_flag:
>>>         assert part['data'] in curr.keys()
>>>         assert part['data'] is key[1]
>>>     else:
>>>         assert part['data'] is curr

Example

>>> # xdoctest: +SKIP("TODO: circular ref detect algo is wrong, fix it")
>>> from kwcoco.util.util_json import *  # NOQA
>>> import pytest
>>> # Test circular reference
>>> data = [[], {'a': []}]
>>> data[1]['a'].append(data)
>>> with pytest.raises(ValueError, match="Circular reference detected at.*1, 'a', 1*"):
...     parts = list(find_json_unserializable(data))
>>> # Should be ok here
>>> shared_data = {'shared': 1}
>>> data = [[shared_data], shared_data]
>>> parts = list(find_json_unserializable(data))

kwcoco.util.util_json.indexable_allclose(dct1, dct2, return_info=False)

Walks through two nested data structures and ensures that everything is roughly the same.

Note

DEPRECATED. Use ubelt.IndexableWalker.allclose() instead.

Parameters:

• dct1 – a nested indexable item

• dct2 – a nested indexable item

Example

>>> from kwcoco.util.util_json import indexable_allclose
>>> dct1 = {
>>>     'foo': [1.222222, 1.333],
>>>     'bar': 1,
>>>     'baz': [],
>>> }
>>> dct2 = {
>>>     'foo': [1.22222, 1.333],
>>>     'bar': 1,
>>>     'baz': [],
>>> }
>>> assert indexable_allclose(dct1, dct2)

class kwcoco.util.util_json.Difference(path: Tuple, value1: Any, value2: Any)

Bases: NamedTuple

A result class of indexable_diff that organizes what the difference between the indexables is.

Create new instance of Difference(path, value1, value2)

path: Tuple

Alias for field number 0

value1: Any

Alias for field number 1

value2: Any

Alias for field number 2

_asdict()

Return a new dict which maps field names to their values.

_field_defaults = {}

_fields = ('path', 'value1', 'value2')

classmethod _make(iterable)

Make a new Difference object from a sequence or iterable

_replace(**kwds)

Return a new Difference object replacing specified fields with new values

kwcoco.util.util_json.indexable_diff(dct1, dct2, rtol=1e-05, atol=1e-08, equal_nan=False)

Walks through two nested data structures finds differences in the structures.

Parameters:

• dct1 – a nested indexable item

• dct2 – a nested indexable item

Note

DEPRECATED. Use ubelt.IndexableWalker.diff() instead.

Returns:

information about the diff with

”similarity”: a score between 0 and 1 “num_differences” being the number of paths not common plus the

number of common paths with differing values.

”unique1”: being the paths that were unique to dct1 “unique2”: being the paths that were unique to dct2 “faillist”: a list 3-tuples of common path and differing values “approximations”:

is the number of approximately equal items (i.e. floats) there were

Return type:

dict

Example

>>> from kwcoco.util.util_json import indexable_diff
>>> dct1 = {
>>>     'foo': [1.222222, 1.333],
>>>     'bar': 1,
>>>     'baz': [],
>>>     'top': [1, 2, 3],
>>>     'L0': {'L1': {'L2': {'K1': 'V1', 'K2': 'V2', 'D1': 1, 'D2': 2}}},
>>> }
>>> dct2 = {
>>>     'foo': [1.22222, 1.333],
>>>     'bar': 1,
>>>     'baz': [],
>>>     'buz': {1: 2},
>>>     'top': [1, 1, 2],
>>>     'L0': {'L1': {'L2': {'K1': 'V1', 'K2': 'V2', 'D1': 10, 'D2': 20}}},
>>> }
>>> info = indexable_diff(dct1, dct2)
>>> print(f'info = {ub.urepr(info, nl=2)}')

kwcoco.util.util_json.coerce_indent(indent)

Example