kwcoco.util.ijson_ext module¶
Add NaN parsing to normal ijson pure-python parsing backend.
References
https://github.com/ICRAR/ijson/issues/33
Example
>>> # xdoctest: +REQUIRES(module:ijson)
>>> import ubelt as ub
>>> from kwcoco.util import ijson_ext as ijson_ext
>>> #
>>> import kwcoco
>>> dset = kwcoco.CocoDataset.demo()
>>> text = dset.dumps()
>>> import io
>>> file = io.StringIO(text)
>>> list(ijson_ext.items(file, 'categories'))
>>> #
>>> text = ub.codeblock(
>>> '''
>>> {"id": 246, "image_id": 123, "flag": true, "empty": null, "category_id": 2, "bbox": [ NaN, 5, 130, 290 ] }
>>> ''')
>>> import io
>>> import json
>>> #
>>> file = io.StringIO(text)
>>> json.load(file)
>>> #
>>> file = io.StringIO(text)
>>> list(ijson_ext.items(file, 'bbox'))
>>> #
>>> import ijson
>>> # Regular ijson does not support NaN
>>> text = ub.codeblock(
>>> '''
>>> {"id": 246, "image_id": 123, "flag": true, "empty": null, "category_id": 2, "bbox": [ null, 5, 130, 290 ] }
>>> ''')
import timerit ti = timerit.Timerit(100, bestof=10, verbose=2) for timer in ti.reset(‘time’):
- with timer:
file = io.StringIO(text) list(ijson.items(file, ‘bbox’))
- for timer in ti.reset(‘time’):
- with timer:
file = io.StringIO(text) list(ijson_ext.items(file, ‘bbox’))
- kwcoco.util.ijson_ext.Lexer(target)[source]¶
Parses lexemes out of the incoming content, and sends them to parse_value. A special EOF result is sent when the data source has been exhausted to give parse_value the possibility of raising custom exceptions due to missing content.
- kwcoco.util.ijson_ext.parse_value(target, multivalue, use_float)[source]¶
Parses results coming out of the Lexer into ijson events, which are sent to target. A stack keeps track of the type of object being parsed at the time (a value, and object or array – the last two being values themselves). A special EOF result coming from the Lexer indicates that no more content is expected. This is used to check for incomplete content and raise the appropriate exception, which wouldn’t be possible if the Lexer simply closed this co-routine (either explicitly via .close(), or implicitly by itself finishing and decreasing the only reference to the co-routine) since that causes a GeneratorExit exception that cannot be replaced with a custom one.