kwcoco.coco_dataset module¶
An implementation and extension of the original MS-COCO API [CocoFormat].
Extends the format to also include line annotations.
The following describes psuedo-code for the high level spec (some of which may
not be have full support in the Python API). A formal json-schema is defined in
kwcoco.coco_schema
.
An informal spec is as follows:
# All object categories are defined here.
category = {
'id': int,
'name': str, # unique name of the category
'supercategory': str, # parent category name
}
# Videos are used to manage collections or sequences of images.
# Frames do not necesarilly have to be aligned or uniform time steps
video = {
'id': int,
'name': str, # a unique name for this video.
'width': int # the base width of this video (all associated images must have this width)
'height': int # the base height of this video (all associated images must have this height)
# In the future this may be extended to allow pointing to video files
}
# Specifies how to find sensor data of a particular scene at a particular
# time. This is usually paths to rgb images, but auxiliary information
# can be used to specify multiple bands / etc...
# NOTE: in the future we will transition from calling these auxiliary items
# to calling these asset items. As such the key will change from
# "auxiliary" to "asset". The API will be updated to maintain backwards
# compatibility while this transition occurs.
image = {
'id': int,
'name': str, # an encouraged but optional unique name
'file_name': str, # relative path to the "base" image data (optional if auxiliary items are specified)
'width': int, # pixel width of "base" image
'height': int, # pixel height of "base" image
'channels': <ChannelSpec>, # a string encoding of the channels in the main image (optional if auxiliary items are specified)
'auxiliary': [ # information about any auxiliary channels / bands
{
'file_name': str, # relative path to associated file
'channels': <ChannelSpec>, # a string encoding
'width': <int> # pixel width of image asset
'height': <int> # pixel height of image asset
'warp_aux_to_img': <TransformSpec>, # tranform from "base" image space to auxiliary/asset space. (identity if unspecified)
'quantization': <QuantizationSpec>, # indicates that the underlying data was quantized
}, ...
]
'video_id': str # if this image is a frame in a video sequence, this id is shared by all frames in that sequence.
'timestamp': str | int # a iso-string timestamp or an integer in flicks.
'frame_index': int # ordinal frame index which can be used if timestamp is unknown.
'warp_img_to_vid': <TransformSpec> # a transform image space to video space (identity if unspecified), can be used for sensor alignment or video stabilization
}
TransformSpec:
The spec can be anything coercable to a kwimage.Affine object.
This can be an explicit affine transform matrix like:
{'type': 'affine': 'matrix': <a-3x3 matrix>},
But it can also be a concise dict containing one or more of these keys
{
'scale': <float|Tuple[float, float]>,
'offset': <float|Tuple[float, float]>,
'skew': <float>,
'theta': <float>, # radians counter-clock-wise
}
ChannelSpec:
This is a string that describes the channel composition of an image.
For the purposes of kwcoco, separate different channel names with a
pipe ('|'). If the spec is not specified, methods may fall back on
grayscale or rgb processing. There are special string. For instance
'rgb' will expand into 'r|g|b'. In other applications you can "late
fuse" inputs by separating them with a "," and "early fuse" by
separating with a "|". Early fusion returns a solid array/tensor, late
fusion returns separated arrays/tensors.
QuantizationSpec:
This is a dictionary of the form:
{
'orig_min': <float>, # min original intensity
'orig_max': <float>, # min original intensity
'quant_min': <int>, # min quantized intensity
'quant_max': <int>, # max quantized intensity
'nodata': <int|None>, # integer value to interpret as nan
}
# Ground truth is specified as annotations, each belongs to a spatial
# region in an image. This must reference a subregion of the image in pixel
# coordinates. Additional non-schma properties can be specified to track
# location in other coordinate systems. Annotations can be linked over time
# by specifying track-ids.
annotation = {
'id': int,
'image_id': int,
'category_id': int,
'track_id': <int | str | uuid> # indicates association between annotations across images
'bbox': [tl_x, tl_y, w, h], # xywh format)
'score' : float,
'prob' : List[float],
'weight' : float,
'caption': str, # a text caption for this annotation
'keypoints' : <Keypoints | List[int] > # an accepted keypoint format
'segmentation': <RunLengthEncoding | Polygon | MaskPath | WKT >, # an accepted segmentation format
}
# A dataset bundles a manifest of all aformentioned data into one structure.
dataset = {
'categories': [category, ...],
'videos': [video, ...]
'images': [image, ...]
'annotations': [annotation, ...]
'licenses': [],
'info': [],
}
Polygon:
A flattened list of xy coordinates.
[x1, y1, x2, y2, ..., xn, yn]
or a list of flattened list of xy coordinates if the CCs are disjoint
[[x1, y1, x2, y2, ..., xn, yn], [x1, y1, ..., xm, ym],]
Note: the original coco spec does not allow for holes in polygons.
We also allow a non-standard dictionary encoding of polygons
{'exterior': [(x1, y1)...],
'interiors': [[(x1, y1), ...], ...]}
TODO: Support WTK
RunLengthEncoding:
The RLE can be in a special bytes encoding or in a binary array
encoding. We reuse the original C functions are in [PyCocoToolsMask]_
in ``kwimage.structs.Mask`` to provide a convinient way to abstract
this rather esoteric bytes encoding.
For pure python implementations see kwimage:
Converting from an image to RLE can be done via kwimage.run_length_encoding
Converting from RLE back to an image can be done via:
kwimage.decode_run_length
For compatibility with the COCO specs ensure the binary flags
for these functions are set to true.
Keypoints:
Annotation keypoints may also be specified in this non-standard (but
ultimately more general) way:
'annotations': [
{
'keypoints': [
{
'xy': <x1, y1>,
'visible': <0 or 1 or 2>,
'keypoint_category_id': <kp_cid>,
'keypoint_category': <kp_name, optional>, # this can be specified instead of an id
}, ...
]
}, ...
],
'keypoint_categories': [{
'name': <str>,
'id': <int>, # an id for this keypoint category
'supercategory': <kp_name> # name of coarser parent keypoint class (for hierarchical keypoints)
'reflection_id': <kp_cid> # specify only if the keypoint id would be swapped with another keypoint type
},...
]
In this scheme the "keypoints" property of each annotation (which used
to be a list of floats) is now specified as a list of dictionaries that
specify each keypoints location, id, and visibility explicitly. This
allows for things like non-unique keypoints, partial keypoint
annotations. This also removes the ordering requirement, which makes it
simpler to keep track of each keypoints class type.
We also have a new top-level dictionary to specify all the possible
keypoint categories.
TODO: Support WTK
Auxiliary Channels / Image Assets:
For multimodal or multispectral images it is possible to specify
auxiliary channels in an image dictionary as follows:
{
'id': int,
'file_name': str, # path to the "base" image (may be None)
'name': str, # a unique name for the image (must be given if file_name is None)
'channels': <ChannelSpec>, # a spec code that indicates the layout of the "base" image channels.
'auxiliary': [ # information about auxiliary channels
{
'file_name': str,
'channels': <ChannelSpec>
}, ... # can have many auxiliary channels with unique specs
]
}
Note that specifing a filename / channels for the base image is not
necessary, and mainly useful for augmenting an existing single-image
dataset with multimodal information. Typically if an image consists of
more than one file, all file information should be stored in the
"auxiliary" or "assets" list.
NEW DOCS:
In an MSI use case you should think of the "auxiliary" list as a
list of single file assets that are composed to make the entire
image. Your assets might include sensed bands, computed features,
or quality information. For instance a list of auxiliary items may
look like this:
image = {
"name": "my_msi_image",
"width": 400,
"height": 400,
"video_id": 2,
"timestamp": "2020-01-1",
"frame_index": 5,
"warp_img_to_vid": {"type": "affine", "scale", 1.4},
"auxiliary": [
{"channels": "red|green|blue": "file_name": "rgb.tif", "warp_aux_to_img": {"scale": 1.0}, "height": 400, "width": 400, ...},
...
{"channels": "cloudmask": "file_name": "cloudmask.tif", "warp_aux_to_img": {"scale": 4.0}, "height": 100, "width": 100, ...},
{"channels": "nir": "file_name": "nir.tif", "warp_aux_to_img": {"scale": 2.0}, "height": 200, "width": 200, ...},
{"channels": "swir": "file_name": "swir.tif", "warp_aux_to_img": {"scale": 2.0}, "height": 200, "width": 200, ...},
{"channels": "model1_predictions:0.6": "file_name": "model1_preds.tif", "warp_aux_to_img": {"scale": 8.0}, "height": 50, "width": 50, ...},
{"channels": "model2_predictions:0.3": "file_name": "model2_preds.tif", "warp_aux_to_img": {"scale": 8.0}, "height": 50, "width": 50, ...},
]
}
Note that there is no file_name or channels parameter in the image
object itself. This pattern indicates that image is composed of
multiple assets. One could indicate that an asset is primary by
giving its information to the parent image, but for better STAC
compatibility, all assets for MSI images should simply be listed
as "auxiliary" items.
Video Sequences:
For video sequences, we add the following video level index:
'videos': [
{ 'id': <int>, 'name': <video_name:str> },
]
Note that the videos might be given as encoded mp4/avi/etc.. files (in
which case the name should correspond to a path) or as a series of
frames in which case the images should be used to index the extracted
frames and information in them.
Then image dictionaries are augmented as follows:
{
'video_id': str # optional, if this image is a frame in a video sequence, this id is shared by all frames in that sequence.
'timestamp': int # optional, timestamp (ideally in flicks), used to identify the timestamp of the frame. Only applicable video inputs.
'frame_index': int # optional, ordinal frame index which can be used if timestamp is unknown.
}
And annotations are augmented as follows:
{
'track_id': <int | str | uuid> # optional, indicates association between annotations across frames
}
Note
The main object in this file is CocoDataset
, which is composed of
several mixin classes. See the class and method documentation for more
details.
Todo
[ ] Use ijson to lazilly load pieces of the dataset in the background or on demand. This will give us faster access to categories / images, whereas we will always have to wait for annotations etc…
[X] Should img_root be changed to bundle_dpath?
[ ] Read video data, return numpy arrays (requires API for images)
[ ] Spec for video URI, and convert to frames @ framerate function.
[ ] Document channel spec
[X] remove videos
References
- CocoFormat
- PyCocoToolsMask
https://github.com/nightrome/cocostuffapi/blob/master/PythonAPI/pycocotools/mask.py
- CocoTutorial
https://www.immersivelimit.com/tutorials/create-coco-annotations-from-scratch/#coco-dataset-format
- class kwcoco.coco_dataset.MixinCocoDepricate[source]¶
Bases:
object
These functions are marked for deprication and may be removed at any time
- class kwcoco.coco_dataset.MixinCocoAccessors[source]¶
Bases:
object
TODO: better name
- delayed_load(gid, channels=None, space='image')[source]¶
Experimental method
- Parameters
gid (int) – image id to load
channels (kwcoco.FusedChannelSpec) – specific channels to load. if unspecified, all channels are loaded.
space (str) – can either be “image” for loading in image space, or “video” for loading in video space.
Todo
- [X] Currently can only take all or none of the channels from each
base-image / auxiliary dict. For instance if the main image is r|g|b you can’t just select g|b at the moment.
- [X] The order of the channels in the delayed load should
match the requested channel order.
[X] TODO: add nans to bands that don’t exist or throw an error
Example
>>> import kwcoco >>> gid = 1 >>> # >>> self = kwcoco.CocoDataset.demo('vidshapes8-multispectral') >>> delayed = self.delayed_load(gid) >>> print('delayed = {!r}'.format(delayed)) >>> print('delayed.finalize() = {!r}'.format(delayed.finalize())) >>> # >>> self = kwcoco.CocoDataset.demo('shapes8') >>> delayed = self.delayed_load(gid) >>> print('delayed = {!r}'.format(delayed)) >>> print('delayed.finalize() = {!r}'.format(delayed.finalize()))
>>> crop = delayed.crop((slice(0, 3), slice(0, 3))) >>> crop.finalize()
>>> # TODO: should only select the "red" channel >>> self = kwcoco.CocoDataset.demo('shapes8') >>> delayed = self.delayed_load(gid, channels='r')
>>> import kwcoco >>> gid = 1 >>> # >>> self = kwcoco.CocoDataset.demo('vidshapes8-multispectral') >>> delayed = self.delayed_load(gid, channels='B1|B2', space='image') >>> print('delayed = {!r}'.format(delayed)) >>> delayed = self.delayed_load(gid, channels='B1|B2|B11', space='image') >>> print('delayed = {!r}'.format(delayed)) >>> delayed = self.delayed_load(gid, channels='B8|B1', space='video') >>> print('delayed = {!r}'.format(delayed))
>>> delayed = self.delayed_load(gid, channels='B8|foo|bar|B1', space='video') >>> print('delayed = {!r}'.format(delayed))
- load_image(gid_or_img, channels=None)[source]¶
Reads an image from disk and
- Parameters
gid_or_img (int | dict) – image id or image dict
channels (str | None) – if specified, load data from auxiliary channels instead
- Returns
the image
- Return type
np.ndarray
Todo
- [ ] allow specification of multiple channels - use delayed image
for this.
- get_image_fpath(gid_or_img, channels=None)[source]¶
Returns the full path to the image
- Parameters
gid_or_img (int | dict) – image id or image dict
channels (str, default=None) – if specified, return a path to data containing auxiliary channels instead
- Returns
full path to the image
- Return type
PathLike
- get_auxiliary_fpath(gid_or_img, channels)[source]¶
Returns the full path to auxiliary data for an image
- Parameters
gid_or_img (int | dict) – an image or its id
channels (str) – the auxiliary channel to load (e.g. disparity)
Example
>>> import kwcoco >>> self = kwcoco.CocoDataset.demo('shapes8', aux=True) >>> self.get_auxiliary_fpath(1, 'disparity')
- load_annot_sample(aid_or_ann, image=None, pad=None)[source]¶
Reads the chip of an annotation. Note this is much less efficient than using a sampler, but it doesn’t require disk cache.
Maybe depricate?
- Parameters
aid_or_int (int | dict) – annot id or dict
image (ArrayLike, default=None) – preloaded image (note: this process is inefficient unless image is specified)
Example
>>> import kwcoco >>> self = kwcoco.CocoDataset.demo() >>> sample = self.load_annot_sample(2, pad=100) >>> # xdoctest: +REQUIRES(--show) >>> import kwplot >>> kwplot.autompl() >>> kwplot.imshow(sample['im']) >>> kwplot.show_if_requested()
- category_graph()[source]¶
Construct a networkx category hierarchy
- Returns
graph: a directed graph where category names are the nodes, supercategories define edges, and items in each category dict (e.g. category id) are added as node properties.
- Return type
Example
>>> import kwcoco >>> self = kwcoco.CocoDataset.demo() >>> graph = self.category_graph() >>> assert 'astronaut' in graph.nodes() >>> assert 'keypoints' in graph.nodes['human']
- object_categories()[source]¶
Construct a consistent CategoryTree representation of object classes
- Returns
category data structure
- Return type
Example
>>> import kwcoco >>> self = kwcoco.CocoDataset.demo() >>> classes = self.object_categories() >>> print('classes = {}'.format(classes))
- class kwcoco.coco_dataset.MixinCocoExtras[source]¶
Bases:
object
Misc functions for coco
- classmethod coerce(key, **kw)[source]¶
Attempt to transform the input into the intended CocoDataset.
- Parameters
key – this can either be an instance of a CocoDataset, a string URI pointing to an on-disk dataset, or a special key for creating demodata.
**kw – passed to whatever constructor is chosen (if any)
Example
>>> # test coerce for various input methods >>> import kwcoco >>> from kwcoco.coco_sql_dataset import assert_dsets_allclose >>> dct_dset = kwcoco.CocoDataset.coerce('special:shapes8') >>> copy1 = kwcoco.CocoDataset.coerce(dct_dset) >>> copy2 = kwcoco.CocoDataset.coerce(dct_dset.fpath) >>> assert assert_dsets_allclose(dct_dset, copy1) >>> assert assert_dsets_allclose(dct_dset, copy2) >>> # xdoctest: +REQUIRES(module:sqlalchemy) >>> sql_dset = dct_dset.view_sql() >>> copy3 = kwcoco.CocoDataset.coerce(sql_dset) >>> copy4 = kwcoco.CocoDataset.coerce(sql_dset.fpath) >>> assert assert_dsets_allclose(dct_dset, sql_dset) >>> assert assert_dsets_allclose(dct_dset, copy3) >>> assert assert_dsets_allclose(dct_dset, copy4)
- classmethod demo(key='photos', **kwargs)[source]¶
Create a toy coco dataset for testing and demo puposes
- Parameters
key (str, default=photos) – Either ‘photos’, ‘shapes’, or ‘vidshapes’. There are also special sufixes that can control behavior.
Basic options that define which flavor of demodata to generate are: photos, shapes, and vidshapes. A numeric suffix e.g. vidshapes8 can be specified to indicate the size of the generated demo dataset. There are other special suffixes that are available. See the code in this function for explicit details on what is allowed.
TODO: better documentation for these demo datasets.
As a quick summary: the vidshapes key is the most robust and mature demodata set, and here are several useful variants of the vidshapes key.
vidshapes8 - the 8 suffix is the number of videos in this case.
vidshapes8-multispectral - generate 8 multispectral videos.
vidshapes8-msi - msi is an alias for multispectral.
(4) vidshapes8-frames5 - generate 8 videos with 5 frames each. (4) vidshapes2-speed0.1-frames7 - generate 2 videos with 7 frames where the objects move with with a speed of 0.1.
**kwargs – if key is shapes, these arguments are passed to toydata generation. The Kwargs section of this docstring documents a subset of the available options. For full details, see
demodata_toy_dset()
andrandom_video_dset()
.
- Kwargs:
image_size (Tuple[int, int]): width / height size of the images
- dpath (str): path to the output image directory, defaults to using
kwcoco cache dir.
aux (bool): if True generates dummy auxiliary channels
- rng (int | RandomState, default=0):
random number generator or seed
verbose (int, default=3): verbosity mode
Example
>>> # Basic demodata keys >>> print(CocoDataset.demo('photos', verbose=1)) >>> print(CocoDataset.demo('shapes', verbose=1)) >>> print(CocoDataset.demo('vidshapes', verbose=1)) >>> # Varaints of demodata keys >>> print(CocoDataset.demo('shapes8', verbose=0)) >>> print(CocoDataset.demo('shapes8-msi', verbose=0)) >>> print(CocoDataset.demo('shapes8-frames1-speed0.2-msi', verbose=0))
Example
>>> import kwcoco >>> dset = kwcoco.CocoDataset.demo('vidshapes5', num_frames=5, >>> verbose=0, rng=None) >>> dset = kwcoco.CocoDataset.demo('vidshapes5', num_frames=5, >>> num_tracks=4, verbose=0, rng=44) >>> # xdoctest: +REQUIRES(--show) >>> import kwplot >>> kwplot.autompl() >>> pnums = kwplot.PlotNums(nSubplots=len(dset.index.imgs)) >>> fnum = 1 >>> for gx, gid in enumerate(dset.index.imgs.keys()): >>> canvas = dset.draw_image(gid=gid) >>> kwplot.imshow(canvas, pnum=pnums[gx], fnum=fnum) >>> #dset.show_image(gid=gid, pnum=pnums[gx]) >>> kwplot.show_if_requested()
Example
>>> import kwcoco >>> dset = kwcoco.CocoDataset.demo('vidshapes5-aux', num_frames=1, >>> verbose=0, rng=None)
Example
>>> import kwcoco >>> dset = kwcoco.CocoDataset.demo('vidshapes1-multispectral', num_frames=5, >>> verbose=0, rng=None) >>> # This is the first use-case of image names >>> assert len(dset.index.file_name_to_img) == 0, ( >>> 'the multispectral demo case has no "base" image') >>> assert len(dset.index.name_to_img) == len(dset.index.imgs) == 5 >>> dset.remove_images([1]) >>> assert len(dset.index.name_to_img) == len(dset.index.imgs) == 4 >>> dset.remove_videos([1]) >>> assert len(dset.index.name_to_img) == len(dset.index.imgs) == 0
- classmethod random(rng=None)[source]¶
Creates a random CocoDataset according to distribution parameters
Todo
[ ] parametarize
- corrupted_images(check_aux=False, verbose=0)[source]¶
Check for images that don’t exist or can’t be opened
- rename_categories(mapper, rebuild=True, merge_policy='ignore')[source]¶
Rename categories with a potentially coarser categorization.
- Parameters
mapper (dict | Callable) – maps old names to new names. If multiple names are mapped to the same category, those categories will be merged.
merge_policy (str) – How to handle multiple categories that map to the same name. Can be update or ignore.
Example
>>> import kwcoco >>> self = kwcoco.CocoDataset.demo() >>> self.rename_categories({'astronomer': 'person', >>> 'astronaut': 'person', >>> 'mouth': 'person', >>> 'helmet': 'hat'}) >>> assert 'hat' in self.name_to_cat >>> assert 'helmet' not in self.name_to_cat >>> # Test merge case >>> self = kwcoco.CocoDataset.demo() >>> mapper = { >>> 'helmet': 'rocket', >>> 'astronomer': 'rocket', >>> 'human': 'rocket', >>> 'mouth': 'helmet', >>> 'star': 'gas' >>> } >>> self.rename_categories(mapper)
- reroot(new_root=None, old_prefix=None, new_prefix=None, absolute=False, check=True, safe=True, verbose=0)[source]¶
Modify the prefix of the image/data paths onto a new image/data root.
- Parameters
new_root (str | None) – New image root. If unspecified the current
self.bundle_dpath
is used. If old_prefix and new_prefix are unspecified, they will attempt to be determined based on the current root (which assumes the file paths exist at that root) and this new root. Defaults to None.old_prefix (str | None) – If specified, removes this prefix from file names. This also prevents any inferences that might be made via “new_root”. Defaults to None.
new_prefix (str | None) – If specified, adds this prefix to the file names. This also prevents any inferences that might be made via “new_root”. Defaults to None.
absolute (bool) – if True, file names are stored as absolute paths, otherwise they are relative to the new image root. Defaults to False.
check (bool) – if True, checks that the images all exist. Defaults to True.
safe (bool) – if True, does not overwrite values until all checks pass. Defaults to True.
verbose (int) – verbosity level, default=0.
CommandLine
xdoctest -m kwcoco.coco_dataset MixinCocoExtras.reroot
Todo
[ ] Incorporate maximum ordered subtree embedding?
Example
>>> import kwcoco >>> def report(dset, name): >>> gid = 1 >>> abs_fpath = dset.get_image_fpath(gid) >>> rel_fpath = dset.index.imgs[gid]['file_name'] >>> color = 'green' if exists(abs_fpath) else 'red' >>> print('strategy_name = {!r}'.format(name)) >>> print(ub.color_text('abs_fpath = {!r}'.format(abs_fpath), color)) >>> print('rel_fpath = {!r}'.format(rel_fpath)) >>> dset = self = kwcoco.CocoDataset.demo() >>> # Change base relative directory >>> bundle_dpath = ub.expandpath('~') >>> print('ORIG self.imgs = {!r}'.format(self.imgs)) >>> print('ORIG dset.bundle_dpath = {!r}'.format(dset.bundle_dpath)) >>> print('NEW bundle_dpath = {!r}'.format(bundle_dpath)) >>> self.reroot(bundle_dpath) >>> report(self, 'self') >>> print('NEW self.imgs = {!r}'.format(self.imgs)) >>> assert self.imgs[1]['file_name'].startswith('.cache')
>>> # Use absolute paths >>> self.reroot(absolute=True) >>> assert self.imgs[1]['file_name'].startswith(bundle_dpath)
>>> # Switch back to relative paths >>> self.reroot() >>> assert self.imgs[1]['file_name'].startswith('.cache')
Example
>>> # demo with auxiliary data >>> import kwcoco >>> self = kwcoco.CocoDataset.demo('shapes8', aux=True) >>> bundle_dpath = ub.expandpath('~') >>> print(self.imgs[1]['file_name']) >>> print(self.imgs[1]['auxiliary'][0]['file_name']) >>> self.reroot(new_root=bundle_dpath) >>> print(self.imgs[1]['file_name']) >>> print(self.imgs[1]['auxiliary'][0]['file_name']) >>> assert self.imgs[1]['file_name'].startswith('.cache') >>> assert self.imgs[1]['auxiliary'][0]['file_name'].startswith('.cache')
- property data_root¶
In the future we will deprecate data_root for bundle_dpath
- property img_root¶
In the future we will deprecate img_root for bundle_dpath
- property data_fpath¶
data_fpath is an alias of fpath
- class kwcoco.coco_dataset.MixinCocoObjects[source]¶
Bases:
object
Expose methods to construct object lists / groups.
This is an alternative vectorized ORM-like interface to the coco dataset
- annots(aids=None, gid=None, trackid=None)[source]¶
Return vectorized annotation objects
- Parameters
aids (List[int]) – annotation ids to reference, if unspecified all annotations are returned.
gid (int) – return all annotations that belong to this image id. mutually exclusive with other arguments.
trackid (int) – return all annotations that belong to this track. mutually exclusive with other arguments.
- Returns
vectorized annotation object
- Return type
Example
>>> import kwcoco >>> self = kwcoco.CocoDataset.demo() >>> annots = self.annots() >>> print(annots) <Annots(num=11)> >>> sub_annots = annots.take([1, 2, 3]) >>> print(sub_annots) <Annots(num=3)> >>> print(ub.repr2(sub_annots.get('bbox', None))) [ [350, 5, 130, 290], None, None, ]
- images(gids=None, vidid=None, names=None)[source]¶
Return vectorized image objects
- Parameters
gids (List[int]) – image ids to reference, if unspecified all images are returned.
vidid (int) – returns all images that belong to this video id. mutually exclusive with gids arg.
names (List[str]) – lookup images by their names.
- Returns
vectorized image object
- Return type
Example
>>> import kwcoco >>> self = kwcoco.CocoDataset.demo() >>> images = self.images() >>> print(images) <Images(num=3)>
>>> self = kwcoco.CocoDataset.demo('vidshapes2') >>> vidid = 1 >>> images = self.images(vidid=vidid) >>> assert all(v == vidid for v in images.lookup('video_id')) >>> print(images) <Images(num=2)>
- categories(cids=None)[source]¶
Return vectorized category objects
- Parameters
cids (List[int]) – category ids to reference, if unspecified all categories are returned.
- Returns
vectorized category object
- Return type
Example
>>> import kwcoco >>> self = kwcoco.CocoDataset.demo() >>> categories = self.categories() >>> print(categories) <Categories(num=8)>
- videos(vidids=None, names=None)[source]¶
Return vectorized video objects
- Parameters
vidids (List[int]) – video ids to reference, if unspecified all videos are returned.
names (List[str]) – lookup videos by their name.
- Returns
vectorized video object
- Return type
Todo
- [ ] This conflicts with what should be the property that
should redirect to
index.videos
, we should resolve this somehow. E.g. all other main members of the index (anns, imgs, cats) have a toplevel dataset property, we don’t have one for videos because the name we would pick conflicts with this.
Example
>>> import kwcoco >>> self = kwcoco.CocoDataset.demo('vidshapes2') >>> videos = self.videos() >>> print(videos) >>> videos.lookup('name') >>> videos.lookup('id') >>> print('videos.objs = {}'.format(ub.repr2(videos.objs[0:2], nl=1)))
- class kwcoco.coco_dataset.MixinCocoStats[source]¶
Bases:
object
Methods for getting stats about the dataset
- property n_annots¶
The number of annotations in the dataset
- property n_images¶
The number of images in the dataset
- property n_cats¶
The number of categories in the dataset
- property n_videos¶
The number of videos in the dataset
- keypoint_annotation_frequency()[source]¶
DEPRECATED
Example
>>> import kwcoco >>> import ubelt as ub >>> self = kwcoco.CocoDataset.demo('shapes', rng=0) >>> hist = self.keypoint_annotation_frequency() >>> hist = ub.odict(sorted(hist.items())) >>> # FIXME: for whatever reason demodata generation is not determenistic when seeded >>> print(ub.repr2(hist)) # xdoc: +IGNORE_WANT { 'bot_tip': 6, 'left_eye': 14, 'mid_tip': 6, 'right_eye': 14, 'top_tip': 6, }
- category_annotation_frequency()[source]¶
Reports the number of annotations of each category
Example
>>> import kwcoco >>> self = kwcoco.CocoDataset.demo() >>> hist = self.category_annotation_frequency() >>> print(ub.repr2(hist)) { 'astroturf': 0, 'human': 0, 'astronaut': 1, 'astronomer': 1, 'helmet': 1, 'rocket': 1, 'mouth': 2, 'star': 5, }
- category_annotation_type_frequency()[source]¶
DEPRECATED
Reports the number of annotations of each type for each category
Example
>>> import kwcoco >>> self = kwcoco.CocoDataset.demo() >>> hist = self.category_annotation_frequency() >>> print(ub.repr2(hist))
- conform(**config)[source]¶
Make the COCO file conform a stricter spec, infers attibutes where possible.
Corresponds to the
kwcoco conform
CLI tool.- KWArgs:
**config :
pycocotools_info (default=True): returns info required by pycocotools
ensure_imgsize (default=True): ensure image size is populated
legacy (default=False): if true tries to convert data structures to items compatible with the original pycocotools spec
workers (int): number of parallel jobs for IO tasks
Example
>>> import kwcoco >>> dset = kwcoco.CocoDataset.demo('shapes8') >>> dset.index.imgs[1].pop('width') >>> dset.conform(legacy=True) >>> assert 'width' in dset.index.imgs[1] >>> assert 'area' in dset.index.anns[1]
- validate(**config)[source]¶
Performs checks on this coco dataset.
Corresponds to the
kwcoco validate
CLI tool.- Parameters
**config – schema (default=True): if True, validate the json-schema
unique (default=True): if True, validate unique secondary keys
missing (default=True): if True, validate registered files exist
corrupted (default=False): if True, validate data in registered files
channels (default=True): if True, validate that channels in auxiliary/asset items are all unique.
require_relative (default=False): if True, causes validation to fail if paths are non-portable, i.e. all paths must be relative to the bundle directory. if>0, paths must be relative to bundle root. if>1, paths must be inside bundle root.
img_attrs (default=’warn’): if truthy, check that image attributes contain width and height entries. If ‘warn’, then warn if they do not exist. If ‘error’, then fail.
verbose (default=1): verbosity flag
fastfail (default=False): if True raise errors immediately
- Returns
- result containing keys -
status (bool): False if any errors occurred errors (List[str]): list of all error messages missing (List): List of any missing images corrupted (List): List of any corrupted images
- Return type
- SeeAlso:
_check_integrity()
- performs internal checks
Example
>>> import kwcoco >>> self = kwcoco.CocoDataset.demo() >>> import pytest >>> with pytest.warns(UserWarning): >>> result = self.validate() >>> assert not result['errors'] >>> assert result['warnings']
- stats(**kwargs)[source]¶
Compute summary statistics to describe the dataset at a high level
This function corresponds to
kwcoco.cli.coco_stats
.- KWargs:
basic(bool, default=True): return basic stats’ extended(bool, default=True): return extended stats’ catfreq(bool, default=True): return category frequency stats’ boxes(bool, default=False): return bounding box stats’
annot_attrs(bool, default=True): return annotation attribute information’ image_attrs(bool, default=True): return image attribute information’
- Returns
info
- Return type
- basic_stats()[source]¶
Reports number of images, annotations, and categories.
- SeeAlso:
kwcoco.coco_dataset.MixinCocoStats.basic_stats()
kwcoco.coco_dataset.MixinCocoStats.extended_stats()
Example
>>> import kwcoco >>> self = kwcoco.CocoDataset.demo() >>> print(ub.repr2(self.basic_stats())) { 'n_anns': 11, 'n_imgs': 3, 'n_videos': 0, 'n_cats': 8, }
>>> from kwcoco.demo.toydata_video import random_video_dset >>> dset = random_video_dset(render=True, num_frames=2, num_tracks=10, rng=0) >>> print(ub.repr2(dset.basic_stats())) { 'n_anns': 20, 'n_imgs': 2, 'n_videos': 1, 'n_cats': 3, }
- extended_stats()[source]¶
Reports number of images, annotations, and categories.
- SeeAlso:
kwcoco.coco_dataset.MixinCocoStats.basic_stats()
kwcoco.coco_dataset.MixinCocoStats.extended_stats()
Example
>>> import kwcoco >>> self = kwcoco.CocoDataset.demo() >>> print(ub.repr2(self.extended_stats()))
- boxsize_stats(anchors=None, perclass=True, gids=None, aids=None, verbose=0, clusterkw={}, statskw={})[source]¶
Compute statistics about bounding box sizes.
Also computes anchor boxes using kmeans if
anchors
is specified.- Parameters
anchors (int) – if specified also computes box anchors via KMeans clustering
perclass (bool) – if True also computes stats for each category
gids (List[int], default=None) – if specified only compute stats for these image ids.
aids (List[int], default=None) – if specified only compute stats for these annotation ids.
verbose (int) – verbosity level
clusterkw (dict) – kwargs for
sklearn.cluster.KMeans
used if computing anchors.statskw (dict) – kwargs for
kwarray.stats_dict()
- Returns
Stats are returned in width-height format.
- Return type
Example
>>> import kwcoco >>> self = kwcoco.CocoDataset.demo('shapes32') >>> infos = self.boxsize_stats(anchors=4, perclass=False) >>> print(ub.repr2(infos, nl=-1, precision=2))
>>> infos = self.boxsize_stats(gids=[1], statskw=dict(median=True)) >>> print(ub.repr2(infos, nl=-1, precision=2))
- find_representative_images(gids=None)[source]¶
Find images that have a wide array of categories.
Attempt to find the fewest images that cover all categories using images that contain both a large and small number of annotations.
- Parameters
gids (None | List) – Subset of image ids to consider when finding representative images. Uses all images if unspecified.
- Returns
list of image ids determined to be representative
- Return type
List
Example
>>> import kwcoco >>> self = kwcoco.CocoDataset.demo() >>> gids = self.find_representative_images() >>> print('gids = {!r}'.format(gids)) >>> gids = self.find_representative_images([3]) >>> print('gids = {!r}'.format(gids))
>>> self = kwcoco.CocoDataset.demo('shapes8') >>> gids = self.find_representative_images() >>> print('gids = {!r}'.format(gids)) >>> valid = {7, 1} >>> gids = self.find_representative_images(valid) >>> assert valid.issuperset(gids) >>> print('gids = {!r}'.format(gids))
- class kwcoco.coco_dataset.MixinCocoDraw[source]¶
Bases:
object
Matplotlib / display functionality
- draw_image(gid, channels=None)[source]¶
Use kwimage to draw all annotations on an image and return the pixels as a numpy array.
- Parameters
gid (int) – image id to draw
channels (kwcoco.ChannelSpec) – the channel to draw on
- Returns
canvas
- Return type
ndarray
- SeeAlso
kwcoco.coco_dataset.MixinCocoDraw.draw_image()
kwcoco.coco_dataset.MixinCocoDraw.show_image()
Example
>>> import kwcoco >>> self = kwcoco.CocoDataset.demo('shapes8') >>> self.draw_image(1) >>> # Now you can dump the annotated image to disk / whatever >>> # xdoctest: +REQUIRES(--show) >>> import kwplot >>> kwplot.autompl() >>> kwplot.imshow(canvas)
- show_image(gid=None, aids=None, aid=None, channels=None, **kwargs)[source]¶
Use matplotlib to show an image with annotations overlaid
- Parameters
gid (int) – image to show
aids (list) – aids to highlight within the image
aid (int) – a specific aid to focus on. If gid is not give, look up gid based on this aid.
**kwargs – show_annots, show_aid, show_catname, show_kpname, show_segmentation, title, show_gid, show_filename, show_boxes,
- SeeAlso
kwcoco.coco_dataset.MixinCocoDraw.draw_image()
kwcoco.coco_dataset.MixinCocoDraw.show_image()
Example
>>> # xdoctest: +REQUIRES(--show) >>> import kwcoco >>> dset = kwcoco.CocoDataset.demo('vidshapes8-msi') >>> dset.show_image(gid=1, channels='B8')
- class kwcoco.coco_dataset.MixinCocoAddRemove[source]¶
Bases:
object
Mixin functions to dynamically add / remove annotations images and categories while maintaining lookup indexes.
- add_video(name, id=None, **kw)[source]¶
Register a new video with the dataset
- Parameters
name (str) – Unique name for this video.
id (None | int) – ADVANCED. Force using this image id.
**kw – stores arbitrary key/value pairs in this new video
- Returns
the video id assigned to the new video
- Return type
Example
>>> import kwcoco >>> self = kwcoco.CocoDataset() >>> print('self.index.videos = {}'.format(ub.repr2(self.index.videos, nl=1))) >>> print('self.index.imgs = {}'.format(ub.repr2(self.index.imgs, nl=1))) >>> print('self.index.vidid_to_gids = {!r}'.format(self.index.vidid_to_gids))
>>> vidid1 = self.add_video('foo', id=3) >>> vidid2 = self.add_video('bar') >>> vidid3 = self.add_video('baz') >>> print('self.index.videos = {}'.format(ub.repr2(self.index.videos, nl=1))) >>> print('self.index.imgs = {}'.format(ub.repr2(self.index.imgs, nl=1))) >>> print('self.index.vidid_to_gids = {!r}'.format(self.index.vidid_to_gids))
>>> gid1 = self.add_image('foo1.jpg', video_id=vidid1, frame_index=0) >>> gid2 = self.add_image('foo2.jpg', video_id=vidid1, frame_index=1) >>> gid3 = self.add_image('foo3.jpg', video_id=vidid1, frame_index=2) >>> gid4 = self.add_image('bar1.jpg', video_id=vidid2, frame_index=0) >>> print('self.index.videos = {}'.format(ub.repr2(self.index.videos, nl=1))) >>> print('self.index.imgs = {}'.format(ub.repr2(self.index.imgs, nl=1))) >>> print('self.index.vidid_to_gids = {!r}'.format(self.index.vidid_to_gids))
>>> self.remove_images([gid2]) >>> print('self.index.vidid_to_gids = {!r}'.format(self.index.vidid_to_gids))
- add_image(file_name=None, id=None, **kw)[source]¶
Register a new image with the dataset
- Parameters
file_name (str | None) – relative or absolute path to image. if not given, then “name” must be specified and we will exepect that “auxiliary” assets are eventually added.
id (None | int) – ADVANCED. Force using this image id.
name (str) – a unique key to identify this image
width (int) – base width of the image
height (int) – base height of the image
channels (ChannelSpec) – specification of base channels. Only relevant if file_name is given.
auxiliary (List[Dict]) – specification of auxiliary assets. See CocoImage.add_auxiliary_item for details
video_id (int) – id of parent video, if applicable
frame_index (int) – frame index in parent video
timestamp (number | str) – timestamp of frame index
**kw – stores arbitrary key/value pairs in this new image
- Returns
the image id assigned to the new image
- Return type
- SeeAlso:
Example
>>> import kwcoco >>> self = kwcoco.CocoDataset.demo() >>> import kwimage >>> gname = kwimage.grab_test_image_fpath('paraview') >>> gid = self.add_image(gname) >>> assert self.imgs[gid]['file_name'] == gname
- add_auxiliary_item(gid, file_name=None, channels=None, **kwargs)[source]¶
Adds an auxiliary / asset item to the image dictionary.
- Parameters
gid (int) – The image id to add the auxiliary/asset item to.
file_name (str | None) – The name of the file relative to the bundle directory. If unspecified, imdata must be given.
channels (str | kwcoco.FusedChannelSpec) – The channel code indicating what each of the bands represents. These channels should be disjoint wrt to the existing data in this image (this is not checked).
**kwargs – See
CocoImage.add_auxiliary_item()
for more details
Example
>>> import kwcoco >>> dset = kwcoco.CocoDataset() >>> gid = dset.add_image(name='my_image_name', width=200, height=200) >>> dset.add_auxiliary_item(gid, 'path/fake_B0.tif', channels='B0', >>> width=200, height=200, >>> warp_aux_to_img={'scale': 1.0})
- add_annotation(image_id, category_id=None, bbox=NoParam, segmentation=NoParam, keypoints=NoParam, id=None, **kw)[source]¶
Register a new annotation with the dataset
- Parameters
image_id (int) – image_id the annoatation is added to.
category_id (int | None) – category_id for the new annotaiton
bbox (list | kwimage.Boxes) – bounding box in xywh format
segmentation (Dict | List | Any) – keypoints in some accepted format, see
kwimage.Mask.to_coco()
andkwimage.MultiPolygon.to_coco()
. Extended types: MaskLike | MultiPolygonLike.keypoints (Any) – keypoints in some accepted format, see
kwimage.Keypoints.to_coco()
. Extended types: KeypointsLike.id (None | int) – Force using this annotation id. Typically you should NOT specify this. A new unused id will be chosen and returned.
**kw – stores arbitrary key/value pairs in this new image, Common respected key/values include but are not limited to the following: track_id (int | str): some value used to associate annotations that belong to the same “track”. score : float prob : List[float] weight (float): a weight, usually used to indicate if a ground truth annotation is difficult / important. This generalizes standard “is_hard” or “ignore” attributes in other formats. caption (str): a text caption for this annotation
- Returns
the annotation id assigned to the new annotation
- Return type
- SeeAlso:
kwcoco.coco_dataset.MixinCocoAddRemove.add_annotation()
kwcoco.coco_dataset.MixinCocoAddRemove.add_annotations()
Example
>>> import kwcoco >>> self = kwcoco.CocoDataset.demo() >>> image_id = 1 >>> cid = 1 >>> bbox = [10, 10, 20, 20] >>> aid = self.add_annotation(image_id, cid, bbox) >>> assert self.anns[aid]['bbox'] == bbox
Example
>>> import kwimage >>> import kwcoco >>> self = kwcoco.CocoDataset.demo() >>> new_det = kwimage.Detections.random(1, segmentations=True, keypoints=True) >>> # kwimage datastructures have methods to convert to coco recognized formats >>> new_ann_data = list(new_det.to_coco(style='new'))[0] >>> image_id = 1 >>> aid = self.add_annotation(image_id, **new_ann_data) >>> # Lookup the annotation we just added >>> ann = self.index.anns[aid] >>> print('ann = {}'.format(ub.repr2(ann, nl=-2)))
Example
>>> # Attempt to add annot without a category or bbox >>> import kwcoco >>> self = kwcoco.CocoDataset.demo() >>> image_id = 1 >>> aid = self.add_annotation(image_id) >>> assert None in self.index.cid_to_aids
Example
>>> # Attempt to add annot using various styles of kwimage structures >>> import kwcoco >>> import kwimage >>> self = kwcoco.CocoDataset.demo() >>> image_id = 1 >>> #-- >>> kw = {} >>> kw['segmentation'] = kwimage.Polygon.random() >>> kw['keypoints'] = kwimage.Points.random() >>> aid = self.add_annotation(image_id, **kw) >>> ann = self.index.anns[aid] >>> print('ann = {}'.format(ub.repr2(ann, nl=2))) >>> #-- >>> kw = {} >>> kw['segmentation'] = kwimage.Mask.random() >>> aid = self.add_annotation(image_id, **kw) >>> ann = self.index.anns[aid] >>> assert ann.get('segmentation', None) is not None >>> print('ann = {}'.format(ub.repr2(ann, nl=2))) >>> #-- >>> kw = {} >>> kw['segmentation'] = kwimage.Mask.random().to_array_rle() >>> aid = self.add_annotation(image_id, **kw) >>> ann = self.index.anns[aid] >>> assert ann.get('segmentation', None) is not None >>> print('ann = {}'.format(ub.repr2(ann, nl=2))) >>> #-- >>> kw = {} >>> kw['segmentation'] = kwimage.Polygon.random().to_coco() >>> kw['keypoints'] = kwimage.Points.random().to_coco() >>> aid = self.add_annotation(image_id, **kw) >>> ann = self.index.anns[aid] >>> assert ann.get('segmentation', None) is not None >>> assert ann.get('keypoints', None) is not None >>> print('ann = {}'.format(ub.repr2(ann, nl=2)))
- add_category(name, supercategory=None, id=None, **kw)[source]¶
Register a new category with the dataset
- Parameters
name (str) – name of the new category
supercategory (str | None) – parent of this category
id (int | None) – use this category id, if it was not taken
**kw – stores arbitrary key/value pairs in this new image
- Returns
the category id assigned to the new category
- Return type
- SeeAlso:
kwcoco.coco_dataset.MixinCocoAddRemove.add_category()
kwcoco.coco_dataset.MixinCocoAddRemove.ensure_category()
Example
>>> import kwcoco >>> self = kwcoco.CocoDataset.demo() >>> prev_n_cats = self.n_cats >>> cid = self.add_category('dog', supercategory='object') >>> assert self.cats[cid]['name'] == 'dog' >>> assert self.n_cats == prev_n_cats + 1 >>> import pytest >>> with pytest.raises(ValueError): >>> self.add_category('dog', supercategory='object')
- ensure_image(file_name, id=None, **kw)[source]¶
Register an image if it is new or returns an existing id.
Like
kwcoco.coco_dataset.MixinCocoAddRemove.add_image()
, but returns the existing image id if it already exists instead of failing. In this case all metadata is ignored.- Parameters
file_name (str) – relative or absolute path to image
id (None | int) – ADVANCED. Force using this image id.
**kw – stores arbitrary key/value pairs in this new image
- Returns
the existing or new image id
- Return type
- ensure_category(name, supercategory=None, id=None, **kw)[source]¶
Register a category if it is new or returns an existing id.
Like
kwcoco.coco_dataset.MixinCocoAddRemove.add_category()
, but returns the existing category id if it already exists instead of failing. In this case all metadata is ignored.- Returns
the existing or new category id
- Return type
- add_annotations(anns)[source]¶
Faster less-safe multi-item alternative to add_annotation.
We assume the annotations are well formatted in kwcoco compliant dictionaries, including the “id” field. No validation checks are made when calling this function.
- Parameters
anns (List[Dict]) – list of annotation dictionaries
- SeeAlso:
Example
>>> import kwcoco >>> self = kwcoco.CocoDataset.demo() >>> anns = [self.anns[aid] for aid in [2, 3, 5, 7]] >>> self.remove_annotations(anns) >>> assert self.n_annots == 7 and self._check_index() >>> self.add_annotations(anns) >>> assert self.n_annots == 11 and self._check_index()
- add_images(imgs)[source]¶
Faster less-safe multi-item alternative
We assume the images are well formatted in kwcoco compliant dictionaries, including the “id” field. No validation checks are made when calling this function.
Note
THIS FUNCTION WAS DESIGNED FOR SPEED, AS SUCH IT DOES NOT CHECK IF THE IMAGE-IDs or FILE_NAMES ARE DUPLICATED AND WILL BLINDLY ADD DATA EVEN IF IT IS BAD. THE SINGLE IMAGE VERSION IS SLOWER BUT SAFER.
- Parameters
imgs (List[Dict]) – list of image dictionaries
- SeeAlso:
kwcoco.coco_dataset.MixinCocoAddRemove.add_image()
kwcoco.coco_dataset.MixinCocoAddRemove.add_images()
kwcoco.coco_dataset.MixinCocoAddRemove.ensure_image()
Example
>>> import kwcoco >>> imgs = kwcoco.CocoDataset.demo().dataset['images'] >>> self = kwcoco.CocoDataset() >>> self.add_images(imgs) >>> assert self.n_images == 3 and self._check_index()
- clear_images()[source]¶
Removes all images and annotations (but not categories)
Example
>>> import kwcoco >>> self = kwcoco.CocoDataset.demo() >>> self.clear_images() >>> print(ub.repr2(self.basic_stats(), nobr=1, nl=0, si=1)) n_anns: 0, n_imgs: 0, n_videos: 0, n_cats: 8
- clear_annotations()[source]¶
Removes all annotations (but not images and categories)
Example
>>> import kwcoco >>> self = kwcoco.CocoDataset.demo() >>> self.clear_annotations() >>> print(ub.repr2(self.basic_stats(), nobr=1, nl=0, si=1)) n_anns: 0, n_imgs: 3, n_videos: 0, n_cats: 8
- remove_annotation(aid_or_ann)[source]¶
Remove a single annotation from the dataset
If you have multiple annotations to remove its more efficient to remove them in batch with
kwcoco.coco_dataset.MixinCocoAddRemove.remove_annotations()
Example
>>> import kwcoco >>> self = kwcoco.CocoDataset.demo() >>> aids_or_anns = [self.anns[2], 3, 4, self.anns[1]] >>> self.remove_annotations(aids_or_anns) >>> assert len(self.dataset['annotations']) == 7 >>> self._check_index()
- remove_annotations(aids_or_anns, verbose=0, safe=True)[source]¶
Remove multiple annotations from the dataset.
- Parameters
anns_or_aids (List) – list of annotation dicts or ids
safe (bool, default=True) – if True, we perform checks to remove duplicates and non-existing identifiers.
- Returns
num_removed: information on the number of items removed
- Return type
Dict
Example
>>> import kwcoco >>> self = kwcoco.CocoDataset.demo() >>> prev_n_annots = self.n_annots >>> aids_or_anns = [self.anns[2], 3, 4, self.anns[1]] >>> self.remove_annotations(aids_or_anns) # xdoc: +IGNORE_WANT {'annotations': 4} >>> assert len(self.dataset['annotations']) == prev_n_annots - 4 >>> self._check_index()
- remove_categories(cat_identifiers, keep_annots=False, verbose=0, safe=True)[source]¶
Remove categories and all annotations in those categories.
Currently does not change any hierarchy information
- Parameters
cat_identifiers (List) – list of category dicts, names, or ids
keep_annots (bool, default=False) – if True, keeps annotations, but removes category labels.
safe (bool, default=True) – if True, we perform checks to remove duplicates and non-existing identifiers.
- Returns
num_removed: information on the number of items removed
- Return type
Dict
Example
>>> import kwcoco >>> self = kwcoco.CocoDataset.demo() >>> cat_identifiers = [self.cats[1], 'rocket', 3] >>> self.remove_categories(cat_identifiers) >>> assert len(self.dataset['categories']) == 5 >>> self._check_index()
- remove_images(gids_or_imgs, verbose=0, safe=True)[source]¶
Remove images and any annotations contained by them
- Parameters
gids_or_imgs (List) – list of image dicts, names, or ids
safe (bool, default=True) – if True, we perform checks to remove duplicates and non-existing identifiers.
- Returns
num_removed: information on the number of items removed
- Return type
Dict
Example
>>> import kwcoco >>> self = kwcoco.CocoDataset.demo() >>> assert len(self.dataset['images']) == 3 >>> gids_or_imgs = [self.imgs[2], 'astro.png'] >>> self.remove_images(gids_or_imgs) # xdoc: +IGNORE_WANT {'annotations': 11, 'images': 2} >>> assert len(self.dataset['images']) == 1 >>> self._check_index() >>> gids_or_imgs = [3] >>> self.remove_images(gids_or_imgs) >>> assert len(self.dataset['images']) == 0 >>> self._check_index()
- remove_videos(vidids_or_videos, verbose=0, safe=True)[source]¶
Remove videos and any images / annotations contained by them
- Parameters
vidids_or_videos (List) – list of video dicts, names, or ids
safe (bool, default=True) – if True, we perform checks to remove duplicates and non-existing identifiers.
- Returns
num_removed: information on the number of items removed
- Return type
Dict
Example
>>> import kwcoco >>> self = kwcoco.CocoDataset.demo('vidshapes8') >>> assert len(self.dataset['videos']) == 8 >>> vidids_or_videos = [self.dataset['videos'][0]['id']] >>> self.remove_videos(vidids_or_videos) # xdoc: +IGNORE_WANT {'annotations': 4, 'images': 2, 'videos': 1} >>> assert len(self.dataset['videos']) == 7 >>> self._check_index()
- remove_annotation_keypoints(kp_identifiers)[source]¶
Removes all keypoints with a particular category
- Parameters
kp_identifiers (List) – list of keypoint category dicts, names, or ids
- Returns
num_removed: information on the number of items removed
- Return type
Dict
- remove_keypoint_categories(kp_identifiers)[source]¶
Removes all keypoints of a particular category as well as all annotation keypoints with those ids.
- Parameters
kp_identifiers (List) – list of keypoint category dicts, names, or ids
- Returns
num_removed: information on the number of items removed
- Return type
Dict
Example
>>> import kwcoco >>> self = kwcoco.CocoDataset.demo('shapes', rng=0) >>> kp_identifiers = ['left_eye', 'mid_tip'] >>> remove_info = self.remove_keypoint_categories(kp_identifiers) >>> print('remove_info = {!r}'.format(remove_info)) >>> # FIXME: for whatever reason demodata generation is not determenistic when seeded >>> # assert remove_info == {'keypoint_categories': 2, 'annotation_keypoints': 16, 'reflection_ids': 1} >>> assert self._resolve_to_kpcat('right_eye')['reflection_id'] is None
- set_annotation_category(aid_or_ann, cid_or_cat)[source]¶
Sets the category of a single annotation
- Parameters
aid_or_ann (dict | int) – annotation dict or id
cid_or_cat (dict | int) – category dict or id
Example
>>> import kwcoco >>> self = kwcoco.CocoDataset.demo() >>> old_freq = self.category_annotation_frequency() >>> aid_or_ann = aid = 2 >>> cid_or_cat = new_cid = self.ensure_category('kitten') >>> self.set_annotation_category(aid, new_cid) >>> new_freq = self.category_annotation_frequency() >>> print('new_freq = {}'.format(ub.repr2(new_freq, nl=1))) >>> print('old_freq = {}'.format(ub.repr2(old_freq, nl=1))) >>> assert sum(new_freq.values()) == sum(old_freq.values()) >>> assert new_freq['kitten'] == 1
- class kwcoco.coco_dataset.CocoIndex[source]¶
Bases:
object
Fast lookup index for the COCO dataset with dynamic modification
- Variables
imgs (Dict[int, dict]) – mapping between image ids and the image dictionaries
anns (Dict[int, dict]) – mapping between annotation ids and the annotation dictionaries
cats (Dict[int, dict]) – mapping between category ids and the category dictionaries
kpcats (Dict[int, dict]) – mapping between keypoint category ids and keypoint category dictionaries
gid_to_aids (Dict[int, List[int]]) – mapping between an image-id and annotation-ids that belong to it
cid_to_aids (Dict[int, List[int]]) – mapping between an category-id and annotation-ids that belong to it
cid_to_gids (Dict[int, List[int]]) – mapping between an category-id and image-ids that contain at least one annotation with this cateogry id.
trackid_to_aids (Dict[int, List[int]]) – mapping between a track-id and annotation-ids that belong to it
vidid_to_gids (Dict[int, List[int]]) – mapping between an video-id and images-ids that belong to it
name_to_video (Dict[str, dict]) – mapping between a video name and the video dictionary.
name_to_cat (Dict[str, dict]) – mapping between a category name and the category dictionary.
name_to_img (Dict[str, dict]) – mapping between a image name and the image dictionary.
file_name_to_img (Dict[str, dict]) – mapping between a image file_name and the image dictionary.
- property cid_to_gids¶
Example: >>> import kwcoco >>> self = dset = kwcoco.CocoDataset() >>> self.index.cid_to_gids
- build(parent)[source]¶
Build all id-to-obj reverse indexes from scratch.
- Parameters
parent (kwcoco.CocoDataset) – the dataset to index
- Notation:
aid - Annotation ID gid - imaGe ID cid - Category ID vidid - Video ID
Example
>>> import kwcoco >>> parent = kwcoco.CocoDataset.demo('vidshapes1', num_frames=4, rng=1) >>> index = parent.index >>> index.build(parent)
- class kwcoco.coco_dataset.MixinCocoIndex[source]¶
Bases:
object
Give the dataset top level access to index attributes
- property anns¶
- property imgs¶
- property cats¶
- property gid_to_aids¶
- property cid_to_aids¶
- property name_to_cat¶
- class kwcoco.coco_dataset.CocoDataset(data=None, tag=None, bundle_dpath=None, img_root=None, fname=None, autobuild=True)[source]¶
Bases:
AbstractCocoDataset
,MixinCocoAddRemove
,MixinCocoStats
,MixinCocoObjects
,MixinCocoDraw
,MixinCocoAccessors
,MixinCocoExtras
,MixinCocoIndex
,MixinCocoDepricate
,NiceRepr
The main coco dataset class with a json dataset backend.
- Variables
dataset (Dict) – raw json data structure. This is the base dictionary that contains {‘annotations’: List, ‘images’: List, ‘categories’: List}
index (CocoIndex) – an efficient lookup index into the coco data structure. The index defines its own attributes like
anns
,cats
,imgs
,gid_to_aids
,file_name_to_img
, etc. SeeCocoIndex
for more details on which attributes are available.fpath (PathLike | None) – if known, this stores the filepath the dataset was loaded from
tag (str) – A tag indicating the name of the dataset.
bundle_dpath (PathLike | None) – If known, this is the root path that all image file names are relative to. This can also be manually overwritten by the user.
hashid (str | None) – If computed, this will be a hash uniquely identifing the dataset. To ensure this is computed see
kwcoco.coco_dataset.MixinCocoExtras._build_hashid()
.
References
http://cocodataset.org/#format http://cocodataset.org/#download
CommandLine
python -m kwcoco.coco_dataset CocoDataset --show
Example
>>> from kwcoco.coco_dataset import demo_coco_data >>> import kwcoco >>> import ubelt as ub >>> # Returns a coco json structure >>> dataset = demo_coco_data() >>> # Pass the coco json structure to the API >>> self = kwcoco.CocoDataset(dataset, tag='demo') >>> # Now you can access the data using the index and helper methods >>> # >>> # Start by looking up an image by it's COCO id. >>> image_id = 1 >>> img = self.index.imgs[image_id] >>> print(ub.repr2(img, nl=1, sort=1)) { 'file_name': 'astro.png', 'id': 1, 'url': 'https://i.imgur.com/KXhKM72.png', } >>> # >>> # Use the (gid_to_aids) index to lookup annotations in the iamge >>> annotation_id = sorted(self.index.gid_to_aids[image_id])[0] >>> ann = self.index.anns[annotation_id] >>> print(ub.repr2(ub.dict_diff(ann, {'segmentation'}), nl=1)) { 'bbox': [10, 10, 360, 490], 'category_id': 1, 'id': 1, 'image_id': 1, 'keypoints': [247, 101, 2, 202, 100, 2], } >>> # >>> # Use annotation category id to look up that information >>> category_id = ann['category_id'] >>> cat = self.index.cats[category_id] >>> print('cat = {}'.format(ub.repr2(cat, nl=1, sort=1))) cat = { 'id': 1, 'name': 'astronaut', 'supercategory': 'human', } >>> # >>> # Now play with some helper functions, like extended statistics >>> extended_stats = self.extended_stats() >>> # xdoctest: +IGNORE_WANT >>> print('extended_stats = {}'.format(ub.repr2(extended_stats, nl=1, precision=2, sort=1))) extended_stats = { 'annots_per_img': {'mean': 3.67, 'std': 3.86, 'min': 0.00, 'max': 9.00, 'nMin': 1, 'nMax': 1, 'shape': (3,)}, 'imgs_per_cat': {'mean': 0.88, 'std': 0.60, 'min': 0.00, 'max': 2.00, 'nMin': 2, 'nMax': 1, 'shape': (8,)}, 'cats_per_img': {'mean': 2.33, 'std': 2.05, 'min': 0.00, 'max': 5.00, 'nMin': 1, 'nMax': 1, 'shape': (3,)}, 'annots_per_cat': {'mean': 1.38, 'std': 1.49, 'min': 0.00, 'max': 5.00, 'nMin': 2, 'nMax': 1, 'shape': (8,)}, 'imgs_per_video': {'empty_list': True}, } >>> # You can "draw" a raster of the annotated image with cv2 >>> canvas = self.draw_image(2) >>> # Or if you have matplotlib you can "show" the image with mpl objects >>> # xdoctest: +REQUIRES(--show) >>> from matplotlib import pyplot as plt >>> fig = plt.figure() >>> ax1 = fig.add_subplot(1, 2, 1) >>> self.show_image(gid=2) >>> ax2 = fig.add_subplot(1, 2, 2) >>> ax2.imshow(canvas) >>> ax1.set_title('show with matplotlib') >>> ax2.set_title('draw with cv2') >>> plt.show()
- property fpath¶
In the future we will deprecate img_root for bundle_dpath
- classmethod from_data(data, bundle_dpath=None, img_root=None)[source]¶
Constructor from a json dictionary
- classmethod from_image_paths(gpaths, bundle_dpath=None, img_root=None)[source]¶
Constructor from a list of images paths.
This is a convinience method.
- Parameters
gpaths (List[str]) – list of image paths
Example
>>> import kwcoco >>> coco_dset = kwcoco.CocoDataset.from_image_paths(['a.png', 'b.png']) >>> assert coco_dset.n_images == 2
- classmethod from_coco_paths(fpaths, max_workers=0, verbose=1, mode='thread', union='try')[source]¶
Constructor from multiple coco file paths.
Loads multiple coco datasets and unions the result
Note
if the union operation fails, the list of individually loaded files is returned instead.
- Parameters
fpaths (List[str]) – list of paths to multiple coco files to be loaded and unioned.
max_workers (int, default=0) – number of worker threads / processes
verbose (int) – verbosity level
mode (str) – thread, process, or serial
union (str | bool, default=’try’) – If True, unions the result datasets after loading. If False, just returns the result list. If ‘try’, then try to preform the union, but return the result list if it fails.
- copy()[source]¶
Deep copies this object
Example
>>> import kwcoco >>> self = kwcoco.CocoDataset.demo() >>> new = self.copy() >>> assert new.imgs[1] is new.dataset['images'][0] >>> assert new.imgs[1] == self.dataset['images'][0] >>> assert new.imgs[1] is not self.dataset['images'][0]
- dumps(indent=None, newlines=False)[source]¶
Writes the dataset out to the json format
- Parameters
newlines (bool) – if True, each annotation, image, category gets its own line
Note
- Using newlines=True is similar to:
print(ub.repr2(dset.dataset, nl=2, trailsep=False)) However, the above may not output valid json if it contains ndarrays.
Example
>>> import kwcoco >>> import json >>> self = kwcoco.CocoDataset.demo() >>> text = self.dumps(newlines=True) >>> print(text) >>> self2 = kwcoco.CocoDataset(json.loads(text), tag='demo2') >>> assert self2.dataset == self.dataset >>> assert self2.dataset is not self.dataset
>>> text = self.dumps(newlines=True) >>> print(text) >>> self2 = kwcoco.CocoDataset(json.loads(text), tag='demo2') >>> assert self2.dataset == self.dataset >>> assert self2.dataset is not self.dataset
Example
>>> import kwcoco >>> self = kwcoco.CocoDataset.coerce('vidshapes1-msi-multisensor', verbose=3) >>> self.remove_annotations(self.annots()) >>> text = self.dumps(newlines=True, indent=' ') >>> print(text)
- dump(file, indent=None, newlines=False, temp_file=True)[source]¶
Writes the dataset out to the json format
- Parameters
file (PathLike | IO) – Where to write the data. Can either be a path to a file or an open file pointer / stream.
newlines (bool) – if True, each annotation, image, category gets its own line.
temp_file (bool | str, default=True) – Argument to
safer.open()
. Ignored iffile
is not a PathLike object.
Example
>>> import tempfile >>> import json >>> import kwcoco >>> self = kwcoco.CocoDataset.demo() >>> file = tempfile.NamedTemporaryFile('w') >>> self.dump(file) >>> file.seek(0) >>> text = open(file.name, 'r').read() >>> print(text) >>> file.seek(0) >>> dataset = json.load(open(file.name, 'r')) >>> self2 = kwcoco.CocoDataset(dataset, tag='demo2') >>> assert self2.dataset == self.dataset >>> assert self2.dataset is not self.dataset
>>> file = tempfile.NamedTemporaryFile('w') >>> self.dump(file, newlines=True) >>> file.seek(0) >>> text = open(file.name, 'r').read() >>> print(text) >>> file.seek(0) >>> dataset = json.load(open(file.name, 'r')) >>> self2 = kwcoco.CocoDataset(dataset, tag='demo2') >>> assert self2.dataset == self.dataset >>> assert self2.dataset is not self.dataset
- union(*, disjoint_tracks=True, **kwargs)[source]¶
Merges multiple
CocoDataset
items into one. Names and associations are retained, but ids may be different.- Parameters
*others – a series of CocoDatasets that we will merge. Note, if called as an instance method, the “self” instance will be the first item in the “others” list. But if called like a classmethod, “others” will be empty by default.
disjoint_tracks (bool, default=True) – if True, we will assume track-ids are disjoint and if two datasets share the same track-id, we will disambiguate them. Otherwise they will be copied over as-is.
**kwargs – constructor options for the new merged CocoDataset
- Returns
a new merged coco dataset
- Return type
CommandLine
xdoctest -m kwcoco.coco_dataset CocoDataset.union
Example
>>> import kwcoco >>> # Test union works with different keypoint categories >>> dset1 = kwcoco.CocoDataset.demo('shapes1') >>> dset2 = kwcoco.CocoDataset.demo('shapes2') >>> dset1.remove_keypoint_categories(['bot_tip', 'mid_tip', 'right_eye']) >>> dset2.remove_keypoint_categories(['top_tip', 'left_eye']) >>> dset_12a = kwcoco.CocoDataset.union(dset1, dset2) >>> dset_12b = dset1.union(dset2) >>> dset_21 = dset2.union(dset1) >>> def add_hist(h1, h2): >>> return {k: h1.get(k, 0) + h2.get(k, 0) for k in set(h1) | set(h2)} >>> kpfreq1 = dset1.keypoint_annotation_frequency() >>> kpfreq2 = dset2.keypoint_annotation_frequency() >>> kpfreq_want = add_hist(kpfreq1, kpfreq2) >>> kpfreq_got1 = dset_12a.keypoint_annotation_frequency() >>> kpfreq_got2 = dset_12b.keypoint_annotation_frequency() >>> assert kpfreq_want == kpfreq_got1 >>> assert kpfreq_want == kpfreq_got2
>>> # Test disjoint gid datasets >>> dset1 = kwcoco.CocoDataset.demo('shapes3') >>> for new_gid, img in enumerate(dset1.dataset['images'], start=10): >>> for aid in dset1.gid_to_aids[img['id']]: >>> dset1.anns[aid]['image_id'] = new_gid >>> img['id'] = new_gid >>> dset1.index.clear() >>> dset1._build_index() >>> # ------ >>> dset2 = kwcoco.CocoDataset.demo('shapes2') >>> for new_gid, img in enumerate(dset2.dataset['images'], start=100): >>> for aid in dset2.gid_to_aids[img['id']]: >>> dset2.anns[aid]['image_id'] = new_gid >>> img['id'] = new_gid >>> dset1.index.clear() >>> dset2._build_index() >>> others = [dset1, dset2] >>> merged = kwcoco.CocoDataset.union(*others) >>> print('merged = {!r}'.format(merged)) >>> print('merged.imgs = {}'.format(ub.repr2(merged.imgs, nl=1))) >>> assert set(merged.imgs) & set([10, 11, 12, 100, 101]) == set(merged.imgs)
>>> # Test data is not preserved >>> dset2 = kwcoco.CocoDataset.demo('shapes2') >>> dset1 = kwcoco.CocoDataset.demo('shapes3') >>> others = (dset1, dset2) >>> cls = self = kwcoco.CocoDataset >>> merged = cls.union(*others) >>> print('merged = {!r}'.format(merged)) >>> print('merged.imgs = {}'.format(ub.repr2(merged.imgs, nl=1))) >>> assert set(merged.imgs) & set([1, 2, 3, 4, 5]) == set(merged.imgs)
>>> # Test track-ids are mapped correctly >>> dset1 = kwcoco.CocoDataset.demo('vidshapes1') >>> dset2 = kwcoco.CocoDataset.demo('vidshapes2') >>> dset3 = kwcoco.CocoDataset.demo('vidshapes3') >>> others = (dset1, dset2, dset3) >>> for dset in others: >>> [a.pop('segmentation', None) for a in dset.index.anns.values()] >>> [a.pop('keypoints', None) for a in dset.index.anns.values()] >>> cls = self = kwcoco.CocoDataset >>> merged = cls.union(*others, disjoint_tracks=1) >>> print('dset1.anns = {}'.format(ub.repr2(dset1.anns, nl=1))) >>> print('dset2.anns = {}'.format(ub.repr2(dset2.anns, nl=1))) >>> print('dset3.anns = {}'.format(ub.repr2(dset3.anns, nl=1))) >>> print('merged.anns = {}'.format(ub.repr2(merged.anns, nl=1)))
Example
>>> import kwcoco >>> # Test empty union >>> empty_union = kwcoco.CocoDataset.union() >>> assert len(empty_union.index.imgs) == 0
Todo
[ ] are supercategories broken?
[ ] reuse image ids where possible
[ ] reuse annotation / category ids where possible
[X] handle case where no inputs are given
[x] disambiguate track-ids
[x] disambiguate video-ids
- subset(gids, copy=False, autobuild=True)[source]¶
Return a subset of the larger coco dataset by specifying which images to port. All annotations in those images will be taken.
- Parameters
gids (List[int]) – image-ids to copy into a new dataset
copy (bool, default=False) – if True, makes a deep copy of all nested attributes, otherwise makes a shallow copy.
autobuild (bool, default=True) – if True will automatically build the fast lookup index.
Example
>>> import kwcoco >>> self = kwcoco.CocoDataset.demo() >>> gids = [1, 3] >>> sub_dset = self.subset(gids) >>> assert len(self.index.gid_to_aids) == 3 >>> assert len(sub_dset.gid_to_aids) == 2
Example
>>> import kwcoco >>> self = kwcoco.CocoDataset.demo('vidshapes2') >>> gids = [1, 2] >>> sub_dset = self.subset(gids, copy=True) >>> assert len(sub_dset.index.videos) == 1 >>> assert len(self.index.videos) == 2
Example
>>> import kwcoco >>> self = kwcoco.CocoDataset.demo() >>> sub1 = self.subset([1]) >>> sub2 = self.subset([2]) >>> sub3 = self.subset([3]) >>> others = [sub1, sub2, sub3] >>> rejoined = kwcoco.CocoDataset.union(*others) >>> assert len(sub1.anns) == 9 >>> assert len(sub2.anns) == 2 >>> assert len(sub3.anns) == 0 >>> assert rejoined.basic_stats() == self.basic_stats()
- view_sql(force_rewrite=False, memory=False)[source]¶
Create a cached SQL interface to this dataset suitable for large scale multiprocessing use cases.
- Parameters
force_rewrite (bool, default=False) – if True, forces an update to any existing cache file on disk
memory (bool, default=False) – if True, the database is constructed in memory.
Note
This view cache is experimental and currently depends on the timestamp of the file pointed to by
self.fpath
. In other words dont use this on in-memory datasets.
- kwcoco.coco_dataset.demo_coco_data()[source]¶
Simple data for testing.
This contains several non-standard fields, which help ensure robustness of functions tested with this data. For more compliant demodata see the
kwcoco.demodata
submoduleExample
>>> # xdoctest: +REQUIRES(--show) >>> from kwcoco.coco_dataset import demo_coco_data, CocoDataset >>> dataset = demo_coco_data() >>> self = kwcoco.CocoDataset(dataset, tag='demo') >>> import kwplot >>> kwplot.autompl() >>> self.show_image(gid=1) >>> kwplot.show_if_requested()