kwcoco.data.grab_spacenet

References

https://medium.com/the-downlinq/the-spacenet-7-multi-temporal-urban-development-challenge-algorithmic-baseline-4515ec9bd9fe https://arxiv.org/pdf/2102.11958.pdf https://spacenet.ai/sn7-challenge/

Module Contents

Classes

Archive

Abstraction over zipfile and tarfile

Functions

unarchive_file(archive_fpath, output_dpath='.', verbose=1, overwrite=True)
grab_spacenet7(data_dpath)	References
convert_spacenet_to_kwcoco(extract_dpath, coco_fpath)	Converts the raw SpaceNet7 dataset to kwcoco
main()

class kwcoco.data.grab_spacenet.Archive(fpath, mode='r')

Bases: object

Abstraction over zipfile and tarfile

SeeAlso:

https://github.com/RKrahl/archive-tools https://pypi.org/project/arlib/

Example

>>> from kwcoco.data.grab_spacenet import *  # NOQA
>>> from os.path import join
>>> dpath = ub.ensure_app_cache_dir('ubelt', 'tests', 'archive')
>>> ub.delete(dpath)
>>> dpath = ub.ensure_app_cache_dir(dpath)
>>> import pathlib
>>> dpath = pathlib.Path(dpath)
>>> #
>>> #
>>> mode = 'w'
>>> self1 = Archive(str(dpath / 'demo.zip'), mode=mode)
>>> self2 = Archive(str(dpath / 'demo.tar.gz'), mode=mode)
>>> #
>>> open(dpath / 'data_1only.txt', 'w').write('bazbzzz')
>>> open(dpath / 'data_2only.txt', 'w').write('buzzz')
>>> open(dpath / 'data_both.txt', 'w').write('foobar')
>>> #
>>> self1.add(dpath / 'data_both.txt')
>>> self1.add(dpath / 'data_1only.txt')
>>> #
>>> self2.add(dpath / 'data_both.txt')
>>> self2.add(dpath / 'data_2only.txt')
>>> #
>>> self1.close()
>>> self2.close()
>>> #
>>> self1 = Archive(str(dpath / 'demo.zip'), mode='r')
>>> self2 = Archive(str(dpath / 'demo.tar.gz'), mode='r')
>>> #
>>> extract_dpath = ub.ensuredir(str(dpath / 'extracted'))
>>> extracted1 = self1.extractall(extract_dpath)
>>> extracted2 = self2.extractall(extract_dpath)
>>> for fpath in extracted2:
>>>     print(open(fpath, 'r').read())
>>> for fpath in extracted1:
>>>     print(open(fpath, 'r').read())

__iter__(self)

add(self, fpath, arcname=None)

close(self)

__enter__(self)

__exit__(self, *args)

extractall(self, output_dpath='.', verbose=1, overwrite=True)

kwcoco.data.grab_spacenet.unarchive_file(archive_fpath, output_dpath='.', verbose=1, overwrite=True)

kwcoco.data.grab_spacenet.grab_spacenet7(data_dpath)

References

https://spacenet.ai/sn7-challenge/

Requires:

awscli

kwcoco.data.grab_spacenet.convert_spacenet_to_kwcoco(extract_dpath, coco_fpath)

Converts the raw SpaceNet7 dataset to kwcoco

Note

• The “train” directory contains 60 “videos” representing a region over time.

• Each “video” directory contains :

  • images - unmasked images

  • images_masked - images with masks applied

  • labels - geojson polys in wgs84?

  • labels_match - geojson polys in wgs84 with track ids?

  • labels_match_pix - geojson polys in pixels with track ids?

  • UDM_masks - unusable data masks (binary data corresponding with an image, may not exist)

File names appear like:

“global_monthly_2018_01_mosaic_L15-1538E-1163N_6154_3539_13”

kwcoco.data.grab_spacenet.main()