Chianti

This is a Python library for loading and augmenting training data asynchronously in the background. It is primarily geared towards pipelines for semantic segmentation where you have a source image and a densely annotated target image.

The library consists of four major components:

  • A set of image loaders that provide the raw image data.
  • A set of iterators that determine the order in which we iterate over a given dataset
  • A set of augmentors for synthetically increasing the diversity of the data set.
  • A data provider that connects all components and returns batches of augmented images.
class DataProvider

This class allows you to load batches of (source, target) pairs asynchronously. Standard transformations:

__init__(augmentor, source_img_loader, target_img_loader, iterator, batch_size)

Initializes a new instance of the DataProvider class.

Parameters:
next()

Returns the next batch of images.

Returns:A tuple of two numpy arrays.
reset()

Resets the underlying iterator to the beginning. This is useful if you want to iterate over a dataset deterministically.

get_num_batches()

Returns the total number of batches per epoch.

Returns:The number of batches per epoch.
Return type:int
class Iterator

A data iterator class.

next()

Returns the next item.

Returns:A tuple of two strings.
reset()

Resets the iterator to its initial state. This also works for iterators that involve randomness.

get_num_elements()

Returns the total number of elements in the structure the we iterate over.

Returns:The number of elements in the underlying structure.
Return type:int
static Sequential(data_list)

Factory method that creates a new sequential iterator. A sequential iterator iterates deterministically over the provided list of elements.

Parameters:data_list (list) – A list of string tuples.
static Random(data_list)

Factory method that creates a new random iterator. A random iterator iterates over the dataset randomly in epochs. This means at the beginning of each epoch, the dataset is shuffled and the iterated over deterministically.

Parameters:data_list (list) – A list of string tuples.
static WeightedRandom(data_list, weights)

Factory method that creates a new weighted random iterator. This iterator draws each training example independently from the dataset with a probability proportional to the associated weights.

Parameters:
  • data_list (list) – A list of string tuples.
  • weights – A list or numpy array of non-negative weights. There must be exactly one weight for each element of the data list.
class Loader

A loader class for providing raw image data.

static RGB()

Factory method for creating a loader that simply loads RGB images from the hard drive.

static Label()

Factory method for creating a loader that loads 8bit class label images from the hard drive.

static ValueMapper(value_map)

Factory method for creating a loader that loads an 8bit class label image from the hard drive and then maps the pixel values according to the provided mapping.

Parameters:value_map – A list of numpy array of exactly 256 elements that represents a permutation.
static ColorMapper(color_map)

Factory method for creating a loader that loads an RGB class label image from the hard drive and then maps the pixel values according to the provided mapping.

Parameters:color_map – A dict that maps colors (RGB tuples) to 8bit integer values.
class Augmentor

A data augmentation class.

static Subsample(factor)

Factory method that creates an augmentor that subsamples the source and the target image by the given factor.

Parameters:factor (int) – Subsampling factor.
static Gamma(strength)

Factory method that creates an augmentor that performs random gamma augmentation on the source image.

Parameters:strength (double) – Strength of the augmentation. Must be a value between 0 and 0.5.
static Translation(factor)

Factory method that creates an augmentor that randomly translates both the source and the target image. On the source image, it uses reflection padding whereas on the target image it uses constant padding with void labels (value -1).

Parameters:offset (int) – Maximum translation offset in any direction.
static Zooming(offset)

Factory method that creates an augmentor that randomly zooms into the image center.

Parameters:factor (double) – Maximum zooming factor.
static Rotation(angel)

Factory method that creates an augmentor that randomly rotates the images.

Parameters:angel (double) – Maximum rotation angel.
static Crop(size, num_classes)

Factory method that creates a crop augmentor. A crop augmentor randomly samples square crops from an image. The probability of a crop being sampled is proportional to the entropy of the class distribution within the crop.

Parameters:
  • size (int) – The size of the crop.
  • num_classes (int) – The total number of classes.
static Saturation(delta_min, delta_max)

Factory method that creates an augmentor that randomly augments the image’s saturation.

Parameters:
  • delta_min (double) – Minimum saturation multiplier.
  • delta_max (double) – Maximum saturation multiplier.
static Hue(delta_min, delta_max)

Factory method that creates an augmentor that randomly augments the image’s hue.

Parameters:
  • delta_min (double) – Minimum hue multiplier.
  • delta_max (double) – Maximum hue multiplier.