atlalign.ml_utils package

Submodules

atlalign.ml_utils.augmentation module

Augmentation related tools.

augmenter_1(p=0.99)[source]

Create augmenter.

Contains no coordinate transforms.

Parameters

p (float) – Number in [0, 1] representing the probability of a random augmentation happening.

Returns

seq – Augmenter where each augmentation was manually inspected and makes sense.

Return type

iaa.Augmenter

atlalign.ml_utils.callbacks module

Callbacks and aggregation functions.

class MLFlowCallback(merged_path, train_original_ixs_path, val_original_ixs_path, freq=10, workers=1, return_inverse=False, starting_step=0, use_validation=True)[source]

Bases: keras.callbacks.Callback

Logs metrics into ML.

Notes

Only runs inside of an mlflow context.

Parameters
  • merged_path (str) – Path to the master h5 file containing all the data.

  • train_original_ixs_path (str) – Path to where original training indices are stored.

  • val_original_ixs_path (str) – Path to where original validation indices are stored.

  • freq (int) – Reports metrics on every freq batch.

  • workers (int) – Number of workers to be used for each of the evaluations.

  • return_inverse (bool) – If True, then generators behave differently.

  • starting_step (int) – Useful when we want to use a checkpointed model and log metrics as of a different step then 1.

  • use_validation (int) – If True, then the custom metrics are computed on the validation set. Otherwise they will be computed on the training set.

static compute_external_metrics(model, gen)[source]

Compute external matrics sample by sample.

Parameters
Returns

metrics – Various metrics.

Return type

dict

on_batch_end(batch, logs=None)[source]

Log metrics to mlflow.

The goal here is two extract 3 types of metrics:
  • train_merged - extracted from logs (it is a running average over epoch)

  • train_original - computed via evaluate_generator

  • val_original - computed via evaluate_generator

on_train_begin(logs=None)[source]

Save model architecture.

get_mlflow_artifact_path(start_char=7)[source]

Get path to the MLFlow artifacts of the active run.

Stupid implementation.

Parameters

start_char (int) – Since the string will start like “file:///actual/path…” we just slice it.

atlalign.ml_utils.io module

Collection of functions dealing with input and output.

class SupervisedGenerator(hdf_path, batch_size=32, shuffle=False, augmenter_ref=None, augmenter_mov=None, return_inverse=False, mlflow_log=False, indexes=None)[source]

Bases: keras.utils.data_utils.Sequence

Generator streaming supervised data from a HDF5 files.

Parameters
  • hdf_path (str or pathlib.Path) – Path to where the hdf5 file is.

  • batch_size (int) – Batch size.

  • shuffle (bool) – If True, then data shuffled at the end of each epoch and also in the constructor.

  • augmenter_ref (None or imgaug.augmenters.Sequential) – If None, no augmentation. If instance of a imgaug Sequential then its a pipeline that will be used to augment all reference images in a batch.

  • augmenter_mov (None or imgaug.augmenters.Sequential) – If None, no augmentation. If instance of a imgaug Sequential then its a pipeline that will be used to augment all moving images in a batch.

  • return_inverse (bool) – If True, then targets are [img_reg, dvf, dvf_inv] else its only [img_reg, dvf].

  • indexes (None or list or str/pathlib.Path) – A user defined list of indices to be used for streaming. This is used to give user the chance to only stream parts of the data. If str or pathlib.Path then read from a .npy file

indexes

List of indices determining the slicing order.

Type

list

volume

Array representing the nissle stain volume of dtype float32.

Type

np.array

times

List of timedeltas representing the for each batch yeild.

Type

list

get_all_data()[source]

Load entire dataset into memory.

on_epoch_end()[source]

Take end of epoch action.

atlalign.ml_utils.layers module

Custom keras layers.

class Affine2DVF(*args, **kwargs)[source]

Bases: keras.engine.base_layer.Layer

Given an affine transformation matrix (2 x 3) generate the corresponding DVF.

call(a, mask=None)[source]

Turn a batch of affine matrices into the corresponding batch of displacement vector fields.

Parameters

a (K.Tensor) – A tensor of shape (batch_size, 2, 3) representing an affine transformation for each sample.

Returns

dvfs – A tensor of shape (batch_size, h, w, 2) representing the corresponding displacement vector fields.

Return type

K.Tensor

compute_output_shape(*args)[source]

Compute the output shape.

Parameters

args (list) – Whatever positional arguments. We already know the expected shape from the self.shape passed into the constructor.

Returns

  • batch_size (int) – Always None since we want to allow for variable batch_size.

  • height (int) – Height of the image.

  • width (int) – Width of the image.

  • n_channels (int) – Always equal to 2 because it represents delta_x and delta_y.

get_config()[source]

Allow for model to be loaded correctly.

We just need to add to the config dictionary all the custom constructor parameters.

class BilinearInterpolation(*args, **kwargs)[source]

Bases: keras.engine.base_layer.Layer

Implementation using tf.contrib.

call(tensors, mask=None)[source]

Perform forward pass.

Parameters
  • tensors (list) –

    A list of two elements - images to we warped and the displacement vector fields. Their shapes are the following

    • images to be warped : (batch_size, h, w, 1)

    • dvfs : (batch_size, h, w, 2)

  • mask (None) –

Returns

output – Warped images with the provided dvfs. Shape (batch_size, h, w, 1).

Return type

K.Tensor

compute_output_shape(input_shapes)[source]

Compute output shape.

Notes

This is a mandatory method to implement for custom Keras layers.

Parameters

input_shapes (list) – A list of the shape of images to be warped and the dvfs respectively. - images to be warped : (batch_size, h, w, 1) - dvfs : (batch_size, h, w, 2)

Returns

  • None – Batch size

  • height (int) – Height of the image.

  • width (int) – Width of the image.

  • n_channels (int) – Number of channels, should be 1.

class BilinearInterpolation_(*args, **kwargs)[source]

Bases: keras.engine.base_layer.Layer

Perform bilinear interpolation as a Keras layer.

Notes

Currently only works with tensorflow backend since we need to use tf.meshgrid.

call(tensors, mask=None)[source]

Perform forward pass.

Parameters
  • tensors (list) –

    A list of two elements - images to we warped and the displacement vector fields. Their shapes are the following

    • images to be warped : (batch_size, h, w, 1)

    • dvfs : (batch_size, h, w, 2)

  • mask (None) –

Returns

output – Warped images with the provided dvfs. Shape (batch_size, h, w, 1).

Return type

K.Tensor

compute_output_shape(input_shapes)[source]

Compute output shape.

Notes

This is a mandatory method to implement for custom Keras layers.

Parameters

input_shapes (list) – A list of the shape of images to be warped and the dvfs respectively. - images to be warped : (batch_size, h, w, 1) - dvfs : (batch_size, h, w, 2)

Returns

  • None – Batch size

  • height (int) – Height of the image.

  • width (int) – Width of the image.

  • n_channels (int) – Number of channels, should be 1.

class DVFComposition(*args, **kwargs)[source]

Bases: keras.engine.base_layer.Layer

Composition of 2 displacement vector fields.

Notes

Computes the DVF of f ∘ g

call(tensors, mask=None)[source]

Composition.

Parameters

tensors (list) – A list of two elements - outer and inner displacement vector field. Their shapes are identical and equal to (batch_size, h, w, 2).

Returns

output – A displacement vector field of the composite transformation of shape (batch_size, h, w, 2).

Return type

K.tensor

compute_output_shape(input_shapes)[source]

Compute output shape.

Parameters

input_shapes (list) – A list of the shape of th 2 dvfs respectively. - dvfs_outer: (batch_size, h, w, 2) - dvfs_inner : (batch_size, h, w, 2)

Returns

  • None – Batch size

  • height (int) – Height of the image.

  • width (int) – Width of the image.

  • n_dim (int) – 2

class ExtractMoving(*args, **kwargs)[source]

Bases: keras.engine.base_layer.Layer

Extract the moving image from the input.

call(x)[source]

Perform forward pass.

compute_output_shape(input_shape)[source]

Compute output shape.

K_meshgrid(x, y)[source]

Generate meshgrid.

Notes

Not part of Keras backend convertor.

class NoOp(*args, **kwargs)[source]

Bases: keras.engine.base_layer.Layer

No operation layer.

call(x)[source]

Perform forward pass.

compute_output_shape(input_shape)[source]

Compute output shape.

block_stn(images, filters=(16, 32, 64), dense_layers=(50,), block_name='STN', return_dvfs=True, return_inverse=True, include_bn_dense=False, include_bn_conv=False)[source]

Spatial transformer network block.

Parameters
  • images (K.tensor) –

    Tensor of shape (batch_size, h, w, 2) representing the batch of images to be warped. The first channel

    represents the atlas image and the second one is the moving image.

  • filters (tuple) – A tuple whose length represents the number of convolutional layers and each element represents the number of filters of that layer. Make sure that h % (2 ** len(filters)) == 0 and w % (2 ** len(filters)) == 0 because of max pooling.

  • dense_layers (tuple) – A tuple whose length represents the number of dense layers after the convolutional layers and each element represents the number of nodes. Note a final dense layer with 8 nodes (number of parameters in the affine transformation) is added automatically everytime.

  • block_name (str) – Name of the block.

  • return_dvfs (bool) – If True then also returns the dsvf tensor then can later be used for composition.

  • return_inverse (bool) – If True, then warping applied to the second channel (moving image), otherwise the first image (atlas).

  • include_bn_dense (bool) – If True, batch normalization is performed after each dense layer layer.

  • include_bn_conv (bool) – If True, batch normalization is performed after each convolutional layer.

Returns

warped_images

Tensor of shape (batch_size, h, w, 1) representing the batch of warped images with the predicted affine

transformation. If return_inverse True then this is the moving image. Else the reference.

dvfsK.tensor

Tensor of shape (batch_size, h, w, 2) representing the batch of delta_x and delta_y displacements. Only returned if return_dvfs is True.

Return type

K.tensor

get_initial_weights(previous_layer_size)[source]

Initialize affine matrix to identity transformation.

Notes

It makes a lot of sense to set the behavior before training to learning an identity transformation.

Parameters

previous_layer_size (int) – Size of the previous dense layer.

Returns

initial_weights

A list of 2 numpy ndarrays:
  1. weight matrix, shape (previous_layer_size, 6)

  2. biases, shape (6,)

Return type

list

atlalign.ml_utils.losses module

keras and Tensorflow implementation of different losses and metrics.

class DVF2IMG(img_loss_callable, weights=(0.5, 0.5))[source]

Bases: object

Class turning image losses into dvf losses.

loss(y_true, y_pred)[source]

Compute loss.

static normalize(img_ref, img_other, c=10)[source]

Divide by a constant.

class Grad(penalty='l1')[source]

Bases: object

Class computing gradient loss with different hyperparameters.

loss(_, y_pred)[source]

Compute loss.

class Mixer(*args, weights=None)[source]

Bases: object

Mixes together multiple different losses.

loss(y_true, y_pred)[source]

Compute loss.

class NCC(win=None, eps=1e-05)[source]

Bases: object

Class computing normalized cross correlations under different hyperparameters.

loss(I, J)[source]

Compute loss.

ncc(I, J)[source]

Compute correlation.

class PerceptualLoss(model='net-lin', net='alex')[source]

Bases: object

Class computing perceptual loss under different hyperparameters.

loss(y_true, y_pred)[source]

Compute loss.

class VDClipper(value=20, power=3)[source]

Bases: object

Clipping class.

loss(y_true, y_pred)[source]

Compute loss.

cross_correlation(y_true, y_pred)[source]

Pearson cross correlation.

Parameters
  • y_true (K.tensor) – Tensor of shape (None, h, w, 1) representing the ground truth image.

  • y_pred (K.tensor) – Tensor of shape (None, h, w, 1) representing the predicted image.

Returns

cc – Scalar tensor representing the 1 - correlation.

Return type

K.Tensor

jacobian(_, y_pred)[source]

Compute loss.

jacobian_distance(y_true, y_pred, norm='l2')[source]

Compute average per pixel distance between jacobians.

Parameters
  • y_true (K.tensor) – True DVF.

  • y_pred (K.tensor) – Pred DVF.

  • norm (str, {'l1', 'l2'}) – Norm to use.

mse_po(y_true, y_pred)[source]

Compute loss.

psnr(y_true, y_pred)[source]

Compute the negative of PSNR.

Parameters
  • y_true (K.tensor) – Tensor of shape (None, h, w, 1) representing the ground truth image.

  • y_pred (K.tensor) – Tensor of shape (None, h, w, 1) representing the predicted image.

Returns

res – Scalar tensor representing the -psnr.

Return type

K.Tensor

ssim(y_true, y_pred, **kwargs)[source]

Compute the negative of structural similarity.

Parameters
  • y_true (K.tensor) – Tensor of shape (None, h, w, 1) representing the ground truth image.

  • y_pred (K.tensor) – Tensor of shape (None, h, w, 1) representing the predicted image.

  • kwargs (dict) – Additional hyperparameters to be passed into the tensorflow implementation of ssim.

Returns

res – Scalar tensor representing the -ssim.

Return type

K.Tensor

References

[1] https://www.tensorflow.org/api_docs/python/tf/image/ssim

vector_distance(y_true, y_pred)[source]

Compute loss.

atlalign.ml_utils.models module

Module containing utilities for manipulation of keras models.

load_model(path, compile=False)[source]

Load a model that uses custom atlalign layers.

The benefit of using this function as opposed to the keras equivalent is that the user does not have to care about how the model was saved (whether architecture and weights were separated). Additionally, all custom possible custom layers are provided.

Parameters
  • path (str or pathlib.Path) – If path is a folder then the folder is expected to have one .h5 file (weights) and one .json (architecture). If path is a file then it needs to be an .h5 and it needs to encapsulate both the weights and the architecture.

  • compile (bool) – Only possible if path refers to a .h5 file.

Returns

Model ready for inference. If compile=True then also ready for continuing the training.

Return type

keras.Model

merge_global_local(model_g, model_l, expose_global=False)[source]

Merge a global and local aligner models into a new one.

Seems to be also changing the input models.

Parameters
  • model_g (keras.Model) – Model performing global alignment.

  • model_l (keras.Model) – Model performing local alignment.

  • expose_global (bool, optional) – If True, then model has 4 outputs where the last two represent image after global alignment and the corresponding dvfs. If False, then only 2 outputs : image after both global and local and the overall dvfs.

Returns

model_gl – Model performing both the local and the global alignment.

Return type

keras.Model

replace_lambda_in_config(input_config, output_format='dict', verbose=False)[source]

Replace Lambda layers with full blown keras layers.

This function only exists because we pretrained a lot of models with 2 different Lambda layers and only after that realized that they cause issues during serialization. One can then use this function to just fix it.

Notes

To make this clear let us define the top dictionary as the one that has keys

  • ‘backend’

  • ‘config’

  • ‘keras_version’

  • ‘class_name’

The bottom dictionary is top[‘config’].

Parameters
  • input_config (dict or str or pathlib.Path or keras.Model) – Config containing an architecture generated by one of the functions in atlalign.nn possibly containing Lambda layers.

  • output_format (str, {'dict', 'keras', 'json'}) –

    What output type to use. See how below how to instantiate a model out of each of the formats

    • ’dict’ - keras.Model.from_config

    • ’json’ - kersa.models.model_from_json

    • ’keras’ - already a model instance

  • verbose (bool) – If True, printing to standard output.

Returns

output_config – Config containing an architecture of the input network but all Lambda layers are replaced by full blown operations.

Return type

dict

save_model(model, path, separate=True, overwrite=True)[source]

Save model.

Parameters
  • model (keras.Model) – Keras model to be saved.

  • path (str or pathlib.Path) – Path to where to save the serialized model. If separate=True then it needs to represent a folder name. Inside of the folder 2 files are created - weights (.h5) and architecture (.json). If separate=False then an extension .h5 is added and architecture + weights are dumped into one file.

  • separate (bool) – If True, then architecture and weights are stored separately. Note that if False then one might encounter issues when loading in in a different Python environment (see references).

  • overwrite (bool) – If True, then possible existing files/folders are overwritten.

References

[1] https://github.com/keras-team/keras/issues/9595

Module contents

Prevent 3 level imports for important objects.

Source

source/api/atlalign.ml_utils.rst