## Submodules¶

Domain adaptation architectures should be indifferent to the task at hand: digits, toy datasets, recsys etc. Take modules as input and organise them into an architecture.

class ada.models.architectures.BaseAdaptTrainer(dataset, feature_extractor, task_classifier, method=None, lambda_init=1.0, adapt_lambda=True, adapt_lr=True, nb_init_epochs=10, nb_adapt_epochs=50, batch_size=32, init_lr=0.001, optimizer=None)[source]

Bases: sphinx.ext.autodoc.importer._MockObject

compute_loss(batch, split_name='V')[source]

Define the loss of the model

Parameters: batch (tuple) – batches returned by the MultiDomainLoader. split_name (str, optional) – learning stage (one of [“T”, “V”, “Te”]). Defaults to “V” for validation. “T” is for training and “Te” for test. This is currently used only for naming the metrics used for logging. a 3-element tuple with task_loss, adv_loss, log_metrics. log_metrics should be a dictionary. NotImplementedError – children of this classes should implement this method.
configure_optimizers()[source]
forward(x)[source]
get_parameters_watch_list()[source]

Update this list for parameters to watch while training (ie log with MLFlow)

method
test_dataloader()[source]
test_epoch_end(outputs)[source]
test_step(batch, batch_nb)[source]
train_dataloader()[source]
training_step(batch, batch_nb)[source]

The most generic of training steps

Parameters: batch (tuple) – the batch as returned by the MultiDomainLoader dataloader iterator: 2 tuples: (x_source, y_source), (x_target, y_target) in the unsupervised setting 3 tuples: (x_source, y_source), (x_target_labeled, y_target_labeled), (x_target_unlabeled, y_target_unlabeled) in the semi-supervised setting batch_nb (int) – id of the current batch. must contain a “loss” key with the loss to be used for back-propagation. see pytorch-lightning for more details. dict
val_dataloader()[source]
validation_epoch_end(outputs)[source]
validation_step(batch, batch_nb)[source]
class ada.models.architectures.BaseDANNLike(dataset, feature_extractor, task_classifier, critic, alpha=1.0, entropy_reg=0.0, adapt_reg=True, batch_reweighting=False, **base_params)[source]
compute_loss(batch, split_name='V')[source]

Define the loss of the model

Parameters: batch (tuple) – batches returned by the MultiDomainLoader. split_name (str, optional) – learning stage (one of [“T”, “V”, “Te”]). Defaults to “V” for validation. “T” is for training and “Te” for test. This is currently used only for naming the metrics used for logging. a 3-element tuple with task_loss, adv_loss, log_metrics. log_metrics should be a dictionary. NotImplementedError – children of this classes should implement this method.
get_parameters_watch_list()[source]

Update this list for parameters to watch while training (ie log with MLFlow)

test_epoch_end(outputs)[source]
validation_epoch_end(outputs)[source]
class ada.models.architectures.BaseMMDLike(dataset, feature_extractor, task_classifier, kernel_mul=2.0, kernel_num=5, **base_params)[source]
compute_loss(batch, split_name='V')[source]

Define the loss of the model

Parameters: batch (tuple) – batches returned by the MultiDomainLoader. split_name (str, optional) – learning stage (one of [“T”, “V”, “Te”]). Defaults to “V” for validation. “T” is for training and “Te” for test. This is currently used only for naming the metrics used for logging. a 3-element tuple with task_loss, adv_loss, log_metrics. log_metrics should be a dictionary. NotImplementedError – children of this classes should implement this method.
forward(x)[source]
test_epoch_end(outputs)[source]
validation_epoch_end(outputs)[source]
class ada.models.architectures.CDANtrainer(dataset, feature_extractor, task_classifier, critic, use_entropy=False, use_random=False, random_dim=1024, **base_params)[source]

compute_loss(batch, split_name='V')[source]

Define the loss of the model

Parameters: batch (tuple) – batches returned by the MultiDomainLoader. split_name (str, optional) – learning stage (one of [“T”, “V”, “Te”]). Defaults to “V” for validation. “T” is for training and “Te” for test. This is currently used only for naming the metrics used for logging. a 3-element tuple with task_loss, adv_loss, log_metrics. log_metrics should be a dictionary. NotImplementedError – children of this classes should implement this method.
forward(x)[source]
class ada.models.architectures.DANNtrainer(dataset, feature_extractor, task_classifier, critic, method=None, **base_params)[source]

This class implements the DANN architecture from Ganin, Yaroslav, et al. “Domain-adversarial training of neural networks.” The Journal of Machine Learning Research (2016) https://arxiv.org/abs/1505.07818

forward(x)[source]
class ada.models.architectures.DANtrainer(dataset, feature_extractor, task_classifier, **base_params)[source]

This is an implementation of DAN Long, Mingsheng, et al. “Learning Transferable Features with Deep Adaptation Networks.” International Conference on Machine Learning. 2015. http://proceedings.mlr.press/v37/long15.pdf code based on https://github.com/thuml/Xlearn.

class ada.models.architectures.FewShotDANNtrainer(dataset, feature_extractor, task_classifier, critic, method, **base_params)[source]

Implements adaptations of DANN to the semi-supervised setting

naive: task classifier is trained on labeled target data, in addition to source data. MME: immplements Saito, Kuniaki, et al. “Semi-supervised domain adaptation via minimax entropy.” Proceedings of the IEEE International Conference on Computer Vision. 2019 https://arxiv.org/pdf/1904.06487.pdf

compute_loss(batch, split_name='V')[source]

Define the loss of the model

Parameters: batch (tuple) – batches returned by the MultiDomainLoader. split_name (str, optional) – learning stage (one of [“T”, “V”, “Te”]). Defaults to “V” for validation. “T” is for training and “Te” for test. This is currently used only for naming the metrics used for logging. a 3-element tuple with task_loss, adv_loss, log_metrics. log_metrics should be a dictionary. NotImplementedError – children of this classes should implement this method.
forward(x)[source]
class ada.models.architectures.JANtrainer(dataset, feature_extractor, task_classifier, kernel_mul=(2.0, 2.0), kernel_num=(5, 1), **base_params)[source]

This is an implementation of JAN Long, Mingsheng, et al. “Deep transfer learning with joint adaptation networks.” International Conference on Machine Learning, 2017. https://arxiv.org/pdf/1605.06636.pdf code based on https://github.com/thuml/Xlearn.

class ada.models.architectures.Method[source]

Bases: enum.Enum

Lists the available methods. Provides a few methods that group the methods by type.

CDAN = 'CDAN'
CDAN_E = 'CDAN-E'
DAN = 'DAN'
DANN = 'DANN'
FSDANN = 'FSDANN'
JAN = 'JAN'
MME = 'MME'
Source = 'Source'
WDGRL = 'WDGRL'
WDGRLMod = 'WDGRLMod'
allow_supervised()[source]
is_cdan_method()[source]
is_dann_method()[source]
is_fewshot_method()[source]
is_mmd_method()[source]
class ada.models.architectures.WDGRLtrainer(dataset, feature_extractor, task_classifier, critic, k_critic=5, gamma=10, beta_ratio=0, **base_params)[source]

Implements WDGRL as described in Shen, Jian, et al. “Wasserstein distance guided representation learning for domain adaptation.” Thirty-Second AAAI Conference on Artificial Intelligence. 2018. https://arxiv.org/pdf/1707.01217.pdf

This class also implements the asymmetric ($eta$) variant described in: Wu, Yifan, et al. “Domain adaptation with asymmetrically-relaxed distribution alignment.” ICML (2019) https://arxiv.org/pdf/1903.01689.pdf

compute_loss(batch, split_name='V')[source]

Define the loss of the model

Parameters: batch (tuple) – batches returned by the MultiDomainLoader. split_name (str, optional) – learning stage (one of [“T”, “V”, “Te”]). Defaults to “V” for validation. “T” is for training and “Te” for test. This is currently used only for naming the metrics used for logging. a 3-element tuple with task_loss, adv_loss, log_metrics. log_metrics should be a dictionary. NotImplementedError – children of this classes should implement this method.
configure_optimizers()[source]
critic_update_steps(batch)[source]
forward(x)[source]
training_step(batch, batch_id)[source]

The most generic of training steps

Parameters: batch (tuple) – the batch as returned by the MultiDomainLoader dataloader iterator: 2 tuples: (x_source, y_source), (x_target, y_target) in the unsupervised setting 3 tuples: (x_source, y_source), (x_target_labeled, y_target_labeled), (x_target_unlabeled, y_target_unlabeled) in the semi-supervised setting batch_nb (int) – id of the current batch. must contain a “loss” key with the loss to be used for back-propagation. see pytorch-lightning for more details. dict
class ada.models.architectures.WDGRLtrainerMod(dataset, feature_extractor, task_classifier, critic, k_critic=5, gamma=10, beta_ratio=0, **base_params)[source]

Implements a modified version WDGRL as described in Shen, Jian, et al. “Wasserstein distance guided representation learning for domain adaptation.” Thirty-Second AAAI Conference on Artificial Intelligence. 2018. https://arxiv.org/pdf/1707.01217.pdf

This class also implements the asymmetric ($eta$) variant described in: Wu, Yifan, et al. “Domain adaptation with asymmetrically-relaxed distribution alignment.” ICML (2019) https://arxiv.org/pdf/1903.01689.pdf

configure_optimizers()[source]
critic_update_steps(batch)[source]
optimizer_step(current_epoch, batch_nb, optimizer, optimizer_i, second_order_closure=None)[source]
training_step(batch, batch_id, optimizer_idx)[source]

The most generic of training steps

Parameters: batch (tuple) – the batch as returned by the MultiDomainLoader dataloader iterator: 2 tuples: (x_source, y_source), (x_target, y_target) in the unsupervised setting 3 tuples: (x_source, y_source), (x_target_labeled, y_target_labeled), (x_target_unlabeled, y_target_unlabeled) in the semi-supervised setting batch_nb (int) – id of the current batch. must contain a “loss” key with the loss to be used for back-propagation. see pytorch-lightning for more details. dict
ada.models.architectures.create_dann_like(method: ada.models.architectures.Method, dataset, feature_extractor, task_classifier, critic, **train_params)[source]
ada.models.architectures.create_fewshot_trainer(method: ada.models.architectures.Method, dataset, feature_extractor, task_classifier, critic, **train_params)[source]
ada.models.architectures.create_mmd_based(method: ada.models.architectures.Method, dataset, feature_extractor, task_classifier, **train_params)[source]
ada.models.architectures.get_aggregated_metrics(metric_name_list, metric_outputs)[source]
ada.models.architectures.get_aggregated_metrics_from_dict(input_metric_dict)[source]
ada.models.architectures.get_metrics_from_parameter_dict(parameter_dict, device)[source]
ada.models.architectures.set_requires_grad(model, requires_grad=True)[source]

class ada.models.layers.ReverseLayerF[source]

Bases: torch.autograd.function.Function

static backward(ctx, grad_output)[source]

Defines a formula for differentiating the operation.

This function is to be overridden by all subclasses.

It must accept a context ctx as the first argument, followed by as many outputs did forward() return, and it should return as many tensors, as there were inputs to forward(). Each argument is the gradient w.r.t the given output, and each returned value should be the gradient w.r.t. the corresponding input.

The context can be used to retrieve tensors saved during the forward pass. It also has an attribute ctx.needs_input_grad as a tuple of booleans representing whether each input needs gradient. E.g., backward() will have ctx.needs_input_grad[0] = True if the first input to forward() needs gradient computated w.r.t. the output.

static forward(ctx, x, alpha)[source]

Performs the operation.

This function is to be overridden by all subclasses.

It must accept a context ctx as the first argument, followed by any number of arguments (tensors or other types).

The context can be used to store tensors that can be then retrieved during the backward pass.

ada.models.layers.weight_init_glorot_uniform(m)[source]

ada.models.losses.compute_mmd_loss(kernel_values, batch_size)[source]
ada.models.losses.cross_entropy_logits(linear_output, label, weights=None)[source]
ada.models.losses.entropy_logits(linear_output)[source]
ada.models.losses.entropy_logits_loss(linear_output)[source]
ada.models.losses.gaussian_kernel(source, target, kernel_mul=2.0, kernel_num=5, fix_sigma=None)[source]

Code from XLearn: computes the full kernel matrix, which is less than optimal since we don’t use all of it with the linear MMD estimate.

ada.models.losses.gradient_penalty(critic, h_s, h_t)[source]

class ada.models.modules.AlexNetFeature[source]

Bases: torch.nn.modules.module.Module

PyTorch model convnet without the last layer adapted from https://github.com/thuml/Xlearn/blob/master/pytorch/src/network.py

forward(x)[source]

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

output_size()[source]
class ada.models.modules.DataClassifierDigits(input_size=128, n_class=10)[source]

Bases: torch.nn.modules.module.Module

forward(input)[source]

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

n_classes()[source]
class ada.models.modules.DomainClassifierDigits(input_size=128, bigger_discrim=False)[source]

Bases: torch.nn.modules.module.Module

forward(input)[source]

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

class ada.models.modules.FFSoftmaxClassifier(input_dim=15, n_classes=2, name='c', hidden=(), activation_fn=<class 'torch.nn.modules.activation.ReLU'>, **activation_args)[source]

Bases: torch.nn.modules.module.Module

extra_repr()[source]

Set the extra representation of the module

To print customized extra information, you should re-implement this method in your own modules. Both single-line and multi-line strings are acceptable.

forward(input_data)[source]

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

n_classes()[source]
class ada.models.modules.FeatureExtractFF(input_dim, hidden_sizes=(15, ), activation_fn=<class 'torch.nn.modules.activation.ReLU'>, **activation_args)[source]

Bases: torch.nn.modules.module.Module

extra_repr()[source]

Set the extra representation of the module

To print customized extra information, you should re-implement this method in your own modules. Both single-line and multi-line strings are acceptable.

forward(input_data)[source]

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

hidden_layer(index=0)[source]
output_size()[source]
class ada.models.modules.FeatureExtractorDigits(num_channels=3, kernel_size=5)[source]

Bases: torch.nn.modules.module.Module

Feature extractor for MNIST-like data

forward(input)[source]

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

output_size()[source]
class ada.models.modules.ModuleType[source]

Bases: enum.Enum

An enumeration.

Classifier = 'classifier'
Critic = 'critic'
Feature = 'feature'
class ada.models.modules.ResNet101Feature[source]

Bases: torch.nn.modules.module.Module

PyTorch model convnet without the last layer adapted from https://github.com/thuml/Xlearn/blob/master/pytorch/src/network.py

forward(x)[source]

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

output_size()[source]
class ada.models.modules.ResNet152Feature[source]

Bases: torch.nn.modules.module.Module

PyTorch model convnet without the last layer adapted from https://github.com/thuml/Xlearn/blob/master/pytorch/src/network.py

forward(x)[source]

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

output_size()[source]
class ada.models.modules.ResNet18Feature[source]

Bases: torch.nn.modules.module.Module

PyTorch model convnet without the last layer adapted from https://github.com/thuml/Xlearn/blob/master/pytorch/src/network.py

forward(x)[source]

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

output_size()[source]
class ada.models.modules.ResNet34Feature[source]

Bases: torch.nn.modules.module.Module

PyTorch model convnet without the last layer adapted from https://github.com/thuml/Xlearn/blob/master/pytorch/src/network.py

forward(x)[source]

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

output_size()[source]
class ada.models.modules.ResNet50Feature[source]

Bases: torch.nn.modules.module.Module

PyTorch model convnet without the last layer adapted from https://github.com/thuml/Xlearn/blob/master/pytorch/src/network.py

forward(x)[source]

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

output_size()[source]

class ada.models.network_factory.NetworkFactory(network_config)[source]
Bases: object
get_critic_network(input_dim)[source]
get_feature_extractor(input_dim, *args)[source]
get_task_classifier(input_dim, n_classes)[source]
ada.models.network_factory.get_module_fn(module_name)[source]