autoaug.autoaugment_learners.RsLearner

class autoaug.autoaugment_learners.RsLearner(num_sub_policies=5, p_bins=11, m_bins=10, exclude_method=[], batch_size=8, toy_size=1, learning_rate=0.1, max_epochs=inf, early_stop_num=30)

This agent tests out randomly generated augmentation policies.

Tests randomly sampled policies from the search space specified by the AutoAugment paper. Random search has been shown to be a hard baseline to beat for many other hyper-parameter optimization tasks (see References below). Hence, this learner acts as a difficult baseline for other AaLearner’s.

Parameters

• num_sub_policies (int, optional) – number of subpolicies per policy. Defaults to 5.

• p_bins (int, optional) – number of bins we divide the interval [0,1] for probabilities. e.g. (0.0, 0.1, … 1.0) Defaults to 11.

• m_bins (int, optional) – number of bins we divide the magnitude space. Defaults to 10.

• exclude_method (list, optional) – list of names(:type:str) of image operations the user wants to exclude from the search space. Defaults to [].

• batch_size (int, optional) – child_network training parameter. Defaults to 32.

• toy_size (int, optional) – child_network training parameter. ratio of original dataset used in toy dataset. Defaults to 0.1.

• learning_rate (float, optional) – child_network training parameter. Defaults to 1e-2.

• max_epochs (Union[int, float], optional) – child_network training parameter. Defaults to float(‘inf’).

• early_stop_num (int, optional) – child_network training parameter. Defaults to 20.

history

list of policies that has been input into self._test_autoaugment_policy as well as their respective obtained accuracies

Type

list

augmentation_space

list of image functions that the user has chosen to include in the search space.

Type

list

References

Ekin D. Cubuk, et al.

“AutoAugment: Learning Augmentation Policies from Data” arXiv:1805.09501

Bergstra James, Yoshua Bengio

“Random Search for Hyper-Parameter Optimization” https://www.jmlr.org/papers/volume13/bergstra12a/bergstra12a.pdf

learn(train_dataset, test_dataset, child_network_architecture, iterations=15)

Runs the main loop (of finding a good policy for the given child network, training dataset, and test(validation) dataset)

Does the loop which is seen in Figure 1 in the AutoAugment paper which is:

1. <generate a random policy>

2. <see how good that policy is>

3. <save how good the policy is in a list/dictionary and (if applicable,) update the controller (e.g. RL agent)>

If child_network_architecture is a <function>, then we make an instance of it. If this is a <nn.Module>, we make a copy.deepcopy of it. We make a copy of it because we we want to keep an untrained (initialized but not trained) version of the child network architecture, because we need to train it multiple times for each policy. Keeping child_network_architecture as a <function> is potentially better than keeping it as a <nn.Module> because every time we make a new instance, the weights are differently initialized which means that our results will be less biased (https://en.wikipedia.org/wiki/Bias_(statistics)).

Parameters

• train_dataset (torchvision.dataset.vision.VisionDataset) –

• test_dataset (torchvision.dataset.vision.VisionDataset) –

• child_network_architecture (Union[function, nn.Module]) – This can be both, for example, LeNet or LeNet()

• iterations (int) – how many different policies do you want to test

Returns

none

Example code:

Listing 11 This is how a child class might implement this method:

for _ in range(15):
    policy = self._generate_new_policy()

    print(policy)
    reward = self._test_autoaugment_policy(policy,
                            child_network_architecture,
                            train_dataset,
                            test_dataset)