# A FINE-GRAINED ANALYSIS ON DISTRIBUTION SHIFT

**Olivia Wiles** **Sven Gowal** **Florian Stimberg** **Sylvestre-Alvise Rebuffi**

**Ira Ktena** **Krishnamurthy (Dj) Dvijotham** **Taylan Cemgil**

DeepMind, London, UK


_{oawiles,sgowal,stimberg,sylvestre,iraktena,taylancemgil}@deepmind.com_ dvij@google.com

ABSTRACT

Robustness to distribution shifts is critical for deploying machine learning models in the real world. Despite this necessity, there has been little work in defining the underlying mechanisms that cause these shifts and evaluating the robustness of algorithms across multiple, different distribution shifts. To this end, we
introduce a framework that enables fine-grained analysis of various distribution
shifts. We provide a holistic analysis of current state-of-the-art methods by evaluating 19 distinct methods grouped into five categories across both synthetic and
real-world datasets. Overall, we train more than 85K models. Our experimental
framework can be easily extended to include new methods, shifts, and datasets.
We find, unlike previous work (Gulrajani & Lopez-Paz, 2021), that progress has
been made over a standard ERM baseline; in particular, pretraining and augmentations (learned or heuristic) offer large gains in many cases. However, the best
methods are not consistent over different datasets and shifts. Code is available at
github.com/deepmind/distribution_shift_framework.

1 INTRODUCTION

If machine learning models are to be ubiquitous in critical applications such as driverless cars (Janai
et al., 2020), medical imaging (Erickson et al., 2017), and science (Jumper et al., 2021), it is pivotal
to build models that are robust to distribution shifts. Otherwise, models may fail surprisingly in ways
that derail trust in the system. For example, Koh et al. (2020); Perone et al. (2019); AlBadawy et al.
(2018); Heaven (2020); Castro et al. (2020) find that a model trained on one set of hospitals may not
generalise to the imaging conditions of another; Alcorn et al. (2019); Dai & Van Gool (2018) find
that a model for driverless cars may not generalise to new lighting conditions or object poses; and
Buolamwini & Gebru (2018) find that a model may perform worse on subsets of the distribution,
such as different ethnicities, if the training set has an imbalanced distribution. Thus, it is important
to understand when we expect a model to generalise and when we do not. This would allow a
practitioner to have confidence in the system (e.g. if a model is demonstrated to be robust to the
imaging conditions of different hospitals, then it can be deployed in new hospitals with confidence).

While domain generalization is a well studied area, Gulrajani & Lopez-Paz (2021); Schott et al.
(2021) have cast doubt on the efficacy of existing methods, raising the question: has any progress
_been made in domain generalization over a standard expectation risk minimization (ERM) algo-_
_rithm? Despite these discouraging results, there are many examples that machine learning models_
_do generalise across datasets with different distributions. For example, CLIP (Radford et al., 2021),_
with well engineered prompts, generalizes to many standard image datasets. Taori et al. (2020)
found that models trained on one image dataset generalise to another, albeit with some drop in
performance; in particular, higher performing models generalise better. However, there is little understanding and experimentation on when and why models generalise, especially in realistic settings
inspired by real-world applications. This begs the following question:

_Can we define the important distribution shifts to be robust to and then systematically evaluate the_

_robustness of different methods?_

To answer the above question, we present a grounded understanding of robustness to distribution
shifts. We draw inspiration from disentanglement literature (see section 6), which aims to separate
images into an independent set of factors of variation (or attributes). In brief, we assume the data


-----

is composed of some (possibly extremely large) set of attributes. We expect models, having seen
some distribution of values for an attribute, to be able to learn invariance to that attribute and so to
generalise to unseen examples of the attribute and different distributions over that attribute. Using
a simple example to clarify the setup, assume our data has two attributes (shape and color) among
others. Given data with some distribution over the set of possible colors (e.g. red and blue) and the
task of predicting shape (e.g. circle or square), we want our model to generalise to unseen colors
(e.g. green) or a different distribution of colors (e.g. there are very few red circles in the training set,
but the samples at evaluation are uniformly sampled from the set of possible colors and shapes).

Using this framework, we evaluate models across three distribution shifts: spurious correlation, low_data drift, and unseen data shift (illustrated in figure 1) and two additional conditions (label noise_
and dataset size). We choose these settings as they arise in the real world and harm generalization
performance. Moreover, in our framework, these distribution shifts are the fundamental blocks of
building more complex distribution shifts. We additionally evaluate models when there is varying
amounts of label noise (as inspired by noise arising from human raters) and when the total size of
the train set varies (to understand how models perform as the number of training examples changes).
The unique ability of our framework to evaluate fine-grained performance of models across different
distribution shifts and under different conditions is of critical importance to analyze methods under
a variety of real-world settings. This work makes the following contributions:

_• We propose a framework to define when and why we expect methods to generalise. We_

use this framework to define three, real world inspired distribution shifts. We then use this
framework to create a systematic evaluation setup across real and synthetic datasets for different distribution shifts. Our evaluation framework is easily extendable to new distribution
shifts, datasets, or methods to be evaluated.

_• We evaluate and compare 19 different methods (training more than 85K models) in these_

settings. These methods span the following 5 common approaches: architecture choice,
data augmentation, domain generalization, adaptive algorithms, and representation learning. This allows for a direct comparison across different areas in machine learning.



_• We find that simple techniques, such as data augmentation and pretraining are often effec-_

tive and that domain generalization algorithms do work for certain datasets and distribution
shifts. However, there is no easy way to select the best approach a-priori and results are
inconsistent over different datasets and attributes, demonstrating there is still much work to
be done to improve robustness in real-world settings.

2 FRAMEWORK TO EVALUATE GENERALIZATION

In this section we introduce our robustness framework for characterizing distribution shifts in a
principled manner. We then relate three common, real world inspired distribution shifts.

2.1 LATENT FACTORISATION

We assume a joint distribution p of inputs x and corresponding attributes y[1], y[2], . . ., y[K] (denoted
as y[1:][K]) with y[k] _2 A[k]_ where A[k] is a finite set. One of these K attributes is a label of interest,
denoted as y[l] (in a mammogram, the label could be cancer/benign and a nuisance attribute y[i] with
_i 6= l could be the identity of the hospital where the mammogram was taken). Our aim is to build a_
classifier f that minimizes the risk R. However, in real-world applications, we only have access to
a finite set of inputs and attributes of size n. Hence, we minimize the empirical risk _R[ˆ] instead:_


_Rˆ(f_ ; p) = [1]


_R(f_ ) = E(x,yl) _p_
_⇠_


_L(y[l], f_ (x))


_L(yi[l][, f]_ [(][x][i][))][.]


(yi[l][,][x][i][)][⇠][p][}][n]i=1
_{_

where L is a suitable loss function. Here, all nuisance attributes y[k] with k 6= l are ignored and we
work with samples obtained from the marginal p(y[l], x). In practice, however, due to selection bias
or other confounding factors in data collection, we are only able to train and test our models on data
collected from two related but distinct distributions: ptrain, ptest. For example, ptrain and ptest may
be concentrated on different subsets of hospitals and this discrepancy may result in a distribution
shift; for example, hospitals may use different equipment, leading to different staining on their cell


-----

(a) ptrain: SC. (b) ptrain: LDD. (c) ptrain: UDS. (d) ptest: y[l], y[a] **are IID.**

Figure 1: Visualization of the joint distribution for the different shifts we consider on the DSPRITES
example. The lighter the color, the more likely the given sample. figure 1a-1c visualise different
shifts over ptrain(y[l], y[a]) discussed in 2.2: spurious correlation (SC), low-data drift (LDD), and un_seen data shift (UDS). figure 1d visualises the test set, where the attributes are uniformly distributed._

images. While we train f on data from ptrain by minimizing _R[ˆ](f_ ; ptrain), we aim to learn a model

that generalises well to data from ptest; that is, it should achieve a small _R[ˆ](f_ ; ptest).

While generalization in the above sense is desirable for machine learning models, it is not clear why
a model f trained on data from ptrain should generalise to ptest. It is worth noting that while ptrain
and ptest can be different, they are both related to the true distribution p. We take inspiration from
disentanglement literature to express this relationship. In particular, that we can view data as being
decomposed into an underlying set of factors of variations. We formalise various distribution shifts
using a latent variable model for the true data generation process:

_z ⇠_ _p(z)_ _y[i]_ _⇠_ _p(y[i]|z)_ _i = 1 . . . K_ **_x ⇠_** _p(x|z)_ (1)

where z denotes latent factors. By a simple refactorization, we can write


_p(y[1:][K], x) = p(y[1:][K])_


_p(x|z)p(z|y[1:][K])dz = p(y[1:][K])p(x|y[1:][K])._


Thus, the true distribution can be expressed as the product of the marginal distribution of the attributes with a conditional generative model. We assume that distribution shifts arise when a new
marginal distribution for the attributes is chosen, such as p(y[1:][K]) = ptrain(y[1:][K]) = ptest(y[1:][K]),
_6_ _6_
but otherwise the conditional generative model is shared across all distributions, i.e., we have
_ptest(y[1:][K], x) = ptest(y[1:][K])_ _p(x_ _z)p(z_ _y[1:][K])dz, and similarly for ptrain._

_|_ _|_

To provide more context, as a running example, we use the colorR DSPRITES dataset (Matthey et al.,
2017); where in our notation y[1] defines the color with A[1] = {red, green, blue}, and y[2] defines the
shape with A[2] = {ellipse, heart, square}. We can imagine that a data collector (intentionally or implicitly) selects some marginal distribution over attributes ptrain(y[1:][K]) when training; for example
they select mostly blue ellipses and red hearts. This induces a new joint distribution over latent factors and attributes: ptrain(z, y[1:][K]) = p(z _y[1:][K])ptrain(y[1:][K]). Consequently, during training, we get_
_|_
images with a different joint distribution ptrain(x, y[1:][K]) = _p(x_ _z)ptrain(z, y[1:][K]). This similarly_

_|_

applies when collecting data for the test distribution. We focus on common cases of distribution

R

shifts visualized in figure 1; we discuss these in more detail in section 2.2.

The goal of enforcing robustness to distribution shifts is to maintain performance when the data
generating distribution ptrain changes. In other words, we would like to minimize risk on p, ptest
given only access to ptrain. We can achieve robustness in the following ways:

_• Weighted resampling._ We can resample the training set using importance weights

_W_ (y[1:][K]) = p(y[1:][K])/ptrain(y[1:][K]). This means that given the attributes, the i-th data point
(yi[1:][K], xi) in the training set is used with probability W (yi[1:][K])/ _i[0]=1_ _[W]_ [(][y]i[1:][0] _[K]) rather_

than 1/n. We refer to this empirical distribution as preweight. This procedure requires access
to the true distribution of attributes p(y[1:][K]), so to avoid bias and improve fairness, it is

[P][n]

often assumed that all combinations of attributes happen uniformly at random.

_• Data Augmentation: Alternatively, we can learn a generative model ˆp(x|y[1:][K]) from_

the training data that aims to approximate _p(x|z)p(z|y[1:][K])dz, as the true conditional_

R


-----

DSPRITES. MPI3D. SHAPES3D.


SMALLNORB. CAMELYON17. IWILDCAM.

Figure 2: Dataset samples. Each row fixes an attribute (e.g. color for DSPRITES, MPI3D,

SHAPES3D; azimuth for SMALLNORB; hospital for CAMELYON17; and location for IWILDCAM).


generator is by our assumption the same over all (e.g. train and test) distributions. If
such a conditional generative model can be learned, we can sample new synthetic data
at training time (e.g. according to the true distribution p(y[1:][K])) to correct for the distribution shift. More precisely, we can generate data from the augmented distribution

_paug = (1 −_ _↵)preweight + ↵pˆ(x|y[1:][K])p(y[1:][K]) and train a supervised classifier on this aug-_

mented dataset. Here, ↵ _2 [0, 1] is the percentage of synthetic data used for training._

_• Representation Learning: An alternative factorization of a data generating distribution_

(e.g. train) is ptrain(y[1:][K], x) = _p(z|x)ptrain(y[1:][K]|z)dz. We can learn an unsupervised_

fier to learn a task specific head that approximatesrepresentation that approximatesR p(z|x) using the training data only, and attach a classi- ptrain(y[l]|z). Again, by our assumption
_p(z|x) / p(x|z)p(z). Given a good guess of the true prior, the learned representation_
would not be impacted by the specific attribute distribution and so generalise to ptest, p.

2.2 DISTRIBUTION SHIFTS

While distribution shifts can happen in a continuum, we consider three types of shifts, inspired by
real-world challenges. We discuss these shifts and two additional, real-world inspired conditions.

**Test distribution ptest.** We assume that the attributes are distributed uniformly: ptest(y[1:][K]) =

1/ _i_

_[|][A][i][|][. This is desirable, as all attributes are represented and a-priori independent.]_

**Shift 1: Spurious correlation – Attributes are correlated under[Q]** _ptrain but not ptest._ Spurious

correlation arises in the wild for a number of reasons including capture bias, environmental factors,
and geographical bias (Beery et al., 2018; Torralba & Efros, 2011). These spurious correlations
lead to surprising results and poor generalization. Therefore, it is important to be able to build
models that are robust to such challenges. In our framework, spurious correlation arises when two
attributes y[a], y[b] are correlated at training time, but this is not true of ptest, for which attributes are
independent: ptrain(y[a] _y[1]_ _. . . y[b]_ _. . . y[K]) > ptrain(y[a]_ _y[1]_ _. . . y[b][−][1], y[b][+1]_ _. . . y[K]). This is especially_
_|_ _|_
problematic when one attribute y[b] is y[l], the label. Using the running DSPRITES example, shape and
color may be correlated and the model may find it easier to predict color. If color is the label, the
model will generalise well. However, if the aim is to predict shape, the model’s reliance on color
will lead to poor generalization.

**Shift 2: Low-data drift – Attribute values are unevenly distributed under ptrain but not under**
_ptest._ Low-data drift arises in the wild (e.g. in (Buolamwini & Gebru, 2018) for different ethnici
ties) when data has not been collected uniformly across different attributes. When deploying models
in the wild, it is important to be able to reason and have confidence that the final predictions will


-----

Figure 3: Spurious Correlation. We use all

correlated samples and vary the number of samples N from the true, uncorrelated distribution.
We plot the percentage change over the baseline
ResNet, averaged over all seeds and datasets.
Blue is better, red worse. CYCLEGAN performs
consistently best while ImageNet augmentation
and pretraining on ImageNet also consistently
boosts performance.


Figure 4: Low-data drift. We use all sam
ples from the high data regions and vary the
number of samples N from the low-data region.
We plot the percentage change over the baseline
ResNet, averaged over all seeds and datasets.
Blue is better, red worse. Pretraining on ImageNet performs consistently best, while CYCLEGAN, most domain generalization methods and
ImageNet augmentation also provide some boost
in performance.


be consistent and fair across different attributes. In the framework above, low-data shifts arise when
certain values in the set A[a] of an attribute y[a] are sampled with a much smaller probability than in
_ptest: ptrain(y[a]_ = v) _ptest(y[a]_ = v). Using the DSPRITES example, only a handful of red shapes
_⌧_
may be seen at training time, yet in ptest all colors are sampled with equal probability.

**Shift 3: Unseen data shift – Some attribute values are unseen under ptrain but are under ptest.**
This is a special case of shift 2: low-data drift which we make explicit due to its important real world
applications. Unseen data shift arises when a model, trained in one setting is expected to work in
another, disjoint setting. For example: a model trained to classify animals on images at certain times
of day should generalise to other times of day. In our framework, unseen data shift arises when
some values in the set A[a] of an attribute y[a] are unseen in ptrain but are in ptest:

_ptrain(y[a]_ = v) = 0 _ptest(y[a]_ = v) > 0 _v_ _ptrain(y[a]_ = v) _> 1_ (2)
_|{_ _|_ _}|_

This is a stronger constraint than in standard out-of-distribution generalization (see section 6), as
multiple values for A[a] must be seen under ptrain, which allows the model to learn invariance to y[a].
In the DSPRITES example, the color red may be unseen at train time but all colors are in ptest.

**Discussion.** We choose these sets of shifts as they are the building blocks of more complex dis
tribution shifts. Consider the simplest case of two attributes: the label and a nuisance attribute. If
we consider the marginal distribution of the label, it decomposes into two terms: the conditional
probability and the probability of a given attribute value: p(y[l]) = _y[a][ p][(][y][l][|][y][a][)][p][(][y][a][)][. The three]_

shifts we consider control these terms independently: unseen data shift and low-data drift control
_p(y[a]) whereas spurious correlation controls p(y[l]_ _y[a]). The composition of these terms describes any_

[P]

_|_
distribution shift for these two variables.

2.3 CONDITIONS


**Label noise. We investigate the change in performance due to noisy information. This can arise**
when there are disagreements and errors among the labellers (e.g. in medical imaging (Castro et al.,
2020)). We model this as an observed attribute (e.g. the label) being corrupted by noise. ˆy[i] _⇠_ _c(y[i]),_

where y[i] _2 A[i]_ is the true label, ˆy[i] _2 A[i]_ the corrupted, observed one, and c the corrupting function.

**Dataset size. We investigate how performance changes with the size of the training dataset. This**
setting arises when it is unrealistic or expensive to collect additional data (e.g. in medical imaging or
in camera trap imagery). Therefore, it is important to understand how performance degrades given
fewer total samples. We do this by limiting the total number of samples from ptrain.


-----

3 MODELS EVALUATED

We evaluate 19 algorithms to cover a broad range of approaches that can be used to improve model
robustness to distribution shifts and demonstrate how they relate to the three ways to achieve robustness, outlined in section 2. We believe this is the first paper to comprehensively evaluate a large set
of different approaches in a variety of settings. These algorithms cover the following areas: architecture choice, data augmentation, domain adaptation, adaptive approaches and representation learning.
Further discussion on how these models relate to our robustness framework is in appendix E.

**Architecture choice. We evaluate the following standard vision models: ResNet18, ResNet50,**
ResNet101 (He et al., 2016), ViT (Dosovitskiy et al., 2021), and an MLP (Vapnik, 1992). We
use weighted resampling preweight to oversample from the parts of the distribution that have a lower
probability of being sampled from under ptrain. Performance depends on how robust the learned
representation is to distribution shift.

**Heuristic data augmentation. These approaches attempt to approximate the true underlying gen-**
erative model p(x|y[1:][K]) in order to improve robustness. We analyze the following augmentation
methods: standard ImageNet augmentation (He et al., 2016), AugMix without JSD (Hendrycks
et al., 2020), RandAugment (Cubuk et al., 2020), and AutoAugment (Cubuk et al., 2019). Performance depends on how well the heuristic augmentations approximate the true generative model.

**Learned data augmentation. These approaches approximate the true underlying generative model**
_p(x|y[1:][K]) by learning augmentations conditioned on the nuisance attribute. The learned augmenta-_
tions can be used to transform any image x to have a new attribute, while keeping the other attributes
fixed. We follow Goel et al. (2020), who use CYCLEGAN (Zhu et al., 2017), but we do not use their
SGDRO objective in order to evaluate the performance of learned data augmentation alone. Performance depends on how well the learned augmentations approximate the true generative model.

**Domain generalization. These approaches aim to recover a representation z that is independent of**
the attribute: p(y[a], z) = p(y[a])p(z) to allow generalization over that attribute. We evaluate IRM
(Arjovsky et al., 2019), DeepCORAL (Sun & Saenko, 2016), domain MixUp (Gulrajani & LopezPaz, 2021), DANN (Ganin et al., 2016), and SagNet (Nam et al., 2021). Performance depends on
the invariance of the learned representation z.

**Adaptive approaches. These works modify p reweight dynamically. We evaluate JTT (Liu et al.,**
2021) and BN-Adapt (Schneider et al., 2020). These methods do not give performance guarantees.

**Representation learning. These works aim to learn a robust representation of z that describes the**
true prior. We evaluate using a β-VAE (Higgins et al., 2017a) and pretraining on ImageNet (Deng
et al., 2009). Performance depends on the quality of the learned representation for the specific task.

4 EXPERIMENTS

We first introduce the datasets and experimental setup. We evaluate the 19 different methods across
these six datasets, three distribution shifts, varying label noise, and dataset size. We plot aggregate
results in figures 3-7 and complete results in the appendix in figures 10-12. We discuss the results
by distilling them into seven concrete takeaways in section 4.1 and four practical tips in section 4.2.

**Datasets.** We evaluate these approaches on six vision, classification datasets – DSPRITES (Matthey

et al., 2017), MPI3D (Gondal et al., 2019), SMALLNORB (LeCun et al., 2004), SHAPES3D (Burgess
& Kim, 2018), CAMELYON17 (Koh et al., 2020; Bandi et al., 2018), and IWILDCAM (Koh et al.,
2020; Beery et al., 2018). These datasets consist of multiple (potentially an arbitrarily large number)
attributes. We select two attributes y[l], y[a] for each dataset and make one y[l] the label. We then use
these two attributes to build the three shifts. Visualizations of samples from the datasets are given
in figure 2 and further description in appendix D.1. We discuss precisely how we set up the shifts,
choose the attributes, and additional conditions for these datasets in appendix D.2.


**Model selection.** When investigating heuristic data augmentation, domain generalization, learned

augmentation, adaptive approaches, and representation learning, we use a ResNet18 for the simpler,
synthetic datasets (DSPRITES, MPI3D, SHAPES3D, and SMALLNORB) but a ResNet50 for the


-----

Figure 5: **Unseen data shift.** We rank the

methods (where best is 1, worst 19) for each
dataset and seed and plot the rankings, with the
overall median rank as the black bar. Pretraining on ImageNet and ImageNet augmentation
perform consistently best. DANN, CycleGAN

and other heuristic augmentations perform consistently well.

more complex, real world ones (CAMELYON17 and IWILDCAM). To perform model selection, we
choose the best model according to the validation set which matches the distribution of the test set.
In the unseen data shift setting for the CAMELYON17 and IWILDCAM, we use the given out-ofdistribution validation set, which is a different, distinct set in that is independent of train, test.
_D_ _D_ _D_
(We consider using the in-distribution validation set in appendix B.4.)

**Hyperparameter choices.** We perform a sweep over the hyperparameters (the precise sweeps are

given in appendix F.8). We run each set of hyperparameters for five seeds for each setting. To
choose the best model for each seed, we perform model selection over all hyperparameters using the
top-1 accuracy on the validation set. In the low-data and spurious correlation settings, we choose a
different set of samples from the low-data region with each seed. We report the mean and standard
deviation over the five seeds.

4.1 TAKEAWAYS

**Takeaway 1: While we can improve over ERM, no one method always performs best. The**
relative performance between methods varies across datasets and shifts. Under spurious correlation
(figure 3), CYCLEGAN consistently performs best but in figure 4, under low-data drift, pretraining
consistently performs best. Under unseen data shift (figure 5), pretraining is again one of the best
performing models. However, if we drill down on the results in figure 10 (appendix B.1), we can
see pretraining performs best on the synthetic datasets, but not on CAMELYON17 (where using
augmentation or DANN is best) or IWILDCAM (where using ViT or an MLP is best).

**Takeaway 2: Pretraining is a powerful tool across different shifts and datasets. While pre-**
training is not always helpful (e.g. in appendix B.1 on CAMELYON17 in figures 10-11, IWILDCAM
in figures 10-11), it often provides a strong boost in performance. This is presumably because the
representation z learned during pretraining is helpful for the downstream task. For example, the
representation may have been trained to be invariant to certain useful properties (e.g. scale, shift,
and color). If these properties are useful on the downstream tasks, then the learned representation
should improve generalization.

**Takeaway 3: Heuristic augmentation improves generalization if the augmentation describes**
**an attribute. In all settings (figures 3-5), ImageNet augmentation generally improves performance.**
However, RandAugment, AugMix, and AutoAugment have more variable performance (as further
shown in figures 10-12). These methods are compositions of different augmentations. We investigate
the impact of each augmentation in RandAugment in appendix B.2 and find variable performance.
Augmentations that approximate the true underlying generative model p(x|y[1:][K]) lead to the best
results; otherwise, the model may waste capacity. For example, on CAMELYON17 (which consists
of cell images), color jitter harms performance but on SHAPES3D and MPI3D it is essential.

**Takeaway 4: Learned data augmentation is effective across different conditions and distribu-**
**tion shifts. This approach is highly effective in the spurious correlation setting (figure 3). It can**
also help in the low-data and unseen data shift settings (figure 4,5) (though the gains for these two
shifts are not as large as for pretraining). The effectiveness of this approach can be explained by the
fact that if the augmentations are learned perfectly, then augmented samples by design are from the
true underlying generative model and can cover missing parts of the distribution.

**Takeaway 5: Domain generalization algorithms offer limited performance improvement. In**
some cases these methods (in particular DANN) do improve performance, most notably in the low_data drift and unseen data shift settings (figures 4-5). However, this depends on the dataset (see_
figures 10-12) and performance is rarely much better than using heuristic augmentation.


-----

Figure 6: Condition 1: Noisy labels. We vary
the amount of noise p in the labels. We plot the
percentage change over the baseline ResNet, averaged over all seeds and datasets.


Figure 7: Condition 2: Fixed data. We vary the
total size of the dataset T . We plot the percentage
change over the baseline ResNet, averaged over
all seeds and datasets.


**Takeaway 6: The best algorithms may differ under the precise conditions. When labels have**
varying noise in figure 6, relative performance is reasonably consistent. When the dataset size
decreases in figure 7, heuristic augmentation methods perform poorly. However, using pretraining
and learned augmentation is consistently robust.

**Takeaway 7: The precise attributes we consider directly impacts the results. For example, on**
DSPRITES, if we make color y[l] and shape y[a], we find that all methods generalise perfectly in the
_unseen data shift setting (as demonstrated in appendix B.3) unlike when shape is y[l]_ (figure 10).

4.2 PRACTICAL TIPS

While there is no free lunch in terms of the method to choose, we recommend the following tips.

**Tip 1: If heuristic augmentations approximate part of the true underlying generative model,**
**use them. Under this constraint, heuristic augmentations can significantly improve performance;**
this should be a first point of call. How to heuristically choose these augmentations without exhaustively trying all possible combinations is an open research question.

**Tip 2: If heuristic augmentations do not help, learn the augmentation. If the true underlying**
generative model cannot be readily approximated with heuristic techniques, but some subset of the
generative model can be learned by conditioning on known attributes, this is a promising way to
further improve performance. How to learn the underlying generative model directly from data and
use this for augmentation is a promising area to explore more thoroughly.

**Tip 3: Use pretraining. In general, pretraining was found to be a useful way to learn a robust rep-**
resentation. While this was not true for all datasets (e.g. CAMELYON17, IWILDCAM), performance
could be dramatically improved by pretraining (DSPRITES, MPI3D, SMALLNORB, SHAPES3D).
An area to be investigated is the utility of self-supervised pre-training.

**Tip 4: More complex approaches lead to limited improvements. Domain generalization, adap-**
tive approaches and disentangling lead to limited improvements, if any, across the different datasets
and shifts. Of these approaches, DANN performs generally best. How to make these approaches
generically useful for robustness is still an open research question.

5 DISCUSSION

Our experiments demonstrate that no one method performs best over all shifts and that performance
is dependent on the precise attribute being considered. This leads to the following considerations.

**There is no way to decide a-priori on the best method given only the dataset. It would be**
helpful for practitioners to be able to select the best approaches without requiring comprehensive
evaluations and comparisons. Moreover, it is unclear how to pinpoint the precise distribution shift
(and thereby methods to explore) in a given application. This should be an important future area of
investigation.

**We should focus on the cases where we have knowledge about the distribution shift. We found**
that the ability of a given algorithm to generalize depends heavily on the attribute and dataset being


-----

considered. Instead of trying to make one algorithm for any possible shift, it makes sense to have
adaptable algorithms which can use auxiliary information if given. Moreover, algorithms should be
evaluated in the context for which we will use them.

**It is pivotal to evaluate methods in a variety of conditions.** Performance varies due to the

number of examples, amount of noise, and size of the dataset. Thus it is important to perform
comprehensive evaluations when comparing different methods, as in our framework. This gives
others a more realistic view of different models’ relative performance in practice.

6 RELATED WORK

We briefly summarize benchmarks on distribution shift, leaving a complete review to appendix C.

**Benchmarking robustness to out of distribution (OOD) generalization.** While a multitude of

methods exist that report improved OOD generalization, Gulrajani & Lopez-Paz (2021) found that
in actuality no evaluated method performed significantly better than a strong ERM baseline on a
variety of datasets. However, Hendrycks et al. (2021) found that, when we focus on better augmentation, larger models and pretraining, we can get a sizeable boost in performance. This can be
seen on the Koh et al. (2020) benchmark (the largest boosts come from larger models and better
augmentation). Our work is complementary to these methods, as we look at a range of approaches
(pretraining, heuristic augmentation, learned augmentation, domain generalisation, adaptive, disentangled representations) on a range of both synthetic and real-world datasets. Moreover, we allow
for a fine-grained analysis of methods over different distribution shifts.

**Benchmarking spurious correlation and low-data drift.** Studies on fairness and bias (surveyed

by Mehrabi et al. (2021)) have demonstrated the pernicious impact of low-data in face recognition
(Buolamwini & Gebru, 2018), medical imaging (Castro et al., 2020), and conservation (Beery et al.,
2018) and spurious correlation in classification (Geirhos et al., 2019) and conservation (Beery et al.,
2020). Arjovsky et al. (2019) hypothesized that spurious correlation may be the underlying reason
for poor generalization of models to unseen data. To our knowledge, there has been no large scale
work focused on understanding the benefits of different methods across these distribution shifts
systematically across multiple datasets and with fine-grained control on the amount of shift. Here
we introduce a framework for creating these shifts in a controllable way to allow such challenges to
be investigated robustly.

**Benchmarking disentangled representations.** A related area, disentangled representation learn
ing, aims to learn a representation where the factors of variation in the data are separated. If this
could be achieved, then models should be able to generalise effortlessly to unseen data as investigated in multiple settings such as reinforcement learning (Higgins et al., 2017b). Despite many
years of work in disentangled representations (Higgins et al., 2017a; Burgess et al., 2017; Kim &
Mnih, 2018; Chen et al., 2018), a benchmark study by Locatello et al. (2019) found that, without
supervision or implicit model or data assumptions, one cannot reliably perform disentanglement;
however, weak supervision appears sufficient to do so (Locatello et al., 2020). Dittadi et al. (2021);
Schott et al. (2021); Montero et al. (2020) further investigated whether representations (disentangled or not) can interpolate, extrapolate, or compose properties; they found that when considering
complex combinations of properties and multiple datasets, representations do not do so reliably.

7 CONCLUSIONS

This work has put forward a general, comprehensive framework to reason about distribution shifts.
We analyzed 19 different methods, spanning a range of techniques, over three distribution shifts –
_spurious correlation, low-data drift, and unseen data shift, and two additional conditions – label_
_noise and dataset size. We found that while results are not consistent across datasets and methods, a_
number of methods do better than an ERM baseline in some settings. We then put forward a number
of practical tips, promising directions, and open research questions. We hope that our framework and
comprehensive benchmark spurs research on in this area and provides a useful tool for practitioners
to evaluate which methods work best under which conditions and shifts.


-----

ACKNOWLEDGMENTS

The authors thank Irina Higgins and Timothy Mann for feedback and discussions while developing
their work. They also thank Irina, Rosemary Ke, and Dilan Gorur for reviewing earlier drafts.

REFERENCES

Ehab A AlBadawy, Ashirbani Saha, and Maciej A Mazurowski. Deep learning for segmentation of

brain tumors: Impact of cross-institutional training and testing. Medical physics, 2018.

Michael A Alcorn, Qi Li, Zhitao Gong, Chengfei Wang, Long Mai, Wei-Shinn Ku, and Anh Nguyen.

Strike (with) a pose: Neural networks are easily fooled by strange poses of familiar objects. In
_Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2019._

Martin Arjovsky, L´eon Bottou, Ishaan Gulrajani, and David Lopez-Paz. Invariant risk minimization.

_arXiv preprint arXiv:1907.02893, 2019._

Peter Bandi, Oscar Geessink, Quirine Manson, Marcory Van Dijk, Maschenka Balkenhol, Meyke

Hermsen, Babak Ehteshami Bejnordi, Byungjae Lee, Kyunghyun Paeng, Aoxiao Zhong, et al.
From detection of individual metastases to classification of lymph node status at the patient level:
the CAMELYON17 challenge. IEEE Transactions on Medical Imaging, 2018.

Sara Beery, Grant Van Horn, and Pietro Perona. Recognition in terra incognita. In Proceedings of

_the European Conference on Computer Vision, 2018._

Sara Beery, Yang Liu, Dan Morris, Jim Piavis, Ashish Kapoor, Neel Joshi, Markus Meister, and

Pietro Perona. Synthetic examples improve generalization for rare classes. In Proceedings of the
_IEEE Workshop on Applications of Computer Vision, 2020._

Joy Buolamwini and Timnit Gebru. Gender shades: Intersectional accuracy disparities in commer
cial gender classification. In Conference on fairness, accountability and transparency, 2018.

Chris Burgess and Hyunjik Kim. 3D shapes dataset. https://github.com/deepmind/3dshapes-dataset/,

2018.

Christopher P Burgess, Irina Higgins, Arka Pal, Loic Matthey, Nick Watters, Guillaume Desjardins,

and Alexander Lerchner. Understanding disentangling in β-VAE. In Workshop on Learning

_Disentangled Representations at the 31st Conference on Neural Information Processing Systems,_
2017.

Fabio M Carlucci, Paolo Russo, Tatiana Tommasi, and Barbara Caputo. Hallucinating agnostic

images to generalize across domains. In Proceedings of the IEEE Conference on Computer Vision
_and Pattern Recognition Workshops, 2019._

Daniel C Castro, Ian Walker, and Ben Glocker. Causality matters in medical imaging. Nature

_Communications, 2020._

Ricky TQ Chen, Xuechen Li, Roger Grosse, and David Duvenaud. Isolating sources of disentangle
ment in variational autoencoders. In Advances in Neural Information Processing Systems, 2018.

Xi Chen, Yan Duan, Rein Houthooft, John Schulman, Ilya Sutskever, and Pieter Abbeel. InfoGAN:

Interpretable representation learning by information maximizing generative adversarial nets. In
_Advances in Neural Information Processing Systems, 2016._

Yunjey Choi, Minje Choi, Munyoung Kim, Jung-Woo Ha, Sunghun Kim, and Jaegul Choo. Star
Gan: Unified generative adversarial networks for multi-domain image-to-image translation. In
_Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2018._

Ekin D Cubuk, Barret Zoph, Dandelion Mane, Vijay Vasudevan, and Quoc V Le. AutoAugment:

Learning augmentation strategies from data. In Proceedings of the IEEE Conference on Computer
_Vision and Pattern Recognition, 2019._


-----

Ekin D Cubuk, Barret Zoph, Jonathon Shlens, and Quoc V Le. RandAugment: Practical auto
mated data augmentation with a reduced search space. In Proceedings of the IEEE Conference on
_Computer Vision and Pattern Recognition Workshops, 2020._

Dengxin Dai and Luc Van Gool. Dark model adaptation: Semantic image segmentation from day
time to nighttime. In International Conference on Intelligent Transportation Systems, 2018.

Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale

hierarchical image database. In Proceedings of the IEEE Conference on Computer Vision and
_Pattern Recognition, 2009._

Andrea Dittadi, Frederik Tr¨auble, Francesco Locatello, Manuel W¨uthrich, Vaibhav Agrawal, Ole

Winther, Stefan Bauer, and Bernhard Sch¨olkopf. On the transfer of disentangled representations

in realistic settings. In Proceedings of the International Conference on Learning Representations,
2021.

Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas

Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, et al. An
image is worth 16x16 words: Transformers for image recognition at scale. In Proceedings of the
_International Conference on Learning Representations, 2021._

Bradley J Erickson, Panagiotis Korfiatis, Zeynettin Akkus, and Timothy L Kline. Machine learning

for medical imaging. Radiographics, 2017.

Chelsea Finn, Pieter Abbeel, and Sergey Levine. Model-agnostic meta-learning for fast adaptation

of deep networks. In Proceedings of the International Conference on Machine Learning, 2017.

Yaroslav Ganin, Evgeniya Ustinova, Hana Ajakan, Pascal Germain, Hugo Larochelle, Franc¸ois

Laviolette, Mario Marchand, and Victor Lempitsky. Domain-adversarial training of neural networks. Journal of Machine Learning Research, 17(1):2096–2030, 2016.

Robert Geirhos, Patricia Rubisch, Claudio Michaelis, Matthias Bethge, Felix A Wichmann, and

Wieland Brendel. Imagenet-trained cnns are biased towards texture; increasing shape bias improves accuracy and robustness. In Proceedings of the International Conference on Learning
_Representations, 2019._

Karan Goel, Albert Gu, Yixuan Li, and Christopher R´e. Model patching: Closing the subgroup

performance gap with data augmentation. arXiv preprint arXiv:2008.06775, 2020.

Muhammad Waleed Gondal, Manuel W¨uthrich, Dorde Miladinovi´c, Francesco Locatello, Martin

Breidt, Valentin Volchkov, Joel Akpo, Olivier Bachem, Bernhard Sch¨olkopf, and Stefan Bauer.

On the transfer of inductive bias from simulation to the real world: a new disentanglement dataset.
_arXiv preprint arXiv:1906.03292, 2019._

Ian J Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair,

Aaron C Courville, and Yoshua Bengio. Generative adversarial nets. In Advances in Neural
_Information Processing Systems, 2014._

Sven Gowal, Chongli Qin, Po-Sen Huang, Taylan Cemgil, Krishnamurthy Dvijotham, Timothy

Mann, and Pushmeet Kohli. Achieving robustness in the wild via adversarial mixing with disentangled representations. In Proceedings of the IEEE Conference on Computer Vision and Pattern
_Recognition, 2020._

Keren Gu, Xander Masotto, Vandana Bachani, Balaji Lakshminarayanan, Jack Nikodem, and Dong

Yin. An instance-dependent simulation framework for learning with label noise. arXiv preprint
_arXiv:2107.11413, 2021._

Ishaan Gulrajani and David Lopez-Paz. In search of lost domain generalization. In Proceedings of

_the International Conference on Learning Representations, 2021._

Bo Han, Quanming Yao, Xingrui Yu, Gang Niu, Miao Xu, Weihua Hu, Ivor Tsang, and Masashi

Sugiyama. Co-teaching: Robust training of deep neural networks with extremely noisy labels. In
_Advances in Neural Information Processing Systems, 2018._


-----

Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recog
nition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition,
2016.

Will Douglas Heaven. Google’s medical ai was super accurate in a lab. real life was a different

story., 2020.

Dan Hendrycks and Thomas Dietterich. Benchmarking neural network robustness to common cor
ruptions and perturbations. In Proceedings of the International Conference on Learning Repre_sentations, 2019._

Dan Hendrycks, Mantas Mazeika, Duncan Wilson, and Kevin Gimpel. Using trusted data to train

deep networks on labels corrupted by severe noise. In Advances in Neural Information Processing
_Systems, 2018._

Dan Hendrycks, Kimin Lee, and Mantas Mazeika. Using pre-training can improve model robustness

and uncertainty. In Proceedings of the International Conference on Machine Learning, 2019.

Dan Hendrycks, Norman Mu, Ekin D Cubuk, Barret Zoph, Justin Gilmer, and Balaji Lakshmi
narayanan. AugMix: A simple data processing method to improve robustness and uncertainty. In
_Advances in Neural Information Processing Systems, 2020._

Dan Hendrycks, Steven Basart, Norman Mu, Saurav Kadavath, Frank Wang, Evan Dorundo, Rahul

Desai, Tyler Zhu, Samyak Parajuli, Mike Guo, et al. The many faces of robustness: A critical
analysis of out-of-distribution generalization. Proceedings of the International Conference on
_Computer Vision, 2021._

Irina Higgins, Loic Matthey, Arka Pal, Christopher Burgess, Xavier Glorot, Matthew Botvinick,

Shakir Mohamed, and Alexander Lerchner. β-VAE: Learning basic visual concepts with a constrained variational framework. In Proceedings of the International Conference on Learning

_Representations, 2017a._

Irina Higgins, Arka Pal, Andrei Rusu, Loic Matthey, Christopher Burgess, Alexander Pritzel,

Matthew Botvinick, Charles Blundell, and Alexander Lerchner. Darla: Improving zero-shot

transfer in reinforcement learning. In Proceedings of the International Conference on Machine
_Learning, 2017b._

Joel Janai, Fatma G¨uney, Aseem Behl, Andreas Geiger, et al. Computer vision for autonomous

vehicles: Problems, datasets and state of the art. Foundations and Trends R⃝ _in Computer Graphics_

_and Vision, 2020._

Fredrik D Johansson, David Sontag, and Rajesh Ranganath. Support and invertibility in domain
invariant representations. In The International Conference on Artificial Intelligence and Statistics.
PMLR, 2019.

John Jumper, Richard Evans, Alexander Pritzel, Tim Green, Michael Figurnov, Olaf Ronneberger,

Kathryn Tunyasuvunakool, Russ Bates, Augustin Z´[ˇ] ıdek, Anna Potapenko, Alex Bridgland,

Clemens Meyer, Simon A. A. Kohl, Andrew J. Ballard, Andrew Cowie, Bernardino RomeraParedes, Stanislav Nikolov, Rishub Jain, Jonas Adler, Trevor Back, Stig Petersen, David Reiman,
Ellen Clancy, Michal Zielinski, Martin Steinegger, Michalina Pacholska, Tamas Berghammer, Sebastian Bodenstein, David Silver, Oriol Vinyals, Andrew W. Senior, Koray Kavukcuoglu, Pushmeet Kohli, and Demis Hassabis. Highly accurate protein structure prediction with AlphaFold.
_Nature, 2021._

Tero Karras, Samuli Laine, and Timo Aila. A style-based generator architecture for generative

adversarial networks. In Proceedings of the IEEE Conference on Computer Vision and Pattern
_Recognition, 2019._

Ashish Khetan, Zachary C Lipton, and Anima Anandkumar. Learning from noisy singly-labeled

data. In Proceedings of the International Conference on Learning Representations, 2018.

Hyunjik Kim and Andriy Mnih. Disentangling by factorising. In Proceedings of the International

_Conference on Machine Learning, 2018._


-----

Diederik P Kingma and Max Welling. Auto-encoding variational bayes. _arXiv preprint_

_arXiv:1312.6114, 2013._

Pang Wei Koh, Shiori Sagawa, Henrik Marklund, Sang Michael Xie, Marvin Zhang, Akshay Bal
subramani, Weihua Hu, Michihiro Yasunaga, Richard Lanas Phillips, Sara Beery, Jure Leskovec,
Anshul Kundaje, Emma Pierson, Sergey Levine, Chelsea Finn, and Percy Liang. WILDS: A
benchmark of in-the-wild distribution shifts. arXiv preprint arXiv:2012.07421, 2020.

Yann LeCun, Fu Jie Huang, and L´eon Bottou. Learning methods for generic object recognition with

invariance to pose and lighting. Proceedings of the IEEE Conference on Computer Vision and
_Pattern Recognition, 2004._

Da Li, Yongxin Yang, Yi-Zhe Song, and Timothy M Hospedales. Deeper, broader and artier domain

generalization. In Proceedings of the International Conference on Computer Vision, 2017.

Ya Li, Xinmei Tian, Mingming Gong, Yajing Liu, Tongliang Liu, Kun Zhang, and Dacheng Tao.

Deep domain generalization via conditional invariant adversarial networks. In Proceedings of the
_European Conference on Computer Vision, pp. 624–639, 2018._

Evan Z Liu, Behzad Haghgoo, Annie S Chen, Aditi Raghunathan, Pang Wei Koh, Shiori Sagawa,

Percy Liang, and Chelsea Finn. Just Train Twice: Improving group robustness without training
group information. In Proceedings of the International Conference on Machine Learning, 2021.

Francesco Locatello, Stefan Bauer, Mario Lucic, Gunnar Raetsch, Sylvain Gelly, Bernhard

Sch¨olkopf, and Olivier Bachem. Challenging common assumptions in the unsupervised learn
ing of disentangled representations. In Proceedings of the International Conference on Machine
_Learning, 2019._

Francesco Locatello, Ben Poole, Gunnar R¨atsch, Bernhard Sch¨olkopf, Olivier Bachem, and Michael

Tschannen. Weakly-supervised disentanglement without compromises. In Proceedings of the
_International Conference on Machine Learning, 2020._

Mingsheng Long, Yue Cao, Jianmin Wang, and Michael Jordan. Learning transferable features with

deep adaptation networks. In Proceedings of the International Conference on Machine Learning,
2015.

Mingsheng Long, Han Zhu, Jianmin Wang, and Michael I Jordan. Deep transfer learning with joint

adaptation networks. In Proceedings of the International Conference on Machine Learning, 2017.

Loic Matthey, Irina Higgins, Demis Hassabis, and Alexander Lerchner. dsprites: Disentanglement

testing sprites dataset. https://github.com/deepmind/dsprites-dataset/, 2017.

Ninareh Mehrabi, Fred Morstatter, Nripsuta Saxena, Kristina Lerman, and Aram Galstyan. A survey

on bias and fairness in machine learning. ACM Computing Surveys (CSUR), 2021.

Milton Llera Montero, Casimir JH Ludwig, Rui Ponte Costa, Gaurav Malhotra, and Jeffrey Bowers.

The role of disentanglement in generalisation. In Proceedings of the International Conference on
_Learning Representations, 2020._

Vinod Nair and Geoffrey E Hinton. Rectified linear units improve restricted boltzmann machines.

In Proceedings of the International Conference on Machine Learning, 2010.

Hyeonseob Nam, HyunJae Lee, Jongchan Park, Wonjun Yoon, and Donggeun Yoo. Reducing do
main gap by reducing style bias. In Proceedings of the IEEE Conference on Computer Vision and
_Pattern Recognition, 2021._

Giorgio Patrini, Alessandro Rozza, Aditya Krishna Menon, Richard Nock, and Lizhen Qu. Making

deep neural networks robust to label noise: A loss correction approach. In Proceedings of the
_IEEE Conference on Computer Vision and Pattern Recognition, 2017._

Xingchao Peng, Qinxun Bai, Xide Xia, Zijun Huang, Kate Saenko, and Bo Wang. Moment matching

for multi-source domain adaptation. In Proceedings of the International Conference on Computer
_Vision, 2019._


-----

Christian S Perone, Pedro Ballester, Rodrigo C Barros, and Julien Cohen-Adad. Unsupervised

domain adaptation for medical imaging segmentation with self-ensembling. NeuroImage, 2019.

Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agar
wal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya
Sutskever. Learning transferable visual models from natural language supervision. arXiv preprint
_arXiv:2103.00020, 2021._

Benjamin Recht, Rebecca Roelofs, Ludwig Schmidt, and Vaishaal Shankar. Do imagenet classifiers

generalize to imagenet? In Proceedings of the International Conference on Machine Learning,
2019.

Danilo Jimenez Rezende, Shakir Mohamed, and Daan Wierstra. Stochastic backpropagation and

approximate inference in deep generative models. In Proceedings of the International Conference
_on Machine Learning, 2014._

Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma, Zhiheng

Huang, Andrej Karpathy, Aditya Khosla, Michael Bernstein, Alexander C. Berg, and Li FeiFei. Imagenet large scale visual recognition challenge. International Journal of Computer Vision,
2015.

Shiori Sagawa, Pang Wei Koh, Tatsunori B Hashimoto, and Percy Liang. Distributionally robust

neural networks for group shifts: On the importance of regularization for worst-case generalization. Proceedings of the International Conference on Learning Representations, 2020.

Steffen Schneider, Evgenia Rusak, Luisa Eck, Oliver Bringmann, Wieland Brendel, and Matthias

Bethge. Improving robustness against common corruptions by covariate shift adaptation. In

_Proceedings of the International Conference on Learning Representations, 2020._

Lukas Schott, Julius von K¨ugelgen, Frederik Tr¨auble, Peter Gehler, Chris Russell, Matthias Bethge,

Bernhard Sch¨olkopf, Francesco Locatello, and Wieland Brendel. Visual representation learning

does not generalize strongly within the same domain. In Proceedings of the International Con_ference on Learning Representations, 2021._

Vaishaal Shankar, Achal Dave, Rebecca Roelofs, Deva Ramanan, Benjamin Recht, and Ludwig

Schmidt. Do image classifiers generalize across time? arXiv preprint arXiv:1906.02168, 2019.

Baochen Sun and Kate Saenko. Deep coral: Correlation alignment for deep domain adaptation. In

_Proceedings of the European Conference on Computer Vision, 2016._

Rohan Taori, Achal Dave, Vaishaal Shankar, Nicholas Carlini, Benjamin Recht, and Ludwig

Schmidt. Measuring robustness to natural distribution shifts in image classification. arXiv preprint
_arXiv:2007.00644, 2020._

Antonio Torralba and Alexei A Efros. Unbiased look at dataset bias. In Proceedings of the IEEE

_Conference on Computer Vision and Pattern Recognition. IEEE, 2011._

Vladimir Vapnik. Principles of risk minimization for learning theory. In Advances in Neural Infor
_mation Processing Systems, 1992._

Hemanth Venkateswara, Jose Eusebio, Shayok Chakraborty, and Sethuraman Panchanathan. Deep

hashing network for unsupervised domain adaptation. In Proceedings of the IEEE Conference on
_Computer Vision and Pattern Recognition, 2017._

Yufei Wang, Haoliang Li, and Alex C Kot. Heterogeneous domain generalization via domain mixup.

In Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing,
2020.

Kai Xiao, Logan Engstrom, Andrew Ilyas, and Aleksander Madry. Noise or Signal: The role of

image backgrounds in object recognition. ArXiv preprint arXiv:2006.09994, 2020.

Cihang Xie and Alan Yuille. Intriguing properties of adversarial training at scale. In Proceedings of

_the International Conference on Learning Representations, 2020._


-----

Minghao Xu, Jian Zhang, Bingbing Ni, Teng Li, Chengjie Wang, Qi Tian, and Wenjun Zhang.

Adversarial domain adaptation with domain mixup. In AAAI Conference on Artificial Intelligence,
2020.

Shen Yan, Huan Song, Nanxiang Li, Lincan Zou, and Liu Ren. Improve unsupervised domain

adaptation with mixup training. arXiv preprint arXiv:2001.00677, 2020.

Hongyi Zhang, Moustapha Cisse, Yann N Dauphin, and David Lopez-Paz. MixUp: Beyond empir
ical risk minimization. In Proceedings of the International Conference on Learning Representa_tions, 2018._

Ling Zhang, Xiaosong Wang, Dong Yang, Thomas Sanford, Stephanie Harmon, Baris Turkbey, Hol
ger Roth, Andriy Myronenko, Daguang Xu, and Ziyue Xu. When unseen domain generalization
is unnecessary? rethinking data augmentation. arXiv preprint arXiv:1906.03347, 2019.

Han Zhao, Remi Tachet Des Combes, Kun Zhang, and Geoffrey Gordon. On learning invariant rep
resentations for domain adaptation. In Proceedings of the International Conference on Machine
_Learning, 2019._

Kaiyang Zhou, Yongxin Yang, Timothy Hospedales, and Tao Xiang. Deep domain-adversarial im
age generation for domain generalisation. In AAAI Conference on Artificial Intelligence, 2020.

Jun-Yan Zhu, Taesung Park, Phillip Isola, and Alexei A Efros. Unpaired image-to-image translation

using cycle-consistent adversarial networks. In Proceedings of the International Conference on
_Computer Vision, 2017._


-----

