# CONTROLLING DIRECTIONS ORTHOGONAL
## TO A CLASSIFIER

**Yilun Xu, Hao He, Tianxiao Shen, Tommi Jaakkola**
Computer Science and Artificial Intelligence Lab,
Massachusetts Institute of Technology
_{ylxu, haohe, tianxiao}@mit.edu;_ tommi@csail.mit.edu

ABSTRACT

We propose to identify directions invariant to a given classifier so that these directions can be controlled in tasks such as style transfer. While orthogonal decomposition is directly identifiable when the given classifier is linear, we formally define a notion of orthogonality in the non-linear case. We also provide a surprisingly simple method for constructing the orthogonal classifier (a classifier utilizing directions other than those of the given classifier). Empirically, we present
three use cases where controlling orthogonal variation is important: style transfer,
domain adaptation, and fairness. The orthogonal classifier enables desired style
transfer when domains vary in multiple aspects, improves domain adaptation with
label shifts and mitigates the unfairness as a predictor. The code is available at
[https://github.com/Newbeeer/orthogonal_classifier.](https://github.com/Newbeeer/orthogonal_classifier)

1 INTRODUCTION

Many machine learning applications require explicit control of directions that are orthogonal to a
predefined one. For example, to ensure fairness, we can learn a classifier that is orthogonal to sensitive
attributes such as gender or race (Zemel et al., 2013; Madras et al., 2018). Similar, if we transfer images
from one style to another, content other than style should remain untouched. Therefore images before
and after transfer should align in directions orthogonal to style. Common to these problems is the task
of finding an orthogonal classifier. Given any principal classifier operating on the basis of principal
_variables, our goal is to find a classifier, termed orthogonal classifier, that predicts the label on the basis_
of orthogonal variables, defined formally later.

The notion of orthogonality is clear in the linear case. Consider a joint distribution PXY over X R[d] and
binary label Y . Suppose the label distribution is Bernoulli, i.e., PY _Y ;0.5_ and class-conditional
distributions are Gaussian, PX _Y_ _y_ _X;_ _µy,σy[2][I][)][, where the means and variances depend on the] ∈_
label. If the principal classifier is linear,∣ = _w1_ Pr _Y_ 1 _θ1[⊺][x][)][, any classifier] = B(_ _[ w][2][, in the set])_ [ W][2][ = {][Pr][(][Y][ =]
1 _θ2[⊺][x][) ∣]_ _[θ]1[⊺][θ][2][ =][ 0][}][, is considered orthogonal to] = N(_ _[ w][1][. Thus the two classifiers][ w][1][,w][2][, with orthogonal]_
decision boundaries (Fig. 1) focus on distinct but complementary attributes for predicting the same label. = ( = ∣
∣

Finding the orthogonal classifier is no longer straightforward in the non-linear case. To rigorously define
what we mean by the orthogonal classifier, we first introduce the notion of mutually orthogonal random
variables that correspond to (conditinally) independent latent variables mapped to observations through a
diffeomorphism (or bijection if discrete). Each r.v. is predictive of the label but represents complementary
information. Indeed, we show that the orthogonal random variable maximizes the conditional mutual
information with the label given the principal counterpart, subject to an independence constraint that
ensures complementarity.

Our search for the orthogonal classifier can be framed as follows: given a principal classifier w1 using
some unknown principal r.v. for prediction, how do we find its orthogonal classifier w2 relying solely on
orthogonal random variables? The solution to this problem, which we call classifier orthogonalization,
turns out to be surprisingly simple. In addition to the principal classifier, we assume access to a full
classifier wx that predicts the same label based on all the available information, implicitly relying on
both principal and orthogonal latent variables. The full classifier can be trained normally, absent of
constraints[1]. We can then effectively “subtract” the contribution of w1 from the full classifier to obtain
the orthogonal classifier w2 which we denote as w2 _wx_ _w1. The advantage of this construction is that_

1The full classifier may fail to depend on all the features, e.g., due to simplicity bias (Shah et al., 2020).

= ∖


-----

|5|Col2|
|---|---|
|5 4 3 2 1 0 1 2|p(x|Y = 0) p(x|Y = 1)|
|||

|5|Col2|
|---|---|
|5 4 3 2 1 0 1 2||
|||

|5|Col2|
|---|---|
|5 4 3 2 1 0 1 2||
|||

|5|Col2|
|---|---|
|5 4 3 2 1 0 1 2||
|||


x1 x1 x1 x1

(a) Distribution pX _Y_ (b) wx _x_ (c) w1 _x_ (d) w2 _x_

Figure 1: An illustrative example of orthogonal classifier in a linear case. (a) is the data distributions in∣

( ) ( ) ( )

two classes; (b,c,d) are the probabilities of the data from class 1 predicted by the full/principal/orthogonal
classifiers. Red and blue colors mean a probability close to 1 or 0. The white color indicates regions with
a probability close to 0.5, which are classifiers’ decision boundaries. Clearly, w1 and w2 have orthogonal
decision boundaries.

we do not need to explicitly identify the underlying orthogonal variables. It suffices to operate only on
the level of classifier predictions.

We provide several use cases for the orthogonal classifier, either as a predictor or as a discriminator. As a
predictor, the orthogonal classifier predictions are invariant to the principal sensitive r.v., thus ensuring
fairness. As a discriminator, the orthogonal classifier enforces a partial alignment of distributions,
allowing changes in the principal direction. We demonstrate the value of such discriminators in 1)
_controlled style transfer where the source and target domains differ in multiple aspects, but we only wish_
to align domain A’s style to domain B, leaving other aspects intact; 2) domain adaptation with label
_shift where we align feature distributions between the source and target domains, allowing shifts in label_
proportions. Our results show that the simple method is on par with the state-of-the-art methods in each
task.

2 NOTATIONS AND DEFINITION


**Symbols. We use the uppercase to denote random variable (e.g., data X, label Y ), the lowercase to**
denote the corresponding samples and the calligraphic letter to denote the sample spaces of r.v., e.g., data
sample space . We focus on the setting where label space is discrete, i.e., 1, _,C_, and denote
the C 1 dimensional probability simplex as ∆[C]. A classifier w ∆[C] is a mapping from sample
space to the simplex. Its X _y-th dimension w_ _x_ _y denotes the predicted probability of label Y_ Y = { ⋯ _y for sample}_ _x._
− ∶X →

**Distributions.** For random variables A,B, we use the notation pA, pA _B, pAB to denote the_

( )

marginal/conditional/joint distribution, i.e., pA _a_ _p_ _A_ _a_, pA _B_ _a_ _b_ _p_ _A_ _a_ _B_ _b_,pAB _a,b_
∣
_p_ _A_ _a,B_ _b_ . Sometimes, for simplicity, we may ignore the subscript if there is no ambiguity, e.g.,
∣
_p_ _a_ _b_ is an abbreviation for pA _B_ _a_ _b_ . ( ) = ( = ) ( ∣ ) = ( = ∣ = ) ( ) =
( = = )

We begin by defining the notion of an orthogonal random variable. We consider continuous∣ _X,Z1,Z2_

( ∣ ) ( ∣ )

and assume their supports are manifolds diffeomorphic to the Euclidean space. The probability density
functions (PDF) are in . Given a joint distribution pXY, we define the orthogonal random variable as
follows:
C[1]

**Definition 1 (Orthogonal random variables). We say Z1 and Z2 are orthogonal random variables w.r.t**
_pXY if they satisfy the following properties:_

_(i) There exists a diffeomorphism f_ 1 2 _such that f_ _Z1,Z2_ _X._

_(ii) Z1 and Z2 are statistically independent given ∶Z_ × Z →X Y, i.e., Z1 (Z2 _Y ._ ) =

The orthogonality relation is symmetric by definition. Note that the orthogonal pair perfectly reconstructs ⊥⊥ ∣
the observations via the diffeomorphism f ; as random variables they are also sampled independently
from class conditional distributions p _Z1_ _Y_ and p _Z2_ _Y_ . For example, we can regard foreground
objects and background scenes in natural images as being mutually orthogonal random variables.
( ∣ ) ( ∣ )

**Remark.** The definition of orthogonality can be similarly developed for discrete variables and discretecontinuous mixtures. For discrete variables, for example, we can replace the requirement of diffeomorphism with bijection.


-----

Since the diffeomorphism f is invertible, we can use z1 1 and z2 2 to denote the two
parts of the inverse mapping so that Z1 _z1_ _X_ and Z2 _z2_ _X_ . Note that, for a given joint distribution
_pXY, the decomposition into orthogonal random variables is not unique. There are multiple pairs of ∶X →Z_ ∶X →Z
random variables that represent valid mutually orthogonal latents of the data. We can further justify our = ( ) = ( )
definition of orthogonality from an information theoretic perspective by showing that the choice of z2
attains the maximum of the following constrained optimization problem.
**Proposition 1. Suppose the orthogonal r.v. of z1** _X_ _w.r.t pXY exists and is denoted as z2_ _X_ _. Then_
_z_ _X_ _z2_ _X_ _is a maximizer of I_ _z_ _X_ ; _Y_ _z1_ _X_ _subject to I_ _z_ _X_ ; _z1_ _X_ _Y_ 0.
( ) ( )

We defer the proof to Appendix( ) = ( ) B.1( . Proposition( ) ∣ ( 1)) shows that the orthogonal random variable maxi-( ( ) ( )∣ ) =
mizes the additional information about the label we can obtain from X while remaining conditionally
independent of the principal random variable. This ensures complementary in predicting the label.

3 CONSTRUCTING THE ORTHOGONAL CLASSIFIER

Let Z1 _z1_ _X_ and Z2 _z2_ _X_ be mutually orthogonal random variables w.r.t pXY . We call Z1 the
principal variable and Z2 the orthogonal variable. In this section, we describe how we can construct the
Bayes optimal classifier operating on features = ( ) = ( ) _Z2 from the Bayes optimal classifier relying on Z1. We_
formally refer to the classifiers of interests as: (1) principal classifier w1 _x_ _y_ _p_ _Y_ _y_ _Z1_ _z1_ _x_ ;
(2) orthogonal classifier w2 _x_ _y_ _p_ _Y_ _y_ _Z2_ _z2_ _x_ ; (3) full classifier wx _x_ _y_ _p_ _Y_ _y_ _X_ _x_ .
( ) = ( = ∣ = ( ))

3.1 CLASSIFIER ORTHOGONALIZATION( ) = ( = ∣ = ( )) ( ) = ( = ∣ = )

Our key idea relies on the bijection between the density ratio and the Bayes optimal classifier (Sugiyama
et al., 2012). Specifically, the ratio of densities pX _Y_ _x_ _i_ and pX _Y_ _x_ _j_, assigned to an arbitrary point

_x, can be represented by the Bayes optimal classifier∣_ ( _w∣_ ) _x_ _i_ Pr∣ (Y ∣ ) i _x_ as _ppXX∣YY (xx∣ji)_ _pY_ _i_ _w_ _x_ _j_ [.]

Similar, the principal classifierover Z1. For any i,j, we have w1 gives us associated density ratios of class-conditional distributionsppZZ11 ∣YY _zz11_ _xx_ _ji_ _pY(_ _i)w =1_ _x_ _j_ [. These can be combined to obtain]( = ∣ ) ∣ ( ∣ ) [=][ p][Y][ (] ([j])[)][w]([(][x])[)][i]
( ( )∣ )

density ratios of class-conditional distributionw2. We additionally rely on the fact that the diffeomorphism ∈Y ∣ ( _p(_ _Z)∣2_ _Y) and subsequently calculate the orthogonal classifier[=][ p][Y][ (] ([j])[)][w] f[1]([(] permits us to change variables between[x])[)][i]_
∣
_x and z1,z2: pX_ _Y_ _x_ _i_ _pZ1_ _Y_ _z1_ _i_ _pZ2_ _Y_ _z2_ _i_ volJf _z1,z2_, where volJf is volume of the
Jacobian (Berger et al., 1987) of the diffeomorphism mapping. Taken together,
∣ ∣ ∣

_pZ2_ _Y_ _z2_ _i_ ( ∣ ) = ( ∣ ) ∗ ( ∣ ) ∗ ( )

_[w][1][(][x][)][i]_ (1)

_pZ2_ ∣Y _z2_ _j_ _pZ1_ _Y_ _z1_ _j_ _pZ2_ _Y_ _z2_ _j_ volJf _z1,z2_ _pZ1_ _Y_ _z1_ _j_ _wx_ _x_ _j_ _w1_ _x_ _j_

( ∣ )

Note that since the diffeomorphism∣ ∣ ∣ _f is shared with all classes, the Jacobian is the same for all label-∣_ /

( ∣ ) [=][ p][Z][1][∣][Y] ([ (][z][1]∣[∣][i][) ∗]) ∗ _[p][Z][2][∣][Y][ (] ([z][2][∣]∣[i][) ∗]) ∗[vol][J][f]_ [(]([z][1][,z][2][)]) [/][ p][Z][1][∣][Y] ([ (][z][1]∣[∣][i][)]) [=][ w][x]([(][x])[)][i] ( )

conditioned distributions on Z1,Z2. Hence the Jacobian volume terms cancel each other in the above
equation. We can then finally work backwards from the density ratios of pZ2∣Y to the orthogonal classifier:

Pr _Y_ _i_ _Z2_ _z2_ _x_ Pr _Y_ _i_ Pr _Y_ _j_ (2)

_w1_ _x_ _i_ _j_ _w1_ _x_ _j_

( = ∣ = ( )) = ( = ) _[w][x][(][x][)][i]_ / ∑ ( ( = ) _[w][x][(][x][)][j]_ )

We call this procedure classifier orthogonalization since it adjusts the full classifier( ) ( w)x to be orthogonal
to the principal classifier w1. The validity of this procedure requires overlapping supports of the classconditional distributions, which ensures the classifier outputs wx _x_ _i,w1_ _x_ _i to remain non-zero for all_
_x_ _,i_ .
( ) ( )

Empirically, we usually have access to a dataset ∈X ∈Y _xt,yt_ _t_ 1 [with][ n][ iid samples from the joint]
distribution pXY . To obtain the orthogonal classifier, we need to first train the full classifier ˆwx based
=
on the dataset . We can then follow the classifier orthogonalization to get an empirical orthogonal D = {( )}[n]
classifier, denoted as w2 ˆwx _w1. We use symbol_ to emphasize that the orthogonal classifier uses
complementary information relative to D _z1. Algorithm 1 summarizes the construction of the orthogonal_
classifier. = ∖ ∖

**Generalization bound. Since wx is trained on a finite dataset, we consider the generalization bound of**
the constructed orthogonal classifier. We denote the population risk as R _w_ EpXY _x,y_ log w _x_ _y_
and the empirical risk as _R[ˆ]_ _w_ _i[,y]i[)∈D][ log][ w][(][x][i][)][y][i]_ [. For a function family][ W] ([ whose elements])
( ) = − [ ( ) ]

map to the simplex ∆[C], we define∣D∣ ˆ[∑]wx[(][x] inf _wx_ _R[ˆ]_ _wx_ _,wx[∗]_ _x[∈W]_ _[R][(][w][x][)][. We further denote]_

( ) = − [1]

the Rademacher complexity of function family with sample size as R .
∈W
X = ( ) [=][ inf] _[w]_ ∣D∣
W ∣D∣ (W)


-----

**Algorithm 1 Classifier Orthogonalization**

**Input: principal classifier w1, dataset** .
Train an empirical full classifier ˆwx on by empirical risk minimization.

Construct an orthogonal classifier ˆwx Dw1 via classifier orthogonalization (Eq. (2)).

**return the empirical orthogonal classifier D** ˆw2 = ˆwx _w1_

∖

**Theorem 1. Assume py is uniform distribution,** _wx_ ∖ _takes values in_ _m,_ 1 _m_ _with m_ 0, [1]2 [)][,]

_and 1_ _pX_ _Y_ _x_ _y_ 0,γ 0, _holds for_ _x_ _,y_ _. Then for any δ_ 0, 1 _with probability_
_at least 1_ _δ, we have:_ ∀ ∈W ( − )[C] ∈(
∣
/ ( ∣ ) ∈( ) ⊂( +∞) ∀ ∈X ∈Y ∈( )
− _R_ _w ˆx_ _w1_ _R_ _wx[∗]_ 2R 2log [1] ¿ _δ_

_m_

⎛ Á ⎞
∣ ( ∖ ) − ( [∖] _[w][1][)∣≤(][1][ +][ γ][)]_ ⎜ ∣D∣(W) + ÁÀ [2log][ 1] ⎟

Theorem 1 shows that the population risk of the empirical orthogonal classifier in Algorithm⎝ ∣D∣ ⎠ 1 would
be close to the optimal risk if the maximum value of the reciprocal of class-conditioned distributions
1 _pX_ _Y_ _x_ _y_ and the Rademacher term are small. Typically, the Rademacher complexity term satisfies
R ∣ 2 (Bartlett & Mendelson, 2001; Gao & Zhou, 2016). We note that the empirical full

classifier may fail in specific ways that are harmful for our purposes. For example, the classifier may/ ( ∣ )

∣D∣

not rely on all the key features due to simplicity bias as demonstrated by ((W) = O(∣D∣[−] [1] ) Shah et al., 2020). There are
several ways that this effect can be mitigated, including Ross et al. (2020); Teney et al. (2021).

3.2 ALTERNATIVE METHOD: IMPORTANCE SAMPLING


An alternative way to get the orthogonal classifier is via importance sampling. For each sample point
_x,y_, we construct its importance φ _x,y_ _pZp1ZY1_ _zz11_ _xx_ _y_
( ( ))

be calculated from the principal classifier by( ) ( ) ∶= φ _x,y∣_ ( ( )∣wp1Y)x[. Via Bayes’ rule, the importance]y _y_ [. We define the importance sampling (IS)][ φ][(][x,y][)][ can]
( )

objective as IS _w_ Ex,y _pXY_ _φ_ _x,y_ log w _x_ _y_ . It can be shown that the orthogonal classifier( ) _w2_
maximizes the IS objective, i.e., w2 arg max( IS. However, the importance sampling method has an) =
∼
Achilles’ heel: variance. A few bad samples with large weights can drastically deviate the estimation, L ( ) ∶= [ ( ) ( ) ]
which is problematic for mini-batch learning in practice. We provide an analysis of the variance of = L
the IS objective. It shows the variance increases with the divergences between Z1’s marginal and its
label-conditioned marginals, _Df_ _pZ1_ _pZ1_ _Y_ _y_ _y_ 1 [even when the learned classifier][ w][ is approaching]
the optimal classifier, i.e., w _w2. Let_ ∣IS =[(][w][) ∶=]= [ 1]n _t_ 1 _[φ][(][x][t][,y][t][)]_ [log][ w][(][x][t][)][y]t [be the empirical IS]
{ ( ∣∣ )}[C]

objective estimated with n iid samples. Clearly, Var IS[(][w]= [)) =][ 1]n [Var][(̂]IS[(][w][))][. While][ Var][(̂]IS[(][w][))]

≈ L[̂][n]

at w _w2 is the following,_ [∑][n]

(L[̂][n] L[1] L[1]

= Var IS[(][w][2][)) =][ E][Y] 1 [∣∣][p]Z1 _Y_ _y[) +][ 1][)]_ [E]Z2 _Y_ _y_ [log][2][ p]Y _Z2_ [(][y][∣][z][2][)] −L(][w][2][)][2]

where Df is the Pearson(L[̂][1] _χ[2]-divergence. The expression indicates that the variance grows linearly with[[(][D][f]_ [(][p][Z] ∣ = ∣ = ∣
the expected divergence. In contrast, the divergences have little effect when training the empirical full
classifier in classifier orthogonalization. We provide further experimental results in section 4.1.2 to
demonstrate that classifier orthogonalization has better stability than the IS objective with increasing
divergences and learning rates.

4 ORTHOGONAL CLASSIFIER FOR CONTROLLED STYLE TRANSFER


In style transfer, we have two domains A,B with distributions PXY,QXY . We use binary label Y to
indicate the domain of data X (0 for domain A, 1 for domain B). Assume Z1 and Z2 are mutually
orthogonal variables w.r.t both PXY and QXY . The goal is to conduct controlled style transfer between
the two domains. By controlled style transfer, we mean transferring domain A’s data to domain B only
via making changes on partial latent (Z1) while not touching the other latent (Z2). Mathematically,
we aim to transfer the latent distribution from PZ1 _PZ2 to QZ1_ _PZ2. This task cannot be achieved by_
traditional style transfer algorithms such as CycleGAN (Zhu et al., 2017), since they directly transfer data
distributions from PX to QX, or equivalently from the perspective of latent distributions, from PZ1 _PZ2_
to QZ1 _QZ2_ . Below we show that the orthogonal classifier wx _w1 enables such controlled style transfer._

Our strategy is to plug the orthogonal classifier into the objective of CycleGAN. The original Cycle
∖

GAN objective defines a min-max game between two generators/discriminator pairs GAB _DAB and_
/


-----

0.55

0.50

0.45

0.40

0.35


0.55

0.50

0.45

0.40

0.35


3000

2000


1000

|Col1|IS wx\ w1|Col3|Col4|Col5|Col6|
|---|---|---|---|---|---|
|||||||
||Oracle|||||
|||||||
|||||||
|||||||
|||||||

|Col1|IS wx|\ w1|Col4|Col5|Col6|Col7|
|---|---|---|---|---|---|---|
||Or|acle|||||
||||||||
||||||||
||||||||
||||||||

|Col1|P(Y|= 1| B|) (wx\ w|1)|Col6|
|---|---|---|---|---|---|
|||= 1| G = 1| B|) (wx\ w ) (IS)|1)||
|||||||
||P(Y|= 1| G|) (IS)|||
|||||||
|||||||


IS
wx\ w1
Oracle


1.2 1.4 1.6 1.8

log y[[][D]f[(][p]Z1[||][p]Z1|Y = y[) + 1]]

(a)


IS
wx\ w1
Oracle


P(Y = 1| B) (wx\ w1)

P(Y = 1| G) (wx\ w1)

P(Y = 1| B) (IS)

P(Y = 1| G) (IS)


3.0 2.5 2.0 1.5 1.0

Learning rate (log-scale)

(b)


0.0 0.2 0.4 0.6 0.8 1.0

Predicted probability

(c)


Figure 2: (a,b): Test loss versus log expected divergence and learning rate. (c): The histogram of
predicted probability of different methods. The two red lines are the ground-truth probabilities for
_P_ _Y_ 1 Brown background and P _Y_ 1 Green background .

_G(BA =DBA∣_ . GAB transforms the images from domain) ( = ∣ _A to domain)_ _B. We use_ _P to denote the generative_
distribution, i.e., GAB _X_ _P for X_ _P_ . Similarly, _Q denotes the generative distribution of generator_
_GBA. The minimax game of CycleGAN is given by/_ [̃]
( ) ∼ [̃] ∼ [̃]

min max GAN[(][G][AB][,D][AB][) + L][BA]GAN[(][G][BA][,D][BA][) + L][cyc][(][G][AB][,G][BA][)] (3)
_GAB_ _,GBA_ _DAB_ _,DBA[L][AB]_

where GAN losses GAN[,] [L][BA]GAN [minimize the divergence between the generative distributions and the]
targets, and the cycle consistency loss cyc is used to regularize the generators (Zhu et al., 2017).
Concretely, the GAN loss L[AB] GAN [is defined as follows,]
L

GAN[(][G][AB] L[,D][AB][AB][) ∶=][ E][x][∼][Q][[][log][ D][AB][(][x][)] +][ E]x _P_

Now we tilt the GAN losses in CycleGAN objective to guide the generators to perform controlled transfer.∼ [̃]

L[AB]

[[][log][(][1][ −] _[D][AB][(][x][))]]_

Specially, we replace GAN [in Eq. (][3][) with the orthogonal GAN loss][ L]OGAN[AB] [when updating the generator]
_GAB:_
L[AB]

OGAN[(][G][AB][,D][AB][) ∶=][ E]x _P_

_DAB_ _x_ _r_ _x_ ∼ [̃]
where φ _DAB_ _x_ _,r_ _xL[AB]_ 1 _DAB_ _x_ _DAB_ _x_ _r_ _x[[][log][and][(][1][ r][ −][(][x][φ][) =][(][D][ w][AB]wx[x][∖][(]w[w][x]1[1][)][(][,r]x[x][)][(]1[0][x][. Consider an extreme case][)))]]_
( ) ( )

where we allow the model to change all latents,( − ( ))+ ( ) ( i.e) _., z1_ _x_ _x. As a result,∖_ ( ) OGAN degenerates to

( ( ) ( )) =

the original GAN since wx _w1_ 2 [and][ φ][(][D][AB][(][x][)][,r][(][x][)) ≡] _[D][AB][(][x][)][. The other orthogonal GAN]_
( ) = L

loss OGAN [can be similarly derived. For a given generator][ G][AB][, the discriminator’s optimum is]
achieved at D L _AB[∗]_ [(][x][) =][ Q][(] ∖[x][)/( ̃]P ≡ _x[1]_ _Q_ _x_ . Assuming the discriminator is always trained to be
optimal, the generator L[BA] _GAB can be viewed as optimizing the following virtual objective:_ OGAN[(][G][AB][) ∶=]
( _P) +x_ ( ))

OGAN[(][G][AB][,D]AB[∗] [) =][ E]x _P_ [log] _P_ _x_ _Q_ _x_ _r_ _x_
̃ ( ) L[AB]

following property.

L[AB] ∼ [̃] ̃ ( )+ ( ) ( ) [. The optimal generator in the new objective satisfies the]

**Proposition 2. The global minimum of** _OGAN[(][G][AB][)][ is achieved if and only if][ ̃]PZ1,Z2_ _z1,z2_
_QZ1_ _z1_ _PZ2_ _z2_ _._
L[AB] ( ) =

We defer the proof to Appendix( ) ( ) B.4. The proposition states that the new objective OGAN [converts the]
original global minimum QZ1 _QZ2 in_ GAN [to the desired one][ Q][Z]1 _[P][Z]2_ [. The symmetric statement holds]
for OGAN[(][G][BA][)][.] L[AB]
L[AB]

4.1 L[BA]EXPERIMENTS

4.1.1 SETUP


**Datasets. (a) CMNIST: We construct C(olors)MNIST dataset based on MNIST digits (LeCun & Cortes,**
2005). CMNIST has two domains and 60000 images with size 3, 28, 28 . The majority of the images
in each domain will have a digit color and a background color correspond to the domain: domain A
“red digit/green background” and domain B “blue digit/brown background”. We define two ( ) _bias_
_degrees λd, λb to control the probability of the images having domain-associated colors, e.g., for an_
image in domain A, with↔ _λd probability, its digit is set to be red; otherwise, its digit color is randomly ↔_
red or blue. The background colors are determined similarly with parameter λb. In our experiments,


-----

Table 1: CMNIST

_Z1 acc._ _Z2 acc._


Table 2: CelebA-GH


_Z1 acc._ _Z2 acc._ _Z1 acc._ _Z2 acc._ FID

Vanilla 90.2 14.3 Vanilla 88.4 16.8 **38.5**

↓

_wx_ _w1 (MLDG)_ 94.9 91.1 _wx_ _w1 (MLDG)_ 90.9 53.0 46.2
_wx_ _w1 (TRM)_ 89.2 96.2 _wx_ _w1 (TRM)_ 92.2 56.5 39.8
_wx ∖_ _w1 (Fish)_ 93.9 98.0 _wx ∖_ _w1 (Fish)_ 88.8 42.9 43.4
∖ ∖

IS (Oracle) 90.0 100 IS (Oracle) 89.1 40.6 44.3

∖ ∖

_wx_ _w1 (Oracle)_ 94.9 100 _wx_ _w1 (Oracle)_ **93.7** **57.4** 39.7
∖ ∖

we set λd 0.9 and λb 0.8. Our goal is to transfer the digit color (Z1) while leaving the background
color (Z2) invariant. (b) CelebA-GH: We construct the CelebA-G(ender)H(air) dataset based on the
gender and hair color attributes in CelebA ( = = Liu et al., 2015). CelebA-GH consists of 110919 images
resized to 3, 128, 128 . In CelebA-GH, domain A is non-blond-haired males, and the domain B is
blond-haired females. Our goal is to transfer the facial characteristic of gender (Z1) while keeping the
hair color ( (Z2) intact. )

**Models. We compare variants of orthogonal classifiers to the vanilla CycleGAN (Vanilla) and the**
importance sampling objective (IS). We consider two ways of obtaining the w1 classifier: (a) Oracle:
The principal classifier w1 is trained on an oracle dataset where Z1 is the only discriminative direction,
_e.g., setting bias degrees λd_ 0.9 and λb 0 in CMNIST such that only digit colors vary across domains.
(b) Domain generalization: Domain generalization algorithms aim to learn the prediction mechanism that
is invariant across environments and thus generalizable to unseen environments ( = = Blanchard et al., 2011;
Arjovsky et al., 2019). The variability of environments is considered as nuisance variation. We construct
environments such that only Y _Z1 is invariant. We defer details of the environments to Appendix E.1._
We use three domain generalization algorithms, Fish (Shi et al., 2021), TRM (Xu & Jaakkola, 2021) and
MLDG (Li et al., 2018), to obtain ∣ _w1. We indicate different approaches by the parentheses after wx_ _w1_
and IS, e.g., wx _w1 (Oracle) is the orthogonal classifier with w1 trained on Oracle datasets._
∖

**Metric. We use three metrics for quantitative evaluation: 1) Z1 accuracy: the success rate of transferring**

∖

an image’s latent z1 from domain A to domain B; 2) Z2 accuracy: percentage of transferred images
whose latents z2 are unchanged; 3) FID scores: a standard metric of image quality (Heusel et al., 2017).
We only report FID scores on the CelebA-GH dataset since it is not common to compute FID on MNIST
images. Z1,Z2 accuracies are measured by two oracle classifiers that output an image’s latents z1 and
_z2 (Appendix E.1.3)._


For more details of datasets, models and training procedure, please refer to Appendix E.

4.1.2 RESULTS

**Comparison to IS. In section 3.2, we demonstrate that IS suffers from high variance when the di-**
vergences of the marginal and the label-conditioned latent distributions are large. We provide further
empirical evidence that our classifier orthogonalization method is more robust than IS. Fig. 2(a) shows
that the test loss of IS increases dramatically with the divergences. Fig. 2(b) displays that IS’s test loss
grows rapidly when enlarging learning rate, which corroborates the observation that gradient variances
are more detrimental to the model’s generalization with larger learning rates (Wang et al., 2013). In
contrast, the test loss of wx _w1 remains stable with varying divergences and learning rates. In Fig. 2(c),_
we visualize the histograms of predicted probabilities of wx _w1 (light color) and IS (dark color)._
We observe that the predicted probabilities of ∖ _wx_ _w1 better concentrates around the ground-truth_
probabilities (red lines). ∖
∖

**Main result. As shown in Table 1 and 2, adding an orthogonal classifier to the vanilla CycleGAN**
significantly improves its z2 accuracy (from 14 to 90 in CMNIST, from 16 to 40 in CelebA-GH)
while incurring a slight loss of the image quality. We observe that the oracle version of orthogonal classifier (wx _w1 (Oracle)) achieves best z2 accuracy on both datasets. In addition, class orthogonalization is+_ +
compatible with domain generalization algorithms when the prediction mechanism Z1 _Y is invariant_
across collected environments. ∖
∣

In Fig. 3, we visualize the transferred samples from domain A. We observe that vanilla CycleGAN
models change the background colors/hair colors along with digit colors/facial characteristics in CMNIST
and CelebA. In contrast, orthogonal classifiers better preserve the orthogonal aspects Z2 in the input
images. We also provide visualizations for domain B in Appendix C.


-----

Input

Vanilla
Oracle MLDG

(b) CelebA-GH: Domain A


Input

Vanilla
Oracle MLDG

(a) CMNIST: Domain A


Figure 3: Style transfer samples on the domain A of CMNIST and CelebA-GH. We visualize the
inputs (top row) and the corresponding transferred samples of different methods.
5 ORTHOGONAL CLASSIFIER FOR DOMAIN ADAPTATION

In unsupervised domain adaptation, we have two domains, source and target with data distribution ps,
_pt. Each sample comes with three variables, data X, task label Y and domain label U_ . We assume U is
uniformly distributed in {0 (source), 1 (target)}. The task label is only accessible in the source domain.
Consider learning a model h _g_ _f that is the composite of the encoder f_ and the predictor
_g_ . A common practice of domain adaptation is to match the two domain’s feature distributions
_ps_ _f_ _X_ _,pt_ _f_ _X_ via domain adversarial training ( = ○ Ganin & Lempitsky, 2015 ∶X →Z; Long et al., 2018; Shu
et al. ∶Z →Y, 2018). The encoder f is optimized to extract useful feature for the predictor while simultaneously
bridging the domain gap via the following domain adversarial objective:( ( )) ( ( ))

min max (4)
_f_ _D_

where discriminator D 0, 1 distinguishes features from two domains. The equilibrium of the

[L(][f,D][) ∶=][ E][x][∼][p][s] [[][log][ D][(][f] [(][x][))] +][ E][x][∼][p][t] [[][log][(][1][ −] _[D][(][f]_ [(][x][)))]]

objective is achieved when the encoder perfectly aligns the two domain, i.e., ps _f_ _x_ _pt_ _f_ _x_ . We
highlight that such equilibrium is not desirable when the target domain has shifted label distribution ( ∶Z →[ ] Azizzadenesheli et al., 2019; des Combes et al., 2020). Instead, when the label shift appears, it is more( ( )) = ( ( ))
preferred to align the conditioned feature distribution,i.e., ps _f_ _x_ _y_ _pt_ _f_ _x_ _y_ _,_ _y_ .

Now we show that our classifier orthogonalization technique can be applied to the discriminator for

( ( )∣ ) = ( ( )∣ ) ∀ ∈Y

making it “orthogonal to” the task label. Specifically, consider the original discriminator as full classifier
_wx, i.e., for a given f_, wx _x_ 0 arg maxD _f,D_ _ps_ _f_ _pxs_ _f_ _pxt_ _f_ _x_
( ( ))

classifier w1 that discriminates the domain purely via the task label( ( ))+ ( ( _Y)),[. We then construct the principle] i.e., w1_ _x_ 0 Pr _U_ 0 _Y_
_y_ _x_ . Note that here we assume the task label( ) = L( _Y can be determined by the data) =_ _X, i.e., Y_ _y_ _X_ . We
focus on the case that Y is discrete so w1 could be directly computed via frequency count. We propose( ) = ( = ∣ =
to train the encoder( )) _f with the orthogonalized discriminator wx_ _w1,_ = ( )

min f _f,_ _wx_ _w1_ 0 Ex _ps_ log _wx_ _w1_ _x_ 0 Ex _pt_ log 1 _wx_ _w1_ _x_ 0 (5)

∖

∼ ∼
L( ( ∖ )(⋅) ) = [ ( ∖ )( ) ] + [ ( −( ∖ )( ) )]

**Proposition 3. Suppose there exists random variable Z2** _z2_ _X_ _orthogonal to y_ _X_ _Y w.r.t_
_p_ _f_ _X_ _,U_ _. Then f achieves global optimum if and only if it aligns all label-conditioned feature_
_distributions, i.e., ps_ _f_ _x_ _y_ _pt_ _f_ _x_ _y_ _,_ _y_ _._ = ( ) ( ) =
( ( ) )

Note that in practice, we have no access to the target domain label prior( ( )∣ ) = ( ( )∣ ) ∀ ∈Y _pt_ _Y_ and the label y _x_ for
target domain data x _pt. Thus we use the pseudo label ˆy as a surrogate to construct the principle_
classifier, where ˆy _x_ arg maxy h _x_ _y is generated by our model h._ ( ) ( )
∼

5.1 EXPERIMENTS( ) = ( )

**Models. We take a well-known domain adaptation algorithm VADA (Shu et al., 2018) as our baseline.**
VADA is based on domain adversarial training and combined with virtual adversarial training and entropy
regularization. We show that utilizing orthogonal classifier can improve its robustness to label shift. We
compare it with two improvements: (1) VADA+wx _w1 which is our method that plugs in the orthogonal_
classifier as a discriminator in VADA; (2) VADA+IW: We tailor the SOTA method for label shifted
domain adaptation, importance-weighted domain adaptation ( ∖ des Combes et al., 2020), and apply it to
VADA. We also incorporate two recent domain adaptation algorithms for images— CyCADA (Hoffman
et al., 2018) and ADDA (Tzeng et al., 2017)—into comparisons.


-----

**Setup. We focus on visual domain adaptation and directly borrow the datasets, architectures and domain**
setups from Shu et al. (2018). To add label shift between domains, we control the label distribution in
the two domains. For source domain, we sub-sample 70% data points from the first half of the classes
and 30% from the second half. We reverse the sub-sampling ratios on the target domain. The label
distribution remains the same across the target domain train and test set. Please see Appendix E.2 for
more details.

**Results. Table 3 reports the test accuracy on seven domain adaptation tasks. We observe that VADA+wx**
_w1 improves over VADA across all tasks and outperforms VADA+IW on five out of seven tasks. We find_
VADA+IW performs worse than VADA in two tasks, MNIST SVHN and MNIST MNIST-M. Our∖
hypothesis is that the domain gap is large between these datasets, hindering the estimation of importance
weight. Hence, VADA-IW is unable to adapt the label shift appropriately. In addition, the results show→ →
that VADA+wx _w1 outperforms ADDA on six out of seven tasks._
∖ Table 3: Test accuracy on visual domain adaptation benchmarks

|Source Target|MNIST SVHN MNIST DIGITS SIGNS CIFAR STL MNIST-M MNIST SVHN SVHN GTSRB STL CIFAR|
|---|---|

|Source-Only ADDA CyCADA|51.8 75.7 34.5 85.0 74.7 68.7 47.7 89.7 78.2 38.4 86.0 90.6 66.8 50.4 - 82.8 39.6 - - - -|
|---|---|


|VADA VADA + IW VADA + w w x∖ 1|77.8 79.0 35.7 90.3 93.6 72.4 53.1 71.2 87.1 34.5 90.7 95.4 74.0 53.8 ↓ ↑ ↓ ↑ ↑ ↑ ↑ 79.1 88.0 40.5 90.6 95.2 74.5 54.1 ↑ ↑ ↑ ↑ ↑ ↑ ↑|
|---|---|


6 ORTHOGONAL CLASSIFIER FOR FAIRNESS

We are given a dataset _xt,yt,ut_ _t_ 1 [that is sampled iid from the distribution][ p][XY U] [, which]
contains the observations x, the sensitive attributes u and the labels y . Our goal is to learn a
=
classifier that is accurate w.r.t D = {( y and fair w.r.t)}[n] u. We frame the fairness problem as finding the orthogonal
classifier of an “totally unfair” classifier ∈X _w1 that only uses the sensitive attributes ∈U_ ∈Y _u for prediction. We_
can directly get the unfair classifier w1 from the dataset statistics, i.e., w1 _x_ _y_ _p_ _Y_ _y_ _U_ _u_ _x_ .
Below we show that the orthogonal classifier of unfair classifier meets equalized odds, one metric for
fairness evaluation. ( ) = ( = ∣ = ( ))
**Proposition 4. If the orthogonal random variable of U** _u_ _X_ _w.r.t pXY exists, then the orthogonal_
_classifier wx_ _w1 satisfies the criterion of equalized odds._
= ( )

We emphasize that, unlike existing algorithms for learning fairness classifier, our method does not require ∖
additional training. We obtain a fair classifier via orthogonalizing an existing model wx which is simply
the vanilla model trained by ERM on the dataset .

6.1 EXPERIMENTS D

**Setup. We experiment on the UCI Adult dataset, which has gender as the sensitive attribute and the UCI**
German credit dataset, which has age as the sensitive attribute (Zemel et al., 2013; Madras et al., 2018;
Song et al., 2019). We compare the orthogonal classifier wx _w1 to three baselines: LAFTR (Madras_
et al., 2018), which proposes adversarial objective functions that upper bounds the unfairness metrics;
**L-MIFR (Song et al., 2019), which uses mutual information objectives to control the expressiveness- ∖**
fairness trade-off; ReBias (Bahng et al., 2020), which minimizes the Hilbert-Schmidt Independence
Criterion between the model representations and biased representations; as well as the Vanilla (wx)
trained by ERM. We employ two fairness metrics – demographic parity distance (∆DP) and equalized
odds distance (∆EO) – defined in Madras et al. (2018). We denote the penalty coefficient of the adversarial
or de-biasing objective as γ, whose values govern a trade-off between prediction accuracy and fairness, in
all baselines. We borrow experiment configurations, such as CNN architecture, from Song et al. (2019).
Please refer to Appendix E.3 for more details.

**Results. Tables 4, 5 report the test accuracy, ∆DP and ∆EO on Adults and German datasets. We observe**
that the orthogonal classifier decreases the unfairness degree of the Vanilla model and has competitive
performances to existing baselines. Especially in the German dataset, compared to LAFTR, our method


-----

has the same ∆EO but better ∆DP and test accuracy. It is surprising that our method outperforms LAFTR,
even though it is not specially designed for fairness. Further, our method has benefit of being training-free,
which allows it to be applied to improve any off-the-shelf classifier’s fairness without additional training.


Table 4: Accuracy v.s. Fairness (Adults)

Acc. ∆DP ∆EO

Vanilla **84.5** 0.19 0.20

↑ ↓ ↓

LAFTR (γ 0.1) 84.2 0.14 0.09
LAFTR (γ 0.5) 84.0 0.12 **0.07**
L-MIFR (γ = 0.05) 81.6 **0.04** 0.15
L-MIFR (γ = 0.1) 82.0 0.06 0.16
ReBias (γ = 100) 84.3 0.15 0.11
ReBias (γ = 50) 84.4 0.17 0.16
=

_wx_ _w1_ 81.6 0.12 0.12

=

∖


Table 5: Accuracy v.s. Fairness (German)

Acc. ∆DP ∆EO

Vanilla **76.0** 0.19 0.33

↑ ↓ ↓

LAFTR (γ 0.1) 73.0 0.11 **0.17**
LAFTR (γ 0.5) 72.7 0.11 0.19
L-MIFR (γ = 0.1) 75.8 0.10 0.21
L-MIFR (γ = 0.05) 75.6 0.08 0.18
ReBias (γ = 100) 73.0 **0.07** **0.17**
ReBias (γ = 50) 75.0 0.10 0.20
=

_wx_ _w1_ 75.4 0.09 0.18

=

∖


7 RELATED WORKS

**Disentangled representation learning Similar to orthogonal random variables, disentangled represen-**
tations are also based on specific notions of feature independence. For example, Higgins et al. (2018)
defines disentangled representations via equivalence and independence of group transformations, Kim &
Mnih (2018) relates disentanglement to distribution factorization, and Shu et al. (2020) characterizes
disentanglement through generator consistency and a notion of restrictiveness. Our work differs in two
key respects. First, most definitions of disentanglement rely primarily on the bijection between latents
and inputs (Shu et al., 2020) absent labels. In contrast, our orthogonal features must be conditionally
independent given labels. Further, in our work orthogonal features remain implicit and they are used
discriminatively in predictors. Several approaches aim to learn disentangled representations in an unsupervised manner (Chen et al., 2016; Higgins et al., 2017; Chen et al., 2018). However, Locatello
et al. (2019) argues that unsupervised disentangled representation learning is impossible without a proper
inductive bias.

**Model de-biasing A line of works focuses on preventing model replying on the dataset biases. Bahng**
et al. (2020) learns the de-biased representation by imposing HSIC penalty with biased representation,
and Nam et al. (2020) trains an unbiased model by up-weighting the high loss samples in the biased
model. Li et al. (2021) de-bias training data through data augmentation. However, these works lack
theoretical definition for the de-biased model in general cases and often require explicit dataset biases.

**Density ratio estimation using a classifier Using a binary classifier for estimating the density ra-**
tio (Sugiyama et al., 2012) enjoys widespread attention in machine learning. The density ratio estimated
by classifier has been applied to Monte Carlo inference (Grover et al., 2019; Azadi et al., 2019), class
imbalance (Byrd & Lipton, 2019) and domain adaptation (Azizzadenesheli et al., 2019).

**Learning a set of diverse classifiers Another line of work related to ours is learning a collection of**
diverse classifiers through imposing penalties that relate to input gradients. Diversity here means that the
classifiers rely on different sets of features. To encourage such diversity, Ross et al. (2018; 2020) propose
a notion of local independence, which uses cosine similarity between input gradients of classifiers as
the regularizer, while in Teney et al. (2021) the regularizer pertains to dot products. Ross et al. (2017)
sequentially trains multiple classifiers to obtain qualitatively different decision boundaries. We defer
more discussions of the advantages of classifier orthogonalization over existing methods to Appendix F.

8 CONCLUSION

We consider finding a discriminative direction that is orthogonal to a given principal classifier. The
solution in the linear case is straightforward but does not generalize to the non-linear case. We define
and investigate orthogonal random variables, and propose a simple but effective algorithm (classifier
orthogonalization) to construct the orthogonal classifier with both theoretical and empirical support.
Empirically, we demonstrate that the orthogonal classifier enables controlled style transfer, improves
existing alignment methods for domain adaptation, and has a lower degree of unfairness.


-----

ACKNOWLEDGEMENTS

The work was partially supported by an MIT-DSTA Singapore project and by an MIT-IBM Grand
Challenge grant. YX was partially supported by the HDTV Grand Alliance Fellowship. We would like
to thank the anonymous reviewers for their valuable feedback.

REFERENCES

Mart´ın Arjovsky, L. Bottou, Ishaan Gulrajani, and David Lopez-Paz. Invariant risk minimization. ArXiv,
abs/1907.02893, 2019.

Samaneh Azadi, Catherine Olsson, Trevor Darrell, Ian J. Goodfellow, and Augustus Odena. Discriminator
rejection sampling. ArXiv, abs/1810.06758, 2019.

K. Azizzadenesheli, Anqi Liu, Fanny Yang, and Anima Anandkumar. Regularized learning for domain
adaptation under label shifts. ArXiv, abs/1903.09734, 2019.

Hyojin Bahng, Sanghyuk Chun, Sangdoo Yun, Jaegul Choo, and Seong Joon Oh. Learning de-biased
representations with biased representations. In ICML, 2020.

Peter L. Bartlett and Shahar Mendelson. Rademacher and gaussian complexities: Risk bounds and
structural results. J. Mach. Learn. Res., 3:463–482, 2001.

M. Berger, Bernard Gostiaux, and S. Levy. Differential geometry: Manifolds, curves, and surfaces. 1987.

G. Blanchard, Gyemin Lee, and C. Scott. Generalizing from several related classification tasks to a new
unlabeled sample. In NIPS, 2011.

Jonathon Byrd and Zachary Chase Lipton. What is the effect of importance weighting in deep learning?
In ICML, 2019.

Tian Qi Chen, Xuechen Li, Roger B. Grosse, and David Kristjanson Duvenaud. Isolating sources of
disentanglement in variational autoencoders. In NeurIPS, 2018.

Xi Chen, Yan Duan, Rein Houthooft, John Schulman, Ilya Sutskever, and P. Abbeel. Infogan: Interpretable representation learning by information maximizing generative adversarial nets. In NIPS,
2016.

Remi Tachet des Combes, Han Zhao, Yu-Xiang Wang, and Geoffrey J. Gordon. Domain adaptation with´
conditional distribution matching and generalized label shift. ArXiv, abs/2003.04475, 2020.

Yaroslav Ganin and V. Lempitsky. Unsupervised domain adaptation by backpropagation. _ArXiv,_
abs/1409.7495, 2015.

Wei Gao and Zhi-Hua Zhou. Dropout rademacher complexity of deep neural networks. Science China
_Information Sciences, 59(7):072104, 2016._

Aditya Grover, Jiaming Song, Alekh Agarwal, Kenneth Tran, Ashish Kapoor, E. Horvitz, and S. Ermon. Bias correction of learned generative models using likelihood-free importance weighting. In
_DGS@ICLR, 2019._

Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and S. Hochreiter. Gans trained
by a two time-scale update rule converge to a local nash equilibrium. In NIPS, 2017.

Irina Higgins, Lo¨ıc Matthey, Arka Pal, Christopher P. Burgess, Xavier Glorot, Matthew M. Botvinick,
Shakir Mohamed, and Alexander Lerchner. beta-vae: Learning basic visual concepts with a constrained
variational framework. In ICLR, 2017.

Irina Higgins, David Amos, David Pfau, Sebastien Racani´ ere, Lo` ¨ıc Matthey, Danilo Jimenez Rezende,
and Alexander Lerchner. Towards a definition of disentangled representations. ArXiv, abs/1812.02230,
2018.

Judy Hoffman, Eric Tzeng, Taesung Park, Jun-Yan Zhu, Phillip Isola, Kate Saenko, Alexei A. Efros, and
Trevor Darrell. Cycada: Cycle-consistent adversarial domain adaptation. In ICML, 2018.


-----

Hyunjik Kim and Andriy Mnih. Disentangling by factorising. In ICML, 2018.

Y. LeCun and Corinna Cortes. The mnist database of handwritten digits. 2005.

Da Li, Yongxin Yang, Yi-Zhe Song, and Timothy M. Hospedales. Learning to generalize: Meta-learning
for domain generalization. ArXiv, abs/1710.03463, 2018.

Yingwei Li, Qihang Yu, Mingxing Tan, Jieru Mei, Peng Tang, Wei Shen, Alan Loddon Yuille, and Cihang
Xie. Shape-texture debiased neural network training. ArXiv, abs/2010.05981, 2021.

Z. Liu, Ping Luo, Xiaogang Wang, and X. Tang. Deep learning face attributes in the wild. 2015 IEEE
_International Conference on Computer Vision (ICCV), pp. 3730–3738, 2015._

Francesco Locatello, Stefan Bauer, Mario Lucic, Sylvain Gelly, Bernhard Scholkopf, and Olivier Bachem.¨
Challenging common assumptions in the unsupervised learning of disentangled representations. ArXiv,
abs/1811.12359, 2019.

Mingsheng Long, Zhangjie Cao, Jianmin Wang, and Michael I. Jordan. Conditional adversarial domain
adaptation. In NeurIPS, 2018.

David Madras, Elliot Creager, T. Pitassi, and R. Zemel. Learning adversarially fair and transferable
representations. In ICML, 2018.

Jun Hyun Nam, Hyuntak Cha, Sungsoo Ahn, Jaeho Lee, and Jinwoo Shin. Learning from failure:
Training debiased classifier from biased classifier. ArXiv, abs/2007.02561, 2020.

Andrew Slavin Ross, Michael C. Hughes, and Finale Doshi-Velez. Right for the right reasons: Training
differentiable models by constraining their explanations. In IJCAI, 2017.

Andrew Slavin Ross, Weiwei Pan, and Finale Doshi-Velez. Learning qualitatively diverse and interpretable rules for classification. ArXiv, abs/1806.08716, 2018.

Andrew Slavin Ross, Weiwei Pan, Leo Anthony Celi, and Finale Doshi-Velez. Ensembles of locally
independent prediction models. In AAAI, 2020.

Harshay Shah, Kaustav Tamuly, Aditi Raghunathan, Prateek Jain, and Praneeth Netrapalli. The pitfalls
of simplicity bias in neural networks. ArXiv, abs/2006.07710, 2020.

Yuge Shi, Jeffrey S. Seely, Philip H. S. Torr, N. Siddharth, Awni Y. Hannun, Nicolas Usunier, and Gabriel
Synnaeve. Gradient matching for domain generalization. ArXiv, abs/2104.09937, 2021.

Rui Shu, H. Bui, H. Narui, and S. Ermon. A dirt-t approach to unsupervised domain adaptation. ArXiv,
abs/1802.08735, 2018.

Rui Shu, Yining Chen, Abhishek Kumar, Stefano Ermon, and Ben Poole. Weakly supervised disentanglement with guarantees. ArXiv, abs/1910.09772, 2020.

Jiaming Song, Pratyusha Kalluri, Aditya Grover, Shengjia Zhao, and S. Ermon. Learning controllable
fair representations. In AISTATS, 2019.

Masashi Sugiyama, Taiji Suzuki, and T. Kanamori. Density ratio estimation in machine learning. 2012.

Damien Teney, Ehsan Abbasnejad, Simon Lucey, and Anton van den Hengel. Evading the simplicity
bias: Training a diverse set of models discovers solutions with superior ood generalization. ArXiv,
abs/2105.05612, 2021.

Eric Tzeng, Judy Hoffman, Kate Saenko, and Trevor Darrell. Adversarial discriminative domain
adaptation. 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 2962–
2971, 2017.

Chong Wang, X. Chen, Alex Smola, and E. Xing. Variance reduction for stochastic gradient optimization.
In NIPS, 2013.

Yilun Xu and T. Jaakkola. Learning representations that support robust transfer of predictors. ArXiv,
abs/2110.09940, 2021.


-----

R. Zemel, Ledell Yu Wu, Kevin Swersky, T. Pitassi, and C. Dwork. Learning fair representations. In
_ICML, 2013._

Jun-Yan Zhu, Taesung Park, Phillip Isola, and Alexei A. Efros. Unpaired image-to-image translation
using cycle-consistent adversarial networks. 2017 IEEE International Conference on Computer Vision
_(ICCV), pp. 2242–2251, 2017._


-----

A MORE PROPERTIES OF ORTHOGONAL RANDOM VARIABLE

In this section we demonstrate the properties of orthogonal random variables. We also discuss the
existence and uniqueness of orthogonal classifier when given the principal classifier. Below we list some
useful properties of orthogonal random variables.
**Proposition 5. Let Z1 and Z2 be any mutually orthogonal w.r.t PXY, then we have**

_(i) Invariance: For any diffeomorphism g, g_ _Z1_ _and g_ _Z2_ _are also mutually orthogonal._

_(ii) Distribution-free: Z1,Z2 are mutually orthogonal w.r.t(_ ) ( _P)_ _X_ _Y if and only if there exists a diffeo-_
_morphism mapping between X and X_ [′]. ′

_(iii) Existence: Suppose the label-conditioned distributions py_ _,_ _supp_ _py_ _,_ _y_ _. For z1_
_and z1_ _is diffeomorphism to Euclidean space, then_ _i_ _, there exists a diffeomorphism map-_
_ping_ _z1_ _X_ _,z2,i_ _X_ _such that z1_ _X_ _z2,i_ _X_ _Y_ _i ∈C. Further if there exists diffeomorphism[1]_ ( ) = X ∀ ∈Y ∈C[1]
_mappings between(X)_ _z2,is, then the orthogonal r.v. of z1_ ∀X ∈Y exists.
( ( ) ( )) ( ) ⊥⊥ ( )∣ =

In the following theorem, we justify the validity of the classifier orthogonalization when( ) _z1_ _x_ satisfies
some regularity conditions.
**Theorem 2. Suppose w1** _x_ _i_ _p_ _Y_ _i_ _z1_ _x_ _. If the label-condition distributions pys, and(_ _z)_ 1 satisfy
_the conditions in Proposition 5.(iii), then wx_ _w1 is the unique orthogonal classifier of w1._
( ) = ( = ∣ ( ))

Note that our construction of wx _w1 does not require the exact expression of ∖_ _z1_ _X_, its orthogonal r.v
or data generation process. The theorem states that the orthogonal classifier constructed by classifier
orthogonalization is the Bayesian classifier for any orthogonal random variables of ∖ ( _z)1_ _X_ .

B PROOFS ( )


B.1 PROOF FOR PROPOSITION 1

**Proposition 1. Suppose the orthogonal r.v. of z1** _X_ _w.r.t pXY exists and is denoted as z2_ _X_ _. Then_
_z_ _X_ _z2_ _X_ _is a maximizer of I_ _z_ _X_ ; _Y_ _z1_ _X_ _subject to I_ _z_ _X_ ; _z1_ _X_ _Y_ 0.
( ) ( )
( ) = ( ) ( ( ) ∣ ( )) ( ( ) ( )∣ ) =

_Proof. For any function z, by chain rule we have I_ _z1_ _X_ _,z2_ _X_ ; _Y_ _I_ _z1_ _X_ ; _Y_
_I_ _z2_ _X_ ; _Y_ _z1_ _X_ and I _z1_ _X_ _,z_ _X_ ; _Y_ _I_ _z1_ _X_ ; _Y_ _I_ _z_ _X_ ; _Y_ _z1_ _X_ . Further, the definition of the orthogonal r.v. ensures that there exists a differmorphism( ( ) f between( ) ) X and= _z(1_ _X(_ _,z)_ 2 ) +X,
which implies( ( ) ∣ _I(_ _z1))X_ _,z_ (f _z(1_ _X)_ _,z(_ 2 )X ) =; _Y_ ( _I(_ _z)1_ _X ) +,z_ (X( ; _Y)_ . ∣ ( ))
( ) ( )

Hence by data processing inequality we have:

( ( ) ( ( ( ) ( ))) ) = ( ( ) ( ) )

_I_ _z1_ _X_ _,z2_ _X_ ; _Y_ _I_ _z1_ _X_ _,z_ _f_ _z1_ _X_ _,z2_ _X_ ; _Y_ _I_ _z1_ _X_ _,z_ _X_ ; _Y_
_I_ _z1_ _X_ ; _Y_ _I_ _z2_ _X_ ; _Y_ _z1_ _X_ _I_ _z1_ _X_ ; _Y_ _I_ _z_ _X_ ; _Y_ _z1_ _X_
( ( ) ( ) ) ≥ ( ( ) ( ( ( ) ( ))) ) = ( ( ) ( ) )
_I_ _z2_ _X_ ; _Y_ _z1_ _X_ _I_ _z_ _X_ ; _Y_ _z1_ _X_
⇔ ( ( ) ) + ( ( ) ∣ ( )) ≥ ( ( ) ) + ( ( ) ∣ ( ))

The above inequality shows that⇔ ( ( ) ∣ z ( z2)) ≥ maximize the mutual information( ( ) ∣ ( )) _I_ _z_ _X_ ; _Y_ _z1_ _X_ . Further,
by definition of the orthogonal r.v., z _z2 is independent of z1 conditioned on Y, i.e. z_ _X_ _z1_ _X_ _Y_
which is equivalent to I _z_ _X_ ; _z1_ =X _Y_ 0. ( ( ) ∣ ( ))
= ( ) ⊥⊥ ( )∣

B.2 PROOF FOR THEOREM( ( ) 1 ( )∣ ) =

**Theorem 1. Assume py is uniform distribution,** _wx_ _takes values in_ _m,_ 1 _m_ _with m_ 0, [1]2 [)][,]

_and 1_ _pX_ _Y_ _x_ _y_ 0,γ 0, _holds for_ _x_ _,y_ _. Then for any δ_ 0, 1 _with probability_
_at least 1_ _δ, we have:_ ∀ ∈W ( − )[C] ∈(
∣
/ ( ∣ ) ∈( ) ⊂( +∞) ∀ ∈X ∈Y ∈( )
− _δ_

_R_ _w ˆx_ _w1_ _R_ _wx[∗]_ 2R 2log [1] ¿

_m_

⎛ Á ⎞
∣ ( ∖ ) − ( [∖] _[w][1][)∣≤(][1][ +][ γ][)]_ ⎜ ∣D∣(W) + ÁÀ [2log][ 1] ⎟
⎝ ∣D∣ ⎠

_Proof. Denote the empirical risk of w as_


_Rˆ_ _w_ _R[ˆ]_ _w_ log w _xi_ _yi_

_xi,yi_

( ) = ( ) = − [1] ( ∑)∈D ( )

∣D∣


-----

and the population risk as

_R_ _w_ Ep(x,y) log w _x_ _y_

_Step 1: bounding excess risk_ _R_ _w ˆx_ (R) = −wx[∗][)∣][.] [ ( ) ]

By the classical result in theorem 8 in Bartlett & Mendelson (2001), we know that for any w,

∣ ( ) − (

_δ_ 0, 1, since log w _x_ _y_ 0, log _m[1]_ [)][, with probability at least][ 1][ −] _[δ][,]_
∈W
∈( _R)w_ _R −w_ (n)in ∈(1 log _w_ _xi_ _yi_ Ep log _w_ _x_ _y_ 2Rn 2log m[1] √ 2logn [1]δ

∣ [ˆ]( ) − ( )∣= ∣ [1] ∑= ( ( )) − [ ( ( ) )]∣≤ (W) +

Since _wˆx_ inf _wx_ _R[ˆ]_ _wx_ _,wx[∗]_ inf _wx_ _R_ _wx_, and _R_ _w ˆx_ _R_ _w ˆx_ 2Rn

2log _m[1]_ 2 log=n _δ[2]_ _,_ _R_ _w∈Wx[∗][) −]([R][(][w])_ _x[∗][)∣≤]_ [2]=[R][n][(W) +]∈W[ 2log]( [ 1]m) 2 logn ∣δ[1], we have[ˆ]( ) − ( )∣ ≤ (W) +

√ √

∣ [ˆ]( _R_ _w ˆx_ _R_ _wx[∗][)∣≤]_ [2][R][n][(W) +][ 2log 1]m √ 2logn [1]δ (6)

∣ ( ) − (

_Step 2: bounding_ _R_ _w ˆx_ _w1_ _R_ _wx[∗]_ _wx_ _R_ _wx[∗][)∣][.]_
∣ ( ∖ ) − ( [∖] _[w][1][)∣]_ _[by][ ∣][R][(][ ˆ]_ ) − (

_w_ _x_ _y_

_w1_ _x_ _y_ _w_ _x_ _y_

Then we have:R(w ∖ _w1) = −Ep(x,y)_ ⎡⎢⎢⎢⎢⎢⎣log ∑y[′] _w(w(1()()xx))yy′′_ ⎤⎥⎥⎥⎥⎥⎦ = R(w) + Ep(x,y) ⎡⎢⎢⎢⎢⎣log w1(x)y∑y[′] _w1((x))y′′_ ⎤⎥⎥⎥⎥⎦

_R_ _w ˆx_ _w1_ _R_ _wx[∗]_
∣ (R _w ˆ ∖x_ ) −R _w(x[∗][)∣+][∖]E[w]p[1][)∣]x,y_ log w1 _x_ _y_ _wˆx_ _x_ _y′_ Ep _x,y_ log w1 _x_ _y_ _wx[∗][(][x][)][y][′]_

≤∣= ∣≤∣RR(((ww ˆ ˆxx) −) −) − _RR(((wwxx[∗][∗][)∣+][)∣+][ max]RRRRRRRRRRRRRRRRRRRRRRRRRREp((x,y⎛⎜⎝RRRRRRRRRRRRRR))E⎡⎢⎢⎢⎢⎣⎡⎢⎢⎢⎢⎢⎣plog(x,y)∑∑⎡⎢⎢⎢⎢⎢⎣yylog max([′][′]_ _wwwwˆ)xx[∗]11((([(]∑yxxx[x]y[′]))[′])[)]yyyy′′w′[′]_ _wwww⎤⎥⎥⎥⎥⎥⎦ˆ1xx[∗]11RRRRRRRRRRRRRR(((([(](xxx[x]x)))[)]))yyyyy′′[′]′′_ ⎤⎥⎥⎥⎥⎥⎦⎤⎥⎥⎥⎥⎦RRRRRRRRRRRRRR−, RRRRRRRRRRRRRREp((x,y))⎡⎢⎢⎢⎢⎣⎡⎢⎢⎢⎢⎢⎣log miny[′]( )wwwwˆxx[∗]11∑((([(]yxxx[x][′] )))[)]yyyyw′′′[′] 1⎤⎥⎥⎥⎥⎥⎦(RRRRRRRRRRRRRRx⎞⎟⎠)y′ ⎤⎥⎥⎥⎥⎦RRRRRRRRRRRR

We define the ratio r _x,y_ _ww[ˆ]xx[∗]_ [(]x[x][)]y[y] [. We define][ y][(][x][) ∶=][ arg min][y][ r][(][x,y][)][,][ ¯]y _x_ arg maxy r _x,y_ .

We have r _x,y_ _x_ ( _y_ _ww) =xx[∗]_ [(]x[x][)]yy′[′]([/]w[w])1[1][(]x[x][)]yy′[′][ ≤] _[r][(][x,][ ¯]y_ _x_ . Let assume _p_ _y1_ _x_ ( ) ∶= ( )

function y[′], we have the following bound, where we have importance weight∑ ′ ( ) / ( ) ( ∣ ) _φ_ _x,y_ _p_ _y1_ _x_

( ( )) ≤ [∑][y][′][ ˆ] ( )) [≤] _[γ][ for all][ x,y][. For any]_

if y _y[′]_ _x_ otherwise 0,
( ∣ )

∶X →Y ( ) ∶=

Ex log r _x,y[′]_ _x_ Ex,yφ _x,y_ log r _x,y_ _γ_ Ex,y log r _x,y_ _γ_ _R_ _w ˆx_ _R_ _wx[∗][))∣]_

= ( )

As a result, we have

∣ ( ( ))∣= ∣ ( ) ( )∣≤ ∣ ( )∣= ∣ ( ) − (

_wx_ _x_ _y_ _w1_ _x_ _y_
E log ′ ′ _y_ _x_ _γ_ _R_ _w ˆx_ _R_ _wx[∗][)∣]_

_y[′] wx[∗][(][x][)][y][′][/][w][1][(][x][)][y][′][ ∣≤]_ [max] [(∣][E] [log][ r][(][x,y][(][x][))∣][,] [∣][E] [log][ r][(][x,][ ¯]

( ) / ( )

Thus∣ [∑][y][′][ ˆ] ( ))∣) ≤ ∣ ( ) − (

∑

_R_ _w ˆx_ _w1_ _R_ _wx[∗]_ _wx_ _R_ _wx[∗][)∣]_

_wˆx_ _x_ _y_ _wˆx_ _x_ _y_

∣ ( ∖ ) − ( [∖] _[w][1][)∣≤∣][R]1[(]+[ ˆ] γ max) −R⎛⎜⎝w ˆRRRRRRRRRRRRRR(Exp(x,yR)_ ⎡⎢⎢⎢⎢⎢⎣wlog maxx[∗][)∣] _y[′]_ _wwwx[∗]11((([(]xx[x])))[)]yyy′′′[′]_ ⎤⎥⎥⎥⎥⎥⎦RRRRRRRRRRRRRR _,_ RRRRRRRRRRRRRREp(x,y) ⎡⎢⎢⎢⎢⎢⎣log miny[′] _wwwx[∗]11(([(](xx[x]))[)])yyy′′[′]′_ ⎤⎥⎥⎥⎥⎥⎦(7)RRRRRRRRRRRRRR⎞⎟⎠

≤( + )∣ ( ) − (


-----

Combining Eq. (6) and Eq. (7), we know that with probability 1 _δ, we have:_
−

_R_ _w ˆx_ _w1_ _R_ _wx[∗]_ 2Rn 2log m[1] √
⎛
∣ ( ∖ ) − ( [∖] _[w][1][)∣≤(][1][ +][ γ][)]_ ⎜ (W) +
⎝

B.3 PROOF FOR PROPOSITION 5 AND THEOREM 2


2log [1]δ


**Proposition 5. Let Z1 and Z2 be any mutually orthogonal w.r.t PXY, then we have**

_(i) Invariance: For any diffeomorphism g, g_ _Z1_ _and g_ _Z2_ _are also mutually orthogonal._

_(ii) Distribution-free: Z1,Z2 are mutually orthogonal w.r.t(_ ) ( _P)_ _X_ _Y if and only if there exists a diffeo-_
_morphism mapping between X and X_ [′]. ′

_(iii) Existence: Suppose the label-conditioned distributions py_ _,_ _supp_ _py_ _,_ _y_ _. For z1_
_and z1_ _is diffeomorphism to Euclidean space, then_ _i_ _, there exists a diffeomorphism map-_
_ping_ _z1_ _X_ _,z2,i_ _X_ _such that z1_ _X_ _z2,i_ _X_ _Y_ _i ∈C. Further if there exists diffeomorphism[1]_ ( ) = X ∀ ∈Y ∈C[1]
_mappings between(X)_ _z2,is, then the orthogonal r.v. of z1_ ∀X ∈Y exists.
( ( ) ( )) ( ) ⊥⊥ ( )∣ =

_Proof. (i) Since Z1 and Z2 are independent, the independence also holds for(_ ) _g_ _Z1_ and g _Z2_ . In
addition, we construct the diffeomorphism between _g_ _Z1_ _,g_ _Z2_ and X as _f[ˆ]_ _g_ _Z1_ _,g_ _Z2_
_f_ _g[−][1]_ _g_ _Z1_ _,g[−][1]_ _g_ _Z2_ _X. Apparently f_ _g[−][1]_ _Z1_ _,g[−][1]_ _Z2_ is a diffeomorphism.( ) ( )
( ( ) ( )) ( ( ) ( )) =

_(ii)( We denote the diffeomorphism between(_ ( )) ( ( ))) = _X_ [′] and( _X( as_ _f)[ˆ]. Then the diffeomorphism mapping between(_ ))
_Z1,Z2_ and X is _f[ˆ]_ _f_ . Thus Z1,Z2 are mutually orthogonal w.r.t X [′].

_(iii) We will use a constructive proof._

( ) ○

_Step 1: Denote dim_ _d,_ dim _z1_ _k, then we can prove that the manifold_ _z1_ _,_ R[d][−][k] is
diffeomorphism of . We denote the diffeomorphism as f, and denote f _x_ _z1_ _x_ _,t_ _x_ where
_z1_ _z1_ _,t_ (X) =R[d][−][k]. ( (X)) = [ (X) ]
X ( ) = [ ( ) ( )]

_Step 2: For X_ _Y_ _y, let Fj_ _t_ _x_ _j_ _t_ _x_ 1, _,t_ _x_ _j_ 1,z1 _x_ _,_ 1 R 0, 1 _,j_ 1,...,d _k_ be

∶X → (X) ∶X →

the CDF corresponding to the distribution of t _x_ _j_ _t_ _X_ 1 _t_ _x_ 1, _,t_ _X_ _j_ 1 _t_ _x_ _j_ 1,z1 _X_
−
_z1_ _x_ _,Y_ _y._ ∣ = ( ( ) ∣ ( ) ⋯ ( ) ( ) ) ∶ →[ ] ∈{ − }
− −
( ) ∣ ( ) = ( ) ⋯ ( ) = ( ) ( ) =

_h_ _t_ _x_ ; _z1_ _x_ 1 _F1_ _t_ _x_ 1 _z1_ _x_ _,y_

( ) =

_h_ _t_ _x_ ; _z1_ _x_ _i_ _Fi_ _t_ _x_ _i_ _t_ _x_ 1, _,t_ _x_ _i_ 1,z1 _x_ _,_ 1 _,i_ 2,...,d
( ( ) ( )) = ( ( ) ∣ ( ) )
−

The conditions p( (C [1]),p (x )) 0 =, _x(_ ( ensure that the PDF of) ∣ ( ) ⋯ ( ) _f(_ )X ) is also in ∈{ and takes values}
in 0, . Thus the above CDFs are all diffeomorphism mapping. By the inverse CDF theorem, for
any z1 _x_, h _t_ _X ∈_ ; _z1_ _X(_ ) > z1 _x∀_ is a uniform distribution in ∈X 0,(1 ) . Hence the random variable C[1]
defined by the mapping ( ∞) _h, i.e., z2,y_ _X_ _h_ _t_ _X_ ; _z1_ _X_, is independent of z1 _X_ given Y 1.
( ) ( ( ) ( ) = ( )) ( )[d][−][k]

In addition, by the construction of( z2) =,y _x_ ( we know that there exists a diffeomorphism( ) ( )) ( ) _f[ˆ] = between_
_z1_ _,t_ and _z1_ _,z2,y_ such that _f[ˆ]_ _z1_ _X_ _,t_ _X_ _z1_ _X_ _,z2,y_ _X_ . Then f
_fˆ_ _z1_ _X_ _,z2,y_ _X_ _X is also a diffeomorphsim mapping.(_ ) [−][1]
( (X) (X)) ( (X) (X)) ( ( ) ( )) = ( ( ) ( )) ○

_Step 3[−][1]_ : From the condition we know that for every y, there exists a diffeomorphism function my such

( ( ) ( )) =

that my _z2,y_ _x_ _z2,1_ _x_ . Apparently _y, my_ _z2,y_ _X_ _z2,1_ _X_ _Y_ _y by z2,y_ _X_ _z1_ _X_ _Y_ _y._
Further, _z1,z2,1_ is a diffeomorphism. Hence z2,1 ∈YX is the orthogonal r.v. of z1 _X_ w.r.t PXY .
○ ( ) = ( ) ∀ ○ ( ) ⊥⊥ ( )∣ = ( ) ⊥⊥ ( )∣ =
( ) ( ) ( )

**Theorem 2. Suppose w1** _x_ _i_ _p_ _Y_ _i_ _z1_ _x_ _. If the label-condition distributions pys, and z1 satisfy_
_the conditions in Proposition 5.(iii), then wx_ _w1 is the unique orthogonal classifier of w1._
( ) = ( = ∣ ( ))

_Proof. By the existence property in Proposition ∖_ 5, we know that for X, there exists a orthogonal r.v. of
_z1_ _X_ and denote it as z2 _X1_ . By the proposition we know that there exists a common diffeomorphism
_f mapping_ _z1_ _X_ _y_ _,z2_ _X_ _y_ to X _y,_ _y_ .
( ) ( )
( ( ∣ ) ( ∣ )) ∣ ∀ ∈Y


-----

Further, by change of variables and the definition of orthogonal r.v., we have _ppj,i,22_ _zz22_

_pi,1_ _z1_ _pi,2_ _z2_ volJf _z1,z2_ _pi_ _x_ _wx_ _x_ _i_ ( )
_pj,1_ _z1_ _pj,2_ _z2_ volJf _z1,z2_ _pj,1_ _z1_ _pj_ _x_ _pj,1_ _z1_ _wx_ _x_ _j_ _w1_ _x_ _j_ [. Thus via the bijection of clas-]( ) =
( ) ( ) ( ) ( ) ( )

sifier and density ratios we know that wx _w1_ _x_ _i_ _p_ _Y_ _i_ _z2_ _x_ .

( ) ( ) ( ) [/][ p][i,][1][(]([z][1][)]) ( ) [/][ p][i,][1][(]([z][1][)]) ( ) ( )

[=] [=] [/][ w][1][(][x][)][i]

B.4 PROOF FOR PROPOSITION 2 ∖ ( ) = ( = ∣ ( ))

**Proposition 2. The global minimum of** _OGAN[(][G][AB][)][ is achieved if and only if][ ̃]PZ1,Z2_ _z1,z2_
_QZ1_ _z1_ _PZ2_ _z2_ _._
L[AB] ( ) =
( ) ( ) _w2_ _x_

_Proof. By definition, r_ _x_ 1 _w2_ _x_ _Q_ _z2_ _x_ OGAN [as the]
( )

following:

− ( ) ( ( )) [. Thus we can reformulate the criterion][ L][AB]

( ) = [=][ P][ (][z][2][(][x][))] _P_ _x_

OGAN[(][G][AB][) =][ E]x _P_ [log]

_P_ _x_ _Q_ _x_ _r_ _x_

∼ [̃] ̃( )
L[AB] _P_ _z1,z2_

Ez1,z2 _P_ [log]̃( ) + ( ) ( )
∼̃ _P_ _z1,z2_ ̃ Q( _z1_ _Q)_ _z2_ _Q[P][ (][z]z[2]2[)]_
=

̃(P _z1,z) +2_ ( Q )z1 (P )z2 ( )

log Ez1,z2 _P_ _P_ _z1,z2_ log 2
∼̃ ̃( ) + ( ) ( )

where we get the lower bound by Jensen’s inequality. The equality holds when≥− [[] ̃( ) ] = −P _z1,z2_ _Q_ _z1_ _P_ _z2_ .

[̃]( ) = ( ) ( )

B.5 PROOF FOR PROPOSITION 3

**Proposition 3. Suppose there exists random variable Z2** _z2_ _X_ _orthogonal to y_ _X_ _Y w.r.t_
_p_ _f_ _X_ _,U_ _. Then f achieves global optimum if and only if it aligns all label-conditioned feature_
_distributions, i.e., ps_ _f_ _x_ _y_ _pt_ _f_ _x_ _y_ _,_ _y_ _._ = ( ) ( ) =
( ( ) )
( ( )∣ ) = ( ( )∣ ) ∀ ∈Y _ps_ _f_ _x_

_Proof. By definition, optimal classifier wx_ _x_ 0 _ps_ _f_ _x_ _pt_ _f_ _x_

_ps_ _y_ _x_ ( ( ))

_ps_ _y_ _x_ ( (pt))y _x_ ( ) = ( ( ))+ ( ( )) [, principle classifier][ w][1][(][x][)][0][ =]
them. It suggests, conditioned on Y, the supports of f _X_ _Y_ _y is non-overlapped for different y in_
( ( ))+ ( ( )) [. By the definition of orthogonal r.v.,][ (][Y,U] [)][ and][ f] [(][X][)][ have a bijection between]
both domains ps and pt. It means if ps _f_ _x_ _y_ 0 then _y[′]_ _y, ps_ _f_ _x_ _y[′]_ 0.
( )∣ =

Thus the orthogonal classifier wx _w1 satisfies:_

( ( )∣ ) > ∀ ≠ ( ( )∣ ) =

_wx_ _w1_ _x_ 0 _[p][s][(][f]_ [(][x][))] ∖

_ps_ _y_ _x_ _ps_ _y_ _x_ _pt_ _y_ _x_

( ∖ )( ) =

( ( )) [/(] _[p][s][(]([f]([(][x])) [))]_ [+][ p][t][(]([f]([(][x])) [))] [)]

_ps_ _y_ _x_ _ps_ _y_ _x_ _pt_ _y_ _x_

= [∑][y][′][ p][s][(][f] [(][x][)∣][y][′][)][p][s][(][y][′][)] / ( [∑][y][′][ p][s][(][f] [(][x][)∣][y][′][)][p][s][(][y][′][)] + [∑][y][′][ p][t][(][f] [(][x][)∣][y][′][)][p][t][(][y][′][)] )

( ( )) ( ( )) ( ( ))

_[p][s][(][f]_ [(][x][)∣][y][(][x][))][p][s][(][y][(][x][))] _[p][t][(][f]_ [(][x][)∣][y][(][x][))][p][t][(][y][(][x][))]

_ps_ _y_ _x_ _ps_ _y_ _x_ _pt_ _y_ _x_

= _ps_ _f_ _x_ _y_ _x_ / ( _[p][s][(][f]_ [(][x][)∣][y][(][x][))][p][s][(][y][(][x][))] + )

( ( )) ( ( )) ( ( ))

_ps_ _f_ _x_ _y_ _x_ _pt_ _f_ _x_ _y_ _x_

( ( )∣ ( ))

Thus the objective in Eq. (= 5) can be reformulated as,
( ( )∣ ( )) + ( ( )∣ ( ))
_f,_ _wx_ _w1_ 0

EL(x _ps(_ log ∖ )(⋅) _p)s_ _f_ _x_ _y_ _x_ _pt_ _f_ _x_ _y_ _x_

_ps_ _f_ _x_ _y_ _x_ _pt_ _f_ _x_ _y_ _x_ _ps_ _f_ _x_ _y_ _x_ _pt_ _f_ _x_ _y_ _x_

∼ ( ( )∣ ( )) ( ( )∣ ( ))
= [

Ey _ps_ Ex _ps_ _x(y_ ( )∣log( _[p])) +[s][(][f]_ [(][x][)∣]( _[y]([) +])∣[ p][t]([(][f]))[(][x][] +][)∣][y][ E][)]_ _[x][∼][p] E[t]_ [[]y[log]pt Ex( _pt(_ _x)∣y_ ( log)) +[p][s][(]([f] [(]([x][)∣])∣[y][) +]( _[ p]))[t][]][(][f]_ [(][x][)∣][y][)]

_ps_ _f_ _x_ _y_ _pt_ _f_ _x_ _y_

∼ ∼ ( ∣ ) ∼ ∼ ( ∣ )
= [− ] + [− ]

Ey _ps_ log Ex _ps_ _x_ _y_ ( ( )∣ ) Ey _pt_ log Ex _pt_ _x_ _y_ ( ( )∣ )

_ps_ _f_ _x_ _y_ _pt_ _f_ _x_ _y_

∼ ∼ ( ∣ ) ∼ ∼ ( ∣ )
≥− 2log 2 [. [ _[p][s][(][f]_ [(][x][)∣][y][) +][ p][t][(][f] [(][x][)∣][y][)] ]] − [ [ _[p][s][(][f]_ [(][x][)∣][y][) +][ p][t][(][f] [(][x][)∣][y][)] ]]

( ( )∣ ) ( ( )∣ )

= −


-----

The lower-bound holds due to Jensen’s inequality. The equality is achieved if and only if _[p]p[s]t_ [(]f[f] [(]x[x][)∣]y[y][)] [is]

invariant to x, for all y, i.e., _y_ _,ps_ _f_ _x_ _y_ _pt_ _f_ _x_ _y_ .
( ( )∣ )
∀ ∈Y ( ( )∣ ) = ( ( )∣ )

B.6 PROOF FOR PROPOSITION 4

**Proposition 4. If the orthogonal random variable of U** _u_ _X_ _w.r.t pXY exists, then the orthogonal_
_classifier wx_ _w1 satisfies the criterion of equalized odds._
= ( )

_Proof. We denote the orthogonal random variables as ∖_ _z2_ _X_, and wx _w1_ _x_ Pr _y_ _z2_ _x_ . Since
_z2_ _X_ _U_ _Y, we know that wx_ _w1_ _X_ _U_ _Y . Hence the prediction of the classifier wx_ _w1 is_
conditional independent of sensitive attribute U on the ground-truth label( ) ∖ Y, which meets the equalized( ) = [ ∣ ( )]
odds metric.( ) ⊥⊥ ∣ ∖ ( ) ⊥⊥ ∣ ∖

C EXTRA SAMPLES

(a) CMNIST: Domain A (b) CMNIST: Domain B

(c) CelebA-GH: Domain A (d) CelebA-GH: Domain B

Input

Vanilla
Oracle MLDG

(a) CMNIST: Domain A

Input

Vanilla
Oracle MLDG

(c) CelebA-GH: Domain A

Input

Vanilla
Oracle MLDG

(b) CMNIST: Domain B

Input

Vanilla
Oracle MLDG

(d) CelebA-GH: Domain B


Figure 4: CMNIST and CelebA-GH

D EXTRA EXPERIMENTS

D.1 STYLE TRANSFER

Table 6: CelebA

_Z1 acc._ _Z2 acc._ FID

Vanilla 75.3 87.0 36.2

↓

_wx_ _w1-MLDG_ 81.8 93.5 35.2
_wx_ _w1-TRM_ 76.3 92.5 33.5
_wx ∖_ _w1-Fish_ 74.2 93.3 33.1
∖

IS-Oracle 73.3 88.1 36.5

∖

_wx_ _w1-Oracle_ **84.0** **95.2** 38.5

We show transferred samples for both domains in Fig. ∖ 4 on CMNIST and CelebA-GH.


-----

D.2 STYLE TRANSFER ON CELEBA

Unlike the split in CelebA-GH dataset, domain A is males, and domain B is females. The two domains
together form the full CelebA (Liu et al., 2015) with 202599 images. Note that there exists large
imbalance of hair colors within the male group, with only around 2% males having blond hair. Table 6
reports the Z1/Z2 accuracies and FID scores on the full CelebA datasets. We find that our method
improves the style transfer on the original CelebA dataset, especially the Z2 accuracy. It shows that our
method can help style transfer in real-world datasets with multiple varying factors.

E EXTRA EXPERIMENT DETAILS

E.1 STYLE TRANSFER

E.1.1 DATASET DETAILS

We randomly sampled two digits colors and two background colors for CMNIST. For CelebA-GH, we
extract two subsets—male with non-blond hair and female with blond hair—by the attributes provided in
CelebA dataset. For all datasets, we use 0.8/0.2 proportions to split the train/test set.

[For CelebA dataset, following the implementation of Zhu et al. (2017) (https://github.com/](https://github.com/junyanz/CycleGAN)
[junyanz/CycleGAN), we use 128-layer UNet as the generator and the discriminator in PatchGAN.](https://github.com/junyanz/CycleGAN)
For CMNIST dataset, we use a 6-layer CNN as the generator and a 5-layer CNN as the discriminator.
We adopt Adam with learning rate 2e-4 as the optimizer and batch size 128/32 for CMNIST/CelebA.

E.1.2 DETAILS OF MODELS

We craft datasets for training the oracle w1 _x_ Pr _Y_ _z1_ _x_ . For CMNIST, oracle dataset has bias
degrees 0.9 0. For CelebA-GH, oracle dataset has {male, female} as classes and the hair colors under
each class are balanced. ( ) = ( ∣ ( ))
/

For each dataset, we construct two environments to train the domain generalization models. For CMNIST,
the two environments are datasets with bias degrees 0.9 0.4 and 0.9 0.8. For CelebA-GH, the two
environments consist of different groups of equal size: one is {non-blond-haired males, blond-haired
females and the other is non-blond-haired males, blond-haired females, non-blond-haired females,/ /
_}_ _{_
blond-haired males . Note that the facial characteristic of gender (Z1) is the invariant prediction
_}_
mechanism across environments, instead of the hair color.

E.1.3 EVALUATION FOR Z1/Z2 ACCURACY

To evaluate the accuracies on Z1, _Z2 latents,_ we train two oracle classifiers w1[∗][(][x][)]
Pr _Y_ _z1_ _x_ _,w2[∗][(][x][) =][ Pr][(][Y][ ∣][z][2][(][x][))][ on two designed datasets. Specifically,][ w][1][/][w][2]_ [are trained on]
datasets 1/ 2 with Z1/Z2 as the only discriminative direction. For CMNIST, 1 is the dataset with=
bias degrees( ∣ ( )) 0.9 0 and 2 is the dataset with bias degrees 0 0.8. For CelebA-GH, 1 is the dataset
with {male, female D D _} as classes and the hair colors under each class are balanced. D_ 2 is the dataset with
blond hair, non-blond hair/ D as classes and the males/females under each class are balanced./ D
_{_ _}_
D

The Z1 accuracy on dataset for transfer model G is:

_Z1 accuracy D_ [1] I arg max _w1[∗][(][x][) /] arg max_ _w1[∗][(][G][(][x][)))]_

_x_ _y_ _y_

And Z2 accuracy is: = ∈D ( =

∣D∣ [∑]

_Z1 accuracy_ [1] I arg max _w2[∗][(][x][) =][ arg max]_ _w2[∗][(][G][(][x][)))]_

_x_ _y_ _y_

where I is the indicator function. = Z1 accuracy measures the success rate of transferring the∈D ( _Z1 and Z2_

∣D∣ [∑]

accuracy measures the success rate of keeping Z2 aspects unchanged.

E.2 DOMAIN ADAPTATION

The full names for the seven datasets are MNIST, MNIST-M, SVHN, GTSRB, SYN-DIGITS (DIGITS),
SYN-SIGNS (SIGNS), CIFAR-10 (CIFAR) and STL-10 (STL). We sub-sample each dataset by the
designated imbalance ratio to construct the pairs.


-----

We use the 18-layer neural network in Shu et al. (2018) with pre-process via instance-normalization.
Following Shu et al. (2018), smaller CNN is used for digits (MNIST, MNIST-M, SVHN, DIGITS) and
traffic signs (SIGNS, GTSRB) and larger CNN is used for CIFAR-10 and STL-10. We use the Adam
optimizer and hyper-parameters in Appendix B in Shu et al. (2018) except for MNIST MNIST-M,
where we find that setting λs 1,λd 1e 1 largely improves the VADA performance over label
shifts. Furthermore, we set λd 1e 2, i.e., the default value suggested by Shu et al. (2018 → ), to enable
adversarial feature alignment. = = −
= −

E.3 FAIRNESS

The two datasets are: the UCI Adult dataset[2] which has gender as the sensitive attribute; the UCI German
credit dataset[3] which has age as the sensitive attribute. We borrow the encoder, decoder and the final fc
layer from Song et al. (2019). For fair comparison, we use the same encoder+fc layer as the function
family of our classifier since our method do not need decoder for reconstruction. We use Adam with
learning rate 1e-3 as the optimizer, and a batch size of 64.
W

The original ReBias method trains a set of biased models to learn the de-biased representations (Bahng
et al., 2020), such as designing CNNs with small receptive fields to capture the texture bias. However, in
the fairness experiments, the bias is explicitly given, i.e., the sensitive attribute. For fair comparison, we
set the HSIC between the sensitive attribute and the representations as the de-bias regularizer.

F LEARNING A DIVERSE SET OF CLASSIFIERS

In this section we discuss the difference and advantages of the classifier orthogonalization, compared
to methods that learn a diverse set of classifiers. We also provide a counter-example to show that the
orthogonal input gradient does not lead to a pair of orthogonal classifiers. We show that orthogonal
classifiers might not in optimal solution set of minimizing input gradient penalty.

The goals of learning diverse classifiers include interpretability (Ross et al., 2017), overcoming simplicity
bias (Teney et al., 2021), improving ensemble performance (Ross et al., 2020), and recovering confounding decision rules (Ross et al., 2018). There is a direct trade-off between diversity and accuracy and
this is controlled by the regularization parameter. In contrast, in our work, given the principal classifier,
our method directly constructs a classifier that uses only the orthogonal variables for prediction. There
is no trade-off to set in this sense. We also focus on novel applications of controlling the orthogonal
directions, such as style transfer, domain adaptation and fairness. Further, our notion of orthogonality is
defined via latent orthogonal variables that relate to inputs via a diffeomorphism (Definition 1) as opposed
to directly in the input space. Although learning diverse classifiers through input gradient penalties
is efficient, it does not guarantee orthogonality in the sense we define it. We provide an illustrative
counter-example in Appendix F, where the input space is not disentangled nor has orthogonal input
gradients but corresponds to a pair of orthogonal classifiers in our sense. We also prove that, given the
principal classifier, minimizing the loss by introducing an input gradient penalty would not necessarily
lead to an orthogonal classifier.

We note that one clear limitation of our classifier orthogonalization procedure is that we need access to
the full classifier. Learning this full classifier can be impacted by simplicity bias (Shah et al., 2020) that
could prevent it from relying on all the relevant features. We can address this limitation by leveraging
previous efforts (Ross et al., 2020; Teney et al., 2021) to mitigate simplicity bias when training the full
classifier.

F.1 RELATIONSHIP TO ORTHOGONAL INPUT GRADIENTS

The orthogonality constraints on input gradients demonstrate good performance in learning a set of
diverse classifiers (Ross et al., 2017; 2018; 2020; Teney et al., 2021). They typically use the dot product
of input gradients between pairs of classifiers as the regularizer. We highlight that when facing the latent
orthogonal random variables, the orthogonal gradient constraints on input space no longer guarantees to
learn the orthogonal classifier. We demonstrate it on a simple non-linear example below.

Consider a binary classification problem with the following data generating process. Label Y is uniformly
distributions in 1, 1 . Conditioned on the label, we have two independent k-dimensional latents

2https://archive.ics.uci.edu/ml/datasets/adult {− }
3https://archive.ics.uci.edu/ml/datasets/Statlog+%28German+Credit+Data%29


-----

_Z1,Z2 such that Z1_ _Y_ _y_ _yµ1,Ik_, Z2 _Y_ _y_ _yµ2,Ik_ . µ1 R[k],µ2 R[m] are the
means of the latent variables, and k _m. The data X is generated from latents via the following_
∣ = ∼N(Z2 ) ∣ = ∼N( ) ∈ ∈

diffeomorphism X _g_ _Z1_ 0 _k_ _m >_

_Z2_

( −( ( − )[))]⎞

We can see that Z1 =,Z[⎛]2 are mutually orthogonal w.r.t the distribution pXY . Now consider the Bayes
optimal classifiers w1⎝,w2 using variable Z⎠ [where]1,Z2. We have[ g][ is an arbitrary non-linear diffeomorphism on] w1 _x_ _pY_ _Z1_ 1 _z1_ _σ_ 2µ[T]1 _[z][1][)][ and][ R][k][.]_
_w2_ _x_ _pY_ _Z2_ 1 _z2_ _σ_ 2µ[T]2 _[z][2][)][, where][ σ][ is the sigmoid function. Apparently, they are a pair of] ∣_
orthogonal classifiers. Besides, we denote the classifier based on the first( ) = _m(_ ∣ dimensions in) = ( _z1 for_
∣
prediction as( ) = _w3(, i.e∣_ _.) =, w3_ _x(_ _pY_ _Z1_ _m_ 1 _z1_ _m_ _σ_ 2 _µ1_ _m[(][z][1][)][∶][m][)][.]_
∶

Given the principal classifier w1, we consider using the loss function in ∣( ) ∶ ∶ Ross et al. (2018; 2020) to learn

( ) = ( ∣( ) ) = ( ( )[T]

the orthogonal classifier of w1, by adding a input gradient penalty term to the standard classification loss:

_w1_ _w_ _c_ _w_ _λE_ cos[2] _xw1_ _x_ _,_ _xw_ _x_

_c is the expected cross-entropy loss, cos is the cosine-similarity and λ is the hyper-parameter for the_

L ( ) = L ( ) + [ (∇ ( ) ∇ ( ))]

gradient penalty term (Ross et al., 2018; 2020). Below we show that (1) the orthogonal classifier w2
Ldoes not necessarily satisfy E cos[2] _xw1_ _x_ _,_ _xw2_ _x_ 0. (2) the orthogonal classifier w2 is not
necessarily the minimizer of _w1_ _w_ .
**Proposition 6. For the above[** (∇Z1,Z2 (, their corresponding Bayes classifiers) ∇ ( ))] = _w1,w2 satisfy_
L ( )
E _xw1_ _x_ _xw2_ _x_ 0 if and only if _µ1_ _m[µ][2]_
( )
∶

_Proof.[∇_ Let( _X)[T]1∇_ _g_ _Z(_ 1)] =0 _kZ2m_ ( )[T] [=][ 0][.]

_x2_

= ( −( ( − )[))][,][ X][2][ =][ Z][2][ are the two components of][ X][. Note that][ z][1][ =][ g][−][1][(][x][1][) +]

0 _k_ _m_
( ( − )[)][. By chain rule, the input gradient of][ w][1][ is]

_xw1_ _x_ _[dz][1]_

_dx_

∇ ( ) = [∇][z][1][ Pr][(][Y][ =][ 1][∣][Z][1][ =][ z][1][(]e[x][−][))][2][µ]1[T] _[z][1][(][x][)]_

_Im_ _k_ _k_ _m_ _k_ 1 _e[−][2][µ]1[T]_ _[z][1][(][x][)]_
= ([∇][x][1] _[g][−]×[1][(][x][1][)])(_ + )× 0 _e[−][2][µT]2_ _[z][2]_ [(][x][)]

where Im _k is the submatrix of Ik_ _k. Similarly we get_ _xw(2_ +x _Im_ _m[)])[2]k[ ⋅]m[2][µ][1]m_ 1 _e[−][2][µT]2_ _[z][2]_ [(][x][)]

Together, the dot product between input gradients is× × ∇ ( ) = ( × ( + )× ( + )[2][ ⋅][2][µ][2][.]

_xw1_ _x_ _xw2_ _x_ 12e[−] e[2][µ][−]1[T][2][µ][z]1[T][1][(][z][x][1][)][(]µ[x][)][T]1 _Im_ _k_ _Tk_ _m_ _k_ _Im0_ _m[)]_ _k_ _m_ _m_ 12e[−] e[2][−][µ][2]2[T][µ][z]2[T][2][(][z][x][2][)][(][x]µ[)]2
∇ ( )[T] ∇ ( ) = 4e[−][2][µ]1[T] _[z][1][(][x][)−][2][µ]2[T]_ _[z][2][(][x][)]_ _µ×1_ _m[µ])[2](_ + )× ( × ( + )×

( + )[2][ (∇][x][1] _[g][−][1][(][x][1][)]_ ( + )[2]

1 _e[−][2][µ]1[T]_ _[z][1][(][x][)]_ 1 _e[−][2][µ]2[T]_ ∶[z][2][(][x][)]

= ( )[T]

4e[−][2][µT]1 _[z][1]_ [(][x][)−][2][µT](2 _[z] +[2]_ [(][x][)] )[2]( + )[2]
Since _m[µ][2]_

1 _e[−][2][µT]1_ _[z][1]_ [(][x][)] 1 _e[−][2][µT]2_ _[z][2]_ [(][x][)]
∶
( + )[2]( + )[2][ >][ 0][,][ E][[∇][x][w][1][(][x][)][T][ ∇][x][w][2][(][x][)]][ if and only if][ (][µ][1][)][T] [=][ 0][.]

Proposition 6 first shows that the expected input gradient product of the two Bayes classifiers of underlying
latents is non-zero, unless the linear decision boundaries on z1,z2 are orthogonal. Hence, given the
principal classifier w1, the expected dot product of input gradients between w1 and its orthogonal
classifier w2 does not have to be zero.
**Proposition 7. The orthogonal classifer w2 does not minimize** _w1_ _w_ _if_ _µ1_ _m_ _µ2 and_
_µ[T]1_ 1 _[g][−][1][(][x][1][)][T][ ∇][x]1_ _[g][−][1][(][x][1][)][k][×][m][(][µ][1][)][∶][m]_ 1 [(][w][3][) < L]∶[w]1 [(][w][2][)][ in this]
_case._ L ( ) ( ) =

[∇][x] [<][ 0][. Particularly, we show that][ L][w]

_Proof. Let w3_ _x_ _pY_ _Z1_ _m_ 1 _z1_ _m_ . The input gradient of w3 _x_ is
∶
∣( ) ∶

_xw3_ _x_ ( ) =xw1 _x_ _m(_ ∣([dz])[1] ) ( )

_dx_

∇ ( ) = (∇ ( ))∶ = [∇][(][z][1][)][∶][m][ Pr][(][Y][ =][ 1][∣(][Z][1][)][∶][m][ = (][z]e[1][−][(][2][x][(][))][µ][1][∶][)][m][T]∶m[)][z][1][(][x][)][∶][m]

_Im_ _m_ _k_ _m_ _m_ 1 _e[−][2][(][µ][1][)][T]m[z][1][(][x][)][∶][m]_
= ([(∇][x][1] _[g][−][1][(]×[x][1][))][k][×][m])(_ + )× ∶

( + )[2][ ⋅] [2][(][µ][1][)][∶][m]


-----

When _µ1_ _m_ _µ2, the dot-product between_ _xw3_ _x_ _,_ _xw1_ _x_ is
( )∶ = 2e[−][2][µ]1[T] _[z][1][(][x][)]µ[T]1_ ∇ ( ) ∇T ( )

_xw1_ _x_ _xw3_ _x_ 1 _e[−][2][µ]1[T]_ _[z][1][(][x][)]_ _Im_ _k_ _k_ _m_ _k_
∇ ( )[T] ∇ ( ) = × )( +e[−])×[2][(][µ][1][)][T]m[z][1][(][x][)][∶][m]

( + )[2][ (∇][x][1] _[g][−][1][(][x][1][)]_ ∶

_Im_ _m_ _k_ _m_ _m_ 1 _e[−][2][(][µ][1][)][T]m[z][1][(][x][)][∶][m]_
([(∇][x]4[1]e[g][−][−][2][1][µ][(]1[T]×[x][z][1][1][(][))][x][)−][k][×][2][m][(][µ])[1]([)][T]m+[z][1])×[(][x][)][∶][m] ∶

( + )[2][ ⋅] [2][(][µ][1][)][∶][m]

∶ 2 1 1 _[g][−][1][(][x][1][)][T][ ∇][x]1_ _[g][−][1][(][x][1][)][k][×][m][(][µ][1][)][∶][m][)]_

1 _e[−][2][µ]1[T]_ _[z][1][(][x][)]_ 1 _e[−][2][(][µ][1][)][T]∶m[z][1][(][x][)][∶][m]_

When _µ1_ _m_ _µ=2, we know that z1_ _x_ _,z2_ _x_ are equally predictive of the label and[+][µ][T] [∇][x]

( + )[2]( + )[2][ (∥] _[µ][2][ ∥][2]_

thus _c_ _w3_ ∶ _c_ _w2_ . Note that if µ[T]1 1 _[g][−][1][(][x][1][)][T][ ∇][x]1_ _[g][−][1][(][x][1][)][k][×][m][(][µ][1][)][∶][m]_
E cos[2] ( _xw)_ 1 _x_ =, _xw2_ _x_ E cos[2] _xw(1_ )x _,_ (xw) 3 _x_ :
L ( ) = L ( ) [∇][x] [<][ 0][, we have]

[ (∇ (E )cos∇[2] _x(w1))] >x_ _,_ [xw2 (∇x ( ) ∇ ( ))]

2

[ (∇ ( ) ∇ ( ))]

_µ2_ 2

⎡ ⎤

E ⎢⎛ ⎞ ⎥

⎢ ⎥

= ⎢⎢⎢⎜⎜⎜ _I_ _k_ _m_ ∥ _k_ _µ ∥1_ [2] 2 _I[)]_ _k_ _m_ _m_ _µ2_ 2 ⎟⎟⎟ ⎥⎥⎥

⎢⎜ ⎟ ⎥ 2
⎢ ∥([∇][x][1] _[g][−][1][(][x][1][)])(_ + )× ∥ ∥([0] ( + )× ∥ ⎥
⎢⎝ ⎠ ⎥
⎣ ⎦

_µ2_ 2 1 1 _[g][−][1][(][x][1][)][T][ (∇][x]1_ _[g][−][1][(][x][1][))][k][×][m][µ][2][)]_

⎡ ⎤

E ⎢⎛ ⎞ ⎥

⎢ ⎥

> ⎢⎢⎢⎜⎜⎜ (∥ _I_ ∥[2] [+][µ]k[T] [∇]m[x] _k_ _µ1_ 2 _Im_ _k_ _k_ _m_ _k_ _µ2_ 2 ⎟⎟⎟ ⎥⎥⎥

⎢⎜ ⎟ ⎥

E ⎢⎢cos⎝ ∥([2] [∇]x[x]w[1] _[g]1[−][1]x[(][x],[1][)])x(w3+_ _x)×_ ∥ ∥([∇][x][1] _[g][−]×[1][(][x][1][)])(_ + )× ∥ ⎠ ⎥⎥

⎣ ⎦

The strict inequality holds by= [ (∇ _µ([T]1_ ) ∇1 _[g][−][1][(][x]([1][)]))][T][ ∇][x]1_ _[g][−][1][(][x][1][)][k][×][m][(][µ][1][)][∶][m]_

_Im_ _k_ _k_ _m_ _k_ _µ2_ 2[∇][x] 0 _k_ _m_ _k_ _µ2_ _Im[0]_ [<]k[ 0][)] _k_ _m_ _k_ _µ2_ 2

1

([∇][x][1] _[g][−]×[1][(][x][1][)])(_ + )× ∥ =∥([∇][x][1] _[g][−][1][(][x][1][)])(_ + )× + ( × ( + )× ∥ 2

_µ2_ 2 _µ2_ 2[)]
0 _k_ _m_ _k_ _Im_ _k[)]_ _k_ _m_ _k_
= (∥([∇][x][1] _[g][−][1][(][x][1][)])(_ + )× ∥[2] [+ ∥(][ 0]× ( + )× ∥[2]

_Im[0]_ _k[)]_ _k_ _m_ _k_ _µ2_ 2
≥∥( × ( + )× ∥

To verify that µ[T]1 1 _[g][−][1][(][x][1][)][T][ ∇][x]1_ _[g][−][1][(][x][1][)][k][×][m][(][µ][1][)][∶][m]_

2 3 3 1
1 1 5 [∇][x] 1 1 1 _[g][−][1][(][x][1][)][T][<][ ∇][ 0][x][ does exist, pick]1_ _[g][−][1][(][x][1][)][k][×][m][(][µ][1][ k][)][∶][m][ =][ 3][,m][ =][ 2][,][ g][−][1][(][x][) =]_
1 −1 1 1

⎛ ⎞ ⎞

− [∇][x] [= −][2][ in this case.]

Together, we have⎝ ⎠ _[x][, and][ µ]w1[1][ = ⎛]w⎝3_ ⎠[. We have]w1 _w2_ _[ µ]._ _[T]_

−

Proposition 7 shows that the orthogonal classifier L ( ) < L ( ) _w2 is not the minimizer of the loss with input gradient_
penalty. The results above also hold for un-normalized version of input gradient penalty in Teney et al.
(2021) by similar analysis.


-----

