# LOCALLY INVARIANT EXPLANATIONS: TOWARDS CAUSAL EXPLANATIONS THROUGH LOCAL INVARI## ANT LEARNING

**Anonymous authors**
Paper under double-blind review

ABSTRACT

Locally interpretable model agnostic explanations (LIME) method is one of the
most popular methods used to explain black-box models at a per example level.
Although many variants have been proposed, few provide a simple way to produce
high fidelity explanations that are also stable and intuitive. In this work, we
provide a novel perspective by proposing a model agnostic local explanation
method inspired by the invariant risk minimization (IRM) principle – originally
proposed for (global) out-of-distribution generalization – to provide such high
fidelity explanations that are also stable and unidirectional across nearby examples.
Our method is based on a game theoretic formulation where we theoretically show
that our approach has a strong tendency to eliminate features where the gradient
of the black-box function abruptly changes sign in the locality of the example we
want to explain, while in other cases it is more careful and will choose a more
conservative (feature) attribution, a behavior which can be highly desirable for
recourse. Empirically, we show on tabular, image and text data that the quality of
our explanations with neighborhoods formed using random perturbations are much
better than LIME and in some cases even comparable to other methods that use
realistic neighbors sampled from the data manifold, where the latter is a popular
strategy to obtain high quality explanations. This is a desirable property given that
learning a manifold to either create realistic neighbors or to project explanations is
typically expensive or may even be impossible. Moreover, our algorithm is simple
and efficient to train, and can ascertain stable input features for local decisions of a
black-box without access to side information such as a (partial) causal graph as has
been seen in some recent works.

1 INTRODUCTION

Deployment and usage of neural black-box models has significantly grown in industry over the last
few years creating the need for new tools to help users understand and trust models (Gunning, 2017).
Even well-studied application domains such as image recognition require some form of prediction
understanding in order for the user to incorporate the model into any important decisions (Simonyan
et al., 2013; Lapuschkin et al., 2016). An example of this could be a doctor who is given a cancer
diagnosis based on an image scan. Since the doctor holds responsibility for the final diagnosis, the
model must provide sufficient reason for its prediction. Even new text categorization tasks (Feng
et al., 2018) are becoming important with the growing need for social media companies to provide
better monitoring of public content. Twitter recently began monitoring tweets related to COVID-19
in order to label tweets containing misleading information, disputed claims, or unverified claims
(Roth & Pickles, 2020). Laws will likely emerge requiring explanations for why red flags were or
were not raised in many examples. In fact, the General Data Protection and Regulation (GDPR)
(Yannella & Kagan, 2018) act passed in Europe already requires automated systems that make
decisions affecting humans to be able to explain them. Given this acute need, a number of methods
have been proposed to explain local decisions (i.e. example specific decisions) of classifiers (Ribeiro
et al., 2016; Lundberg & Lee, 2017; Simonyan et al., 2013; Lapuschkin et al., 2016; Dhurandhar
et al., 2018a). Locally interpretable model-agnostic explanations (LIME) is arguably the most
well-known local explanation method that requires only query (or black-box) access to the model.


-----

Although LIME is a popular method, it is known to be sensitive to certain design choices such as
i) (random) sampling to create the (perturbation) neighborhood[1], ii) the size of this neighborhood
(number of samples) and iii) (local) fitting procedure to learn the explanation model (Molnar,
2019; Zhang et al., 2019b). The first, most serious issue could lead to nearby examples having
drastically different explanations making effective recourse a challenge. One possible mitigation
is to increase the neighborhood size, however, one cannot arbitrarily do so as it not only leads to
higher computational cost, but in today’s cloud computing-driven world it could have direct monetary
implications where every query to a black-box model has an associated cost (Dhurandhar et al., 2019).

There have been variants suggested to overcome some of
these limitations (Botari et al., 2020; Shrotri et al., 2021; Coefficient inconsistency for LIME
Plumb et al., 2018) primarily through mechanisms that
create realistic neighborhoods or through adversarial train- 4.0
ing (Lakkaraju et al., 2020), however, their efficacy is 3.5
restricted to certain settings and modalities based on their
assumptions and training strategies. 3.0


In this paper we introduce a new method called Locally 2.5
INvariant EXplanations (LINEX) inspired by the invari- 2.0
ant risk minimization (IRM) principle (Arjovsky et al., 4.5 5.0 5.5 6.0 6.5 7.0 7.5 8.0
2019), that produces feature based explanations that are sepal length (cm)
robust to neighborhood sampling and can recover faithful 4.5 Coefficient inconsistency for LINEX
(i.e. mimic black-box behavior), stable (i.e. similar for
closeby examples) and unidirectional (i.e. same sign attri- 4.0
butions for closeby examples, see section 4.1) explanations 3.5
across tabular, image, and text modalities. In particular, we
show that our method performs better than the competitors 3.0
for random as well as realistic neighborhood generation, sepal width (cm)2.5
where in some cases even with the prior strategy our explanation quality is close to methods that employ the latter. 2.0
Qualitatively, our method highlights (local) features as sepal length (cm)
important that in the particular locality i) have consistently

Coefficient inconsistency for LIME

4.5

4.0

3.5

3.0

sepal width (cm)2.5

2.0

4.5 5.0 5.5 6.0 6.5 7.0 7.5 8.0

sepal length (cm)

Coefficient inconsistency for LINEX

4.5

4.0

3.5

3.0

sepal width (cm)2.5

2.0

4.5 5.0 5.5 6.0 6.5 7.0 7.5 8.0

sepal length (cm)

high gradient with respect to (w.r.t.) the black-box function Figure 1: Above we visualize for the
and ii) where the gradient does not change significantly, IRIS dataset the Coefficient Inconsisespecially in sign. Such stable behavior for LINEX is tency (CI) (see Section 5 for exact defvisualized in Figure 1, where we get similar explanations inition and setup details) between the
for nearby examples in the IRIS dataset. The (in)fidelity explanation (top two features) for an exof LINEX is still similar to LIME (see Table 3), but of ample and its nearest neighbor in the
course our explanations are much more stable. dataset. Each circle denotes an exam
ple and a rainbow colormap depicts the

2 RELATED WORK degree of inconsistency w.r.t. its nearest

neighbor where red implies least incon
Posthoc explanations can typically be partitioned into two sistency, while violet implies the most.
broad categories global and local. Global explainabil- As can be seen LINEX explanations are
ity avers to trying to understand a black-box model at a much more consistent than LIME’s.
holistic level where the typical tact is knowledge transfer
(Hinton et al., 2015; Dhurandhar et al., 2018b; 2020) where (soft/hard) labels of the black-box model
are used to train an interpretable model such as a decision tree or rule list (Rudin, 2019). Local
explanations on the other hand avers to understanding individual decisions. These explanations are
typically in two forms, either exemplar based or feature based. For exemplar based as the name
suggests similar but diverse examples (Kim et al., 2016; Gurumoorthy et al., 2019) are provided as
explanations for the input in question. While for feature based (Ribeiro et al., 2016; Lundberg & Lee,
2017; Dhurandhar et al., 2018a; Lapuschkin et al., 2016), which is the focus of this work, important
features are returned as being important for the decision made for the input. There are some methods
that do both (Plumb et al., 2018). Moreover, there are methods which provide explanations that are
local, global as well as at a group level (Ramamurthy et al., 2020). All of these methods though may
not still provide stable and robust local feature based explanations which can be desirable in practice
(Ghorbani et al., 2019).

1By perturbation neighborhood – referred to as simply neighborhood – we mean neighborhoods generated
for local explanations. By exemplar neighborhood, we mean nearest examples in a dataset to a given example.


-----

Given this there have been more recent works that try to learn either robust or even causal explanations.
In (Lakkaraju et al., 2020) the authors try to learn robust and stable local explanations relative to
distribution shifts and adversarial attacks. However, the distribution shifts they consider are linear
shifts and adversarial training is performed which can be slow and sometimes unstable (Zhang et al.,
2019a). Moreover, the method seems to be applicable primarily to tabular data. Works on causal
explanations (Frye et al., 2020; Heskes et al., 2020) mainly modify SHAP and assume access to a
partial causal graph. Some others (Vig et al., 2020) assume white-box access. In this work we do not
assume availability of such additional information. There are also works which show that creating
realistic neighborhoods by learning the data manifold for LIME (Botari et al., 2020; Shrotri et al.,
2021) can lead to better quality explanations, where in a particular work (Anders et al., 2020) it is
suggested that projecting explanations themselves on to the manifold can also make them more robust.
The need for stability in a exemplar neighborhood for LIME like methods has been highlighted in
(Zhang et al., 2019b), with the general desire for stable explanations being also expressed in prior
works (Yeh et al., 2019; Visani et al., 2020).

3 PRELIMINARIES

**Invariant Risk Minimization: Given a collection of training datasets D = {De}e∈Etr gathered**
from a set of environments Etr, where De = {x[i]e[, y]e[i] _[}][n]i=1[e]_ [is the dataset gathered from environment]
_e ∈Etr and ne is the number of points in environment e. The feature value for data point i is_
**_xin environment[i]e_** _[∈X][ and the corresponding label is] e is drawn i.i.d from a distribution[ y]e[i]_ _[∈Y][, where] Pe. Define a predictor[ X ⊆]_ [R][d][ and][ Y ⊆] f[R] :[. Each point] X → R. The goal[ (][x]e[i] _[, y]e[i]_ [)]
of IRM is to use these collection of datasets D to construct a predictor f that performs well across
many unseen environments _all, where_ _all_ _tr. Define the risk achieved by f in environment e as_
_Re(f_ ) = Ee _ℓ(f_ (Xe), Ye) E, where ℓ is the square loss when E _⊇E_ _f_ (Xe) is the predicted value and Ye is
the corresponding label, (Xe, Ye) Pe and the expectation Ee is defined w.r.t. the distribution of
  _∼_
points in environment e.

An invariant predictor is composed of two parts a representation Φ ∈ R[d][×][n] and a predictor (with the
constant term) w ∈ R[d][×][1]. We say that a data representation Φ elicits an invariant predictor w[T]Φ
across the set of environments Etr if there is a predictor w that achieves the minimum risk for all
the environments w arg minw ˜ R[d][×][1][ R][e][( ˜]w[T]Φ), _e_ _tr. IRM may be phrased as the following_
_∈_ _∈_ _∀_ _∈E_
constrained optimization problem (Arjovsky et al., 2019):


_Re(w[T]Φ)_ s.t. w arg min **_w[T]Φ),_** _e_ _tr_ (1)
_eX∈Etr_ _∈_ **_w˜∈R[d][×][1][ R][e][( ˜]_** _∀_ _∈E_


min
**Φ∈R[d][×][n],w∈R[d][×][1]**


If w[T]Φ solves the above, then it is an invariant predictor across the training environments _tr._
_E_

**Local explainability setup vs IRM setup: There are multiple differences between IRM setup and**
the setup we have in this paper. First, IRM is meant to learn global models directly from the dataset.
This also applies to the game theoretic version proposed in (Ahuja et al., 2020), while for local
explainability we are trying to learn local models that explain (per example) a given black-box model.
Second, given that we want to explain a black-box model which typically is a function of the input
features, we want to highlight features in our explanation that may be spurious from the domain
perspective, but nonetheless the black-box model uses them to make decisions. Third, the IRM
model does not have to be interpretable as its learned representation Φ can be arbitrary. Fourth, the
IRM learning procedure is quite inefficient as it tries to solve a bi-level optimization problem with a
highly non-convex constraint. Fifth, IRM assumes that one has access to multiple environments (viz.
multiple data sources), but in our case we have to figure out how to appropriately create them for
each example that we want to explain. Last, since a black-box model is trained on the input features
all or a subset of them are causal to its prediction, unlike the standard IRM setup where a predictor is
expected to model the true underlying mechanism. Hence in our case, it is sufficient to limit ourselves
to some interpretable representation, which in many cases is just the input feature space (i.e. Φ = X ).

Thus IRM cannot be directly applied to our problem and we propose a novel game theoretic approach
that is suitable for local explainability consistent with the above points. With our approach it is
interesting that we are able to obtain similar style theoretical results for our setting that have been
seen in out-of-distribution generalization type of works (Ahuja et al., 2021), thus justifying our choice
of leveraging such frameworks.


-----

**Nash Equilibrium (NE): To understand how certain key aspects of our method function let us revisit**
the notion of Nash Equilibrium (Dutta, 1999). A standard normal form game is written as a tuple
Ω= ( _,_ _ui_ _i_ _,_ _i_ _i_ ), where is a finite set of players. Player i takes actions from a
_N_ _{_ _}_ _∈N_ _{S_ _}_ _∈N_ _N_ _∈N_
strategy set Si. The utility of player i is ui : S → R, where we write the joint set of actions of all the
players asi is si and the strategy of the rest of players is S = Πi∈N Si. The joint strategy of all the players is given as s−i = (si[′] )i[′] ≠ _i._ **_s ∈S, the strategy of player_**

**Definition 1. A strategy s[†]** _∈S is said to be a pure strategy Nash equilibrium (NE) if it satisfies_

_ui(s[†]i_ _[,][ s][†]_ _i[)][ ≥]_ _[u][i][(][k,][ s][†]_ _i[)][,][ ∀][k][ ∈S][i][,][ ∀][i][ ∈N]_
_−_ _−_

NE thus identifies a state where each player is using the best possible strategy in response to the rest
of the players leaving no incentive for any player to alter their strategy. In seminal work by (Debreu,
1952) it was shown that for a special class of games called concave games such a pure NE always
exists. This is relevant because the game implied by Algorithm 1 falls in this category.

4 METHODOLOGY

We first define certain desirable properties we would like our explanation method to have. The
first three have been seen in previous works, while the last Unidirectionality is something new we
propose. We then describe our method where the goal is to explain a black-box model f : X → R
for individual inputs x based on predictors w by looking at their corresponding components, also
termed as feature attributions.

4.1 DESIRABLE PROPERTIES

We now discuss certain properties we would like our explainability method to have in order to provide
robust explanations that could potentially be used for recourse.

**Fidelity: This is the most standard property which all proxy model based explanation methods are**
evaluated against (Ribeiro et al., 2016; Lundberg & Lee, 2017; Lakkaraju et al., 2020) as it measures
how well the proxy model simulates the behavior of the black-box it is attempting to explain.

**Stability: This is also a popular notion (Hancox-Li, 2020; Ramamurthy et al., 2020; Yeh et al., 2019)**
to evaluate robustness of explanations. Largely, stability can be measured at three levels. One is
prediction stability, which measures how much the predictions of an explanation model change for
the same example subject to different randomizations within the method or across close by examples.
The second is the variance in the feature attributions again for the same or close by examples. It is
good for a method to showcase stability w.r.t. both even though in many cases the latter might imply
the former. An interesting third notion of stability is the correlation between the feature attributions
of an explanation model and average feature values of examples belonging to a particular class. This
measures how much does the explanation method pick features that are important for the class, rather
than spurious ones that seem important for the example of interest.

**Black-box Invariance: This is the same as implementation invariance defined in (Sundararajan et al.,**
2017). Essentially, if two models have exactly the same behavior on all inputs then their explanations
should also be the same. Since, our method is model agnostic with only query access to the model it
is easy to see that it satisfies this property if the same environments are created.

**Unidirectionality: This is a new property, but as we argue that this is a natural one to have. Loosely**
speaking, unidirectionality would measure how consistently the sign of the predictor for a feature is
maintained for the same or close by examples by an explanation method. This is a natural metric
(Miller, 2018), which from an algorithmic recourse (Karimi et al., 2021) perspective is also highly
desirable. For instance, recommending a person to increase their salary to get a loan and then
recommending to another person with a very similar profile to decrease their salary for the same
outcome makes little sense.

We define the unidirectionality Υ as a measure of how consistent the sign of the attribution for a
particular feature in a local explanation is when varying neighborhoods for the same example or when
considering different close by examples. In particular, given m attributions for each of d features


-----

**Algorithm 1: Locally Invariant EXplanations (LINEX) method.**
**Input: example to explain x, black-box predictor f** (.), number of environments to be created k,
(l∞) threshold γ > 0, (l1) threshold t > 0 and convergence threshold ϵ > 0
**Initialize: ∀i ∈{1, ..., k} ˜wi = 0 and ∆= 0**
Let ξ1(.), ..., ξk(.) be k environment creation functions as described in section 4.2.2
**do**

**for i = 1 to k do**

**_w˜_** [+]i [=][ P]j 1,...,k _,j=i_ **_w[˜]j_**
_−_ _∈{_ _}_ _̸_

**_w˜i[prev]_** = ˜wi

2

**_w˜i = arg minw˜_** **_x˜∈ξi(x)_** _f_ (˜x) − **_w˜−[+]i[T]_** **_x[˜] −_** **_w˜_** [T]x˜ s.t. | ˜w−[+]i [+ ˜]w|1 ≤ _t and | ˜w|∞_ _≤_ _γ_

∆= max ( ˜wi[prev]P **_w˜i_** 2, ∆) 
_|_ _−_ _|_
**end**
**while ∆** _≥_ _ϵ;_
**Output: w =** _i_ 1,...,k **_w[˜]i_**

_∈{_ _}_

[P]

denoted by w1[(1)][, ..., w]m[(][d][)] [the unidirectionality metric for an example is:]


sgn _wj[(][i][)]_
_j=1_

X 


(2)


Υ =


_md_


_i=1_


where |.| stands for absolute value. Clearly, the more consistent the signs for the attribution of a
particular feature across m realizations/explanations the higher the value, where the maximum value
can be one. If equal number of attributions have different signs for all features then Υ will be zero, the
lowest possible value. This property thus measures how intuitively consistent (ignoring magnitude)
the explanations are and compliments the other properties mentioned above.

4.2 METHOD

4.2.1 DESCRIPTION

In Algorithm 1, we show the steps of our method LINEX. The input to the method is the example
we want to explain x, the black-box predictor, a few thresholds that we describe next and k (local)
environments whose creation is described in Section 4.2.2.

In the algorithm we iteratively learn a constrained least squares predictor for each environment, where
the final (local) linear predictor is the sum of these individual predictors. In each iteration when computing the contribution of environment ei to the final summed predictor, the most recent contributions
of the other predictors are summed and the residual is optimized subject to the constraints. The first
constraint is a standard lasso type constraint which tries to keep the final predictor sparse as is also
seen in LIME.

**Why l** **constraint? The second constraint is more unique and is a l** constraint on the predictor of
_∞_ _∞_
just the current environment. This constraint as we prove in Section 4.3 is essential for obtaining
robust predictors. To intuitively understand why this is the case consider we have two environments.
In this case if the optimal predictors for a feature in each environment have opposite signs, then
the Nash equilibrium (NE) is when each predictor takes +γ or −γ values as they try to force the
sum to have the same sign as them. In other words, features that have a disagreement in even the
_direction of their impact are eliminated by our method. LIME type methods on the other hand would_
simply choose some form of average value of the predictors which may be a risky choice especially
for actionability/recourse given that the directions change so abruptly. On the other hand, if the
optimal predictors for a feature in the two environments have the same sign, the lower absolute valued
predictor would be chosen (assuming γ is greater) making it a careful choice. The reasoning for this
and a discussion involving more than two environments is given in Section 4.3.

The overall algorithm resembles a (simultaneous) game where each environment is a player trying to
find the best predictor for its environment given all the other predictors and constraints. Also note
that the optimization problem solved by each player is convex as norms are convex.


-----

4.2.2 CREATING LOCAL ENVIRONMENTS

In the standard IRM framework environments are assumed to be given, however, in our case of local
explainability we have to decide how to produce them. We offer a few options for the environment
creation functions ξi _i_ 1, ..., k in Algorithm 1.
_∀_ _{_ _}_

**Random Perturbation: This possibly is the simplest approach and similar to what LIME employs.**
We could perturb the input example by adding zero mean gaussian noise to create the base environment
(used by LIME) and then perform bootstrap sampling to create the k different environments. This
will efficiently create neighbors in each environment, although they may be unrealistic in the sense
that they could correspond to low probability points w.r.t. the underlying distribution.

**Realistic Generation/Selection: One could also create neighbors using data generators such as done**
in MeLIME (Botari et al., 2020) or select neighboring examples from the training set as done in
MAPLE (Plumb et al., 2018) to create the base environment following which bootstrap sampling
could be done to form the k different environments. This approach may provide more realistic
neighbors than the previous one, but may be much more computationally expensive.

Other than bootstrapping one could also over sample and try to find the optimal hard/soft partition
through various clustering type objectives (Aggarwal & Reddy, 2013; Creager et al., 2020).

4.3 THEORETICAL RESULTS

In this section, we analyze the output of Algorithm 1 when there are two environments. The
extension to multiple environments is discussed following this result, where the general intuition is
still maintained but some special cases arise depending on whether there are an even or odd number
of environments. To prove our main result we make two assumptions.

**Assumption 1 The feature values for each of the dimensions in the samples created forming the local**
_environments are independent._

This assumption is satisfied by the most standard way of creating neighborhoods/environments, where
random gaussian noise is used to create them as described in Section 4.2.2.

**Assumption 2 t ≥** _γd, where d is the dimensionality of the feature vector._

Making this assumption ensures that we closely analyze the role of the ℓ penalty, which is one of
_∞_
the main novelties in our method.

**Definition 2 Let the explanation that each environment ξi arrives at for an example x based on**
unconstrained least squares minimization be wi[∗] [where,]

**_wi[∗]_** _[∈]_ [arg min]w˜∈R[d] Ex˜∈ξi(x)[[(][f] [(˜]x)[2] _−_ _w˜[T]x˜)[2]]_ (3)

The expectation is taken w.r.t the environment generation distribution.
**Theorem 1. The output of Algorithm 1 under Assumptions 1, 2 and equation 3 is given by:**

**_w =_** **_w1[∗]_** _|w2[∗][|≥|][w]1[∗][|][ +][ w]2[∗]_ _|w1[∗][|][>][|][w]2[∗][|]_ **1w1[∗][⊙][w]2[∗][≥][0]** (4)

_[⊙]_ **[1]** _[⊙]_ **[1]**

_where ⊙_ _is element wise product and_ **1 is the indicator function.** 

_Proof Sketch. The above expression describes the NE of the game played between the two local_
environments each trying to move w towards their least squares optimal solution. Given assumptions
1 and 2, we witness the following behavior of our method. Let the i[th] feature of the predictors ˜w1
and ˜w2 from Algorithm 1 be ˜w1i and ˜w2i respectively. Let the corresponding least squares optimal
predictors for the i[th] feature have the following relation: w1[∗]i _[> w]2[∗]i_ [and][ |][w]1[∗]i[|][ >][ |][w]2[∗]i[|][. Then the]
two environments will push the ensemble predictor, ˜w1i + ˜w2i, in opposite directions during their
turns, with the first environment increasing its weight, ˜w1i, and the second environment decreasing its
weight, ˜w2i. Eventually, the environment with a higher absolute value (ξ1 = 1 since |w1[∗]i[|][ >][ |][w]2[∗]i[|][)]
reaches the boundary ( ˜w1i = γ) and cannot move any further due to the l constraint. The other
_∞_
environment ξ2 best responds, where it either hits the other end of the boundary ( ˜w2i = −γ), in
which case the weight of the ensemble for component i is zero, a case which occurs if w1[∗]i [and][ w]2[∗]i
have opposite signs; or gets close to the other boundary while staying in the interior ( ˜w2i = w2[∗]i
in which case the weight of the ensemble for feature i is w2[∗]i[, a situation which occurs if][ w]1[∗]i [and][−][ w][γ]2[∗][),]i
have the same sign.


-----

Table 1: Datasets, models and neighborhoods used in experiments. RF→ Random Forest, NN→
Neural Network and NB→ Naive Bayes.

Dataset Modality Black-box model acc/R[2], Realistic neighborhood creation methods

IRIS tabular RF classifier, 93% KDEGen (Botari et al., 2020), RF (Plumb
et al., 2018)
MEPS tabular RF regressor, 0.325 (Plumb et al., 2018)
FMNIST image NN classifier, 87% VAEGen (Botari et al., 2020)
Rotten Tomatoes text NB classifier, 75% Word2VecGen (Botari et al., 2020)

**Implications of the Theorem 1: The following are the main takeaways from Theorem 1: (1) If the**
signs of the explanations for unconstrained least squares for the two environments differ for some
feature, then the algorithm outputs a zero as the attribution for that feature. (2) If the signs of the
explanations for the two environments are the same, then the algorithm outputs the lesser magnitude
of the two. These two properties are highly desirable from an algorithmic recourse or actionability
perspective, where the first biases us to not rely on features where the black-box function changes
direction rapidly (unidirectionality). The second, provides a reserved estimate so that we do not
incorrectly over rely on the particular feature (stability).

**Behavior for More than Two Environments: Given Assumptions 1 and 2 we now discuss the**
behavior of our method for more than two environments. If the number of environments is odd,
then using similar logic to that discussed in the proof sketch one can see that the feature attribution
would be equal to the median of the feature attributions across all the environments. Essentially,
all environments with optimal least squares attributions above the median would be at +γ, while
those below it would be at −γ. The one at the median would remain so with no incentive for any
environment to alter its attribution making it a NE. This is a stable choice that is also likely to be
faithful as we have no more information to decide otherwise. On the other hand if we have an even
number of environments the situation is a bit different. The final attribution in this case depends on
the middle two environments in the same manner as the two environment case proved in Theorem 1.
Thus, if the optimal least squares attributions of the middle two environments have opposite sign,
then the final attribution is zero, else its the lower of the two attributions in terms of the numerical
value. This happens because the NE for the other environments is +γ or −γ depending on if their
optimal least squares attributions are above or below those of the middle two environments. This
again is a stable and likely to be faithful choice, where also unidirectionality is preferred.

5 EXPERIMENTS

We test our method on four real world datasets covering all three modalities: IRIS (Tabular) (Dheeru
& Karra Taniskidou, 2017), Medical Expenditure Panel Survey (Tabular) (Agency for Healthcare
Research and Quality, 2019), Fashion MNIST (Image) (Xiao et al., 2017), and Rotten Tomatoes
reviews (Text) (Pang et al., 2002) with LIME-like random (rand) and MeLIME-like realistic neighborhood generation (real) or MAPLE-like realistic neighborhood selection (mpl). The summary of
black-box classifier accuracies, and type of realistic perturbation used for the datasets are provided in
Table 1. In all cases except FMNIST which comes with its own test partition we randomly split the
datasets into 80/20% train/test partition and average results for the local explanations over this test
partition. We chose FMNIST over other higher resolution image datasets as MeLIME performs best
w.r.t. it, making it an acid test for our method. For LINEX we produce two environments where the
two environments are formed by performing bootstrap sampling on the base environment which is
created either by rand, real or mpl type neighborhood generation described above. Thus in all cases
the union of the environments is the same as a single neighborhood used to produce explanations
for the competitors making it a fair comparison. Behavior of LINEX with more environments is in
Appendix C.

Given the neighborhood generation schemes we compare LINEX with LIME, Smoothed LIME
(S-LIME), MeLIME and MAPLE, where for S-LIME we average the explanations of LIME across
the LINEX environments. SHAP’s results are in Appendix F, since it is not a natural fit here.

**Metrics: We evaluate the methods using five simple metrics: Infidelity (INFD), Generalized Infidelity**
(GI), Coefficient Inconsistency (CI), Class Attribution Consistency (CAC) and Unidirectionality (Υ).
The first two evaluate faithfulness, the next two stability and the last goodness for recourse.


-----

(4a) MeLIME:0 r = 0.292 0(4b) LINEX:r = 0.633 0 (4c) Original 0(d) MeLIME:r = 0.297 0 (e) LINEX:r = 0.962 0 (f) Original

5 5 5 5 5 5

10 10 10 10 10 10

15 15 15 15 15 15

20 20 20 20 20 20

25 25 25 25 25 25

0 10 20 0 10 20 0 10 20 0 10 20 0 10 20 0 10 20


Figure 2: Sample results using FMNIST dataset for two classes. (a-c): Class Dress, (d-f): Class
_Sandal. (a, d): MeLIME explanations. (b, d): LINEX explanations. (c, f): Original images. We_
observe that LINEX explanations capture important artifacts and thus exhibit significantly higher
correlation with the original images, where in aggregate too the correlations are high w.r.t. images
belonging to a particular class, thus showcasing higher stability (i.e. high CAC) as is seen in Table 3.
More examples are shown in Appendix E

Table 2: Below we see three example positive sentiment sentences from the Rotten Tomatoes dataset.
Green and red indicate the most important word highlighted by MeLIME and LINEX respectively.
As can be seen LINEX highlights stronger positive sentiment words. More examples in Appendix D.

|Example 1|Example 2|Example 3|
|---|---|---|
|one-of-a-kind near-masterpiece|moving tale of love and destruction in unexpected places, unexamined lives|spare yet audacious . . .|


Example 1 Example 2 Example 3

moving tale of love and destruction in
one-of-a-kind near-masterpiece spare yet audacious . . .
unexpected places, unexamined lives


Let Dt denote a (test) dataset with examples (x, y) where yb(x) is the black-box models prediction on
_x and ye[x][′]_ [(][x][)][ is the prediction on][ x][ (][∈X] [) using the explanation model at][ x][′][. The feature attributions]
(or coefficients) for the explanation model at x are denoted by c[x]e [, and][ N][x] [denotes the exemplar]
neighborhood of x with |.|card denoting cardinality. We now define the above metrics.

_Infidelity (INFD): This is the most commonly used metric to validate the faithfulness of explanation_
models (Ribeiro et al., 2016). Here we define it as the MAE between the black-box and explanation
model predictions across all the test points: INFD = _|Dt1|card_ (x,y)∈Dt _e_ [(][x][)][|][.]

_[|][y][b][(][x][)][ −]_ _[y][x]_

_Generalized Infidelity (GI): This metric has also been used in previous works (Ramamurthy et al.,P_
2020) to measure the generalizability of local explanations to neighboring test points. It is defined as:
GI = _|Dt1|card_ (x,y)∈Dt _|Nx1|card_ _x[′]∈Nx_ _e_ [(][x][)][|][.]

_[|][y][b][(][x][)][ −]_ _[y][x][′]_

_Coefficient Inconsistency (CI):P_ This notion has been used before (Hancox-Li, 2020) to measure howP
robust an explanation method is. We define it to be the MAE between the the linear coefficients of all
the test points and their respective neighbors: CI = _|Dt1|card_ (x,y)∈Dt _|Nx1|card_ _x[′]∈Nx_ _[|][c]e[x]_ _[−]_ _[c]e[x][′]_ _[|][1][.]_

_Class-Attribution Consistency (CAC): For local explanations of classification black-boxes, we expectP_ P
certain important features to be highlighted across most of the explanations of a class. This is codified
by this metric which is defined as follows: CAC = _|Y|1card_ _y∈Y_ _[r][(][µ]e[y][, µ][y][)][, where][ Y][ denotes the set of]_

class labels in the dataset, µy the mean (vector) of all inputs in class y, µ[y]e [the mean explanation]

P _∈Y_

for class y and r the Pearson’s correlation coefficient. This metric quantifies the consistency between
the important features for a class and attributions provided by the explanations.


_Coefficient Unidirectionality (Υ): This is motivated and defined in equation 2 in section 4.1._

We report the above metrics in Table 3. Each result in Table 3 is mean ± standard error of the mean
over five kernel sizes τ _√d generally, where τ = {0.05, 0.1, 0.25, 0.5, 0.75}. Test neighborhoods do_

not make sense for random perturbations with FMNIST and Rotten Tomatoes because the features
(viz. superpixels) used by neighboring test examples are different. Also, we do not use realistic
perturbations with MEPS since KDE and VAE generators do not work well with categorical data. In
addition, since MEPS data uses regression black-box, CAC cannot be computed. All these justify
the missing entries in Table 3. The results were generated on Linux machines with 56 cores and 242
GB RAM. More details regarding the exact perturbation schemes for LIME/MeLIME/MAPLE, the
perturbation neighborhood sizes and the time taken by the different methods are Appendix A and B.

**Observations: We see that in terms of CAC, LINEX is better than baselines in all cases which**
indicates that on average the local explanations with LINEX highlight the important features characterzing the entire class making them more stable. This is further verified by looking at the Υ and CI


-----

Table 3: Comparing the different methods using metrics infidelity (INFD), generalized infidelity
(GI), coefficient inconsistency (CI), class attribution consistency (CAC) and unidirectionality (Υ). ↑
indicates higher value for the metric is better, and ↓ indicates lower is better. Results better by ≥ 1%
are bolded. LINEX is better than baselines in 18 out of 35 cases, and worse only in 5 cases. Plots
showing behavior of these metrics with varying neighborhood size, number of environments and
kernel width are in Appendix C.

|Dataset|Method|INFD ↓|GI ↓|CI ↓|Υ ↑|CAC ↑|
|---|---|---|---|---|---|---|
|IRIS|LIME S-LIME LINEX/rand|0.015 ± 0.011 0.015 ± 0.010 0.013 ± 0.009|0.132 ± 0.042 0.077 ± 0.011 0.052 ± 0.008|0.319 ± 0.132 0.143 ± 0.045 0.044 ± 0.013|0.646 ± 0.040 0.704 ± 0.037 0.802 ± 0.043|0.667 ± 0.167 0.878 ± 0.034 0.921 ± 0.042|
||MeLIME LINEX/real|0.008 ± 0.003 0.009 ± 0.003|0.049 ± 0.018 0.029 ± 0.003|0.219 ± 0.108 0.024 ± 0.002|0.629 ± 0.013 0.744 ± 0.044|0.464 ± 0.100 0.942 ± 0.023|
||MAPLE LINEX/mpl|0.009 ± 0.001 0.013 ± 0.000|0.038 ± 0.004 0.020 ± 0.000|0.261 ± 0.033 0.026 ± 0.002|0.458 ± 0.032 0.694 ± 0.008|0.586 ± 0.035 0.929 ± 0.004|
|MEPS|LIME S-LIME LINEX/rand|0.158 ± 0.066 0.158 ± 0.066 0.130 ± 0.052|0.214 ± 0.041 0.214 ± 0.042 0.164 ± 0.021|0.005 ± 0.001 0.005 ± 0.001 0.003 ± 0.001|0.981 ± 0.006 0.974 ± 0.008 0.979 ± 0.006|NA|
||MAPLE LINEX/mpl|0.063 ± 0.000 0.098 ± 0.001|0.067 ± 0.000 0.094 ± 0.001|0.007 ± 0.000 0.007 ± 0.000|0.957 ± 0.000 0.950 ± 0.000|NA|
|FMNIST|LIME S-LIME LINEX/rand|0.162 ± 0.003 0.142 ± 0.003 0.149 ± 0.002|NA|NA|NA|NA|
||MeLIME LINEX/real|0.001 ± 0.000 0.100 ± 0.002|0.277 ± 0.000 0.304 ± 0.001|0.007 ± 0.000 0.002 ± 0.000|0.769 ± 0.000 0.780 ± 0.000|0.327 ± 0.000 0.649 ± 0.001|
|Rotten Tomatoes|LIME S-LIME LINEX/rand|0.079 ± 0.036 0.075 ± 0.035 0.069 ± 0.032|NA|NA|NA|NA|
||MeLIME LINEX/real|0.029 ± 0.001 0.053 ± 0.000|0.391 ± 0.000 0.361 ± 0.000|0.000 ± 0.000 0.000 ± 0.000|0.999 ± 0.000 1.000 ± 0.000|0.909 ± 0.000 0.953 ± 0.001|


_Dataset_ _Method_ INFD ↓ GI ↓ CI ↓ Υ ↑ CAC ↑

LIME 0.015 ± 0.011 0.132 ± 0.042 0.319 ± 0.132 0.646 ± 0.040 0.667 ± 0.167

S-LIME 0.015 ± 0.010 0.077 ± 0.011 0.143 ± 0.045 0.704 ± 0.037 0.878 ± 0.034

LINEX/rand 0.013 ± 0.009 0.052 ± 0.008 0.044 ± 0.013 0.802 ± 0.043 0.921 ± 0.042

_IRIS_ MeLIME 0.008 ± 0.003 0.049 ± 0.018 0.219 ± 0.108 0.629 ± 0.013 0.464 ± 0.100

LINEX/real 0.009 ± 0.003 0.029 ± 0.003 0.024 ± 0.002 0.744 ± 0.044 0.942 ± 0.023

MAPLE 0.009 ± 0.001 0.038 ± 0.004 0.261 ± 0.033 0.458 ± 0.032 0.586 ± 0.035

LINEX/mpl 0.013 ± 0.000 0.020 ± 0.000 0.026 ± 0.002 0.694 ± 0.008 0.929 ± 0.004

LIME 0.158 ± 0.066 0.214 ± 0.041 0.005 ± 0.001 0.981 ± 0.006

S-LIME 0.158 ± 0.066 0.214 ± 0.042 0.005 ± 0.001 0.974 ± 0.008 NA

_MEPS_ LINEX/rand 0.130 ± 0.052 0.164 ± 0.021 0.003 ± 0.001 0.979 ± 0.006

MAPLE **0.063** **0.000 0.067** **0.000 0.007** 0.000 0.957 0.000
_±_ _±_ _±_ _±_ NA
LINEX/mpl 0.098 ± 0.001 0.094 ± 0.001 0.007 ± 0.000 0.950 ± 0.000

LIME 0.162 ± 0.003

S-LIME 0.142 ± 0.003 NA NA NA NA

_FMNIST_ LINEX/rand 0.149 ± 0.002

MeLIME **0.001 ± 0.000 0.277 ± 0.000 0.007 ± 0.000** 0.769 ± 0.000 0.327 ± 0.000

LINEX/real 0.100 ± 0.002 0.304 ± 0.001 0.002 ± 0.000 0.780 ± 0.000 0.649 ± 0.001

LIME 0.079 ± 0.036

S-LIME 0.075 0.035 NA NA NA NA

_Rotten_ _±_

LINEX/rand 0.069 0.032

_Tomatoes_ _±_

MeLIME **0.029 ± 0.001 0.391 ± 0.000** 0.000 ± 0.000 0.999 ± 0.000 0.909 ± 0.000

LINEX/real 0.053 ± 0.000 0.361 ± 0.000 0.000 ± 0.000 1.000 ± 0.000 0.953 ± 0.001


metrics where LINEX is similar or better than others. For GI and INFD metrics, the results are more
evenly spread which implies that LINEX’s main advantage is obtaining stable and unidirectional
explanations that are faithful to a similar degree.

We show an example MeLIME and LINEX/realistic explanation on FMNIST in Figure 2, where we
see that LINEX explanations are more coherent and also delineate the outline of the image more
clearly compared to MeLIME explanations leading to better CAC. Even on the text data we see more
reasonable attributions in Table 2, where “masterpiece”, “moving” and “audacious” are highlighted
as the most important words indicative of positive sentiment in the three examples.

An interesting observation is that when it comes to the stability metrics (CI and CAC) and unidirectionality LINEX with even random perturbation model is better than MeLIME in some cases. This
is very promising as it means LINEX could be potentially be trusted without the need to generate
realistic perturbations which may be computationally expensive or not even possible.

6 DISCUSSION

In this paper we have provided a method based on a game theoretic formulation and inspired by the
invariant risk minimization principle to provide faithful, stable and unidirectional explanations. We
have defined the latter property and argued that it is somewhat of a necessity (may not be sufficient)
for recourse. We have theoretically shown that our method has a strong tendency to be stable and
unidirectional as we will mostly eliminate features where the black-box models gradient changes
abruptly and in other cases choose a conservative value. Empirically, we have verified this where
we outperform competitors in majority of the cases on these metrics. An interesting observation is
also that in some cases our method provides more stable and unidirectional explanations with just a
random perturbation model relative to more expensive methods that use realistic neighbors.

In the future, it would be worth experimenting with more varied strategies to form environments and
if possible find the optimal ones (Creager et al., 2020), which may lead to picking even more relevant
features that are “causal” to the local decision.


-----

ETHICS STATEMENT

With the wide adoption of deep learning technologies, explaining or understanding the reasons behind
their decisions has become extremely important in many critical applications (Arya et al., 2019).
Numerous explainability methods have been proposed in literature to explain individual decisions
of black-box models (Ribeiro et al., 2016; Plumb et al., 2018; Botari et al., 2020; Dhurandhar
et al., 2018a; Lundberg & Lee, 2017). Although LINEX is more stable and undirectional than other
competing approaches, it still is a posthoc explainability method that may not be completely faithful
to the black-box model. This of course is not just a limitation of our approach, but nonetheless
should be taken into account before a user makes a decision. Our method could also be used to
divulge information by exposing the inner workings of the black-box leading to privacy concerns.
One possible mitigation strategy in this case would be to keep the sensitive attributes hidden from the
explainer.

REPRODUCIBILITY STATEMENT

Experimental details are provided in Section 5 of the main paper and Appendix B. All datasets are
public. Code will be provided during the discussion phase through an anonymized link.

REFERENCES

[Agency for Healthcare Research and Quality. Medical Expenditure Panel Survey (MEPS). https:](https://meps.ahrq.gov/mepsweb/)
[//meps.ahrq.gov/mepsweb/, 2019. Last accessed 2019-08.](https://meps.ahrq.gov/mepsweb/)

C. Aggarwal and C. Reddy. Data Clustering: Algorithms and Applications. CRC Press, 2013.

Kartik Ahuja, Karthikeyan Shanmugam, Kush Varshney, and Amit Dhurandhar. Invariant risk
minimization game. In International Conference on Machine Learning, 2020.

Kartik Ahuja, Karthikeyan Shanmugam, and Amit Dhurandhar. Linear regression games: Convergence guarantees to approximate out-of-distribution solutions. In AISTATS, 2021.

Elvio Amparore, Alan Perotti, and Paolo Bajardi. To trust or not to trust an explanation: using leaf to
evaluate local linear xai methods. PeerJ Computer Science, 7:e479, 2021.

Christopher J. Anders, Plamen Pasliev, Ann-Kathrin Dombrowski, Klaus-Robert Muller, and Pan
Kessel. Fairwashing explanations with off-manifold detergent. In Intl. Conference on Machine
_Learning (ICML), 2020._

Martin Arjovsky, Leon Bottou, Ishaan Gulrajani, and David Lopez-Paz. Invariant risk minimization.´
_arXiv preprint arXiv:1907.02893, 2019._

Vijay Arya, Rachel K. E. Bellamy, Pin-Yu Chen, Amit Dhurandhar, Michael Hind, Samuel C. Hoffman, Stephanie Houde, Q. Vera Liao, Ronny Luss, Aleksandra Mojsilovic, Sami Mourad, Pablo´
Pedemonte, Ramya Raghavendra, John Richards, Prasanna Sattigeri, Karthikeyan Shanmugam,
Moninder Singh, Kush R. Varshney, Dennis Wei, and Yunfeng Zhang. One explanation does not fit
all: A toolkit and taxonomy of AI explainability techniques. arXiv:1909.03012, 2019.

Tiago Botari, Frederik Hvilshøj, Rafael Izbicki, and Andre C. P. L. F. de Carvalho. Melime:
Meaningful local explanation for machine learning models, 2020.

Elliot Creager, Jorn-Henrik Jacobsen, and Richard Zemel. Environment inference for invariant¨
learning. In e ICML 2020 Workshop on Uncertainty and Robustness in Deep Learning, 2020.

Gerard Debreu. A social equilibrium existence theorem. Proceedings of the National Academy of
_Sciences, 38(10):886–893, 1952._

[Dua Dheeru and Efi Karra Taniskidou. UCI machine learning repository, 2017. URL http:](http://archive.ics.uci.edu/ml)
[//archive.ics.uci.edu/ml.](http://archive.ics.uci.edu/ml)


-----

Amit Dhurandhar, Pin-Yu Chen, Ronny Luss, Chun-Chen Tu, Paishun Ting, Karthikeyan Shanmugam,
and Payel Das. Explanations based on the missing: Towards contrastive explanations with pertinent
negatives. In Advances in Neural Information Processing Systems, pp. 592–603, 2018a.

Amit Dhurandhar, Karthikeyan Shanmugam, Ronny Luss, and Peder Olsen. Improving simple models
with confidence profiles. In Advances in Neural Information Processing Systems, pp. 10296–10306,
2018b.

Amit Dhurandhar, Tejaswini Pedapati, Avinash Balakrishnan, Kartik Ahuja Pin-Yu Chen,
Karthikeyan Shanmugam, and Ruchir Puri. Model agnostic contrastive explanations for structured
data. https://arxiv.org/abs/1906.00117, 2019.

Amit Dhurandhar, Karthikeyan Shanmugam, and Ronny Luss. Enhancing simple models by exploiting
what they already know. In Intl. Conference on Machine Learning, 2020.

Prajit K. Dutta. Strategies and games: theory and practice. MIT Press, 1999.

Shi Feng, Eric Wallace, Alvin Grissom II, Mohit Iyyer, Pedro Rodriguez, and Jordan Boyd-Graber.
Pathologies of neural models make interpretations difficult. In Proceedings of the 2018 Conference
_on Empirical Methods in Natural Language Processing, pp. 3719–3728, Brussels, Belgium,_
October-November 2018. Association for Computational Linguistics.

Christopher Frye, Colin Rowat, and Ilya Feige. Asymmetric shapley values: incorporating causal
knowledge into model-agnostic explainability. NeurIPS, 2020.

Amirata Ghorbani, Abubakar Abid, and James Zou. Interpretation of neural networks is fragile, 2019.

David Gunning. Explainable artificial intelligence (xai). In Defense Advanced Re_search_ _Projects_ _Agency,_ 2017. URL [https://www.darpa.mil/program/](https://www.darpa.mil/program/explainable-artificial-intelligence)
[explainable-artificial-intelligence.](https://www.darpa.mil/program/explainable-artificial-intelligence)

Karthik Gurumoorthy, Amit Dhurandhar, Guillermo Cecchi, and Charu Aggarwal. Efficient data
representation by selecting prototypes with importance weights. In Proceedings of the IEEE
_International Conference on Data Mining, 2019._

Leif Hancox-Li. Robustness in machine learning explanations: does it matter? In Proceedings of the
_2020 Conference on Fairness, Accountability, and Transparency. ACM, 2020._

Tom Heskes, Evi Sijben, Ioan Gabriel Bucur, and Tom Claassen. Causal shapley values: Exploiting
causal knowledge to explain individual predictions of complex models. Advances in Neural
_Information Processing Systems, 2020._

Geoffrey Hinton, Oriol Vinyals, and Jeff Dean. Distilling the knowledge in a neural network.
arXiv:1503.02531, 2015.

Amir-Hossein Karimi, Bernhard Scholkopf, and Isabel Valera. Algorithmic recourse: from coun-¨
terfactual explanations to interventions. In ACM conference on Fairness, Accountability and
_Transparency (FAccT), 2021._

Been Kim, Rajiv Khanna, and Oluwasanmi Koyejo. Examples are not enough, learn to criticize!
Criticism for interpretability. In In Advances of Neural Inf. Proc. Systems, 2016.

Himabindu Lakkaraju, Nino Arsov, and Osbert Bastani. Robust and stable black box explanations.
_ICML, 2020._

Sebastian Lapuschkin, Alexander Binder, Gregoire Montavon, Klaus-Robert M´ uller, and Wojciech¨
Samek. The lrp toolbox for artificial neural networks. Journal of Machine Learning Research, 17
[(114):1–5, 2016. URL http://jmlr.org/papers/v17/15-618.html.](http://jmlr.org/papers/v17/15-618.html)

Scott M. Lundberg and Su-In Lee. A unified approach to interpreting model predictions. In Advances
_in Neural Information Processing Systems, pp. 4765–4774, 2017._

Tim Miller. Explanation in artificial intelligence: Insights from the social sciences. Artificial
_Intelligence, 2018._


-----

[Christoph Molnar. Interpretable machine learning, 2019. URL https://christophm.github.](https://christophm.github.io/interpretable-ml-book/index.html)
[io/interpretable-ml-book/index.html.](https://christophm.github.io/interpretable-ml-book/index.html)

Bo Pang, Lillian Lee, and Shivakumar Vaithyanathan. Thumbs up? sentiment classification using
machine learning techniques. In Proceedings of EMNLP, pp. 79–86, 2002.

Gregory Plumb, Denali Molitor, and Ameet Talwalkar. Model agnostic supervised local explanations.
In Advances in Neural Information Processing Systems, pp. 2520–2529, 2018.

Karthikeyan Natesan Ramamurthy, Bhanukiran Vinzamuri, Yunfeng Zhang, and Amit Dhurandhar.
Model agnostic multilevel explanations, 2020.

Marco Tulio Ribeiro, Sameer Singh, and Carlos Guestrin. “Why should I trust you?”: Explaining the
predictions of any classifier. In Proceedings of the ACM SIGKDD International Conference on
_Knowledge Discovery and Data Mining, pp. 1135–1144, 2016._

Yoel Roth and Nick Pickles. Updating our approach to misleading information. 2020. Pub[lished 2020-05-11 at https://blog.twitter.com/en_us/topics/product/2020/](https://blog.twitter.com/en_us/topics/product/2020/updating-our-approach-to-misleading-information.html)
[updating-our-approach-to-misleading-information.html.](https://blog.twitter.com/en_us/topics/product/2020/updating-our-approach-to-misleading-information.html)

Cynthia Rudin. Stop explaining black box machine learning models for high stakes decisions and use
interpretable models instead. Nature Mach. Intell., 1(5):206–215, May 2019.

Aditya Aniruddha Shrotri, Nina Narodytska, Alexey Ignatiev, Joao Marques-Silva, Kuldeep S. Meel,
and Moshe Vardi. Constraint-driven explanations of black-box {ml} models, 2021.

Karen Simonyan, Andrea Vedaldi, and Andrew Zisserman. Deep inside convolutional networks:
Visualising image classification models and saliency maps. arXiv:1312.6034, 2013.

Mukund Sundararajan, Ankur Taly, and Qiqi Yan. Axiomatic attribution for deep networks. ICML,
2017.

Jesse Vig, Sebastian Gehrmann, Yonatan Belinkov, Sharon Qian, Daniel Nevo, Simas Sakenis, Jason
Huang, Yaron Singer, and Stuart Shieber. Causal mediation analysis for interpreting neural nlp:
The case of gender bias. NeurIPS, 2020.

Giorgio Visani, Enrico Bagli, Federico Chesani, Alessandro Poluzzi, and Davide Capuzzo. Statistical stability indices for lime: obtaining reliable explanations for machine learning models.
_arXiv:2001.11757, 2020._

Han Xiao, Kashif Rasul, and Roland Vollgraf. Fashion-mnist: a novel image dataset for benchmarking
machine learning algorithms, 2017.

Philip N. Yannella and Odia Kagan. Analysis: Article 29 working party guidelines on automated
decision making under gdpr. 2018. https://www.cyberadviserblog.com/2018/01/analysis-article29-working-party-guidelines-on-automated-decision-making-under-gdpr/.

Chih-Kuan Yeh, Cheng-Yu Hsieh, Arun Suggala, David I Inouye, and Pradeep K Ravikumar. On
the (in) fidelity and sensitivity of explanations. In Advances in Neural Information Processing
_Systems, pp. 10965–10976, 2019._

Huan Zhang, Hongge Chen, Zhao Song, Duane Boning, Inderjit S. Dhillon, and Cho-Jui Hsieh.
The limitations of adversarial training and the blind-spot attack. In International Conference on
_Learning Representations (ICLR), 2019a._

Yujia Zhang, Kuangyan Song, Yiming Sun, Sarah Tan, and Madeleine Udell. Why should you trust
my explanation? ICML-AI for Social Good, 2019b.


-----

APPENDIX

A EFFICIENCY OF LINEX

It is important to note that the query complexity (i.e. number of times we query the black box to
obtain an explanation) of LINEX is the same as that of LIME since the union of the environments is
the same as a LIME perturbation neighborhood. This is important in todays cloud-driven world where
models may exist on different cloud platforms and posthoc explanations are an independent service
where each call to the model has an associated cost. In terms of running time for two environments,
convergence was fast and running time was approximately 2.5 times that of LIME (LINEX took
2.5 seconds on IRIS for 30 examples as opposed to 1 second by LIME, LINEX took 47 seconds on
MEPS for 500 examples as opposed to 18 seconds by LIME), which is very similar to Smoothed
LIME (S-LIME) (took 2.3 seconds on IRIS and 40 seconds on MEPS) that we still outperform in
majority of the cases.

Realistic neighborhood generation can be time consuming especially for MeLIME since generators
have to be trained which may take up to an hour using a single GPU for datasets such as FMNIST.
After the generator is trained and neighborhood sampled MeLIME takes the same amount of time as
LIME since the model fitting procedure is the same. MAPLE took 1.5 seconds for the IRIS dataset
for 30 examples and 27 seconds for 500 MEPS examples.

B EXPERIMENTAL DETAILS

B.1 DATASET DETAILS AND HYPERPARAMETER SPECIFICATIONS

We describe the datasets and the hyperparameters used for each. We set perturbation neighborhood
sizes 10 (IRIS), 500 (MEPS), 100 (FMNIST-random), 500 (FMNIST-realistic), 100 (Rotten tomatoes)
for generating local explanations. We also use 3, 10, 10, 5 as examplar neighborhood sizes to compute
GI, CI and Υ metrics for the four datasets respectively. We also use 5−sparse explanations for all
cases except FMNIST with realistic perturbations where we follow MeLIME and generate a dense
explanation using ridge penalty with penalty multiplier value of 0.001. The ℓ bound γ in Algorithm
_∞_
1 is set as the maximum absolute value of linear coefficient computed by running LIME/MeLIME
in the two individual environments. Please look at IRIS dataset first since it contains some of the
common details used across others.

**IRIS (Tabular):** This dataset has 150 instances with four numerical features representing the sepal
and petal width and length in centimeters. The task is to classify instances of Iris flowers into three
species: setosa, versicolor, and virginica. A random forest classifier was trained with a train/test
split of 0.8/0.2 and yielded a test accuracy of 93%. We provide local explanations for the prediction
probabilities for class setosa. For both random and realistic perturbations, we use a perturbation
neighborhood size of n. For random perturbations, we used the same approach followed by LIME
and sample from a Gaussian around each data point. Realistic perturbations (with the same number
_n) were generated using KDEGen Botari et al. (2020), a kernel density estimator (KDE) with the_
Gaussian kernel fitted on the training dataset to sample data around a sample point. For both random
and realistic perturbations, we weight the neighborhood using a Gaussian kernel of width τ _√d, where_

_d is the dimension of the feature vector and τ = {0.05, 0.1, 0.25, 0.5, 0.75}, and this corresponded_
to kernel widths {0.1, 0.2, 0.5, 1.0, 1.5}. We also perform a weighted version of realistic selection
where we use MAPLE Plumb et al. (2018) to assign weights to all the test examples and pick the top
_n weighted examples to use as the perturbation neighborhood. For random/realistic perturbations and_
realistic selection, the corresponding environments (of size n each) for LINEX are created by drawing
_k bootstrap samples where k = {2, 3, 4, 5} in our experiments. We test for n = {10, 20, 30, 40, 50}_
with this dataset.

**Medical Expenditure Panel Survey (Tabular):** The Medical Expenditure Panel Survey (MEPS)
dataset is produced by the US Department of Health and Human Services. It is a collection of surveys
of families of individuals, medical providers, and employers across the country. We choose Panel 19
of the survey which consists of a cohort that started in 2014 and consisted of data collected over 5
rounds of interviews over 2014 _−_ 2015. The outcome variable was a composite utilization feature that


-----

quantified the total number of healthcare visits of a patient. The features used included demographic
features, perceived health status, various diagnosis, limitations, and socioeconomic factors. We filter
out records that had a utilization (outcome) of 0, and log-transformed the outcome for modeling.
These pre-processing steps resulted in a dataset with 11136 examples and 32 categorical features.
We train a random forest regressor that has a test R[2] of 0.325 in this dataset. We provide local
explanations of the predictions. With MEPS, we do not use realistic perturbations since KDE and
VAE generators do not work well with categorical data. Otherwise the setting is similar as IRIS
data, except that we use n = {50, 100, 200, 300, 400, 500}. The kernel widths in this case were
_{0.28, 0.57, 1.41, 2.83, 4.24}. We use k = {2, 3, 4, 5} for this dataset._

**Fashion MNIST (Images):** This dataset has 28 × 28 grayscale images of fashion articles with
60,000 train and 10,000 test samples. The task is to classify these into 10 classes corresponding
to coat, shoe, and so on. A neural network trained with test accuracy of 87%. Explanations are
generated for the prediction probabilities corresponding to the predicted class for each example.
We choose 1000 test examples to generate explanations. Realistic perturbations were generated
using VAEGen (Botari et al., 2020), a Variational Auto Encoder (VAE) fitted on the training dataset.
For random perturbations, the we chose n from {50, 100, 200, 300, 400, 500} and kernel sizes were
_{0.43, 0.85, 2.14, 4.27, 6.41}. For realistic perturbations we chose n from {250, 500, 750, 1000} and_
the kernel widths were {1.4, 2.8, 7.0, 14.0, 21.0}. We use k = {2, 3, 4, 5} for this dataset.

**Rotten Tomatoes (Text):** This dataset contains 10662 movie reviews from rotten tomatoes website
along with their sentiment polarity, i.e., positive or negative reviews and the task is to classify the
sentiment of the reviews into positive or negative. The review sentences were vectorized using
CountVectorizer and TfidfTransformer and a sklearn Naive Bayes classifier was fitted on training dataset which yielded a test accuracy of 75%. Explanations are generated for the prediction
probabilities corresponding to the predicted class for each example. Realistic perturbations were
generated using Word2VecGen (Botari et al., 2020), wherein word2vec embeddings are first trained
using the training corpus and new sentences are generated by randomly replacing a sentence word
whose distance in the embedding space lies within the radius of the neighbourhood. For both
random and realistic perturbations, n was chosen from {25, 50, 75, 100}. The kernel sizes were
_{0.42, 1.06, 2.12, 3.18} for random perturbations (kernel size 0.21 resulted in numerical issues), and_
_{0.21, 0.42, 1.06, 2.12, 3.18} for realistic perturbations. We use k = {2, 3, 4, 5} for this dataset._

C RESULTS WITH ALL DATASETS AND HYPERPARAMETER COMBINATIONS
FOR RANDOM AND REALISTIC PERTURBATIONS

We present results with all hyperparameter combinations for random and realistic perturbations.
Results for LIME with random perturbations (LIME), smoothed LIME (S-LIME), LINEX with
random perturbations (LINEX/rand), MeLIME (MeLIME), LINEX with MeLIME-like realistic
neighborhoods (LINEX/real), MAPLE (MAPLE), LINEX with MAPLE-like realistic neighborhoods
(LINEX/mpl) are presented in figures 4-18. The legend for these figures are given in Figure 3.

For the four datasets, we perform ablations by varying one of perturbation neighborhood size (Figures
4-8), number of environments (Figures 9-13), and kernel width (Figures 14-18). Each point in
these figures are averaged over all possible values for the two parameters that are not ablated. For
example, each point in Figure 4 is averaged over all possible values for kernel widths and number of
environments for a given perturbation neighborhood size. Standard errors of the mean are also plotted
in the same color with lesser opacity. Lower values of Infidelity (INFD), Generalized Infidelity (GI),
Coefficient Inconsistency (CI) are better whereas for Unidirectionality (Υ) and Class Attribution
Consistency (CAC) higher values are better.

Figures 4-8 show ablations with respect to perturbation neighborhood sizes. Considering all datasets,
the stability/recourse metrics (CI, Υ, CAC) are clearly better for LINEX compared to its counterparts.
For LINEX methods (LINEX/rand, LINEX/real, LINEX/mpl), the metrics get better or stays approximately the same generally as perturbation neighborhood size increases keeping with the intuition
that larger perturbation neighborhood sizes should produce explanations that are more stable in the
exemplar neighborhood. Υ for FMNIST is somewhat of an exception which could be because of the
high dimensionality as well as the quality of MeLIME perturbations.


-----

Turning to the fidelity metrics (INFD and GI) in tabular datasets, we see that the results still favor
LINEX, but less heavily compared to the stability/recourse metrics. This is in line with what we
observe in Table 3. In IRIS and MEPS, LINEX is close to or outperforms the corresponding baselines
in the GI measure (except for LINEX/mpl with MEPS). This gap closes a bit with INFD, but we note
that GI is a better measure since it estimates how faithful explanations are in a exemplar neighborhood.
With the text dataset, LINEX variants are slightly more favored, whereas with the image dataset, the
baselines have an edge.

Considering Figures 9-13, we see that variations are less stark with respect to number of environments
overall for LINEX variants. Note that except for S-LIME, other baselines do not use multiple
environments, and hence stay constant. The slight variations in MAPLE are due to the effect of
random seeds. In the stability/recourse metrics, again LINEX variants emerge as the clear winner
across datssets. With the faithfulness metrics (GI and INFD), in the text dataset, LINEX variants
generally perform better, whereas the baselines have a better performance in the image dataset.

Finally, we study the variation of the performance measures with respect to kernel width in Figures
14-18. We see that the stability/recourse metrics flatten out in all cases with large kernel widths.
This behaviour holds true for faithfulness metrics (GI and INFD) as well except in some cases.
GI and INFD measures also increase before they flatten out since the fit becomes poorer at larger
kernel widths. The stability/recourse metrics become better or remain approximately the same since
explanations generally improve or preserve their stability properties as kernel widths increase. Note
that very small kernel widths can lead to unexpected behavior that does not fit the trend as seen with
the tabular datasets since explanations can become hyper-local. MAPLE and LINEX/mpl stay the
same at different kernel widths since they use a different weighting scheme. As with other ablations,
we see that LINEX variants are similar or better in stability/recourse metrics overall, while with the
faithfulness metrics the results are more mixed.

Note that we do not compute MeLIME perturbations with MEPS since KDE and VAE generators
do not work well with categorical data, and do not use compute CAC since the task is regression.
Further, the features used in explanations for different test examples are not comparable for random
perturbations with FMNIST and Rotten Tomatoes, hence we cannot compute CAC for those cases as
well. This explains the missing curves/plots.

LIME
S-LIME
LINEX/rand
MeLime
LINEX/real
MAPLE
LINEX/mpl


Figure 3: Legend for figures 4-18



0.1

|Col1|Col2|Col3|Col4|Col5|Col6|Col7|Col8|Col9|Col10|Col11|Col12|Col13|Col14|Col15|Col16|Col17|Col18|Col19|
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
||||||||||||||||||||
||||||||||||||||||||
||||||||||||||||||||
||||||||||||||||||||
||||||||||||||||||||
||||||||||||||||F||||
||||||||||||||||||||
||||||||||||||||||||
|||||||||||||||inconsistency 5 0.0105 0.008 0.006 ient|0||||
|||||||o|3 d|5 s|iz|4 e|0||4||||||



0.0


0.004


10 15 20 25 30 35 40 45 50

Neighborhood size

(a) Iris


100 200 300 400 500

50

Neighborhood size

(b) MEPS


0.007 0.00024

0.006 0.00023

0.00022

0.005 0.00021

0.004 0.00020

0.00019

0.003 0.00018

Coefficient inconsistency0.002 Coefficient inconsistency0.00017

300 400 500 600 700 800 900 1000 0.00016 30 40 50 60 70 80 90 100

Neighborhood size Neighborhood size


(c) FMNIST


(d) Rotten Tomatoes


Figure 4: Coefficient inconsistency (CI) vs. Perturbation neighborhood size.


-----

1.0

0.65 0.96

0.9 0.60 0.95

0.8 0.55

0.7 0.50 0.94

0.6 0.45 0.93

0.5 0.40 0.92

Class attribution consistency0.4 Class attribution consistency0.35 Class attribution consistency0.91

10 15 20 25 30 35 40 45 50 300 400 500 600 700 800 900 1000 30 40 50 60 70 80 90 100

Neighborhood size Neighborhood size Neighborhood size


(a) Iris


(b) FMNIST


(c) Rotten Tomatoes


Figure 5: Class attribution consistency (CAC) vs. Perturbation neighborhood size.


0.9 0.98 0.785 0.999600

0.999575

0.8 0.97 0.780 0.999550

0.7 0.96 0.775 0.9995250.999500

0.6 0.95 0.770 0.999475

0.5 0.94 0.999450

10 15 20 25 30 35 40 45 50 0.93 100 200 300 400 500 0.765 300 400 500 600 700 800 900 1000 0.9994250.999400 30 40 50 60 70 80 90 100

Neighborhood size Neighborhood size Neighborhood size Neighborhood size


(a) Iris

0.2000.175 0.225 0.36 0.39

0.150 0.200 0.34 0.38

0.1250.100 0.1750.150 0.32 0.37

0.075 0.125 0.30 0.36

Generalized infidelity0.050 Generalized infidelity0.100 Generalized infidelity0.28 Generalized infidelity0.35

0.025 0.075 0.26

10 15 20 25 30 35 40 45 50 100 200 300 400 500 300 400 500 600 700 800 900 1000 30 40 50 60 70 80 90 100

Neighborhood size Neighborhood size Neighborhood size Neighborhood size


(a) Iris


(b) MEPS


(c) FMNIST


(d) Rotten Tomatoes


Figure 6: Unidirectionality (Υ) vs. Perturbation neighborhood size.


(b) MEPS


(c) FMNIST


(d) Rotten Tomatoes


Figure 7: Generalized infidelity (GI) vs. Perturbation neighborhood size.


0.0400.0350.030 0.180.160.14 0.350.30 0.2000.1750.1500.125

0.025 0.12 0.25 0.100

Infidelity0.0200.015 Infidelity0.100.08 Infidelity0.20 Infidelity0.0750.050

0.010 0.06 0.15 0.025

0.005 0.04 0.000

10 15 20 25 30 35 40 45 50 100 200 300 400 500 100 200 300 400 500 300 400 500 600 700 800 900 1000

Neighborhood size Neighborhood size Neighborhood size Neighborhood size


(a) Iris

0.30

0.25 0.009 0.007 0.00024

0.200.15 0.0080.007 0.0060.005 0.000230.000220.00021

0.10 0.006 0.004 0.00020

Coefficient inconsistency0.05 Coefficient inconsistency0.0050.004 Coefficient inconsistency0.0030.002 Coefficient inconsistency0.000190.000180.00017

2.0 2.5 3.0 3.5 4.0 4.5 5.0 2.0 2.5 3.0 3.5 4.0 4.5 5.0 2.0 2.5 3.0 3.5 4.0 4.5 5.0 2.0 2.5 3.0 3.5 4.0 4.5 5.0

Number of environments Number of environments Number of environments Number of environments


(a) Iris


(b) MEPS

0.0400

0.09 0.0375

0.08 0.03500.0325

0.07 0.0300

Infidelity0.06 Infidelity0.02750.0250

0.05 0.02250.0200

30 40 50 60 70 80 90 100 30 40 50 60 70 80 90 100

Neighborhood size Neighborhood size


(e) Rotten Tomatoes
(random)


(c) FMNIST (random)

0.0400

0.0375

0.0350

0.0325

0.0300

0.0275

0.0250

0.0225

0.0200

30 40 50 60 70 80 90

Neighborhood size

(f) Rotten Tomatoes
(realistic)


(d) FMNIST (realistic)


Figure 8: Infidelity (INFD) vs. Perturbation neighborhood size.


0.30


(b) MEPS


(c) FMNIST


(d) Rotten Tomatoes


Figure 9: Coefficient inconsistency (CI) vs. Number of environments.


-----

0.65 0.960

0.9 0.60 0.955

0.8 0.55 0.9500.945

0.7 0.500.45 0.9400.935

0.6 0.40 0.930

Class attribution consistency0.5 Class attribution consistency0.35 Class attribution consistency0.9250.920

2.0 2.5 3.0 3.5 4.0 4.5 5.0 2.0 2.5 3.0 3.5 4.0 4.5 5.0 2.0 2.5 3.0 3.5 4.0 4.5 5.0

Number of environments Number of environments Number of environments


(a) Iris


(b) FMNIST


(c) Rotten Tomatoes


Figure 10: Class attribution consistency (CAC) vs. Number of environments.


0.90 0.975 0.782 0.99957

0.850.800.750.70 0.9700.9650.9600.9550.950 0.7800.7780.7760.7740.7720.770 0.999560.999550.999540.999530.999520.999510.99950

0.65 0.945 0.768 0.99949

2.0 2.5 3.0 3.5 4.0 4.5 5.0 2.0 2.5 3.0 3.5 4.0 4.5 5.0 2.0 2.5 3.0 3.5 4.0 4.5 5.0 2.0 2.5 3.0 3.5 4.0 4.5 5.0

Number of environments Number of environments Number of environments Number of environments


(a) Iris

0.22 0.35

0.12 0.20 0.34 0.38

0.10 0.18 0.33

0.08 0.160.14 0.320.31 0.37

0.06 0.12 0.30 0.36

Generalized infidelity0.04 Generalized infidelity0.10 Generalized infidelity0.29 Generalized infidelity0.35

0.02 0.08 0.28

2.0 2.5 3.0 3.5 4.0 4.5 5.0 2.0 2.5 3.0 3.5 4.0 4.5 5.0 2.0 2.5 3.0 3.5 4.0 4.5 5.0 2.0 2.5 3.0 3.5 4.0 4.5 5.0

Number of environments Number of environments Number of environments Number of environments


(a) Iris


(b) MEPS


(c) FMNIST


(d) Rotten Tomatoes


Figure 11: Unidirectionality (Υ) vs. Number of environments.

0.22 0.35


(b) MEPS


(c) FMNIST


(d) Rotten Tomatoes


Figure 12: Generalized infidelity (GI) vs. Number of environments.


0.32 0.16

0.030 0.16 0.30 0.14

0.14 0.28 0.12

0.025 0.26 0.10

0.020 0.12 0.24 0.08

Infidelity0.015 Infidelity0.100.08 Infidelity0.220.20 Infidelity0.060.04

0.010 0.18 0.02

0.06 0.16 0.00

2.0 2.5 3.0 3.5 4.0 4.5 5.0 2.0 2.5 3.0 3.5 4.0 4.5 5.0 2.0 2.5 3.0 3.5 4.0 4.5 5.0 2.0 2.5 3.0 3.5 4.0 4.5 5.0

Number of environments Number of environments Number of environments Number of environments

(c) FMNIST (random)


0.16

0.14

0.12

0.10

0.08

0.06

0.04

0.02

0.00

2.0 2.5 3.0 3.5 4.0 4.5

Number of environments

(d) FMNIST (realistic)


(a) Iris

1.00

0.9 0.99 0.780 0.99955

0.8 0.98 0.778 0.99954

0.97 0.776 0.99953

0.7 0.96 0.774 0.99952

0.6 0.950.94 0.7720.770 0.999510.99950

0.5 0.93 0.768 0.99949

0.2 0.4 0.6 0.8 1.0 1.2 1.4 0.5 1.0 1.5 2.0 2.5 3.0 3.5 4.0 2.5 5.0 7.5 10.0 12.5 15.0 17.5 20.0 0.5 1.0 1.5 2.0 2.5 3.0

Kernel width Kernel width Kernel width Kernel width


(a) Iris


(b) MEPS

0.085 0.0375

0.080 0.0350

0.075 0.0325

0.070 0.0300

Infidelity0.065 Infidelity0.0275

0.060 0.0250

0.055 0.0225

0.050 0.0200

2.0 2.5 3.0 3.5 4.0 4.5 5.0 2.0 2.5 3.0 3.5 4.0 4.5 5.0

Number of environments Number of environments


(e) Rotten Tomatoes
(random)


(f) Rotten Tomatoes
(realistic)


Figure 13: Infidelity (INFD) vs. Number of environments.

1.00


(b) MEPS


(c) FMNIST


(d) Rotten Tomatoes


Figure 14: Unidirectionality (Υ) vs. Kernel width.


-----

0.6 0.010 0.007 0.00024

0.5 0.009 0.006 0.00023

0.4 0.008 0.00022

0.30.2 0.0070.0060.005 0.0050.004 0.000210.00020

Coefficient inconsistency0.10.0 0.2 0.4 0.6 0.8 1.0 1.2 1.4 Coefficient inconsistency0.0040.003 0.5 1.0 1.5 2.0 2.5 3.0 3.5 4.0 Coefficient inconsistency0.0030.002 2.5 5.0 7.5 10.0 12.5 15.0 17.5 20.0 Coefficient inconsistency0.000190.000180.00017 0.5 1.0 1.5 2.0 2.5 3.0

Kernel width Kernel width Kernel width Kernel width


(a) Iris

1.0

0.60 0.955

0.8 0.55 0.950

0.945

0.6 0.50 0.940

0.4 0.45 0.935

0.40 0.930

Class attribution consistency0.2 Class attribution consistency0.35 Class attribution consistency0.9250.920

0.2 0.4 0.6 0.8 1.0 1.2 1.4 2.5 5.0 7.5 10.0 12.5 15.0 17.5 20.0 0.5 1.0 1.5 2.0 2.5 3.0

Kernel width Kernel width Kernel width

0.25 0.30 0.32 0.39

0.20 0.25 0.31 0.38

0.15 0.20 0.30 0.37

0.10 0.15 0.29 0.36

Generalized infidelity0.05 Generalized infidelity0.10 Generalized infidelity0.28 Generalized infidelity0.35

0.2 0.4 0.6 0.8 1.0 1.2 1.4 0.5 1.0 1.5 2.0 2.5 3.0 3.5 4.0 2.5 5.0 7.5 10.0 12.5 15.0 17.5 20.0 0.5 1.0 1.5 2.0 2.5 3.0

Kernel width Kernel width Kernel width Kernel width


(a) Iris


(b) MEPS


(c) FMNIST


(d) Rotten Tomatoes


Figure 15: Coefficient inconsistency (CI) vs. Kernel width.


(a) Iris


(b) FMNIST


(c) Rotten Tomatoes


Figure 16: Class attribution consistency (CAC) vs. Kernel width.

0.39


0.25


(b) MEPS


(c) FMNIST


(d) Rotten Tomatoes


Figure 17: Generalized infidelity (GI) vs. Kernel width.


0.08 0.30 0.26 0.12

0.25 0.24 0.10

0.06 0.20 0.22 0.08

0.04 0.15 0.20 0.06

Infidelity Infidelity0.10 Infidelity0.18 Infidelity0.04

0.02 0.05 0.16 0.02

0.00 0.00 0.00

0.2 0.4 0.6 0.8 1.0 1.2 1.4 0.5 1.0 1.5 2.0 2.5 3.0 3.5 4.0 1 2 3 4 5 6 2.5 5.0 7.5 10.0 12.5 15.0 17.5 20.0

Kernel width Kernel width Kernel width Kernel width


(a) Iris


(b) MEPS

0.175

0.029

0.150 0.028

0.125 0.027

0.100 0.026

Infidelity0.075 Infidelity0.025

0.050 0.024

0.025 0.023

0.000 0.022

0.5 1.0 1.5 2.0 2.5 3.0 0.5 1.0 1.5 2.0 2.5 3.0

Kernel width Kernel width


(e) Rotten Tomatoes
(random)


(c) FMNIST (random)

0.029

0.028

0.027

0.026

0.025

0.024

0.023

0.022

0.5 1.0 1.5 2.0 2.5 3.0

Kernel width

(f) Rotten Tomatoes
(realistic)


(d) FMNIST (realistic)


Figure 18: Infidelity (INFD) vs. Kernel width.


D EXAMPLE FEATURE ATTRIBUTIONS IN TEXT DATA: MELIME VS LINEX

Below we see sample attributions by the two methods along with the magnitude of the attributions.
Attribution magnitudes are printed with a precision of 10[−][3] and shown along with the corresponding
words in descending order.


D.1 POSITIVE SENTIMENT

enticing and often funny documentary .
MeLIME: documentary funny and enticing often
LINEX : documentary funny often enticing and
MeLIME: 0.517 0.446 0.333 0.317 0.311


-----

LINEX : 0.416 0.377 0.342 0.331 0.330

one-of-a-kind near-masterpiece .
MeLIME: kind near masterpiece
LINEX : masterpiece kind one
MeLIME: 0.832 0.695 0.182
LINEX : 0.712 0.384 0.381

a fast, funny, highly enjoyable movie .
MeLIME: enjoyable highly funny fast movie
LINEX : enjoyable highly fast funny movie
MeLIME: 0.550 0.432 0.412 0.389 0.198
LINEX : 0.409 0.389 0.372 0.350 0.326

ferrara’s strongest and most touching movie of recent years .
MeLIME: touching years most strongest and
LINEX : touching most recent strongest and
MeLIME: 0.735 0.490 0.450 0.443 0.427
LINEX : 0.490 0.488 0.450 0.444 0.407

saved from being merely way-cool by a basic, credible compassion .
MeLIME: cool basic credible merely from
LINEX: cool credible merely compassion from
MeLIME: 1.514 0.050 0.040 0.029 0.026
LINEX : 0.358 0.308 0.304 0.299 0.293

really quite funny .
MeLIME: funny quite really
LINEX : funny quite really
MeLIME: 0.559 0.417 0.233
LINEX : 0.462 0.368 0.275

spare yet audacious . . .
MeLIME: spare yet audacious
LINEX : audacious spare yet
MeLIME: 0.626 0.447 0.395
LINEX : 0.501 0.431 0.422

an engrossing and infectiously enthusiastic documentary .
MeLIME: engrossing documentary and enthusiastic an
LINEX : engrossing documentary an enthusiastic and
MeLIME: 0.593 0.455 0.358 0.354 0.333
LINEX : 0.461 0.407 0.374 0.357 0.350

a wildly funny prison caper .
MeLIME: funny caper wildly prison
LINEX : funny caper prison wildly
MeLIME: 0.541 0.364 0.214 0.193
LINEX : 0.403 0.335 0.245 0.239

this charming but slight tale has warmth, wit
and interesting characters compassionately portrayed .
MeLIME: charming compassionately and interesting portrayed
LINEX : charming compassionately has tale portrayed
MeLIME: 0.690 0.507 0.456 0.444 0.424
LINEX : 0.464 0.435 0.431 0.430 0.429

thoughtful, provocative and entertaining .
MeLIME: thoughtful entertaining and provocative


-----

LINEX : thoughtful entertaining and provocative
MeLIME: 0.612 0.517 0.402 0.395
LINEX : 0.505 0.461 0.415 0.404

the film is quiet, threatening and unforgettable .
MeLIME: quiet unforgettable and film the
LINEX : unforgettable quiet film and is
MeLIME: 0.597 0.483 0.412 0.325 0.303
LINEX : 0.421 0.416 0.388 0.378 0.338

a moving tale of love and destruction in unexpected places, unexamined lives .
MeLIME: unexpected moving love tale lives
LINEX : moving unexpected places lives in
MeLIME: 0.692 0.662 0.577 0.538 0.499
LINEX : 0.538 0.530 0.521 0.513 0.501

though frodo’s quest remains unfulfilled, a hardy group of
determined new zealanders has proved its creative mettle .
MeLIME: creative group proved has new
LINEX : creative quest its proved determined
MeLIME: 0.602 0.441 0.424 0.402 0.393
LINEX : 0.410 0.392 0.390 0.385 0.381

D.2 NEGATIVE SENTIMENT

originality is sorely lacking .
MeLIME: lacking sorely is originality
LINEX : lacking sorely originality is
MeLIME: 0.543 0.381 0.296 0.278
LINEX : 0.430 0.356 0.314 0.271

an ugly, pointless, stupid movie .
MeLIME: stupid pointless ugly movie an
LINEX : stupid pointless ugly movie an
MeLIME: 0.543 0.499 0.385 0.365 0.276
LINEX : 0.446 0.411 0.373 0.360 0.350

so devoid of pleasure or sensuality that it cannot even be dubbed hedonistic .
MeLIME: devoid even be dubbed of
LINEX : devoid so dubbed be cannot
MeLIME: 0.666 0.416 0.413 0.372 0.344
LINEX : 0.400 0.392 0.387 0.380 0.368

neither revelatory nor truly edgy--merely crassly flamboyant
and comedically labored .
MeLIME: edgy neither nor labored revelatory
LINEX : edgy neither nor labored truly
MeLIME: 1.256 0.338 0.277 0.204 0.021
LINEX : 0.439 0.398 0.398 0.369 0.349

occasionally funny, sometimes inspiring, often boring .
MeLIME: boring occasionally inspiring sometimes often
LINEX : boring occasionally sometimes often inspiring
MeLIME: 0.669 0.242 0.218 0.210 0.182
LINEX : 0.377 0.266 0.266 0.250 0.236

a cumbersome and cliche-ridden movie greased
with every emotional device known to man .
MeLIME: cliche every device movie with


-----

LINEX : cliche every man cumbersome emotional
MeLIME: 0.695 0.449 0.327 0.280 0.268
LINEX : 0.385 0.361 0.354 0.349 0.309

ponderous, plodding soap opera disguised as a feature film .
MeLIME: plodding soap ponderous opera disguised
LINEX : plodding soap film ponderous feature
MeLIME: 0.579 0.522 0.421 0.408 0.382
LINEX : 0.442 0.440 0.418 0.406 0.377

kitschy, flashy, overlong soap opera .
MeLIME: soap flashy opera overlong kitschy
LINEX : soap flashy opera overlong kitschy
MeLIME: 0.499 0.397 0.391 0.358 0.230
LINEX : 0.389 0.362 0.360 0.346 0.300

[a] poorly executed comedy .
MeLIME: poorly comedy executed
LINEX : poorly comedy executed
MeLIME: 0.653 0.348 0.257
LINEX : 0.502 0.335 0.309

a bad movie that happened to good actors .
MeLIME: bad happened movie to that
LINEX : bad happened to movie actors
MeLIME: 0.692 0.396 0.371 0.367 0.242
LINEX : 0.442 0.384 0.367 0.361 0.344

a complete waste of time .
MeLIME: waste complete time of
LINEX : waste complete time of
MeLIME: 0.614 0.425 0.313 0.247
LINEX : 0.480 0.381 0.348 0.278

don’t waste your money .
MeLIME: waste money don your
LINEX : waste money don your
MeLIME: 0.592 0.497 0.408 0.309
LINEX : 0.483 0.450 0.411 0.337

witless and utterly pointless .
MeLIME: pointless witless and utterly
LINEX : pointless witless utterly and
MeLIME: 0.652 0.491 0.263 0.245
LINEX : 0.506 0.444 0.311 0.269

E EXAMPLE FEATURE ATTRIBUTIONS IN IMAGE DATA: MELIME VS LINEX

We show feature attributions for individual example images with MeLIME and LINEX with MeLIME
perturbations in Figure 19. In Figure 20 we show class-wise mean feature attributions along with
mean images. Clearly, LINEX explanations provide more meaningful feature attributions.

F RESULTS FOR ALL METHODS INCLUDING SHAP

In Table 4, we provide the results for SHAP along with all methods for easy comparison. Note
that SHAP does not have standard errors since it is computed only once per test point. The INFD
values for SHAP are miniscule since SHAP values add up to the predictions by definition. In order to


-----

(1a) MeLIME:r = 0.279 (1b) LINEX:r = 0.603 (1c) Original (2a) MeLIME:r = 0.434 (2b) LINEX:r = 0.689 (2c) Original

0 0 0 0 0 0

5 5 5 5 5 5

10 10 10 10 10 10

15 15 15 15 15 15

20 20 20 20 20 20

25 25 25 25 25 25

0 10 20 0 10 20 0 10 20 0 10 20 0 10 20 0 10 20

(3a) MeLIME:r = 0.259 (3b) LINEX:r = 0.711 (3c) Original (4a) MeLIME:r = 0.292 (4b) LINEX:r = 0.633 (4c) Original

0 0 0 0 0 0

5 5 5 5 5 5

10 10 10 10 10 10

15 15 15 15 15 15

20 20 20 20 20 20

25 25 25 25 25 25

0 10 20 0 10 20 0 10 20 0 10 20 0 10 20 0 10 20

(5a) MeLIME:r = 0.271 (5b) LINEX:r = 0.701 (5c) Original (6a) MeLIME:r = 0.297 (6b) LINEX:r = 0.962 (6c) Original

0 0 0 0 0 0

5 5 5 5 5 5

10 10 10 10 10 10

15 15 15 15 15 15

20 20 20 20 20 20

25 25 25 25 25 25

0 10 20 0 10 20 0 10 20 0 10 20 0 10 20 0 10 20

(7a) MeLIME:r = 0.482 (7b) LINEX:r = 0.820 (7c) Original (8a) MeLIME:r = 0.177 (8b) LINEX:r = 0.788 (8c) Original

0 0 0 0 0 0

5 5 5 5 5 5

10 10 10 10 10 10

15 15 15 15 15 15

20 20 20 20 20 20

25 25 25 25 25 25

0 10 20 0 10 20 0 10 20 0 10 20 0 10 20 0 10 20

(9a) MeLIME:0 r = 0.247 0(9b) LINEX:r = 0.729 0 (9c) Original (10a) MeLIME:0 r = 0.381(10b) LINEX:0 r = 0.792 0 (10c) Original

5 5 5 5 5 5

10 10 10 10 10 10

15 15 15 15 15 15

20 20 20 20 20 20

25 25 25 25 25 25

0 10 20 0 10 20 0 10 20 0 10 20 0 10 20 0 10 20


Figure 19: Results using individual samples for realistic perturbations for FMNIST dataset for all
classes:1-10 (T-shirt/top, Trouser, Pullover, Dress, Coat, Sandal, Shirt, Sneaker, Bag and Ankle boot).
(a) MeLIME feature attributions for an image. (b) LINEX feature attributions for an image. (c)
Original image in the class. The r values show Pearson’s correlation between feature attributions
and the original image from the respective class. We observe that LINEX attributions/explanations
exhibit significantly higher correlation with the original image belonging to a particular class (i.e.
high CAC).

compute GI, CI, Υ, CAC, we convert the SHAP values to SHAP attributions (Amparore et al., 2021,
equation 4) first and follow the same approach used by other explanation methods.

G ERROR ANALYSIS OF LINEX

We perform error analysis for LINEX to gain better understanding about the method. We choose
FMNIST dataset for doing this since: (a) This is the highest dimensional dataset (784 dimensions)
that we have. (b) LINEX/real under performs MeLIME in terms of the INFD measure here (see
Table 5) more heavily compared to other datasets. (c) It is easier to visualize the explanations for this
dataset.

We start by observing that even though LINEX/real underperforms in the INFD metric, the gap is not
so great in the GI metric, which suggests that MeLIME may be overfitting explanations here. We
also note that in terms of CI, Υ, and CAC metrics, LINEX/real clearly outperforms MeLIME.


-----

(1a) MeLIME:r=0.269 (1b) LINEX:r=0.549 (1c) Original (2a) MeLIME:r=0.486 (2b) LINEX:r=0.809 (2c) Original

0 0 0 0 0 0

5 5 5 5 5 5

10 10 10 10 10 10

15 15 15 15 15 15

20 20 20 20 20 20

25 25 25 25 25 25

0 10 20 0 10 20 0 10 20 0 10 20 0 10 20 0 10 20

(3a) MeLIME:r=0.311 (3b) LINEX:r=0.619 (3c) Original (4a) MeLIME:r=0.418 (4b) LINEX:r=0.860 (4c) Original

0 0 0 0 0 0

5 5 5 5 5 5

10 10 10 10 10 10

15 15 15 15 15 15

20 20 20 20 20 20

25 25 25 25 25 25

0 10 20 0 10 20 0 10 20 0 10 20 0 10 20 0 10 20

(5a) MeLIME:r=0.323 (5b) LINEX:r=0.696 (5c) Original (6a) MeLIME:r=0.419 (6b) LINEX:r=0.800 (6c) Original

0 0 0 0 0 0

5 5 5 5 5 5

10 10 10 10 10 10

15 15 15 15 15 15

20 20 20 20 20 20

25 25 25 25 25 25

0 10 20 0 10 20 0 10 20 0 10 20 0 10 20 0 10 20

(7a) MeLIME:r=0.379 (7b) LINEX:r=0.783 (7c) Original (8a) MeLIME:r=0.432 (8b) LINEX:r=0.840 (8c) Original

0 0 0 0 0 0

5 5 5 5 5 5

10 10 10 10 10 10

15 15 15 15 15 15

20 20 20 20 20 20

25 25 25 25 25 25

0 10 20 0 10 20 0 10 20 0 10 20 0 10 20 0 10 20

(9a) MeLIME:r=0.373 (9b) LINEX:r=0.757 (9c) Original (10a) MeLIME:r=0.441 (10b) LINEX:r=0.852 (10c) Original

0 0 0 0 0 0

5 5 5 5 5 5

10 10 10 10 10 10

15 15 15 15 15 15

20 20 20 20 20 20

25 25 25 25 25 25

0 10 20 0 10 20 0 10 20 0 10 20 0 10 20 0 10 20


Figure 20: Results using realistic perturbations for FMNIST dataset with mean feature importances
for all classes:1-10 (T-shirt/top, Trouser, Pullover, Dress, Coat, Sandal, Shirt, Sneaker, Bag and Ankle
_boot). (a) Mean feature attributions of all images in the class using MeLIME. (b) Mean feature_
attributions of all images in the class using LINEX. (c) Mean of all images in the class. The r values
show Pearson’s correlation between average feature attributions and mean of the original images
from the respective classes. We observe that LINEX explanations/attributions exhibit significantly
higher correlation with the original images belonging to a particular class (i.e. high CAC).

We now choose a sample of images from the dataset where LINEX/real has highest instancelevel infidelity numbers and display them in Figure 21. Just looking at the explanations and the
corresponding original images visually, it is evident that LINEX/real highlights the prominent features
like stripes in the t-shirt, handles of the bags, outlines of the boots/shoes, even though the infidelity
values are high. However, MeLIME misses out on some of these prominent features and focuses only
on optimizing the fit. The fact that LINEX zeroes in on important features also provides additional
evidence for the closeness of GI metrics between the two methods, and the better performance of
LINEX/real with CI, Υ, and CAC metrics.


-----

Table 4: Comparing the different methods using metrics infidelity (INFD), generalized infidelity
(GI), coefficient inconsistency (CI), class attribution consistency (CAC) and unidirectionality (Υ).
_↑_ indicates higher value for the metric is better, and ↓ indicates lower is better. Results better by ≥
1% are bolded. LINEX is better than baselines in 18 out of 35 cases, and worse only in 5 cases. We
set neighborhood sizes 10 (IRIS), 500 (MEPS), 100 (FMNIST-random), 500 (FMNIST-realistic),
100 (Rotten tomatoes) for generating local explanations. We also use 3, 10, 10, 5 as exemplar
neighborhood sizes to compute GI, CI and Υ metrics for the four datasets respectively.

|Dataset|Method|INFD ↓|GI ↓|CI ↓|Υ ↑|CAC ↑|
|---|---|---|---|---|---|---|
|IRIS|LIME S-LIME LINEX/rand|0.015 ± 0.011 0.015 ± 0.010 0.013 ± 0.009|0.132 ± 0.042 0.077 ± 0.011 0.052 ± 0.008|0.319 ± 0.132 0.143 ± 0.045 0.044 ± 0.013|0.646 ± 0.040 0.704 ± 0.037 0.802 ± 0.043|0.667 ± 0.167 0.878 ± 0.034 0.921 ± 0.042|
||MeLIME LINEX/real|0.008 ± 0.003 0.009 ± 0.003|0.049 ± 0.018 0.029 ± 0.003|0.219 ± 0.108 0.024 ± 0.002|0.629 ± 0.013 0.744 ± 0.044|0.464 ± 0.100 0.942 ± 0.023|
||MAPLE LINEX/mpl|0.009 ± 0.001 0.013 ± 0.000|0.038 ± 0.004 0.020 ± 0.000|0.261 ± 0.033 0.026 ± 0.002|0.458 ± 0.032 0.694 ± 0.008|0.586 ± 0.035 0.929 ± 0.004|
||SHAP|0.007|0.197|0.248|0.664|0.524|
|MEPS|LIME S-LIME LINEX/rand|0.158 ± 0.066 0.158 ± 0.066 0.130 ± 0.052|0.214 ± 0.041 0.214 ± 0.042 0.164 ± 0.021|0.005 ± 0.001 0.005 ± 0.001 0.003 ± 0.001|0.981 ± 0.006 0.974 ± 0.008 0.979 ± 0.006|NA|
||MAPLE LINEX/mpl|0.063 ± 0.000 0.098 ± 0.001|0.067 ± 0.000 0.094 ± 0.001|0.007 ± 0.000 0.007 ± 0.000|0.957 ± 0.000 0.950 ± 0.000|NA|
||SHAP|0.000|0.091|0.009|0.940|NA|
|FMNIST|LIME S-LIME LINEX/rand|0.162 ± 0.003 0.142 ± 0.003 0.149 ± 0.002|NA|NA|NA|NA|
||MeLIME LINEX/real|0.001 ± 0.000 0.100 ± 0.002|0.277 ± 0.000 0.304 ± 0.001|0.007 ± 0.000 0.002 ± 0.000|0.769 ± 0.000 0.780 ± 0.000|0.327 ± 0.000 0.649 ± 0.001|
||SHAP|0.000|1.962|0.589|0.551|0.038|
|Rotten Tomatoes|LIME S-LIME LINEX/rand|0.079 ± 0.036 0.075 ± 0.035 0.069 ± 0.032|NA|NA|NA|NA|
||MeLIME LINEX/real|0.029 ± 0.001 0.053 ± 0.000|0.391 ± 0.000 0.361 ± 0.000|0.000 ± 0.000 0.000 ± 0.000|0.999 ± 0.000 1.000 ± 0.000|0.909 ± 0.000 0.953 ± 0.001|
||SHAP|0.000|0.384|0.008|0.999|0.015|


_Dataset_ _Method_ INFD ↓ GI ↓ CI ↓ Υ ↑ CAC ↑

LIME 0.015 ± 0.011 0.132 ± 0.042 0.319 ± 0.132 0.646 ± 0.040 0.667 ± 0.167

S-LIME 0.015 ± 0.010 0.077 ± 0.011 0.143 ± 0.045 0.704 ± 0.037 0.878 ± 0.034

LINEX/rand 0.013 ± 0.009 0.052 ± 0.008 0.044 ± 0.013 0.802 ± 0.043 0.921 ± 0.042

MeLIME 0.008 0.003 0.049 0.018 0.219 0.108 0.629 0.013 0.464 0.100

_IRIS_ _±_ _±_ _±_ _±_ _±_

LINEX/real 0.009 ± 0.003 0.029 ± 0.003 0.024 ± 0.002 0.744 ± 0.044 0.942 ± 0.023

MAPLE 0.009 ± 0.001 0.038 ± 0.004 0.261 ± 0.033 0.458 ± 0.032 0.586 ± 0.035

LINEX/mpl 0.013 ± 0.000 0.020 ± 0.000 0.026 ± 0.002 0.694 ± 0.008 0.929 ± 0.004

SHAP 0.007 0.197 0.248 0.664 0.524

LIME 0.158 ± 0.066 0.214 ± 0.041 0.005 ± 0.001 0.981 ± 0.006

S-LIME 0.158 ± 0.066 0.214 ± 0.042 0.005 ± 0.001 0.974 ± 0.008 NA

LINEX/rand 0.130 **0.052 0.164** **0.021 0.003** 0.001 0.979 0.006

_MEPS_ _±_ _±_ _±_ _±_

MAPLE **0.063** **0.000 0.067** **0.000 0.007** 0.000 0.957 0.000
_±_ _±_ _±_ _±_ NA
LINEX/mpl 0.098 ± 0.001 0.094 ± 0.001 0.007 ± 0.000 0.950 ± 0.000

SHAP 0.000 0.091 0.009 0.940 NA

LIME 0.162 ± 0.003

S-LIME 0.142 0.003 NA NA NA NA

_FMNIST_ _±_

LINEX/rand 0.149 ± 0.002

MeLIME **0.001 ± 0.000 0.277 ± 0.000 0.007 ± 0.000** 0.769 ± 0.000 0.327 ± 0.000

LINEX/real 0.100 ± 0.002 0.304 ± 0.001 0.002 ± 0.000 0.780 ± 0.000 0.649 ± 0.001

SHAP 0.000 1.962 0.589 0.551 0.038

LIME 0.079 ± 0.036

_Rotten_ S-LIME 0.075 ± 0.035 NA NA NA NA
_Tomatoes_ LINEX/rand 0.069 ± 0.032

MeLIME **0.029 ± 0.001 0.391 ± 0.000** 0.000 ± 0.000 0.999 ± 0.000 0.909 ± 0.000

LINEX/real 0.053 ± 0.000 0.361 ± 0.000 0.000 ± 0.000 1.000 ± 0.000 0.953 ± 0.001

SHAP 0.000 0.384 0.008 0.999 0.015


-----

Table 5: Comparing the different methods using metrics infidelity (INFD), generalized infidelity
(GI), coefficient inconsistency (CI), class attribution consistency (CAC) and unidirectionality (Υ).
_↑_ indicates higher value for the metric is better, and ↓ indicates lower is better. Results that are
statistically significantly better based on paired t-test are bolded. LINEX is better than baselines in
19 out of 35 cases, and worse only in 6 cases. Plots showing behavior of these metrics with varying
neighborhood size, number of environments and kernel width are in Appendix C.

|Dataset|Method|INFD ↓|GI ↓|CI ↓|Υ ↑|CAC ↑|
|---|---|---|---|---|---|---|
|IRIS|LIME S-LIME LINEX/rand|0.015 ± 0.011 0.015 ± 0.010 0.013 ± 0.009|0.132 ± 0.042 0.077 ± 0.011 0.052 ± 0.008|0.319 ± 0.132 0.143 ± 0.045 0.044 ± 0.013|0.646 ± 0.040 0.704 ± 0.037 0.802 ± 0.043|0.667 ± 0.167 0.878 ± 0.034 0.921 ± 0.042|
||MeLIME LINEX/real|0.008 ± 0.003 0.009 ± 0.003|0.049 ± 0.018 0.029 ± 0.003|0.219 ± 0.108 0.024 ± 0.002|0.629 ± 0.013 0.744 ± 0.044|0.464 ± 0.100 0.942 ± 0.023|
||MAPLE LINEX/mpl|0.009 ± 0.001 0.013 ± 0.000|0.038 ± 0.004 0.020 ± 0.000|0.261 ± 0.033 0.026 ± 0.002|0.458 ± 0.032 0.694 ± 0.008|0.586 ± 0.035 0.929 ± 0.004|
|MEPS|LIME S-LIME LINEX/rand|0.158 ± 0.066 0.158 ± 0.066 0.130 ± 0.052|0.214 ± 0.041 0.214 ± 0.042 0.164 ± 0.021|0.005 ± 0.001 0.005 ± 0.001 0.003 ± 0.001|0.981 ± 0.006 0.974 ± 0.008 0.979 ± 0.006|NA|
||MAPLE LINEX/mpl|0.063 ± 0.000 0.098 ± 0.001|0.067 ± 0.000 0.094 ± 0.001|0.007 ± 0.000 0.007 ± 0.000|0.957 ± 0.000 0.950 ± 0.000|NA|
|FMNIST|LIME S-LIME LINEX/rand|0.162 ± 0.003 0.142 ± 0.003 0.149 ± 0.002|NA|NA|NA|NA|
||MeLIME LINEX/real|0.001 ± 0.000 0.100 ± 0.002|0.277 ± 0.000 0.304 ± 0.001|0.007 ± 0.000 0.002 ± 0.000|0.769 ± 0.000 0.780 ± 0.000|0.327 ± 0.000 0.649 ± 0.001|
|Rotten Tomatoes|LIME S-LIME LINEX/rand|0.079 ± 0.036 0.075 ± 0.035 0.069 ± 0.032|NA|NA|NA|NA|
||MeLIME LINEX/real|0.029 ± 0.001 0.053 ± 0.000|0.391 ± 0.000 0.361 ± 0.000|0.000 ± 0.000 0.000 ± 0.000|0.999 ± 0.000 1.000 ± 0.000|0.909 ± 0.000 0.953 ± 0.001|


_Dataset_ _Method_ INFD ↓ GI ↓ CI ↓ Υ ↑ CAC ↑

LIME 0.015 ± 0.011 0.132 ± 0.042 0.319 ± 0.132 0.646 ± 0.040 0.667 ± 0.167

S-LIME 0.015 ± 0.010 0.077 ± 0.011 0.143 ± 0.045 0.704 ± 0.037 0.878 ± 0.034

LINEX/rand 0.013 ± 0.009 0.052 ± 0.008 0.044 ± 0.013 0.802 ± 0.043 0.921 ± 0.042

_IRIS_ MeLIME 0.008 ± 0.003 0.049 ± 0.018 0.219 ± 0.108 0.629 ± 0.013 0.464 ± 0.100

LINEX/real 0.009 ± 0.003 0.029 ± 0.003 0.024 ± 0.002 0.744 ± 0.044 0.942 ± 0.023

MAPLE **0.009 ± 0.001 0.038 ± 0.004** 0.261 ± 0.033 0.458 ± 0.032 0.586 ± 0.035

LINEX/mpl 0.013 ± 0.000 0.020 ± 0.000 0.026 ± 0.002 0.694 ± 0.008 0.929 ± 0.004

LIME 0.158 ± 0.066 0.214 ± 0.041 0.005 ± 0.001 0.981 ± 0.006

S-LIME 0.158 ± 0.066 0.214 ± 0.042 0.005 ± 0.001 0.974 ± 0.008 NA

_MEPS_ LINEX/rand 0.130 ± 0.052 0.164 ± 0.021 0.003 ± 0.001 0.979 ± 0.006

MAPLE **0.063** **0.000 0.067** **0.000 0.007** 0.000 0.957 0.000
_±_ _±_ _±_ _±_ NA
LINEX/mpl 0.098 ± 0.001 0.094 ± 0.001 0.007 ± 0.000 0.950 ± 0.000

LIME 0.162 ± 0.003

S-LIME 0.142 ± 0.003 NA NA NA NA

_FMNIST_ LINEX/rand 0.149 ± 0.002

MeLIME **0.001 ± 0.000 0.277 ± 0.000 0.007 ± 0.000** 0.769 ± 0.000 0.327 ± 0.000

LINEX/real 0.100 ± 0.002 0.304 ± 0.001 0.002 ± 0.000 0.780 ± 0.000 0.649 ± 0.001

LIME 0.079 ± 0.036

S-LIME 0.075 0.035 NA NA NA NA

_Rotten_ _±_

LINEX/rand 0.069 0.032

_Tomatoes_ _±_

MeLIME **0.029 ± 0.001 0.391 ± 0.000** 0.000 ± 0.000 0.999 ± 0.000 0.909 ± 0.000

LINEX/real 0.053 ± 0.000 0.361 ± 0.000 0.000 ± 0.000 1.000 ± 0.000 0.953 ± 0.001


-----

MeLime: r=0.121, INFD=0.001 LINEX/real: r=0.334, INFD=0.169 Original

MeLime: r=0.093, INFD=0.001 LINEX/real: r=0.338, INFD=0.216 Original

MeLime: r=0.112, INFD=0.001 LINEX/real: r=0.539, INFD=0.201 Original

MeLime: r=0.114, INFD=0.001 LINEX/real: r=0.412, INFD=0.189 Original

MeLime: r=0.393, INFD=0.001 LINEX/real: r=0.713, INFD=0.180 Original

MeLime: r=0.162, INFD=0.001 LINEX/real: r=0.528, INFD=0.161 Original


Figure 21: Error analysis for a chosen set of examples in FMNIST using MeLIME and LINEX/real
methods. The three columns are the MeLIME feature attributions, LINEX/real feature attributions,
and the original images. The rows correspond to different examples. We show the Pearson’s
correlation coefficient between feature attributions and mean of the original images from the respective
classes (r) and instance-level infidelity (INFD) measures. LINEX seems to highlight important
features like stripes in the t-shirt, handles of the bags, outlines of the boots/shoes more prominently,
while MeLIME seems to overfit to the data while missing out on highlighting some key features
prominently


-----

