#### RELATING TRANSFORMERS TO MODELS AND NEURAL
##### REPRESENTATIONS OF THE HIPPOCAMPAL FORMATION

**James C.R. Whittington[∗]** **Joseph Warren, Timothy E.J. Behrens**
University of Oxford & Stanford University University of Oxford & University College London

ABSTRACT

Many deep neural network architectures loosely based on brain networks have recently been shown to replicate neural firing patterns observed in the brain. One
of the most exciting and promising novel architectures, the Transformer neural
network, was developed without the brain in mind. In this work, we show that
transformers, when equipped with recurrent position encodings, replicate the precisely tuned spatial representations of the hippocampal formation; most notably
place and grid cells. Furthermore, we show that this result is no surprise since
it is closely related to current hippocampal models from neuroscience. We additionally show the transformer version offers dramatic performance gains over the
neuroscience version. This work continues to bind computations of artificial and
brain networks, offers a novel understanding of the hippocampal-cortical interaction, and suggests how wider cortical areas may perform complex tasks beyond
current neuroscience models such as language comprehension.

1 INTRODUCTION

The last ten years have seen dramatic developments using deep neural networks, from computer
vision (Krizhevsky et al., 2012) to natural language processing and beyond (Vaswani et al., 2017).
During the same time, neuroscientists have used these tools to build models of the brain that explain
neural recordings at a precision not seen before (Yamins et al., 2014; Banino et al., 2018; Whittington et al., 2020). For example, representations from convolutional neural networks (Lecun et al.,
1998) predict neurons in visual and inferior temporal cortex (Yamins et al., 2014; Khaligh-Razavi &
Kriegeskorte, 2014), representations from transformer neural networks (Vaswani et al., 2017) predict
brain representations in language areas (Schrimpf et al., 2020), and lastly recurrent neural networks
(Cueva & Wei, 2018; Banino et al., 2018; Sorscher et al., 2019) have been shown to recapitulate
grid cells (Hafting et al., 2005) from medial entorhinal cortex. Being able to use models from machine learning to predict brain representations provides a deeper understanding into the mechanistic
computations of the respective brain areas, and offers deeper insight into the nature of the models.

As well as using off-the-shelf machine learning models, neuroscience has developed bespoke deep
learning models (mixing together recurrent networks with memory networks) that learn neural representations that mimic the exquisite spatial representations found in hippocampus and entorhinal
cortex (Whittington et al., 2020; Uria et al., 2020), including grid cells (Hafting et al., 2005), band
cells (Krupic et al., 2012), and place cells (O’Keefe & Dostrovsky, 1971). However, since these
models are bespoke, it is not clear whether they, and by implication the hippocampal architecture,
are capable of the general purpose computations of the kind studied in machine learning.

In this work we 1) show that transformers (with a little twist) recapitulate spatial representations
found in the brain; 2) show a close mathematical relationship of this transformer to current hippocampal models from neuroscience (with a focus on Whittington et al. (2020) though the same is
true for Uria et al. (2020)); 3) offer a novel take on the computational role of the hippocampus, and
an instantiation of hippocampal indexing theory (Teyler & Rudy, 2007); 4) offer novel insights on
the role of positional encodings in transformers. 5) discuss whether similar computational principles
might apply to broader cognitive domains, such as language, either in the hippocampal formation or
in neocortical circuits.

_∗Correspondence to: jcrwhittington@gmail.com_


-----

Note, we are not saying the brain is closely related to transformers because it learns the same neural
representations, instead we are saying the relationship is close because we have shown a mathematical relationship between transformers and carefully formulated neuroscience models of the hippocampal formation. This relationship helps us get a better understanding of hippocampal models, it
also suggests a new mechanism for place cells that would not be possible without this mathematical
relationship, and finally it tells us something formal about position encodings in transformers.

2 TRANSFORMERS

Transformer Neural Networks (Vaswani et al., 2017) are highly successful machine learning algorithms. Originally developed for language, transformers perform well on other tasks that can be
posed sequentially, such as mathematical understanding, logic problems (Brown et al., 2020), and
image processing (Dosovitskiy et al., 2020).

Transformers accept a set of observations;bedding or image patch etc), and aim to predict missing elements of that set. The missing ele- X = {x1, x2, x3, · · ·, xT } (xt could be a word emments could be in the future, i.e. xt>T, or could be a missing part of a sentence or image, i.e.
_{x1 = the, x2 = cat, x3 = sat, x4 = ?, x5 = the, x6 = mat}._

**Self-attention. The core mechanism of transformers is self-attention. Self-attention allows each**
element to ‘attend’ to all other elements, and update itself accordingly. In the example data-set
above, the 4[th] element (?) could attend to the 2[nd] (cat), 3[rd] (sat), and 6[th] (mat) to understand
it should be on. Formally, to attend to another element each element (xt is a row vector) emits a
query (qt = xtWq) and compares it to other elements keys (kτ = xtWk). Each element is then
updated using yt = _τ_ _[κ][(][q][t][,][ k][τ]_ [)][v][τ] [, where][ κ][(][q][t][,][ k][τ] [)][ is kernel describing the similarity of][ q][t][ to]

**_kτ and vτ is the value computed by each element vτ = xtWv. Intuitively, the similarity measure_**
_κ(qt, kτ_ ) places more emphasis on the elements that are relevant for prediction; in this example,

[P]

the keys may contain information about whether the word is a noun, verb or adjective, while the
query may ‘ask’ for any elements that are nouns or verbs - elements that match this criteria (large
_κ(qt, kτ_ ), i.e. cat, sat, mat) are ‘attended’ to and therefore contribute more to the output yt.
Typically, the similarity measure is a softmax i.e. κ(qt, kτ ) = _e[β][q][t]_ _[·][k][τ]_

_τ_ _[′][ e][β][q][t]_ _[·][k][τ]_ _[′][ .]_
These equations can be succinctly expressed in matrix form, with all elements updated simultane-P
ously:

**_yt = softmax(_** **_[q]√[t][K]dk[T]_** )V _→_ **_Y = softmax(_** **_[QK]√dk[T]_** )V (1)

Here Q, K, V are matrices with rows filled by qt, kt, vt respectively, and the softmax is taken
independently for each row. After this update, each yt is then sent through a deep network (fθ(· · · ))
typically consisting of residual (He et al., 2016) and layer-normalisation (Ba et al., 2016b) layers to
produce zt = fθ(yt). Z is the output of the transformer which can then be used for prediction, or
sent through subsequent transformer blocks.

**Position encodings. Self-attention is permutation invariant and so tells you nothing about order of**
the inputs. Should the data be sequential (i.e. meaning depends on the order of elements, such as
in language, or navigation as we will see later!), it is necessary to additionally encode the position/
where x is in the sequence. This is typically done by adding a ‘position encoding’ that uniquely
identifies each time-step (the position embedding can be appended i.e.et - typically sines and cosines) to each input: ht = [xt, et], with self attention then performed using xt ← **_xt_** +et. Alternatively
**_ht as input._**

3 TRANSFORMERS LEARN ENTORHINAL REPRESENTATIONS

Here we show that transformers (with a small modification) recapitulate spatial representations grid and band cells - when trained on tasks that require abstract spatial knowledge.

**Spatial understanding task. The task (more detail in Appendix) is to predict upcoming sensory ob-**
servations xt+1 conditioned on taking an action at while moving around spatial environments (Figure 1a). For example, after seeing {(x1 = cat, a1 = North), (x2 = dog, a2 = East), (x3 =
frog, a3 = South), (x4 = pig, a4 = West), (x5 = ?, a5 = · · · )}, the aim is to predict


-----

**a** **Environment 1** **Environment 2** **b** **Transformer** y3

**(with causal attention mask)**

**A** **B** **A** **B** **D** **C**
**Key, Value** k0 v0 k1 v1 k2 v2

**B** **C** **D** **C** **A** **D** **iWk** **iWv**

**Query** q0 q1 q2 q3

**D** **A** **B** **A** **D** **C** **iWq**

**position encodingInput with** e0 x0 e1 x1 e2 x2 e3 ?

**A** **B** **A** **D** **?** **Observations (x)** **D** **A** **C** **A** **?**

**Actions (a)** W(a0) W(a1) W(a2)

**Recurrent connections**

**Real grid cell**

**c** **rate maps** **d** **With linear activations** **e** **With ReLu activations** **f** **Band cells**

**Cell rate maps** **Autocorrelations** **Cell rate maps** **Autocorrelations**

Figure 1: (a) Sequence prediction in spatial navigation tasks test abstract spatial understanding since
some sensory predictions can only be done by knowing (generalising) certain rules e.g. North +
East + South + West = 0 or Parent + Sibling + Niece = 0. Note, we use sequences drawn
from much larger graphs. (b) Transformer with recurrent position encodings. (c) Real grid cell
rate-maps (Hafting et al., 2005). (d-f) Learned position embedding rate-maps (i.e. average activity
at each spatial location; plots are spatially smoothed). (d-e) Resembling grid cells with (e) linear
activation or (e) ReLu activation post transition. (f) Resembling band cells (Krupic et al., 2012).

**_x5 = cat. For simplicity, we treat sensory observations as one-hot vectors, thus the prediction_**
problem is a classification problem.

When faced with an unseen stimulus-action pair (e.g. x4 = pig, a4 = West above; an action you
have never taken at that stimulus before), successful prediction requires more than just remembering
specific sequences of stimulus-action pairs; knowledge of the rules of space must be known; i.e.
North + East + South + West = 0 allows prediction of x5 = cat. Crucially, such rules
_generalise to any 2D spaces and may therefore be transferred to aid prediction in entirely novel_
2D environments. This is powerful, since unobserved relations between observed stimuli can be
inferred in a zero-shot manner.

However, these relational rules are not ‘known’ a priori and therefore must be learnt. We therefore
train across multiple different spatial environments which share the same underlying 4-connected
Euclidean structure (Figure 1a) - this means the model must learn and generalise the abstract structure of space to use for prediction in new environments.

To perform on these tasks, the three modifications to the transformer are:

1. Recall equation 1; yt = softmax( **_[q]√[t][K]dk[T] )V, where Q = HWq, K = HWk, V =_**

**_HWv, and H is a matrix of inputs and position encodings (i.e. its rows are ht = [xt, et])._**
We restrict these weight matrices such that queries (Q) and keys (K) are the same; Q, K =
**_EWe. We refer to this matrix as_** **_E[˜]. Thus the keys and queries only focus on position_**
encodings. Meanwhile, values are exclusively dependent on the stimulus component of H
i.e. V = XWx. We refer to this matrix as **_X[˜]_** .

**_etE[˜][T]_**

**_yt = softmax(_** **_[q]√[t][K]dk[T]_** )V _→_ **_yt = softmax(_** **[˜]√dk** )X[˜] (2)


|Col1|Col2|Col3|Col4|Col5|Col6|Col7|a Enviro A|Col9|Col10|nment 1 B A|Col12|Col13|Col14|Col15|Col16|E B|Col18|nvironment 2 D C|Col20|Col21|b|Col23|Col24|Col25|Col26|Col27|Col28|Col29|Col30|T (with c|Col32|Col33|ransform ausal attentio|Col35|Col36|er n mask)|Col38|y3|Col40|Col41|Col42|Col43|Col44|Col45|Col46|Col47|
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
||||||||B|||C||D||||C||A||D|||Key Q|, Value uery|||k0 v iWk q0||0 iWv|k1 q1|||v1||k q|2 v2 2||q3|||||||||
||||||||||||||||||||||||||||||||||||||||||||||||
||||||||D A||B|A A D||B ?|Observ||ation|A s (x) D||D A C||C A|pos ?||Inp ition|ut with encodin||g|iWq e0 x||0|e1|||x1||e|2 x2||e3|||||||||
||||||||||||||||||||||||||||||||||||||||?||||||||
||||||||||||||||||||||||||||||||||||||||||||||||
||||||||cRea rat||l grid e ma|cell d ps|||Acti Cell rat||ons e ma|(a) With linear ps||activations Auto||corr|elations|||e||Ce|W ll rate ma||W(a ith ps|0) Recu ReLu acti||rre vat|W(a1) nt connec ions Autocorrel||tio ati|W(a2 ns ons|) f|Band cell|s||||||||
||||||||||||||||||||||||||||||||||||||||||||||||
||||||||||||||||||||||||||||||||||||||||||||||||
|||||||Fi|g|ure|1:|(a|) S|eq|ue|nce|pr|ed|icti|on in|sp|at|ial|na|vig|ati|on|ta|sk|s te|st|abs|tra|ct|sp|atia|l|unders|ta|nding|s|ince|||||||
|||||||so E fr|m as o|e s t m|en + S muc|so o h|ry ut lar|pre h + ge|di W r g|cti es ra|ons t phs|c = .|an o 0 or (b)|nly Par Tra|be en nsf|do t or|ne + me|by Si r w|k bl it|no i h r|wi ng ecu|ng + rr|(g Ni en|ene ec t p|ra e osi|lis = 0 tio|ing . N n e|) o n|cer te, cod|tain we ing|r u s.|ules e se seq (c)|.g ue Re|. Nor nces d al gri|t r d|h + awn cell|||||||
|||||||ra at ac|te e ti|-m ach vat|aps sp ion|( at o|Ha ial r (e|ftin lo ) R|g ca e|et tio Lu|al., n; act|2 plo iv|005 ts a atio|). (d re s n po|-f) pat st t|L ial ra|ear ly nsi|ned sm tio|p oo n.|os th (f)|itio ed) Re|n . se|em (d- m|be e) blin|dd Re g|ing se ba|ra mb nd|te lin ce|-m g lls|aps gri (K|( d ru|i.e. av cells w pic et|er i al|age ac th (e) ., 201|ti li 2)|vity near .|||||||
|||||||x|5|=|ca|t.|F|or|si|mp|lic|ity|, w|e tre|at|se|nso|ry|ob|se|rva|tio|ns|as|o|ne-|ho|t|vec|tor|s,|thus t|h|e pred|ic|tion|||||||
|||||||pr W|o h|ble en|m i fac|s a ed|cl w|ass ith|ifi an|cat u|io nse|n p en|rob sti|lem. mulu|s-a|ct|ion|pa|ir|(e.|g.|x 4|=|p|ig|, a|4|=|We|st|a|bove;|an|actio|n|you|||||||
|||||||ha sp N|v e or|e n cifi t|eve c s h +|r t eq E|ak ue as|en nce t|at s +|tha of So|t st sti u|im mu th|ulu lus +|s be -acti Wes|for on t|e), pa =|su irs 0 a|cce ; k llo|ss no ws|ful wl p|pr ed red|ed ge ict|ict of io|ion th n o|re e f|qu rul x 5|ire es =|s of|mo sp ca|re t ace t.|ha C|n just must b ruciall|re e y|memb know , such|e n; r|ring i.e. ules|||||||
|||||||ge 2 in|n D fe|era en rre|lis viro d i|e t n n a|o a me ze|ny nts ro|2 . -sh|D Th ot|spa is i m|ce s an|s a pow ner.|nd m erfu|ay l,|t sin|her ce|efo un|re ob|be se|tr rve|an d|sf rel|erre atio|d n|to s b|aid etw|e|pre en|dic obs|tio e|n in e rved s|n ti|tirely muli c|n a|ovel n be|||||||
|||||||H tr|o ai|wev n a|er, cro|th ss|es m|e re ulti|la pl|tio e d|nal iff|r er|ules ent|are spati|not al|‘k en|no vir|wn on|’ me|a p nt|rio s w|ri hi|an ch|d t sh|he ar|ref e th|ore e s|m a|us me|t be un|l de|earnt. rlying|W|e ther 4-conn|e e|fore cted|||||||
|||||||E tu To|uc re p|lid of er|ean sp for|s ac m|tru e t on|ctu o u th|re se ese|(F for ta|igu pr sk|re ed s, t|1a) icti he t|- th on i hree|is n n m|me ew od|an en ifi|s th vir cati|e on on|mo m s t|de ent o t|l m s. he|us tra|t le nsf|ar or|n a me|nd r a|g re|en :|eral|is|e the a|b|stract|st|ruc-|||||||
|||||||||1|.|Re|cal|l e|qu|ati|on|1;|y t|=|so|ft|ma|x(|q √t|KT|)V|,|w|her|e|Q|=|H|W|, q||K =|H|W , k|V|=|||||||
||||||||||||||||||||||||dk||||||||||||||||||||||||
||||||||||H We E||W re W e|, an v strict . W||d H th e r|is a ese w efer t||ma eig o th|trix ht m is m|of i atri atr|nput ces s ix as||s an uch E˜|d po that . Th||siti qu us|on eri th|e es|nco (Q|di ) a|ngs nd|(i. ke|e. ys|its (K|ro ) a|w r|s are h e the sa|t m|= [x t e; Q,|, K|e ]). t =|||||||
||||||||||||||||||||||||||||e|key|s|and|q|ue|rie|s o|nl|y focu|s|on po|si|tion|||||||
||||||||||||||||||||||||||||epende|||nt on th|||e stimul|||us comp||onent o||f H|||||||
||||||||||||||||||||||||||||y t|||= soft|||e˜ t max( √|||E˜T ) X˜ d k||||(2)|||||||
||||||||||||||||||||||||||||, in tran d querie|||sformer s, but n|||s, best p ot value|||erforma s.||nce is w||hen|||||||
||||||||||||||||||||||||||||matrice l previo e agent|||s contai us time wander|||n the pr -steps ( s the en|||ojected i.e. e < vironme||position and x t nt accu||en- ). <t mu-|||||||


_previous_


time-steps (i.e.


**_e<t_**


and x


_<t)._


-----

Figure 2: (a) The TEM model, with a path integration component (equation 3) and a memory
network component (equation 5 and 6). TEM path integrates g and makes sensory predictions x via
its memory network (dashed lines are additional connections for inference). (b) TEM recapitulates
a host of empirically described cell representations (Whittington et al., 2020). Top/bottom row:
example TEM MEC/Hippocampal representations (plots are spatially smoothed). Figures adapted
from Whittington et al. (2020). (c) Schematic of TEM (adapted from Sanders et al. (2020)), showing
that the same cortical representations (LEC and MEC) are reused in different environments allowing
for generalisation, facilitated by different hippocampal combinations. (d) The TEM hippocampal
conjunction is an outer product - cells receive input from particular MEC and LEC cells.

lating new experiences (not-yet-experienced stimulus-position pairs are inaccessible to the
agent). Meanwhile the query at each time-point is the present positional encoding et.

3. The position encodings are recurrently generated (as in Wang et al. (2019); Liu et al.
(2020)); et+1 = σ(etWa), where Wa is a learnable action-dependent weight matrix, and
_σ(· · · ) is a non-linear activation function. This means that unlike traditional transformers,_
position encodings can be optimised and not the same for every sequence. It now becomes
interesting to see what representations are learned.

These modifications are sufficient to learn spatial representations, in the position encodings, that
mimic representations observed in the brain (Figure 1C; see Appendix for model and training details). The rest of this paper now explains why this is not a surprising result; namely we show that a
transformer with recurrent positional encodings is closely related to current neuroscience models of
the hippocampus and surrounding cortex (Whittington et al., 2020; Uria et al., 2020). Here we focus
on the Tolman-Eichenbaum Machine (TEM) (Whittington et al., 2020), though the same principles
apply for Uria et al. (2020).

The critical points are: 1) the memory component of TEM can be viewed as a transformer selfattention, since the TEM memory network is analogous to a Hopfield network (Hopfield, 1982)
which have recently been shown to be closely related to transformers (Ramsauer et al., 2020); 2)
TEM path integration (see below) can be viewed as a way to learn a position encoding.

4 TEM

The Tolman-Eichenbaum Machine (TEM; Figure 2, further details in Appendix) is a neuroscience
model that captures many known neural phenomena in hippocampus (HPC) and entorhinal cortex


-----

(medial/lateral; MEC/LEC). TEM is a sequence learner trained on tasks exactly like the one described in the previous section. TEM consists of two parts;

**1) A module that aims to understand where it is in space, using a representation g to represent**
location. To update its location, TEM uses path-integration - the accumulation of self movement
vectors a - enacted in a recurrent neural network:

**_gt+1 = σ(gtWa)_** (3)

Where Wa is a learnable action dependent weight matrix and σ(· · · ) is a non-linear activation
function. It is in this path-integrating representationg that TEM learns grid and other entorhinal
cells for self-localisation (Figure 2b).

**2) To make sensory predictions, location representations g alone are not enough; they must each**
link to a sensory observation x, corresponding to the stimulus at that position. Note that these links
are specific to an environment, since each environment consists of a different arrangement of stimuli
in space (i.e. different stimulus-position pairings).

The linking is done by binding every element of g with every element of x, in other words an outer
product that is flattened back into a vector;

**_p = flatten(x[T]_** **_g)_** (4)

These conjunctive p representations are stored in ‘fast weights’ [1]via Hebbian learning;


_t−1_

**_p[T]τ_** **_[p][τ]_** (5)
_τ_ =1

X


**_Mt =_**


And they can later be retrieved using an attractor network (a continuous version of the Hopfield
network). Here a query vector q (details next paragraph) is inputted into the network and updated
via;
**_q_** _σ(qMt)_ (6)
_←_

where σ(· · · ) is a non-linear activation function; a ReLu in TEM. Crucially, because the memories
are formed using both g and x, they can be retrieved (pattern-completed) using just one of those
representations alone i.e. ‘what did I see the last time I was here’ or ‘where was I the last time I
saw this’. To retrieve a memorised conjunction p, TEM imagines (path-integrates) the next location
**_g and provides this as input to the attractor network in the form q = flatten(1[T]_** **_g). Equation 6 is_**
then iterated until a memory is retrieved.

Finally, to make sensory predictions, the retrieved conjunctive memory (p[retrieved]t ) is ‘deconjunctified’ into sensory and location components. The sensory component is obtained by unflattening
**_p[retrieved]t_** and summing over the g dimension (Figure 8);

**_x[retrieved]t_** = sum(unflatten(p[retrieved]t ), 1) (7)

Finally, to make the sensory prediction x[retrieved]t is fed through a MLP zt = fθ(x[retrieved]t )) to
classify (predict) the upcoming sensory observation.

It is also possible, and often helpful, to project g and x via Wg and Wx; ˜g = gWg and ˜x = xWx
before they are combined conjunctively [2].

5 TEM AS A TRANSFORMER

Here we show that the above equations of TEM can be written so that: 1) the memory retrieval
components looks like a transformer self-attention; 2) the path integration representation, g look
like position encodings.

1We note such ‘fast weights’ have previously been thought of as an alternative to the LSTM (Ba et al.,
2016a).
2In fact, in the TEM code online, Wg and Wx are set as fixed weight matrices, where Wg sub-samples g
and Wx transforms (compresses) x from a one-hot to a two-hot representation.


-----

**Transformer**


**TEM**


k0 v0 k1 v1 k2 v2 **Key, Value** k0 v0 k1 v1 k2 v2

q0 q1 q2 q3 **Query** q0 q1 q2 q3

e0 x0 e1 x1 e2 x2 e3 ? **position encodingInput with** g0 x0 g1 x1 g2 x2 g3 ?

**(with causal attention mask)** y3

k0 v0 k1 v1 k2 v2

**iWk** **iWv**

q0 q1 q2 q3

**iWq**

e0 x0 e1 x1 e2 x2 e3 ?

x3

k0 v0 k1 v1 k2 v2

q0 q1 q2 q3

g0 x0 g1 x1 g2 x2 g3 ?


Figure 3: Self-attention in (a) Transformers and (b) TEM.

1) When considering the TEM memory retrieval process more closely (in this analysis, for direct
comparison, we are only considering 1 attractor step in TEM with no non-linearity), we see that the
attractor update qtMt = qt _tτ_ **_[p]τ[T]_** **_[p][τ]_** [is simply equal to]

_t_

P

**_p[retrieved]t_** = [qtp[T]τ []][p][τ] (8)

X


Since [qtp[T]τ []][ is just a dot-product (][[][q][t]
weighted by their similarity (dot product) to the query. As noted by Ramsauer et al. (2020), this is[·][ p][τ] []][), a single step of the attractor just retrieves memories]
exactly like a transformer but without the softmax scaling the dot-products. Thus the TEM memory
retrieval process behaves like transformer self-attention.

2) We can however go further since TEM’s input to the transformer (i.e. the TEM memories) are
special; they are learnable and built from an outer product between ˜g and ˜x (pτ = flatten(x˜[T]τ **_g[˜]τ_** )
), and these memories can be retrieved by a query based on ˜g or ˜x alone (e.g. qt = flatten(1[T] **_g˜t))._**
Together, these properties mean we can reduce the above dot product even further;



[qtp[T]τ [] = ¯]x˜τ [g˜t · ˜gτ ] _→_ **_p[retrieved]t_** = ˜gtG[˜][T] **ΛxP** (9)

Where _x[¯]˜ =_ _i[(]x[˜]τ_ )i and Λx is a diagonal matrix with elements _x[¯]˜τ (see Appendix for an alternative_

derivation using vector elements). Thus to retrieve a conjunctive p memory, all that was necessary
is weighting past[P] **_p representations via ‘self-attention’ of ˜gt to past representations_** **_G[˜]._**

To simplify this even further, we consider what happens when we ‘deconjunctify’ p[retrieved]t to
obtain the sensory component of the memory. Following the TEM procedure described above
(Figure 8);


**_x˜τ_** _g˜[¯]τ_ _x˜[¯]τ_ [g˜t · ˜gτ ] = ˜gtG[˜][T] **ΛgΛxX[˜]** (10)


**_x˜[retrieved]t_** = sum(unflatten(p[retrieved]t ), 1) =


Where Λg is a diagonal matrix with elements _g[¯]˜τ =_ _i[(]g[˜]τ_ )i. Now all that is necessary to retrieve

the sensory component of the memory is weighting past ˜x representations with via ‘self-attention’
of ˜gt to past representations **_G[˜]. This equation is now very similar to equation 1 except without the[P]_**
softmax and with additional weightings Λx and Λg. These weighting however are likely learned to
be constant (α) because otherwise some memories will be preferentially retrieved. In this case TEM
is retrieving memories using

**_gtG[˜][T]_**
**_x˜[retrieved]t_** = (αg˜tG[˜][T] )X[˜] _cf._ _softmax(_ **[˜]** )X[˜] (11)

_√dk_

Which can be seen to be very closely related to the transformer equation (shown on the right), and
diagrammatically shown in Figure 3. The model presented in this paper utilises the full transformer
softmax rule.

**The TEM-transformer. Thus the TEM-transformer (TEM-t; from Section 3) is this transformer**
that is directly analogous to TEM. Additional modelling details (analogous to modelling details
in TEM) can be found in the Appendix. TEM-t offers dramatic performance improvements over
the original TEM model (Figure 4; code will be released on publication). In particular, 1) Sample


-----

###### a


Figure 4: TEM-t is a more efficient learner than TEM, both in (a) sample efficiency and (b) time
per gradient step. Zero-shot accuracy is prediction accuracy when taking links it has never taken
before, but to a state it has visited before. Successful accuracy here is only possible with learned and
generalised spatial knowledge. We have used the code from TEM from the TEM authors original
[code https://github.com/djcrw/generalising-structural-knowledge, and](https://github.com/djcrw/generalising-structural-knowledge)
so have not optimised it for speed of learning etc, so we cannot claim this to be a fair comparison, nevertheless the difference is stark. We note that in the TEM paper, the authors say it takes up
to 50,000 gradient updates for full training, whereas we stopped at 20,000.

efficiency is increased; TEM-t requires many fewer data samples than TEM, and thus training time
is reduced 2) TEM-t can tackle much larger problems, with the ability to store and retrieve many
more memories (not shown here). Additionally to improved performance, TEM-t learns grid cells
(Figure 1) and has potential implications for what place cells are (see next section).

**Path integrating position encodings. This leads us to an interesting observation; we see that TEM’s**
representations for path integration g plays the role of position encodings in transformers. However
the structure of these positional encodings are not hard-coded, but instead learned via path integration (the structure of space!), with the particular position encoding depending on the particular
sequence of actions taken. Other (non-spatial) structural representations could also be learned depending on the task at hand, i.e. grammar for language. This is a very different (and we think
fruitful) re-understanding of position encodings; representing ‘location’ in a (learned) structure that
can be inferred on the fly.

6 PLACE CELLS IN TRANSFORMERS

Here we discuss, and demonstrate, how TEM-t offers a new interpretation of place representations.
To do so we utilise a recent suggestion of how the transformer update can be performed in biological
hardware (Krotov & Hopfield, 2020). In particular, self-attention (equation 1) can be split into two
steps which correspond to two pools of neurons (Figure 5A); 1) calculate softmax( **_[q]√[t][K]dk[T] ). 2)_**

multiply by V . In this light, K and V can simply be seen as weight matrices between feature
neuron (representing the query) and memory neurons (computing the softmax).

Since memory neurons are sparsely activated due to the softmax, they appear to have a spatial tuning
for each environment resembling hippocampal place cells (Figure 5D-E; note Krotov & Hopfield
(2020) stated memory neurons may correspond to place cells but without simulation). Similarly to
experimentally recorded place cells, these neurons remap randomly between environments i.e. place
cells being neighbours in one environment is not predictive of them being neighbours in another
(unlike grid cells which maintain their phase neighbours across environments).

We can curate this architecture for the specifics of TEM-t. TEM-t explicitly considers factorised g
and x representations (e.g. MEC and LEC), which project to feature neurons in hippocampus (or
still in cortex). Thus the feature neurons consist of two separate sub-populations, ˜g = gWg and
**_x˜ = xWx, but which can connect to the same memory neurons in hippocampus (Figure 5B-C)._**
These feature sub-populations are updated alternately rather than simultaneously, depending on the
direction of retrieval; for example, when retrieving ˜x the ˜g feature neurons stay constant while the ˜x


-----

**F**

**G**

**Memory neurons**

**B**

**C**

**˜**

**˜**

**K**

**V**

**_G_**

**_X˜_**

**_g˜1_** **_g˜2_** **_x˜2x˜3_**

**_g˜3_** **_x˜1_**

**_g˜_**

**_x˜_**

**_g˜_**

**_x˜_**

**_g˜_** **_x˜_**

**Feature neurons**

**Environment 1**

**E** **Environment 2**

_W_

_g_

**_g_**

**MEC**

**_x_**

**LEC**

**_c˜_**

**_c˜1_** **_c˜2_** **_c˜3_**

**_g˜1_** **_g˜2_** **_x˜2x˜3_**

**_g˜3_** **_x˜1_**

**_g˜_** **_x˜_**

Krotov & Hopfield (2020) describe a neu
Figure 5: TEM-Transformer neural architecture.

**(a)**

**_q_**

**_K[T]_**

).

between ‘feature’ neurons (i.e.

**_h) and memory neurons (i.e. softmax(_**

across brain regions.

**(d)**

extended for TEM-t, but now the feature neurons are not all updated simultaneously, but only those
Memory neurons resemble hippocampal place cells and

A possible architecture where cortical neurons project to feature

domly across environments.

**(f)**

**(g)**

neurons in hippocampus which in turn project to memory neurons in hippocampus.
brain regions can be included easily in this architecture with minimal increase in hippocampal neu
ron number.

neurons are updated (in turn updating

**_x_**

cortical representations in potentially disparate brain areas. Thus TEM-t instantiates hippocampal
indexing theory (Teyler & Rudy, 2007), which states that hippocampus provides an index that binds

together cortical patterns across different brain regions.

know that place cell remapping is not random; instead individual place cells preferentially remap to

|Col1|Col2|Col3|Col4|Col5|Col6|Col7|Col8|Col9|Col10|Col11|Col12|Col13|Col14|Col15|Col16|Col17|Col18|Col19|Col20|Col21|Col22|Col23|
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
|Memo|ry neu|rons|B||||C|||||F||||||G|||||
||||||||||||||||||||||||
||||||||g˜|1g˜2|x˜|x˜3|||H|PC G˜||˜|||||||
|K|V||G˜||X˜|||g˜3|2 x˜1|||||||X|||||||
|||||g˜|x˜|||g˜|x˜|||||g˜||x˜||||c||˜1 c˜2|
|Featur|e neur Envi|ons ronme|nt 1|||E||Envi|ronme|nt 2||W||||W||||g˜1|g˜2||
||||||Rema|pping||||||||g|||x|||||g˜3 x˜|
|||||||||||||||g MEC||x LEC||||||g˜|
||||||||||||||||||||||||
|re 5|: T|EM-|Tran|sfor|mer|neur|al ar|chit|ectur|e. (|a) K|roto||v &||Ho||pfiel|d (2|020)||des|
|pla een|usib ‘fea|le ar ture|chite ’ ne|ctur uron|al in s (i.e|stan . h|tiatio ) and|n th me|e ‘H mor|opfi y ne|eld n uron|etw s (i.||orks e. so||is al ftm||l yo ax(q|u ne KT t|ed’ ). (|||
|nde ss b|d for rain|TE regi|M-t, ons.|but n (d)|ow t Me|he f mory|eatu neu|re ne ron|uron s res|s ar emb|e not le hi|all ppo||upda camp||ted s al p||imu lace|ltane cell|ousl s an|||
|ly a|cros|s en|viron|men|ts.|(f) A|pos|sibl|e arc|hite|cture|wh||ere c||orti||cal n|euro|ns p|||
|rons n reg|in h ions|ippo can|cam be i|pus nclu|whic ded|h in easil|turn y in|proj this|ect t arch|o m itect|emor ure|y ne with||uron mini||s in mal||hipp incr|oca ease|mpu in h||s. (g ippo|
|num|ber.||||||||||||||||||||||
||||||||||||||||||||||||
|rons|are|upda|ted (|in tu|rn u|pdat|ing|x in|LEC|). In|this|vein||, hip||poc||amp|al m|emor||ies l|
|ical xing|repr the|esent ory (|atio Teyl|ns in er &|pot Rud|entia y, 2|lly d 007)|ispa, wh|rate ich s|brai tates|n ar that|eas. hip||Thu poca||s TE mpu||M-t s pro|inst vide|antia s an||tes h ind|
|ther|cort|ical|patte|rns|acros|s di|ffere|nt br|ain r|egio|ns.||||||||||||
|rand|oml|y re|mapp|ing|plac|e cel|ls de|scrib|ed o|ne p|arag|raph||ago||can||not b|e the|full||pict|


The randomly remapping place cells described one paragraph ago cannot be the full picture since we

locations consistent particular grid cell firing (as predicted by conjunctive memory cells p in TEM
and verified experimentally in Whittington et al. (2020)). However another mechanism for this
phenomena born from TEM-t could be as follows. Should the feature neurons exist in hippocampus
(Figure 5F) then there will be hippocampal spatial cells ˜g that maintain their relationship to grid
cells across different environment (as they are inherited from g via a projection ˜g = gWg). Thus
across the population of hippocampal cells, there will be those that maintain their relationship to
grid cells (e.g. ˜g and those that don’t (e.g. memory neurons and ˜x), but the population effect will
exist, just like what is experimentally observed.

As a note, the particular relationship of our model to the model of Krotov & Hopfield (2020), is
what they refer to as a ‘type B’ model. These are models with contrastive normalisation on the
memory neurons (via a softmax in our case), as opposed to ‘type B’ models which have a power
activation function on the memory neurons. TEM (left hand side of Equation 11) corresponds to a
linear activation function on the memory neurons, and is directly analogous to the original Hopfield
energy. Secondly, the notion that there are two types of feature neurons that can be bound together in
the same memory, was explored in Krotov & Hopfield (2016) where pixel intensities were associated
with labels of those images. In TEM-t, one of the feature vectors, g, is learned via a RNN and
structures itself according to the underlying task structure.

As an additional aside, we note that Krotov & Hopfield (2020) architectures does not solve the
scaling problem of conventional Hopfield networks; the number of memories that original Hopfield
networks could store scaled linearly with the dimensionality of the recurrent attractor network (Amit
et al., 1985). While recent analytical work has shown with exponential power activation functions,
_N_
the number of memories that can be stored to scale as 2 2, where N is the dimensionality of the

feature neurons (Demircigil et al., 2017). This is a considerably more favourable scaling. However,
unfortunately the architecture from Krotov & Hopfield (2020) instead requires a growing number of
memory neurons (one for each memory), so the number of memories is still linear with the number
of neurons! We note that mathematically derived scaling law was for an exponential activation
function, not with a softmax as we use here.


-----

7 DISCUSSION

We have shown that TEM, a current model of the hippocampal formation, is closely related to
a transformer with recurrent position encodings. We now consider some wider implications for
neuroscience.

**Multiple cortical inputs to hippocampus. TEM considers hippocampal conjunctions between two**
cortical regions (g and x). It is, however, possible to consider conjunctions of more than two brain
regions. Indeed hippocampal neurons often respond to more than two task variables (McKenzie
et al., 2014). In TEM, the naive approach of a ‘triple’ (or higher) conjunction would increase the
number of hippocampal neurons would increase by a factor of nc; the number of neurons from brain
region ˜c. TEM-t does not scale so badly. Instead it just requires an additional nc feature neurons,
and the number of memory neurons can stay the same since the each hippocampal memory neuron
can simply index a memory across three (or more), rather than two, brain regions (Figure 5G).

With multiple inputs to hippocampus [x˜, ˜g, ˜c, . . . ], any subset of those brain areas can reinstate a
memory in the other brain regions i.e. ˜x and ˜g can reinstate a ˜c memory or ˜g alone could reinstate ˜x
and ˜c memories. As an analogy to the TEM triple conjunction, TEM-t proposes that ˜ct is updated via
**_c˜t ←_** _softmax((g˜tG[˜][T]_ ) ⊙ (x˜tX[˜] _[T]_ ))C[˜], where ⊙ is an element wise product. We note an alternate,
and perhaps more intuitive, option could be ˜ct = softmax(g˜tG[˜][T] + ˜xtX[˜] _[T]_ )C[˜] - this is equivalent
to two ‘double’ conjunctions in TEM, one between ˜g and ˜c, the other between ˜x and ˜c. This is
important for TEM as it requires many fewer hippocampal neurons than a ‘triple’ conjunction.

**Beyond hippocampus: Cortex as a Transformer. We have considered transformers as a model**
of hippocampus and its connections. We know, however, that transformer representations predict
language areas (Schrimpf et al., 2020), and that patients can talk and comprehend just fine with major
hippocampal deficits (Elward & Vargha-Khadem, 2018). This indicates that the transformer, and
TEM-like models, may also model other brain regions, such as language areas, that are seemingly
independent from hippocampus (related ideas discussed in Hawkins et al. (2019); Lewis (2021) but
specifically for grid cells in neocortex). This raises two questions. Firstly what is the analogue
of spatial positional encodings for higher order tasks such as language, and secondly what takes
the role of the memory neurons if not hippocampus. We offer some thoughts in the following two
paragraphs.

In spatial tasks, TEM and TEM-t learn positional encodings that mirror the structure of space. The
implication is that positional encoding should reflect the abstract underlying properties of the task at
hand. In language for example, this structure is grammar. This contrasts to the typical positional encodings in Transformers - sines and cosines - which represent a linear structure. It is our contention
that positional encodings that are inferred on the fly and consist of previously learned structures
(like the spatial case we have considered) would offer an interesting and potentially fruitful research
direction in problems of language, maths, and logic.

If the transformer were solely instantiated in cortex, then what about the memory neurons? It is
possible that the memory neuron equivalent exists in cortex too, but for these tasks, since it is not
necessary to store long term memories or bind knowledge across multiple brain areas hippocampus
is not required; so short term cortical memory neurons suffice.

8 CONCLUSION

We have shown that transformers with recurrent positional encodings reproduce neural representations found in rodent entorhinal cortex and hippocampus. We then showed these transformers
are close mathematical cousins to models of hippocampus that neuroscientists have developed over
the last few years. We hope this work brings neuroscience and machine learning closer together,
and offers understanding for both sides; for neuroscientists a road map to understanding cortical
areas beyond the hippocampal formation; for machine learners a greater understanding of positional
encodings in transformers.


-----

REFERENCES

Daniel J. Amit, Hanoch Gutfreund, and H. Sompolinsky. Storing infinite numbers of patterns in a
spin-glass model of neural networks. Physical Review Letters, 55(14):1530–1533, 1985. ISSN
00319007. doi: 10.1103/PhysRevLett.55.1530.

Jimmy Ba, Geoffrey Hinton, Volodymyr Mnih, Joel Z. Leibo, and Catalin Ionescu.
Using Fast Weights to Attend to the Recent Past. _Advances in Neural Informa-_
_tion_ _Processing_ _Systems_ _29,_ 29:4331–4339, 10 2016a. ISSN 10495258. URL
[http://arxiv.org/abs/1610.06258http://papers.nips.cc/paper/](http://arxiv.org/abs/1610.06258 http://papers.nips.cc/paper/6057-using-fast-weights-to-attend-to-the-recent-past.pdf)
[6057-using-fast-weights-to-attend-to-the-recent-past.pdf.](http://arxiv.org/abs/1610.06258 http://papers.nips.cc/paper/6057-using-fast-weights-to-attend-to-the-recent-past.pdf)

Jimmy Lei Ba, Jamie Ryan Kiros, and Geoffrey E. Hinton. Layer Normalization. arXiv preprint,
2016b. ISSN 1607.06450. doi: 10.1038/nature14236. [URL http://arxiv.org/abs/](http://arxiv.org/abs/1607.06450)
[1607.06450.](http://arxiv.org/abs/1607.06450)

Andrea Banino, Caswell Barry, Benigno Uria, Charles Blundell, Timothy Lillicrap, Piotr Mirowski, Alexander Pritzel, Martin J Chadwick, Thomas Degris, Joseph Modayil,
Greg Wayne, Hubert Soyer, Fabio Viola, Brian Zhang, Ross Goroshin, Neil Rabinowitz,
Razvan Pascanu, Charlie Beattie, Stig Petersen, Amir Sadik, Stephen Gaffney, Helen King, Koray Kavukcuoglu, Demis Hassabis, Raia Hadsell, and Dharshan Kumaran.
Vector-based navigation using grid-like representations in artificial agents. _Nature, 557_
(7705):429–433, 5 2018. ISSN 0028-0836. doi: 10.1038/s41586-018-0102-6. URL
[http://dx.doi.org/10.1038/s41586-018-0102-6http://www.nature.](http://dx.doi.org/10.1038/s41586-018-0102-6 http://www.nature.com/articles/s41586-018-0102-6)
[com/articles/s41586-018-0102-6.](http://dx.doi.org/10.1038/s41586-018-0102-6 http://www.nature.com/articles/s41586-018-0102-6)

Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal,
Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M.
Ziegler, Jeffrey Wu, Clemens Winter, Christopher Hesse, Mark Chen, Eric Sigler, Mateusz Litwin,
Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam McCandlish, Alec Radford,
Ilya Sutskever, and Dario Amodei. Language models are few-shot learners. arXiv preprint, 2020.
ISSN 23318422.

Yoram Burak and Ila R. Fiete. Accurate path integration in continuous attractor network models
of grid cells. PLoS Computational Biology, 5(2):e1000291, 2 2009. ISSN 1553734X. doi: 10.
[1371/journal.pcbi.1000291. URL https://dx.plos.org/10.1371/journal.pcbi.](https://dx.plos.org/10.1371/journal.pcbi.1000291)
[1000291.](https://dx.plos.org/10.1371/journal.pcbi.1000291)

Christopher J. Cueva and Xue-Xin Wei. Emergence of grid-like representations by training recurrent
neural networks to perform spatial localization. International Conference on Learning Represen_[tations, 0:1–19, 3 2018. URL http://arxiv.org/abs/1803.07770.](http://arxiv.org/abs/1803.07770)_

Mete Demircigil, Judith Heusel, Matthias L¨owe, Sven Upgang, and Franck Vermet. On a Model of
Associative Memory with Huge Storage Capacity. Journal of Statistical Physics, 168(2):288–299,
2017. ISSN 00224715. doi: 10.1007/s10955-017-1806-y.

Yedidyah Dordek, Daniel Soudry, Ron Meir, and Dori Derdikman. Extracting grid cell characteristics from place cell inputs using non-negative principal component analysis. _eLife, 5_
(MARCH2016):1–36, 2016. ISSN 2050084X. doi: 10.7554/eLife.10094.

Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas
Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszkoreit, and Neil Houlsby. An Image is Worth 16x16 Words: Transformers for Image Recognition at
[Scale. arXiv preprint, pp. 1–21, 2020. URL http://arxiv.org/abs/2010.11929.](http://arxiv.org/abs/2010.11929)

Rachael L. Elward and Faraneh Vargha-Khadem. Semantic memory in developmental amnesia.
_Neuroscience Letters, 680(April):23–30, 2018. ISSN 18727972. doi: 10.1016/j.neulet.2018.04._
[040. URL https://doi.org/10.1016/j.neulet.2018.04.040.](https://doi.org/10.1016/j.neulet.2018.04.040)

Dileep George, Rajeev V. Rikhye, Nishad Gothoskar, J. Swaroop Guntupalli, Antoine
Dedieu, and Miguel L´azaro-Gredilla. Clone-structured graph representations enable


-----

flexible learning and vicarious evaluation of cognitive maps. _Nature Communications,_
12(1):2392, 12 2021. ISSN 2041-1723. doi: 10.1038/s41467-021-22559-5. URL
[http://dx.doi.org/10.1038/s41467-021-22559-5https://linkinghub.](http://dx.doi.org/10.1038/s41467-021-22559-5 https://linkinghub.elsevier.com/retrieve/pii/B0123693985003418 http://www.nature.com/articles/s41467-021-22559-5)
[elsevier.com/retrieve/pii/B0123693985003418http://www.nature.](http://dx.doi.org/10.1038/s41467-021-22559-5 https://linkinghub.elsevier.com/retrieve/pii/B0123693985003418 http://www.nature.com/articles/s41467-021-22559-5)
[com/articles/s41467-021-22559-5.](http://dx.doi.org/10.1038/s41467-021-22559-5 https://linkinghub.elsevier.com/retrieve/pii/B0123693985003418 http://www.nature.com/articles/s41467-021-22559-5)

Nicholas J. Gustafson and Nathaniel D. Daw. Grid Cells, Place Cells, and Geodesic Generalization
for Spatial Reinforcement Learning. PLoS Computational Biology, 7(10):e1002235, 10 2011.
[ISSN 1553-7358. doi: 10.1371/journal.pcbi.1002235. URL https://dx.plos.org/10.](https://dx.plos.org/10.1371/journal.pcbi.1002235)
[1371/journal.pcbi.1002235.](https://dx.plos.org/10.1371/journal.pcbi.1002235)

Torkel Hafting, Marianne Fyhn, Sturla Molden, May-britt Britt Moser, and Edvard I. Moser. Microstructure of a spatial map in the entorhinal cortex. Nature, 436(7052):801–806, 2005. ISSN
00280836. doi: 10.1038/nature03721.

Jeff Hawkins, Marcus Lewis, Mirko Klukas, Scott Purdy, and Subutai Ahmad. A Framework for
Intelligence and Cortical Function Based on Grid Cells in the Neocortex. Frontiers in Neural
_Circuits, 12(January):1–15, 1 2019. ISSN 1662-5110. doi: 10.3389/fncir.2018.00121. URL_
[https://www.numenta.com/assets/pdf/research-publications/papers/](https://www.numenta.com/assets/pdf/research-publications/papers/Companion-paper-to-A-Framework-for-Intelligence-and-Cortical-Function-Based-on-Grid-Cells-in-the-Neocortex.pdf https://www.frontiersin.org/article/10.3389/fncir.2018.00121/full)
[Companion-paper-to-A-Framework-for-Intelligence-and-Cortical-Function-Based-on-Gri](https://www.numenta.com/assets/pdf/research-publications/papers/Companion-paper-to-A-Framework-for-Intelligence-and-Cortical-Function-Based-on-Grid-Cells-in-the-Neocortex.pdf https://www.frontiersin.org/article/10.3389/fncir.2018.00121/full)
[pdfhttps://www.frontiersin.org/article/10.3389/fncir.2018.00121/](https://www.numenta.com/assets/pdf/research-publications/papers/Companion-paper-to-A-Framework-for-Intelligence-and-Cortical-Function-Based-on-Grid-Cells-in-the-Neocortex.pdf https://www.frontiersin.org/article/10.3389/fncir.2018.00121/full)
[full.](https://www.numenta.com/assets/pdf/research-publications/papers/Companion-paper-to-A-Framework-for-Intelligence-and-Cortical-Function-Based-on-Grid-Cells-in-the-Neocortex.pdf https://www.frontiersin.org/article/10.3389/fncir.2018.00121/full)

Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep Residual Learning for Image
Recognition. Proc. Computer Vision and Pattern Recognition (CVPR), pp. 770–778, 12 2016.
[URL http://arxiv.org/abs/1512.03385.](http://arxiv.org/abs/1512.03385)

J J Hopfield. Neural networks and physical systems with emergent collective computational abilities (associative memory/parallel processing/categorization/content-addressable memory/fail-soft
devices). Biophysics, 79(April):2554–2558, 1982.

Seyed Mahdi Khaligh-Razavi and Nikolaus Kriegeskorte. Deep Supervised, but Not Unsupervised,
Models May Explain IT Cortical Representation. PLoS Computational Biology, 10(11), 2014.
ISSN 15537358. doi: 10.1371/journal.pcbi.1003915.

Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton. ImageNet Classification with Deep Convolutional Neural Networks. Advances In Neural Information Processing Systems, pp. 1–9, 2012.
ISSN 10495258. doi: http://dx.doi.org/10.1016/j.protcy.2014.09.007.

Dmitry Krotov and John Hopfield. Large Associative Memory Problem in Neurobiology and Ma[chine Learning. arXiv preprint, pp. 1–12, 2020. URL http://arxiv.org/abs/2008.](http://arxiv.org/abs/2008.06996)
[06996.](http://arxiv.org/abs/2008.06996)

Dmitry Krotov and John J. Hopfield. Dense Associative Memory for Pattern Recognition. Advances
_in Neural Information Processing Systems, 0(Nips):1180–1188, 6 2016. ISSN 10495258. URL_
[http://arxiv.org/abs/1606.01164.](http://arxiv.org/abs/1606.01164)

Julia Krupic, Neil Burgess, and John O’Keefe. Neural Representations of Location Composed
of Spatially Periodic Bands. _Science, 337(6096):853–857, 8 2012._ ISSN 0036-8075. doi:
10.1126/science.1222403. URL [http://www.sciencemag.org/content/337/](http://www.sciencemag.org/content/337/6096/853.full.pdf https://www.sciencemag.org/lookup/doi/10.1126/science.1222403)
[6096/853.full.pdfhttps://www.sciencemag.org/lookup/doi/10.1126/](http://www.sciencemag.org/content/337/6096/853.full.pdf https://www.sciencemag.org/lookup/doi/10.1126/science.1222403)
[science.1222403.](http://www.sciencemag.org/content/337/6096/853.full.pdf https://www.sciencemag.org/lookup/doi/10.1126/science.1222403)

Yann Lecun, Le’on Bottou, Yoshua Bengio, and Parick Haffner. Gradient-based learning applied to
document recognition. proc. OF THE IEEE, 1998.

Marcus Lewis. Hippocampal Spatial Mapping As Fast Graph Learning. arXiv preprint, 0, 7 2021.
[URL http://arxiv.org/abs/2107.00567.](http://arxiv.org/abs/2107.00567)

Xuanqing Liu, Hsiang Fu Yu, Inderjit S. Dhillon, and Cho Jui Hsieh. Learning to encode position
for transformer with continuous dynamical model. 37th International Conference on Machine
_Learning, ICML 2020, PartF16814:6283–6291, 2020._


-----

Sam McKenzie, Andrea J. Frank, Nathaniel R. Kinsky, Blake Porter, Pamela D Rivie, Pamela D.
Rivi`ere, Howard Eichenbaum, Pamela D Rivie, Pamela D. Rivi`ere, Howard Eichenbaum,
Pamela D Rivie, Pamela D. Rivi`ere, and Howard Eichenbaum. Hippocampal representation of
related and opposing memories develop within distinct, hierarchically organized neural schemas.
_Neuron, 83(1):202–215, 7 2014. ISSN 10974199. doi: 10.1016/j.neuron.2014.05.019. URL_
[https://linkinghub.elsevier.com/retrieve/pii/S089662731400405X.](https://linkinghub.elsevier.com/retrieve/pii/S089662731400405X)

John O’Keefe and J. Dostrovsky. The hippocampus as a spatial map. Preliminary evidence
from unit activity in the freely-moving rat. Brain Research, 34(1):171–175, 11 1971. ISSN
[00068993. doi: 10.1016/0006-8993(71)90358-1. URL http://linkinghub.elsevier.](http://linkinghub.elsevier.com/retrieve/pii/0006899371903581)
[com/retrieve/pii/0006899371903581.](http://linkinghub.elsevier.com/retrieve/pii/0006899371903581)

Hubert Ramsauer, Bernhard Sch¨afl, Johannes Lehner, Philipp Seidl, Michael Widrich, Lukas Gruber, Markus Holzleitner, Milena Pavlovi´c, Geir Kjetil Sandve, Victor Greiff, David Kreil, Michael
Kopp, G¨unter Klambauer, Johannes Brandstetter, and Sepp Hochreiter. Hopfield networks is all
you need. arXiv preprint, 2020. ISSN 23318422.

Honi Sanders, Matthew Wilson, Mirko Klukas, Sugandha Sharma, and Ila Fiete. Efficient Inference in Structured Spaces. Cell, 183(5):1147–1148, 11 2020. ISSN 00928674. doi: 10.1016/j.
[cell.2020.11.008. URL https://doi.org/10.1016/j.cell.2020.11.008https:](https://doi.org/10.1016/j.cell.2020.11.008 https://linkinghub.elsevier.com/retrieve/pii/S0092867420315191)
[//linkinghub.elsevier.com/retrieve/pii/S0092867420315191.](https://doi.org/10.1016/j.cell.2020.11.008 https://linkinghub.elsevier.com/retrieve/pii/S0092867420315191)

Martin Schrimpf, Idan Blank, Greta Tuckute, Carina Kauf, Eghbal Hosseini, Nancy Kanwisher,
Joshua Tenenbaum, and Evelina Fedorenko. The neural architecture of language: Integrative
reverse-engineering converges on a model for predictive processing. bioRxiv preprint, 2020. doi:
10.1101/2020.06.26.174482.

Ben Sorscher, Gabriel C Mel, Surya Ganguli, and Samuel A Ocko. A unified theory for the origin
of grid cells through the lens of pattern formation. Advances in Neural Information Processing
_Systems 32, 32(NeurIPS):10003–10013, 2019._

Kimberley L. Kimberly L. Kimberley L. Kimberly L. Stachenfeld, Matthew M. Botvinick, and
Samuel J. Gershman. The hippocampus as a predictive map. Nature Neuroscience, 20(11):1643–
1653, 2017. ISSN 15461726. doi: 10.1038/nn.4650.

Timothy J Teyler and Jerry W Rudy. The hippocampal indexing theory and episodic memory: Updating the index. Hippocampus, 17(12):1158–1169, 12 2007. ISSN 10509631. doi:
10.1002/hipo.20350. [URL https://onlinelibrary.wiley.com/doi/10.1002/](https://onlinelibrary.wiley.com/doi/10.1002/hipo.20350 http://doi.wiley.com/10.1002/hipo.20350)
[hipo.20350http://doi.wiley.com/10.1002/hipo.20350.](https://onlinelibrary.wiley.com/doi/10.1002/hipo.20350 http://doi.wiley.com/10.1002/hipo.20350)

Benigno Uria, Borja Ibarz, Andrea Banino, Vinicius Zambaldi, Dharshan Kumaran, Demis Hassabis, Caswell Barry, and Charles Blundell. The spatial memory pipeline: A model of egocentric to allocentric understanding in mammalian brains. bioRxiv preprint, pp. 1–52, 2020. ISSN
26928205. doi: 10.1101/2020.11.11.378141.

Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez,
Łukasz Kaiser, and Illia Polosukhin. Attention is all you need. Advances in Neural Informa_tion Processing Systems, 2017-Decem(Nips):5999–6009, 2017. ISSN 10495258._

Zhiwei Wang, Yao Ma, Zitao Liu, and Jiliang Tang. R-Transformer: Recurrent Neural Network En[hanced Transformer. arXiv preprint, 0:1–11, 2019. URL http://arxiv.org/abs/1907.](http://arxiv.org/abs/1907.05572)
[05572.](http://arxiv.org/abs/1907.05572)

James CR R. Whittington, Timothy H. Muller, Shirley Mark, Caswell Barry, Neil Burgess, Timothy
E.J. EJ Behrens, Guifen Chen, Caswell Barry, Neil Burgess, and Timothy E.J. EJ Behrens. The
Tolman-Eichenbaum Machine: Unifying Space and Relational Memory through Generalization
in the Hippocampal Formation. Cell, 183(5):1249–1263, 11 2020. ISSN 00928674. doi: 10.1016/
[j.cell.2020.10.024. URL https://doi.org/10.1016/j.cell.2020.10.024https:](https://doi.org/10.1016/j.cell.2020.10.024 https://linkinghub.elsevier.com/retrieve/pii/S009286742031388X)
[//linkinghub.elsevier.com/retrieve/pii/S009286742031388X.](https://doi.org/10.1016/j.cell.2020.10.024 https://linkinghub.elsevier.com/retrieve/pii/S009286742031388X)

Daniel L K Yamins, Ha Hong, Charles F. Cadieu, Ethan A. Solomon, Darren Seibert, and James J.
DiCarlo. Performance-optimized hierarchical models predict neural responses in higher visual
cortex. Proceedings of the National Academy of Sciences of the United States of America, 111
(23):8619–8624, 2014. ISSN 10916490. doi: 10.1073/pnas.1403112111.


-----

A APPENDIX

A.1 THE MATHS USING ELEMENTS

Here we derive the main results using vector and matrix elements. Since each element in p is
multiplicative combination of every pair of elements in ˜x and ˜g, then

_pij = ˜gix˜j_ (12)

Where we label the elements of vector p with two indices even though it is a vector. Similarly, the
memory matrix uses 4 indices;
_Mijkl =_ _g˜i[τ]_ _x[˜][τ]j_ _g[˜]k[τ]_ _x[˜][τ]l_ (13)

_τ_

X

**Retrieving a memory via path integration. Querying the network becomes qij = ˜gi[P I]**, which
reduces to


_g˜i[P I]_ _g˜i[τ]_ _x[˜][τ]j_ _g[˜]k[τ]_ _x[˜][τ]l_
_ijk_

X


_qijMijkl =_
_ijk_

X


(14)


_x˜[τ]l_ [(]


_g˜k[τ]_ [)(]


_g˜i[P I]_ _g˜i[τ]_ [)(]


_x˜[τ]j_ [)]


**_g˜t[P I]_** **_X˜_** _[T]_ **ΛgΛxG[˜]**
_→_

**Retrieving a memory using a sensory observation alone. Similarly when the query is the sensory**
observation alone qij = xj


_x˜jg˜i[τ]_ _x[˜][τ]j_ _g[˜]k[τ]_ _x[˜][τ]l_
_ijlτ_

X


_qijMijkl =_
_ijl_

X


(15)


_g˜k[τ]_ [(]


_x˜[τ]l_ [)(]


_g˜i[τ]_ [)(]


_x˜j ˜x[τ]j_ [)]


_→_ **_x˜tX[˜]_** _[T]_ **ΛgΛxG[˜]**

**Retrieving a memory using a sensory observation and path integration. When the query is**
_qij = gi[P I]_ _xj, and we want to retrieve a position encoding memory_


_g˜i[P I]_ _x˜jg˜i[τ]_ _x[˜][τ]j_ _g[˜]k[τ]_ _x[˜][τ]l_
_ijlτ_

X


_qijMijkl =_
_ijl_

X


(16)


_g˜k[τ]_ [(]


_x˜[τ]l_ [)(]


_g˜i[P I]_ _g˜i[τ]_ [)(]


_x˜j ˜x[τ]j_ [)]


(x˜tX[˜] _[T]_ **_g˜t[P I]_** **_G˜[T]_** )ΛgG[˜]
_→_ _⊙_

A.2 DETAILS OF IMPLEMENTATION

Code will be made available at [https://github.com/djcrw/](https://github.com/djcrw/generalising-structural-knowledge)
[generalising-structural-knowledge.](https://github.com/djcrw/generalising-structural-knowledge)

**Scaling beta. Since the number of memories is not constant, we use an adaptive β parameter in the**
softmax. This is because normalisation term in the softmax sums the number of memories, and so
more memories down-weights probabilities. We want a self-attention value to not be affected by the
number of elements in the set. In particular, we weight the softmax by log(nmemories).

**Losses. We use same set of losses as in TEM;**

-  a one-step sensory prediction cross entropy loss, i.e. using gt[P I] as input to the memory
retrieval process


-----

-  a sensory prediction cross entropy loss using gt as input to the memory retrieval process

-  a squared error loss between gt and gt[P I]

-  l2 weight regularisation

-  l2 regularisation on gt

**Normalisation. We find that using layernorm (Ba et al., 2016b) on the positional encodings (not**
in the RNN, but on the input to transformer) to be beneficial, since the memory retrieval process
can then be standardised - no one memory is up-weighted relative to others. Using layernorm before
self-attention in transformers is common practice (Vaswani et al., 2017). For simplicity, we use fixed
weights on the layer norm, i.e. is is just a z score of g, Since we use a one-hot encodings of our
sensory representations, they are already normalised.

**Stabilising position representations. Recurrently generated positional encodings accumulate noise**
and drift. While bespoke path integration networks from neuroscience mitigate noise by enforcing
their neural representations to stay close to a neural manifold (Burak & Fiete, 2009), this can not
be guaranteed in learned recurrent networks. One method of stabilisation is via sensory landmarks

-  i.e. ‘what positional encoding did I have the last time I saw this landmark’. In this vein TEM
uses the following query to the memory network; qt = flatten(x˜[T]t [1][)][, and][ memories of positional]
**encodings are retrieved as;**


**_g˜τ_** _g˜[¯]τ_ _x˜[¯]τ_ [x˜t · ˜xτ ] = ˜xtX[˜] _[T]_ **ΛgΛxG[˜]** (17)


**_g˜t[retrieved]_** = sum(unflatten(qtMt), 0) =


The final position encoding, g is computed on a basis of path integration (gt[P I] ; equation 3) and stored
landmark information (gt[retrieved]; equation 17), i.e. gt = gt[P I] + fθ(gt[retrieved], gt[P I] )(gt[retrieved]
_−_
**_gt[P I]_** ), where g[retrieved] = fθ(g˜[retrieved]) and fθ( ) are different MLPs.

_· · ·_

We note that a better query to the memory network would be qt = flatten(x˜[T]t **_[g]t[P I]_** ) since sensory
observations may be aliased, including gt[P I] can help disambiguate such aliasing. In this case, the
retrieved memory is (x˜tX _[T]_ **_gt[P I]_** **_G)ΛxG. This is perhaps, different to what would be anticipated;_**
_⊙_
using an element wise product rather than an addition. Translating this other TEM memory retrieval
process into transformers we get gt[retrieved] = fθ(softmax(x˜tX _[T]_ **_gt[P I]_** **_G)G), this can be thought_**
_⊙_
of as another attention head i.e. from input [g, x], the key and query ‘attends’ to x while the value
‘attends’ to g.

**When to add memories. Memories should not be added at every step, otherwise the self-attention**
mechanism would be biased towards memories (locations) that have been visited more frequently.
This is not a problem typically addressed in transformer, but here to mitigate this issue memories are
only added when existing memories for that particular conjunction do not already exist i.e. if there
is a memory already with the present combination of g and x (similarity determined by dot product)
then no new memory is added [3].

**Multiple iterations: Though in our simulations we only used a single iteration of the transformer**
block, it is possible to use multiple iterations - just like a Hopfield network (as in Ramsauer et al.
(2020)). Here the retrieved sensory memory ˜x[retrieved]t can be fed into the next iteration, along
with positional encoding i.e. the path integrated g. In this case (see Appendix for derivation),
TEM suggests that ˜xt is iteratively updated via ˜xt ← _softmax(x˜tX[˜]_ _[T]_ _⊙_ **_g˜tG[˜])X[˜]_**, but with the first

-  it may be thought that it should be additive instead.iteration via ˜xt ← _softmax(g˜tG[˜])X[˜]_ . The multiplicative term in the softmax is perhaps unexpected

**Place-like representations without a softmax: Here we chose a softmax activation function on**
the memory neurons to make the relationship between TEM and transformers exact. However was
this choice necessary to observe place cells in the memory neurons? While we have not examined
this experimentally, we suspect place-like representations would emerge for a variety of activations,
since memories must still be sparsely activated for correct prediction. We leave this for future
work to verify, but note again that power and exponential activations have been shown to have a

3The TEM model actually does something very much like this - memories are only added if they did not
already exist - M = _τ_ [(][p][τ][ −] **_p[ˆ]τ_** )[T] (pτ + ˆpτ ), where ˆpτ is the memory that was retrieved at τ, i.e. only the

un-predicted components of the memory pτ are added.

[P]


-----

if you know that a right → _down →_ _left →_ _up means you’re back in the same place. Knowledge_

**Hidden structure**

# …

**Passive actions**

### … …
 … … … …

**Sensory observations**

### … … ?…

**Trials**

**Environments**

Figure 6: Learning to predict the next sensory observation in environments that share the same
structure but differ in their sensory observations. TEM only sees the sensory observations and
associated action taken, it is not told about the underlying structure - this must be learned.

much greater memory capacity (Krotov & Hopfield, 2016; Demircigil et al., 2017) than traditional
Hopfield nets with linear activation.

ADDITIONAL DETAILS OF THE SPATIAL TASK

We formalise a task type that not only relates to known hippocampal function, but also tests the
learning and generalising of abstract structural knowledge. We formalise this via relational understanding on graph structures (a graph is a set of nodes that relate to each other).

Should one passively move on a graph (e.g. Figure 6), where each node is associated with a nonunique sensory observation (e.g. an image of a banana), then predicting the subsequent sensory
observation tests whether you understand the graph structure you are in. For example, if you return
to a previously visited node (Figure 6 pink) by a new direction - it is only possible to predict correctly
if you know that a right _down_ _left_ _up means you’re back in the same place. Knowledge_

of such loop closures is equivalent to understanding the structure of the graph.

We thus train our models on these graphs with it trying to predict the next sensory observation.
Our models are trained on many environments sharing the same structure, e.g. 2D graphs (Figure
6), however the stimulus distribution is different (each vertex is randomly assigned a stimulus).
Should it be able to learn and generalise this structural knowledge, then it should be able to enter
new environments (structurally similar but with different stimulus distributions) and perform feats
of loop closure on first presentation.

Formally, given data of the form D = (x[k] _T_ _[,][ a][k]_ _T_ [)][}][ with][ k][ ∈{][1][,][ · · ·][, N] _[}][ (which environment it]_
_{_ _≤_ _≤_
is in), where x _T and a_ _T are a sequence of sensory observations and associated actions/relations_
_≤_ _≤_
(Figure 6), N is the number of environments in the dataset, and T is the duration of time-steps in
each environment, our model should maximise its probability of observing the sensory observations
for each environment.

The sensory stimuli are chosen randomly, with replacement, at each node. We understand that this
is not like the real world, where adjacent locations have sensory correlations - most notable in space
(though names in a family tree will be less correlated). Sensory correlations help with sensory
predictions, thus if we use environments with sensory correlations, we would not know what was
causing the learned representations, sensory correlations, or transition structure. To answer this
question cleanly, and to know that transition structure is the sole cause, we do not use environments
with sensory correlations.


-----

A.4 ADDITIONAL DETAILS OF TEM

We present a more detailed model schematic of TEM in Figure 7. We see there are two components
to TEM - a RNN for understanding position (g, in green top of Figure 7) that also indexes memories
via ‘queries’ q = Wgg. A memory network that binds together x and g, via an outer product
(middle green in 7, with more detail in Figure 8A). When the memory network is queried (red in
Figure 7), it undergoes attractor dynamics to retrieve the full memory. To make a sensory prediction,
the retrieved memory is ‘deconjunctified’ into a sensory representation (Figure 8C).

**Model flow. TEM transitions through time and infers gt and pt at each time-step. gt is inferred**
before forming each new memory pt. In other words variables g and p are inferred in the following
order gt, pt, gt+1, pt+1, gt+2 . This flow of information is shown in a schematic in Figure 7.
_· · ·_

Independently, at each time-step, the model model asks ‘are the inferred variables what I would have
predicted given my current understanding of the world (weights)’. I.e. 1) Is the inferred gt the one
I would have predicted from gt 1. 2) Is the inferred pt the one I would have predicted from gt. 3)
_−_
Is xt what I would have predicted from pt. This leads to errors (at each timestep) between inferred
and predicted variables gt and pt, and between sensory data xt and its prediction.

At then end of a sequence, these errors are accumulated, with model parameters updated along the
gradient (from back-propagation through time) that matches each others variables and also matches
the data.

Since the model runs along uninterrupted, it’s activity at one time-step influence those at later timesteps. Thus when learning (using back-propagation through time - BPTT), gradient information
flows backwards in time. This is important as, should a bad memory be formed at one-time step, it
will have consequences for later predictions - thus BPTT allows us to learn how to form memories
and latent representations such that they will be useful many steps into the future.

A.5 RELATED WORK

Here we discuss and compare other models that recapitulate spatial representations found in the
brain. These models fall into several categories. 1) Auto-encoder like models trained on spatial
representations (Dordek et al., 2016). 2) Graph representation models (Gustafson & Daw, 2011;
Stachenfeld et al., 2017). 3) Latent state inference models (George et al., 2021). 4) Path integrating
models with RNNs trained on spatial representations (Cueva & Wei, 2018; Banino et al., 2018). 5)
Models mixing RNNs and memory networks that are trained on sequences of sensory observations.

We note that models (1-4) are learned using curated spatial representation, i.e. the modeller has
already worked how how space works, and the model is finding an alternate representation of that
space. This is not how you or I learn. Instead, we learn by extracting regularities from the sensory
world, and transfer/generalise this knowledge from domain to domain. Model category (5) does
unsupervised learning by predicting sequences of sensory observations alone. From just sensory
observations, it can slowly build up a picture of how space is structured, and then transfer this
knowledge to situations that share the same underlying spatial structure. Our model TEM-t fall in
the model (5) category, along with TEM (Whittington et al., 2020) and other similar models Uria
et al. (2020).

For model to make sensory predictions as fast as possible, it is necessary to have two components.
a) A RNN that understand position, and b) a memory network that can remember what you see and
where you see them. Now the model can be asked what ’what will you see when heading East’
and correctly respond Cat even it it has never gone East at that location before. This is because
it can simulate going East via its RNN and then retrieve the memory it had stored at that location
(it must have visited the Cat location before thought!). The other models, e.g. an autoencoder,
could not solve this task since it learns slow statistical structures between sensory observations over
many many training examples. Thus it would be clueless to the entirely new configuration of sensory
observations in the new environment. In sum, it is necessary to have an understanding of position,
**and the ability to make and retrieve memories to successfully make sensory predictions as fast**
**as possible.**

We describe the relationship between TEM (and thus the hippocampal-entorhinal system) as close
**not because it produces the same representations, instead we are saying the relationship is close**


-----

**+ action at+1**

gt

Wa

Wg qt

Wg

Mt pt

Wx

xt

**Step = t**

Transition with

weight matrix

**Path integrate new g**

**Query**

**memory**

**Form memory** **network, and**

**p from g and** **retrieve**

**x, via outer** **memory p via**

**product, then** **attractor**

**change** **dynamics**

**weights M**

**Predict sensory**
**Sensory data**

**from p**

**Step = t+1**

Figure 7: Schematic to show the model flow. Depiction of TEM at two time-points, with each
time-point described at a different level of detail. Timepoint t shows network implementation, t + 1
describes each computation in words. Red is for model predictions, green is for updating model
variables. We do not show the stabilising position encodings module here. Circles depict neurons
(blue is g, red is x, blue/red is p); shaded boxes depict computation steps; arrows show learnable
weights; looped arrows describe recurrent attractor. Black lines between neurons in attractor describe Hebbian weights M . Wa are learnable, action dependent, transition weights. Wg and Wx
are learnable projection matrices. Yellow arrows show training errors.

because we have shown a mathematical relationship between the two models. This could not
happen for most ML models as most ML systems do not have both RNN structured representations
and a memory system and so is not mathematically relatable to current models of the hippocampal
formation. However, equipped with the mathematical relationship, we can say that the memory
part of the transformer is related to the hippocampal memory system in TEM. And the positional
encodings are related to the entorhinal grid RNN system in TEM.

A.6 FURTHER LEARNED CELL REPRESENTATIONS


-----

Figure 8: Memory formation and retrieval in TEM. (a-b) Memory formation. (a) Projected sensory
sensory code ˜x and projected grid code ˜g are combined via an outer-product ˜x[T] **_g˜, which is flattened_**
to obtain a vector of place cells p. Each place cell (denoted by a single diagonally divided cell) is
a conjunction of an element from each of ˜x and ˜g (denoted by the two colours composing each
cell). The activity of the place cell is the product of the values of these elements. (b) A new Hebbian
memory p[T] **_p is added to the existing memory matrix M_** . (c-d) Memory retrieval. (c) Multiplication
of the query q with the memory matrix M retrieves a place code p. This retrieved code is the sum
of previously experienced codes, weighted by their similarity to the present query. This may be
repeated iteratively to converge to the stored p that is most similar to q. (d) The retrieved place code
**_p is reshaped and summed along the rows to average-out the g components. The result is_** _g[¯]˜x˜._


-----

**A**

**B**

Figure 9: Learned grid cells ordered by grid score. We only show cells that are both active and have
a grid score above 0. A gird score of 0.3-0.5 is generally considered to be a grid cell. The panels on
the right hand side are the show the grid scores for the whole population of cells (though in some
cases the grid score was not calculable e.g. if the cell has no activity; these cells are ignored in the
analysis). (a) Grid cells learned with a linear activation function. (a) Grid cells learned with a ReLu
activation function.



Figure 10: Learned grid cells in hexagonal 6-connected world.


-----

between TEM-t RNN neurons (grid cells), and TEM-t memory neurons (place cells).


**A** **B**

Figure 11: Place cells ordered by novel place cell metric. This metric assess how place-like the firing
of each cell is. In particular, we look at the connected components of the firing rate map, and our
metric is the ratio of ‘firing mass’ in the largest connected component versus all connected compo-
nents. This metric is 1 if all the firing is in a single component, and it is lower if the firing is spread
between components. (a) TEM-t learned memory place cells. (b) Our novel metric distinguishes


-----

