Part II — Dataset audits (facts first, interpretation second)

II.1 Dataset A: FTLE heterogeneity $G_{\lambda }$

Dataset A — $G_{\lambda }$ grid (NL_Glambda_trend.csv)

A.1.1 What the dataset actually contains (fact check)

From your description and usage:

• Grid over:

  • width $N\in \{10,50,100,200\}$
  • depth $L\in \{2,6,8,12\}$

• For each $(N,L)$:

  • 16 runs from gain $\times$ base learning-rate combinations

• Recorded:

  • $G_{\lambda }$
  • ${\log }_{10}{G}_{\lambda }$

This is internally consistent.

II.1.1 Experimental grid

We evaluate the FTLE field ${\lambda }_T(x)$ on a fixed 2D grid and compute:

Parameters:

• Width $N\in 10,50,100,200$
• Depth $L\in 2,6,8,12$
• 16 runs per $(N,L)$ from gain $\times$ learning-rate combinations

II.1.2 Empirical facts

From the data:

1. Width dependence
   For fixed $L$:

2. Depth dependence at moderate/large width
   For fixed $N\ge 50$:

3. Narrow-width anomaly ($N=10$)

• $G_{\lambda }$ is non-monotone in $L$ (a “U-turn”).
• Variability across hyperparameters is largest at $N=10$.

These are purely observational statements.

II.1.3 Immediate but safe interpretation

Because $G_{\lambda }$ measures variance of ${\lambda }_T(x)$ over the grid:

• $G_{\lambda }\downarrow$ means the FTLE field becomes more spatially uniform with respect to the grid measure.
• At large width, increasing depth appears to homogenize the FTLE field.
• At small width, training dynamics are sensitive and geometry is unstable.

No claim is made yet about feature learning.

II.2 Dataset B: Kernel and representation alignment (KA / RA)

Dataset B — RA / KA grid (RA_KA_NL_dataset.csv)

A.2.1 What the dataset actually contains

• Same $(N,L)$ grid
• One $\mathrm{RA}$ and one $\mathrm{KA}$ per point
• These are alignments between initial and final geometry

This is important: RA/KA measure total rotation over training, not instantaneous dynamics.

II.2.1 Definitions (reminder)

Kernel alignment:

Representation alignment:

Both measure rotation away from initialization, not magnitude change.

II.2.2 Empirical facts

From the data:

1. Width dependence

2. Depth dependence

with the strongest effect at small $N$.

3. Wide-but-deep regime

• Even at $N=200$, increasing depth produces nontrivial rotation:

II.2.3 Meaning (strictly limited)

• $\mathrm{RA},\mathrm{KA}$ quantify feature motion / geometry rotation.
• They do not encode where in input space the changes occur.
• They do not imply localization or ridges.

II.3 Cross-dataset summary (no synthesis yet)

From Part II alone:

• Width suppresses both:
  • feature motion (RA/KA $\uparrow$),
  • FTLE heterogeneity ($G_{\lambda }\downarrow$).
• Depth:
  • promotes feature motion (RA/KA $\downarrow$),
  • but often suppresses FTLE heterogeneity at large width.

This establishes a tension, not a contradiction:

• feature rotation and FTLE heterogeneity are not the same axis.

This tension motivates the theory development in later parts.

Status after Part II (progress bar)

• ✔ Definitions fixed and invariant properties established (Part I)
• ✔ Empirical trends recorded cleanly (Part II)
• ⏳ Theory explaining why these axes decouple (next)