## `eigenkit.orthogonal.torch.modules`

PyTorch modules for orthogonal networks: the angles are an `nn.Parameter`, and the orthogonal matrix is rebuilt from them, so it stays orthogonal at every training step.

Numerical contracts. Each module documents the exact sequence of torch operations it runs, and the tests pin it bit for bit. Changing one of them is a breaking change, because it changes the last bits of every result and so the trajectory of any training run.

Example

```python
import torch

from eigenkit.orthogonal.torch import Orthogonal

rotation = Orthogonal(8, "butterfly", init="random", std=0.5)
x = torch.randn(4, 8)
y = rotation(x)  # x @ R.T
torch.allclose(y.norm(dim=1), x.norm(dim=1))  # True
```

### `MAX_MATERIALISE = 2 ** 31`

The largest `$\mathrm{depth} \cdot n^2$` for which `mode="materialise"` builds the dense matrix.

### `Orthogonal`

Bases: `_Rotation`

An `$n \times n$` orthogonal matrix parameterised by the angles of a network of RBS (Givens) rotations.

A Givens rotation is orthogonal for every angle, and a product of orthogonal matrices is orthogonal, so the matrix is orthogonal at every training step with no projection and no penalty. On quantum hardware each rotation is one RBS gate, so this is the classical simulation of a quantum orthogonal layer.

The `state_dict` holds only `theta`. The pair indices are buffers that are not saved, so `.to(device)` moves them and a checkpoint stays small. They are rebuilt from the topology whenever the module moves, so a module built on the `meta` device, then `to_empty()` and `load_state_dict()`, works.

Numerical contracts:

- Initialisation: `init="identity"` sets `theta = torch.zeros(n_angles)` and draws no random numbers; `init="random"` sets `theta = torch.randn(n_angles) * std`, one call on the global generator, made before any other random draw of the module.
- The matrix: starting from `W = torch.eye(n)` in the dtype and on the device of `theta`, each layer takes the next `k` angles, `c = cos(theta[at:at+k]).unsqueeze(1)`, `s = sin(...).unsqueeze(1)`, and sets `W = W.index_copy(0, i, c*Wi + s*Wj).index_copy(0, j, -s*Wi + c*Wj)` with `Wi = W.index_select(0, i)`, `Wj = W.index_select(0, j)`. With a reflected topology the last row is negated at the end.
- Caching: the matrix is cached only when `not self.training and not torch.is_grad_enabled()`, under the key `(theta._version, theta.device, theta.dtype)`, and it is reused only while `theta` still holds the values it was built from (`torch.equal`), so a write through `theta.data` cannot leave it stale. The cache decides only when the matrix is rebuilt, never its values. On the `meta` device nothing is cached. A frozen matrix (`freeze_matrix`) takes precedence.

`mode="materialise"` builds the dense matrix, at a cost of `$O(\mathrm{depth} \cdot n^2)$`, and applies it as one matrix product. It is refused when `$\mathrm{depth} \cdot n^2$` exceeds `MAX_MATERIALISE` (`$2^{31}$`), which rules out a pyramid wider than 1024. `mode="apply"` applies the rotations to the input layer by layer, at a cost of `$O(\mathrm{depth} \cdot n \cdot B)$` for a batch of `$B$`, and never builds the matrix in `forward`; it has no contract with the materialised path, whose last bits it does not reproduce.

Parameters:

| Name       | Type           | Description                                                                                            | Default                                                        |
| ---------- | -------------- | ------------------------------------------------------------------------------------------------------ | -------------------------------------------------------------- |
| `n`        | `int`          | The width.                                                                                             | *required*                                                     |
| `topology` | \`TopologyName | Topology\`                                                                                             | A layout name, or a Topology on n wires. Default: "butterfly". |
| `init`     | `Init`         | "identity" for zero angles, "random" for normal angles of standard deviation std. Default: "identity". | `'identity'`                                                   |
| `std`      | `float`        | The standard deviation of the random angles. Default: 0.1.                                             | `0.1`                                                          |
| `mode`     | `Mode`         | "materialise" or "apply". Default: "materialise".                                                      | `'materialise'`                                                |

#### `n_angles`

The number of angles, one per RBS gate.

#### `forward(x, dim=-1)`

Rotate `x` along one axis: `$y = Rx$` on that axis.

In `mode="materialise"`, or with a frozen matrix: `R = self.matrix().to(x.dtype)`, then `x @ R.t()` for a 2-D `x` along its last axis and `torch.einsum("oc,bc...->bo...", R, x)` for an `x` of 3 or more dimensions along `dim=1`. In `mode="apply"` the rotations act on `x` directly.

Parameters:

| Name  | Type     | Description                                                 | Default    |
| ----- | -------- | ----------------------------------------------------------- | ---------- |
| `x`   | `Tensor` | A 2-D batch (B, n), or a tensor (B, n, ...).                | *required* |
| `dim` | `int`    | -1 (or 1) for a 2-D input, 1 for a larger one. Default: -1. | `-1`       |

Returns:

| Type     | Description                                          |
| -------- | ---------------------------------------------------- |
| `Tensor` | torch.Tensor: The rotated tensor, of the shape of x. |

Raises:

| Type         | Description                                 |
| ------------ | ------------------------------------------- |
| `ValueError` | For any other combination of shape and dim. |

#### `matrix()`

The materialised `$n \times n$` orthogonal matrix (see the class contract).

Returns:

| Type     | Description                                                        |
| -------- | ------------------------------------------------------------------ |
| `Tensor` | torch.Tensor: The matrix, in the dtype and on the device of theta. |

#### `reset_parameters()`

Initialise the angles again, by the rule of `init`.

`init="identity"` sets them to zero and draws nothing; `init="random"` draws `torch.randn(n_angles) * std` from the global generator, as the constructor does, and copies it into `theta` in place. The estimators call this after seeding, so `random_state` controls the random angles, and an identity start stays the identity.

### `OrthogonalConv1x1`

Bases: `Orthogonal`

An orthogonal 1x1 convolution: `Orthogonal` applied to the channels of a feature map, plus an optional bias.

Numerical contract: `forward(x) = F.conv2d(x, W.reshape(n, n, 1, 1), bias)` with `W = self.matrix().to(x.dtype)` and `bias = self.bias.to(x.dtype)`, as `Orthogonal.forward` casts its matrix to the dtype of the input. The bias parameter is created after `theta`, so the `state_dict` holds `theta` and `bias`.

Parameters:

| Name       | Type           | Description                                                          | Default                                                        |
| ---------- | -------------- | -------------------------------------------------------------------- | -------------------------------------------------------------- |
| `n`        | `int`          | The number of channels.                                              | *required*                                                     |
| `topology` | \`TopologyName | Topology\`                                                           | A layout name, or a Topology on n wires. Default: "butterfly". |
| `init`     | `Init`         | "identity" or "random". Default: "identity".                         | `'identity'`                                                   |
| `std`      | `float`        | The standard deviation of the random angles. Default: 0.1.           | `0.1`                                                          |
| `bias`     | `bool`         | Whether to add a trainable bias, initialised to zero. Default: True. | `True`                                                         |
| `mode`     | `Mode`         | "materialise" or "apply". Default: "materialise".                    | `'materialise'`                                                |

#### `forward(x)`

Apply the orthogonal 1x1 convolution.

Parameters:

| Name | Type     | Description                                                                                | Default    |
| ---- | -------- | ------------------------------------------------------------------------------------------ | ---------- |
| `x`  | `Tensor` | A feature map of shape (B, n, H, W), or (n, H, W) without a batch axis, as F.conv2d takes. | *required* |

Returns:

| Type     | Description                                  |
| -------- | -------------------------------------------- |
| `Tensor` | torch.Tensor: The output, of the shape of x. |

#### `reset_parameters()`

Initialise the angles again by the rule of `init` (see `Orthogonal.reset_parameters`), and the bias to zero, as the constructor does.

### `PartitionedOrthogonal`

Bases: `_Rotation`

An orthogonal `$\mathrm{width} \times \mathrm{width}$` matrix for any width, from orthogonal blocks on the binary parts of the width, mixed by fixed permutations.

The butterfly needs a power-of-two width. This module splits the width into its binary parts (`$588 = 512 + 64 + 8 + 4$`), rotates every part of size 2 or more with its own `Orthogonal`, and repeats this over `stages` stages, `$S$` of them, with a fixed channel permutation `$P_s$` in between:

```math
R = (P_S^\top B_S P_S) \cdots (P_2^\top B_2 P_2) \, B_1,
\qquad B_s = \operatorname{blockdiag}(\text{parts of stage } s)
```

Every factor is orthogonal, so `$R$` is too. All angles start at zero, so `$R = I$` at initialisation. A part of size 1 (an odd width) is left as it is.

Numerical contracts:

- Construction: stages outer, parts inner, one `Orthogonal(part, topology)` with `init="identity"` per part of size 2 or more, so no random numbers are drawn; then, for `s = 0 .. stages - 2`, the buffers `perm_s = mixing_permutation(width, perm_seed * 1000 + s)` and its inverse `inv_s = torch.argsort(perm_s)`, not saved in the `state_dict` and rebuilt the same way whenever the module moves (so `to_empty()` after a `meta`-device build works).
- Stage matrix: `torch.block_diag(*part_matrices)`, with `torch.ones(1, 1)` in the dtype and on the device of the other parts appended for a part of size 1.
- The matrix: `R = stage_0`; then for `s >= 1`, `R = B_s[inv][:, inv] @ R` with `inv = inv_{s-1}`, the stored `torch.argsort(perm_{s-1})`.
- `forward(x, dim)` follows the contract of `Orthogonal.forward` in `mode="materialise"`.

The `state_dict` keys are `blocks.<stage>.<part>.theta`.

Parameters:

| Name        | Type           | Description                                                  | Default       |
| ----------- | -------------- | ------------------------------------------------------------ | ------------- |
| `width`     | `int`          | The width, at least 2.                                       | *required*    |
| `stages`    | `int`          | The number of block-diagonal stages, at least 1. Default: 2. | `2`           |
| `perm_seed` | `int`          | The seed of the mixing permutations. Default: 0.             | `0`           |
| `topology`  | `TopologyName` | The layout of every part. Default: "butterfly".              | `'butterfly'` |

#### `n_angles`

The number of angles over every stage and part.

#### `matrix()`

The materialised `$\mathrm{width} \times \mathrm{width}$` orthogonal matrix (see the class contract).

Returns:

| Type     | Description               |
| -------- | ------------------------- |
| `Tensor` | torch.Tensor: The matrix. |

### `binary_parts(width)`

The powers of two that sum to `width`, largest first: 588 gives `[512, 64, 8, 4]`.

Parameters:

| Name    | Type  | Description         | Default    |
| ------- | ----- | ------------------- | ---------- |
| `width` | `int` | A positive integer. | *required* |

Returns:

| Type        | Description                                                |
| ----------- | ---------------------------------------------------------- |
| `list[int]` | list\[int\]: The binary digits of width, as powers of two. |

Raises:

| Type         | Description                         |
| ------------ | ----------------------------------- |
| `ValueError` | If width is not a positive integer. |

### `mixing_permutation(width, random_state)`

A fixed random permutation of `width` channels, from its own CPU generator.

Numerical contract: `torch.randperm(width, generator=torch.Generator().manual_seed(random_state))`. The global generator is not touched.

Parameters:

| Name           | Type  | Description                | Default    |
| -------------- | ----- | -------------------------- | ---------- |
| `width`        | `int` | The number of channels.    | *required* |
| `random_state` | `int` | The seed of the generator. | *required* |

Returns:

| Type     | Description                                                      |
| -------- | ---------------------------------------------------------------- |
| `Tensor` | torch.Tensor: A permutation of 0 .. width - 1, as a long tensor. |
