[{"content":"Abstract This is the proof of correctness of the stenographic encoding algorithm described in 1. The paper provides empirical results to demonstrate the effectiveness, but not a full theoretical argument. I derived this to convince myself that it is indeed correct.\nIntroduction The message is randomized with a key and partitioned into $n$ blocks: $X_{1}, \\ldots, X_{n}$. Each block is uniform $X_{i}\\sim\\text{Unif}[2^{b}]$ where $b$ is the parameter block size. The covertext generator is a distribution $\\mathcal{C}(c\\mid c_{1},...,c_{m}):=P(C_{m+1}=c\\mid C_{1}=c_{1},...,C_{m}=c_{m})$ specified autoregressively, where the random vector $(C_{1},...,C_{m})$ is the \u0026ldquo;context\u0026rdquo; in the LLM sense and $C_{m+1}$ is the next generated \u0026ldquo;token\u0026rdquo;.\nThe algorithm proceeds as follows:\nFor $i\\in 1\\ldots n$, initialize $\\mu_{i}$ to uniform distributions $\\set{0,1}^{b}\\rightarrow [0,1]$. For $j\\in 1\\ldots m$: $i^{*}:=\\arg\\max_{i}H(\\mu_{i})$ $\\gamma_{j}:=$ MEC of $\\mu_{i^{*}}$ and $\\mathcal{C}(C_{j}\\mid C_{1:j-1}=S_{1:j-1})$, the autoregressive distribution of next token $S_{j}\\sim \\gamma_{j}(C_{j}\\mid X_{i^{*}}=x_{i^{*}})$, the distribution of next token conditional on the $i^{*}$th block of ciphertext $\\mu_{i^{*}}\\leftarrow \\gamma_{j}(X_{i^{*}}\\mid C_{j}=S_{j})$ The objective is to show that $(C_{1},...,C_{m})\\overset{D}{=}(S_{1},\\ldots S_{m})$, that is, the random vectors of $m$-sequences of tokens produced naturally and those produced by the algorithm are equivalent in distribution, i.e., the encoding does not introduce any statistical bias.\nAs a shorthand, I will write events $\\set{S_{j}=s_{j}}$ as $\\set{s_{j}}$ and $\\set{S_{1:j-1}=s_{j-1}}$ as $\\set{s_{1:j-1}}$. The notation $[n]$ denotes the set $\\set{1,...,n}$.\nProof The proof proceeds in two steps. First, we must show that at the beginning of each iteration $j$, for all $i\\in[n]$ and all values of $x$, we have $$P(X_{i}=x\\mid s_{1:j-1} )=\\mu_{i}(x)$$ that is, the $\\mu_{i}$ distributions maintained by the algorithm are the accurate distributions of $X_{i}$ conditional on the previously produced tokens. This is not entirely trivial, since the $\\mu_{i}$ are updated manually. Secondly, we show the equality in distributions between natural and encoded sequences of tokens.\n◆ Lemma For each iteration $j$, for all $i\\in[n]$ and values of $x$, the following hold: $$\\newcommand{\\indep}{\\mathrel{\\perp\\!\\!\\!\\perp}} \\begin{align*} \u0026(i)\\qquad P(X_{i}=x\\mid s_{1:j-1})=\\mu_{i}(x) \\\\ \u0026(ii)\\qquad X_{1} \\indep X_{2} \\indep \\ldots \\indep X_{n}\\;\\mid s_{1:j-1} \\end{align*}$$ The second claim is that $\\set{X_{i}}_{n}$ are mutually conditionally independent given $s_{1:j-1}$. We will use this statement as a sort of inventor\u0026rsquo;s paradox to establish the first.\nProof. Proceed by induction on $j$. At the start of the first iteration, $X_{i}\\sim\\text{Unif}[2^{b}]=\\mu_{i}$ by definition. All $X_{i}$ are mutually independent, so the claims hold trivially.\nIn the inductive case, we assume the claims hold at the start of step $j$. The maximum entropy block $i^{*}$ is chosen. We prove $(i)$ first; the aim is to show that for all $i$, $\\mu'_{i}$, defined to be the adjusted $\\mu$ at the end of the step, satisfies $$\\mu'_{i}(x)=P(X_{i}=x\\mid s_{1:j})\\quad \\forall x.$$ Note that only for $i=i^{*}$ does the distribution actually change; the rest are left untouched. We therefore consider two cases:\nCase 1: $i=i^{*}$ Breaking down the conditioning as $P(\\ldots \\mid s_{1:j-1}\\cap s_{j})$ and rewriting, we get $$\\begin{align*} P(X_{i^{*}}=x\\mid s_{1:j})\u0026=\\frac{P(X_{i^{*}}=x\\mid s_{1:j-1})P(s_{j}\\mid X_{i^{*}}=x,\\; s_{1:j-1})}{P(s_{j}\\mid s_{j-1})}\\\\ \u0026= \\frac{\\mu_{i^{*}}(x)\\gamma_{j}(s_{j}\\mid x)}{\\mathcal{C}(s_{j}\\mid s_{j-1})}\\\\ \u0026= \\frac{\\gamma_{j}(x,s_{j})}{\\mathcal{C}(s_{j}\\mid s_{j-1})}\\\\ \u0026=\\gamma_{j}(x\\mid s_{j})= \\mu'_{i^{*}}(x). \\end{align*}$$ In the above derivation, the coupling $\\gamma_{j}$ twice allows us to change conditioning via chain rule with its marginal (respectively $\\mu_{i^{*}}$ and $\\mathcal{C}$).\nCase 2: $i\\neq i^{*}$ Similarly, we write $$P(X_{i}=x\\mid s_{1:j})=\\frac{P(X_{i}=x\\mid s_{1:j-1})P(S_{j}=s_{j}\\mid X_{i}=x,\\; s_{1:j-1})}{P(s_{j}\\mid s_{1:j-1})}$$ Since $S_{j}\\sim\\gamma_{j}(C_{j}\\mid X_{i^{*}}=x_{i^{*}})$, where the next token $C_{j}\\sim\\mathcal{C}(\\cdot \\mid s_{1:j-1})$ is independent of $X_{i}$ trivially and $X_{i^{*}}$ is conditionally independent of $X_{i}$ by the inductive hypothesis, $S_{j}$ is also conditionally independent of $X_{i}$. Thus the second term in the numerator can drop the conditioning on $X_{i}=x$, simplifying to $$P(X_{i}=x\\mid s_{1:j}) = \\frac{\\mu_{i}(x)\\mathcal{C}(s_{j}\\mid s_{j-1})}{\\mathcal{C}(s_{j}\\mid s_{j-1})}=\\mu_{i}(x)=\\mu'_{i}(x).$$Lastly, we show that $(ii)$ holds at the end of each step. Using the same identity and the conditional independence given $s_{1:j-1}$: $$\\begin{align*} P\\bigg(\\bigcap_{i\\in[n]}X_{i}=x_{i}\\mid s_{1:j}\\bigg) \u0026=\\frac{P(s_{j}\\mid s_{1:j-1}\\cap \\bigcap_{i}X_{i}=x_{i})}{\\mathcal{C}(s_{j}\\mid s_{j-1})} \\prod_{i\\in[n]} P(X_{i}=x_{i}\\mid s_{1:j-1}) \\end{align*}$$ Once again the independence of $S_{j}$ allows us to drop the conditioning on all $X_{i}$ except $X_{i^{*}}$ and simplify: $$\\begin{align*} \\ldots \\;\u0026= \\frac{\\gamma_{j}(s_{j}\\mid x_{i^{*}})}{\\mathcal{C}(s_{j}\\mid s_{j-1})}\\prod_{i\\in[n]} P(X_{i}=x_{i}\\mid s_{1:j-1})\\\\ \u0026= \\frac{\\gamma_{j}(x_{i^{*}}\\mid s_{j})}{\\mu_{i^{*}}(x_{i^{*}})}\\prod_{i\\in[n]}\\mu_{i}(x_{i})\\\\ \u0026= \\mu'_{i^{*}}(x_{i^{*}})\\prod_{i\\in[n]\\setminus\\set{i^{*}}} \\mu'_{i}(x_{i})\\\\ \u0026= \\prod_{i\\in[n]} \\mu'_{i}(x_{i}) \\\\ \u0026=\\prod_{i\\in[n]}P(X_{i}=x_{i}\\mid s_{1:j}). \\end{align*}$$ This concludes the proof of $(i)$ and $(ii)$ at the beginning of step $j+1$.\n◆ Theorem Let $(S_{1},...,S_{m})$ be the random sequence of tokens produced by the algorithm (stegotext), and let $(C_{1},...,C_{m})$ be the random sequence of tokens generated naturally (covertext) in an autoregressive manner, i.e. $C_{j}\\sim\\mathcal{C}(\\cdot \\mid C_{1:j-1}=c_{1:j-1})$. Then: $$(S_{1},...,S_{m})\\overset{D}{=}(C_{1},...,C_{m}).$$ Proof. Let $j$ be fixed and consider some value $s$ of the next token $S_{j}$. $$P(S_{j}=s\\mid s_{1:j-1})=\\sum\\limits_{x}\\gamma_{j}(X_{i^{*}}=x,s)=\\sum\\limits_{x}\\gamma_{j}(s\\mid x)\\mu_{i^{*}}(x)=\\mathcal{C}(s\\mid s_{1:j-1})$$ The first equality averages over all possible values of $X_{i^{*}}$ and the rest follows by definition of coupling. Now, taking the product over all $j$ with the chain rule: $$\\begin{align*} P(S_{1:m}=s_{1:m})\u0026= \\prod_{j\\in[m]}P(S_{j}=s_{j}\\mid s_{1:j-1}) \\\\ \u0026= \\prod_{j\\in[m]}\\mathcal{C}(s_{j}\\mid s_{1:j-1})\\\\ \u0026= P(C_{1:m}=s_{1:m}). \\end{align*}$$ Thus, the random vectors have equal distribution.\n◆ Corollary The algorithm constructs a coupling between a factorable uniform distribution and the autoregressive conditional $\\mathcal{C}(\\cdot\\mid\\cdot)$. Let $\\text{A}(\\mathbf{x}, \\mathbf{s})$ be the joint distribution induced by the algorithm between the space of $n$-block ciphertexts and $m$-token text samples, writing $\\text{A}(\\mathbf{x},\\mathbf{s}):=P(\\mathbf{X}=\\mathbf{x},\\;\\mathbf{S}=\\mathbf{s})$ and $\\text{A}(\\mathbf{s}\\mid \\mathbf{x})$ for probability of outputting stegotext $\\mathbf{s}$ given input ciphertext $\\mathbf{x}$, where $\\mathbf{X}=(X_{1},...,X_{n})$ and $\\mathbf{S}=(S_{1},...,S_{m})$.\nObserve that for any $\\mathbf{x}$ and $\\mathbf{s}$, we have $\\text{A}(\\mathbf{x},\\mathbf{s})=\\text{A}(\\mathbf{s}\\mid \\mathbf{x})P(\\mathbf{X}=\\mathbf{x})=\\mathcal{C}(\\mathbf{s})P(\\mathbf{X}=\\mathbf{x})$ by above. Now fixing $\\mathbf{x}$ and summing over $\\mathbf{s}$ yields $P(\\mathbf{X}=\\mathbf{x})$, and likewise fixing $\\mathbf{s}$ and summing over $\\mathbf{x}$ gives $\\mathcal{C}(\\mathbf{s})$, which are the exact marginals, showing $\\text{A}(\\cdot\\mid \\cdot)$ is a coupling.\nAppendix The proof makes repeated use of this basic identity: $$P(A\\mid B\\cap C)=\\frac{P(A\\mid B)P(C\\mid A\\cap B)}{P(C\\mid B)}$$ https://arxiv.org/abs/2210.14889\u0026#160;\u0026#x21a9;\u0026#xfe0e;\n","date":"2026-08-30T00:00:00Z","permalink":"https://saan-volta.github.io/post/proof_of_perfect_secrecy/","title":"Proof of Correctness of MEC Steganography"},{"content":"Introduction The field of steganography is concerned with hiding secret messages in innocuous-seeming data. It is a sort of dual to cryptography, but rather than encrypt the content of the message, one tries to conceal the message\u0026rsquo;s presence in the first place. The primary instrument for detecting such concealments is statistical analysis - looking for patterns within data that should not be there, if it contained no secrets. This post and project is about an algorithm that is impervious to such attacks - it leverages an LLM to embed the secret within the choice of tokens, but its output is statistically indistinguishable from naturally generated text.\nThis technique can also be used for the inverse task - marking LLM-generated text with a watermark embedded in the tokens themselves, rather than any metadata. Not too long ago, Anthropic introduced this feature into their models. If they relied on this algorithm, they can provably claim this will not affect their models\u0026rsquo; performance.\nAnother property of this encoding is that it is maximally efficient from the perspective of information theory: it takes the fewest tokens per bit of information. And more broadly, this algorithm can be used to encode data efficiently in other mediums, such as images and sample paths of RL agents.\nThis post is an overview of this idea, its applications, and some of the implementation logic. It is partially technical but not fully rigorous; my aim is to give an actionable introduction. Material from the following papers is adapted here: 1 | 2 | 3 | 4 |.\nFinally, I built a live demo you can use to try this method out: https://cover.markovian.net! Here\u0026rsquo;s also my implementation of the algorithms described in this post: https://github.com/saan-volta/Cover.\n1. The problem setting What is perfect secrecy? Our task is to embed a message within a random signal without disturbing its statistical properties. We define $\\mathbb{C}$ to be the space of covertexts - data which we will use for, well, cover, and $\\mathbb{M}$ is the space of messages which we might want to encode. To reason about statistical properties, we define $\\mathcal{M}$ to be the distribution of messages and $\\mathcal{C}$ to be the distribution of true covertexts (over $\\mathbb{C}$). For example, if we want to embed a message within text data, we\u0026rsquo;ll use a cover distribution that captures typical patterns of natural language. Meanwhile, we denote with $\\mathcal{S}$ the distribution (also over $\\mathbb{C}$) of stegotexts: pieces of data which appear as covertexts, but in fact contain a hidden message. So the encoding is is a (randomized) map $f:\\mathbb{M}\\leadsto \\mathbb{C}$, converting a message into a piece of stegotext (existing in the space of covertexts).\nIntuitively, we achieve perfect secrecy when $\\mathcal{C}$ and $\\mathcal{S}$ are identical, i.e. have KL-divergence zero. Another, more technical way of saying it is when $$E_{M\\sim\\mathcal{M}}\\bigg[P(f(M)=c)\\bigg]=\\mathcal{C}(c).$$ This says that on average, the probability that the random message $M$ encodes to stegotext $c$ is equivalent to the probability of $c$ appearing as an innocuous covertext.\nInformation and uncertainty We are also concerned with the encoding efficiency of our scheme: the amount of covertext data needed to embed our message. To quantify this, we introduce several definitions from information theory. ◇ Definition: Entropy Let $X$ be a random variable. Then its entropy is $H(X)=-E[\\log p_{X}(X)]$, where $p_{X}(x)=P(X=x)$. If $X$ is discrete, this is equivalently $H(X)=-\\sum_{x}p_{X}(x)\\log p_{X}(x)$. Entropy is the measure of chaos of a random variable; it is increased with the unpredictability of its outcome. The highest entropy is achieved in the uniform distribution (any outcome is equally likely), and the lowest (zero) in the Dirac delta distribution $\\delta_{x}$, where the entire mass is concentrated on the single point $x$ (the outcome is fully predictable).\nWe consider similarly the joint entropy on a vector of random variables $$H(X,Y)=-E[\\log p_{X,Y}(X,Y)]$$ and the conditional entropy $H(X\\mid Y)$ as the uncertainty of $X$ conditioned on the realization of $Y$. These two forms are tied by the identity $$H(X,Y)=H(X)+H(Y\\mid X)=H(Y)+H(X\\mid Y).$$ There\u0026rsquo;s one more definition we need to state our objective. ◇ Definition: Mutual information Let $X,Y$ be random variables. Then $I(X;Y)=H(X)-H(X\\mid Y)$ is the mutual information between $X$ and $Y$, and the expected amount of uncertainty about $X$ eliminated by knowing $Y$. Combining these formulae, we get $$I(X;Y)=H(X)+H(Y)-H(X,Y).$$ Then the core principle is this: if we want to maximize the mutual information between the two variables, we must minimize the joint entropy $H(X,Y)$. This is our only option since the marginal entropies $H(X)$ and $H(Y)$ are not modifiable.\nIn the context of steganography, to achieve maximal encoding efficiency, we aim to increase $I(M;S)$, where $M$ is the secret message and $S$ the stegotext; this will give us the best ratio of bits encoded per output token. In effect, this is a measure of compression. But how do we optimize it?\n2. The MEC The point and crux of the algorithm we will examine is in the concept of minimum-entropy coupling (MEC). In a sentence, it\u0026rsquo;s a constructed joint distribution with the lowest $H(\\cdot,\\cdot)$ measure. More concretely: ◇ Definition: Minimum entropy coupling Let $X\\sim\\mathcal{X}$ and $Y\\sim\\mathcal{Y}$. A coupling of $\\mathcal{X}$ and $\\mathcal{Y}$ is a joint distribution $\\gamma(\\cdot,\\cdot)$ that maintains the marginals of $X$ and $Y$. That means for all values $x$ of $X$ $$\\sum\\limits_{y}\\gamma(x,y)=\\mathcal{X}(x),$$ and for all values $y$ of $Y$ $$\\sum\\limits_{x}\\gamma(x,y)=\\mathcal{Y}(y).$$ Note that in general there are many coupling with set marginals; we use $\\Gamma(\\mathcal{X,Y})$ to denote their set. The minimum-entropy coupling is the coupling $\\gamma^{\\star}\\in\\Gamma(\\mathcal{X,Y})$ such that $\\forall \\gamma\\neq \\gamma^{\\star}$, we have $H(\\gamma^{\\star})\\leq H(\\gamma)$. In other words, it is the coupling with the smallest joint entropy.\nLet\u0026rsquo;s see an example. Here are two variables $X$ and $Y$ with their corresponding marginal distributions: A trivial example of coupling is the independent coupling, defined as $\\gamma(x,y):=\\mathcal{X}(x)\\cdot\\mathcal{Y}(y).$\nFor both of these produced couplings, the sum along each axis is the corresponding marginal. However, the right one is \u0026ldquo;packed more tightly\u0026rdquo; and has far less noise. The coupling minimizes the mutual information between $X$ and $Y$ by more than 3 bits over the independent.\nWhere did I get this second joint distribution? The underlying MEC algorithm I used here is developed in (Kocaoglu et al, 2016). This algorithm is actually approximate as this problem is considered NP-hard. The actual implementation of it is beyond the scope of this post, but one element of note is that this paper considers this task is a completely different context - it is about the problem of inferring causal relationship between random variables based on observed data.\nBut what does this have to do with steganography?\n3. First of its kind Here we arrive at the first key contribution of (Schroeder de Witt, 2023). ◇ Definition: Encoding induced by coupling A steganographic encoding procedure $f:\\mathbb{M}\\leadsto\\mathbb{C}$ is induced by a coupling $\\gamma\\in\\Gamma(\\mathcal{M},\\mathcal{C})$, where $\\mathcal{M,C}$ are respectively the distributions of messages and covertext, if for all $m\\in\\mathbb{M}$ and $c\\in\\mathbb{C}$ $$P(f(M)=c)=\\gamma(C=c\\mid M=m).$$ A note on what $\\gamma(C=c\\mid M=m)$ means: since $\\gamma$ is a joint distribution of $(M,C)$, this can be visualized as taking row $m$ of the joint matrix, normalizing it to a probability distribution, and finding the probability value at index $c$.\n◆ Theorem A steganographic encoding procedure is perfectly secure if and only if it is induced by a coupling. Among encoding procedures that are perfectly secure, a procedure maximizes the mutual information $I(M;S)$ if and only if it is induced by a minimum-entropy coupling. These two proofs in the paper are remarkably concise. The best way I could show them would be to copy it line for line, but I will not do that.\nSo this is what we want - a way to produce a coupling between the distributions of covertext and the message space (for the moment, both of them being natural language). However\u0026hellip; this is somewhat tricky. The MEC algorithm can couple two explicitly-defined discrete distributions with relatively small supports, but $\\mathcal{C}$ and $\\mathcal{M}$ are incredibly complex. How does one couple the distributions over all sequences of natural text?\nThe second contribution the authors present is the algorithm to produce an implicit coupling between a factorable uniform distribution and a distribution specified autoregressively. The trick is this: given a message $m$, we XOR it with a random key to obtain $X:=m\\oplus K$, which is distributed uniformly. We then partition it into $n$ blocks of $b$ bits each, so $X=(X_{1},...,X_{n})$. Meanwhile, we take $\\mathcal{C}(\\cdot\\mid c_{1},...,c_{j})$ as the distribution of the next token (word) conditioned on a context of length $j$; this distribution is specified by a large language model, which we will assume to accurately represent natural text. The greedy algorithm, called Iterative MEC, is this: Here\u0026rsquo;s what this does in plain terms. We maintain a list of distributions $\\mu_{1},...,\\mu_{n}$, each for a block $X_{i}$. At each iteration, we fetch the block with the highest entropy, couple it to the distribution of the next token $\\mathcal{C}(C_{j}\\mid \\;...)$ using the simple MEC procedure, and emit the next stegotext token conditional on the actual value of $X_{{i}^{*}}$, and finally update the corresponding distribution. This procedure ends when all $\\mu_{i}$ collapse to the deterministic single-point distribution $\\delta_{x_{i}}$ - as they are initialized to uniform, observe that they transform from the highest- to the lowest-entropy distribution. At the end, we receive a sequence of stegotext tokens $(S_{1},...,S_{m})$; the length $m$ is exactly how many iterations it takes to eliminate all uncertainty from all the blocks $\\set{\\mu_{i}}_{n}$. In essence, this algorithm manipulates the choice of tokens at each step, but the actual distribution of the generated text is not changed as it is an encoding induced by a coupling.\nTo decode, essentially the exact same algorithm is run - the tokens $\\set{S_{j}}_{m}$ are consumed to similarly update the block distributions $\\set{\\mu_{i}}_{n}$ until they converge to the zero entropy Dirac deltas. Then, we simply sample from them in order to reconstruct $\\set{\\hat{x}_{i}}_{n}$, the estimation of the keyed message, and finally XOR with the key again to recover $m$.\n4. Other applications The iMEC method is not limited to text data. In place of the LLM-given $\\mathcal{C}(\\cdot\\mid c_{1},...,c_{j})$ next-token-based-on-context distribution, one could similarly encode data within audio and images. A particularly cool medium expplored in (Sokota et al., 2022) is encoding a message within the sample trajectory of a reinforcement learning agent in a Markovian decision process: A more LLM-oriented application is watermarking: certifying that a piece of data was generated artificially by embedding a signature in the tokens themselves. In this case, the performance of the model will provably be unaffected.\nMore broadly, MEC is applied in entropic causal inference: given two random variables $X,Y$, determining whether there exists an outside variable $E$ such that $Y=f(X,E)$, i.e. there exists a deterministic function implying causal direction from $X$ to $Y$. It is intuitive that if this is so, the entropy in the true causal direction will be small. Furthermore, the optimal coupling should be able to identify $E$. The authors of (Cicalese at el, 2019) explore some other applications of this method, as well as develop a MEC algorithm that approximates the true minimum-entropy coupling with error margin of 1 bit.\nSchroeder de Witt et al., 2023: https://arxiv.org/abs/2210.14889\u0026#160;\u0026#x21a9;\u0026#xfe0e;\nSokota et al., 2022: https://arxiv.org/abs/2107.08295\u0026#160;\u0026#x21a9;\u0026#xfe0e;\nCicalese et al., 2019: https://arxiv.org/abs/1901.07530\u0026#160;\u0026#x21a9;\u0026#xfe0e;\nKocaoglu et al., 2016: https://arxiv.org/abs/1611.04035v2\u0026#160;\u0026#x21a9;\u0026#xfe0e;\n","date":"2026-08-26T00:00:00Z","permalink":"https://saan-volta.github.io/post/perfect_secrecy/","title":"Perfect Secrecy and Entropy Minimization"},{"content":"Introduction A bit ago, I made a post about a result characterizing $k$-colorability of a graph using ring theory of ideals: ◆ Theorem: Theorem 1 For a graph $G=(V,E)$ with $|V|=n$, define its adjacency polynomial $f_{G}(x_{1},...,x_{n})=\\prod_{ij\\in E}(x_{i}-x_{j})$, and define the ideal $I_{k}=\\langle x_{1}^{k}-1,...,x_{n}^{k}-1\\rangle$ for some integer $k$ in the ring $\\mathbb{C}[x_{1},...,x_{n}]$. Then $G$ is $k$-colorable if and only if $f_{G}\\in I_{k}$. Thus graph coloring problem is reduced to an ideal membership problem. I think this result is cool for its own sake, but the next natural question is \u0026ldquo;does this make this problem any easier to solve?\u0026rdquo; I spent quite a bit of time seeing if I could make it work as a computational task, and this post is a sort of journal covering the hoops I had to jump through to try and make it feasible.\n1. Smooth sailing Disclaimer: this section covers mathematical background that I hope is accessible, but it is not strictly necessary to understand the implementation. At the end of the section, I outline the actual thing that needs to be computed, you may skip to it directly.\nHere\u0026rsquo;s the nice thing. In general, to solve an ideal membership problem, we have to compute a Gröbner basis of said ideal. An ideal can be represented with many bases (generating sets), but some of them are much more amenable to doing any sort of computation, and Gröbner bases are the best of them. The downside is that computing such a basis is really hard by itself (at least as hard as graph coloring, for certain). The upside is that we don\u0026rsquo;t need to, and I will give a brief overview to explain this.\nWe begin with polynomial division. In a univariate polynomial ring, such as $\\mathbb{R}[x]$, we have what is called the \u0026ldquo;division algorithm\u0026rdquo;: for any polynomials $f$ and $g$, we can find polynomials $q,r$ such that $f=qg+r$, where $\\deg(r)\u003c\\deg(g)$, that is, we divide $f$ by $g$ with remainder. This works always because a univariate ring is a lovely and handsome Principal Ideal Domain, which means that any ideal $I$ can be generated by only one element, like $I=\\langle x\\rangle$; this is like how the set of even numbers in $\\mathbb{Z}$ can be represented by $\\langle 2\\rangle$, i.e. all multiples of 2. But a multivariate ring, like $\\mathbb{R}[x,y]$, does not have this property. So how do we divide polynomials in 2 variables?\nWhen we divide $f$ by $g$, what we actually do is divide by the ideal $\\langle g\\rangle$. If you\u0026rsquo;ve taken any sort of abstract algebra, you may be familiar with this action as the canonical homomorphism $R\\rightarrow R/\\langle g\\rangle$ sending each element of the ring to the coset of the ideal to which it belongs (the remainder is the coset representative). But in $\\mathbb{R}[x,y]$, some (most) ideals are not principal; we have to write them as $I=\\langle g_{1},g_{2}\\rangle$. If you attempt to run the standard division algorithm to divide by $g_{1}$ and $g_{2}$ trying to find the quotients and the remainder in $f=q_{1}g_{1}+q_{2}g_{2}+r$, you may find that the order of $g_{1}$ and $g_{2}$ affects not only the quotients, but also the value of the remainder itself, so the \u0026ldquo;canonical homomorphism\u0026rdquo; is actually not a function at all.\nA Gröbner basis is what lets us fix this, since their essential property is that they always produce the same minimal remainder $r$ when used as a divisor, regardless of the permutation of elements within. Thus we can actually compute $(f\\mod I)$ when $I$ is expressed as a Gröbner basis $\\langle g_{1},...,g_{n}\\rangle$.\nI haven\u0026rsquo;t really defined what constitutes a Gröbner basis, and actually I\u0026rsquo;m not gonna do that here at all, because the main takeaway that matters is that $I_{k}=\\langle x_{i}^{k}-1\\mid i\\in[n]\\rangle$ is already Gröbner. In particular, reducing a polynomial $f\\mod I_{k}$ is simply reducing its every exponent mod $k$. Thus our problem is: ▲ Problem: The Big Thing For a given graph $G=(V,E)$ and integer $k$, compute the adjacency polynomial $f_{G}=\\prod_{ij\\in E}(x_{i}-x_{j})$ and reduce every exponent mod $k$. $G$ is $k$-colorable if and only if the result is $0$. Examples Before we begin, here are some examples of graphs and their corresponding polynomials, just to get a feel for it: $$\\begin{align} \u0026P_{2}\\rightarrow \\quad x_{1}-x_{2}.\\\\ \u0026P_{3}\\rightarrow \\quad x_{1}x_{2}-x_{1}x_{3}-x_{2}^{2}+x_{2}x_{3}.\\\\ \u0026C_{3}\\rightarrow \\quad -x_1^2 x_2 + x_1 x_2^2 + x_1^2 x_3 - x_2^2 x_3 - x_1 x_3^2 + x_2 x_3^2.\\\\ \u0026S_{4}\\rightarrow \\quad x_1^4 - x_1^3 x_2 - x_1^3 x_3 + x_1^2 x_2 x_3 - x_1^3 x_4 + x_1^2 x_2 x_4 + x_1^2 x_3 x_4 - x_1 x_2 x_3 x_4 \\\\\u0026- x_1^3 x_5 + x_1^2 x_2 x_5 + x_1^2 x_3 x_5 - x_1 x_2 x_3 x_5 + x_1^2 x_4 x_5 - x_1 x_2 x_4 x_5 \\\\\u0026- x_1 x_3 x_4 x_5 + x_2 x_3 x_4 x_5.\\\\ \u0026C_{4}\\rightarrow \\quad x_1^2 x_2 x_3 - x_1 x_2^2 x_3 - x_1^2 x_3^2 + x_1 x_2 x_3^2 - x_1^2 x_2 x_4 + x_1 x_2^2 x_4 + x_1^2 x_3 x_4 \\\\\u0026-2 x_1 x_2 x_3 x_4 + x_2^2 x_3 x_4 + x_1 x_3^2 x_4 - x_2 x_3^2 x_4 + x_1 x_2 x_4^2 - x_2^2 x_4^2 \\\\\u0026- x_1 x_3 x_4^2 + x_2 x_3 x_4^2.\\\\ \u0026K_{4}\\rightarrow \\quad x_1^3 x_2^2 x_3 - x_1^2 x_2^3 x_3 - x_1^3 x_2 x_3^2 + x_1 x_2^3 x_3^2 + x_1^2 x_2 x_3^3 - x_1 x_2^2 x_3^3 \\\\\u0026- x_1^3 x_2^2 x_4 + x_1^2 x_2^3 x_4 + x_1^3 x_3^2 x_4 - x_2^3 x_3^2 x_4 - x_1^2 x_3^3 x_4 + x_2^2 x_3^3 x_4 + x_1^3 x_2 x_4^2\\\\\u0026 - x_1 x_2^3 x_4^2 - x_1^3 x_3 x_4^2 + x_2^3 x_3 x_4^2 + x_1 x_3^3 x_4^2 - x_2 x_3^3 x_4^2 - x_1^2 x_2 x_4^3 \\\\\u0026+ x_1 x_2^2 x_4^3 + x_1^2 x_3 x_4^3 - x_2^2 x_3 x_4^3 - x_1 x_3^2 x_4^3 + x_2 x_3^2 x_4^3. \\end{align}$$As you can see, these get large rather quickly.\n2. The First Look Since have a (theoretical, at this point) algorithm for checking whether $G$ is $k$-colorable, to find the minimum such $k$, or the chromatic number $\\chi(G)$, we will simply run binary search; this works because is_colorable(G,k) is a monotonic function with respect to $k$.\ndef chromatic_number(G, k): low, high = 2, maxdegree(G)+1 # assume |E| \u0026gt; 0. med = (low+high)//2 while low \u0026lt; high: bool k_valid = is_colorable(G, med) if k_valid: low, high = low, med else: low, high = med+1, high return med This solves for $\\chi(G)$ using $O(\\log_{2}(\\max\\deg(G)-1))$ queries. We use $\\max\\deg(G)+1$ as that is the upper bound on $\\chi(G)$ by Brooks\u0026rsquo; theorem (and unless $G$ is $K_{n}$ or $C_{n}$ with $n$ odd, you can drop the +1). So going forward, all focus is on optimizing the query is_colorable.\nThe task therefore is repeated multiplication of $n$-variate polynomials. The main bottleneck here is storing and updating terms. My first choice here is processing the edges one by one and reducing exponents mod $k$ at each step, since that caps the number of possible terms to $k^{n}$ (an astronomical number, but at least it\u0026rsquo;s not unbounded). At each step we multiply the previous iteration of the polynomial $f_{t}$ by some $(x_{i}-x_{j})$:\nf_0 = 1 for (i, j) in E: f_{t+1} = (f_t * x_i) - (f_t * x_j) f_{t+1} %= I_k The second line in the loop reduces each exponent in each term mod $k$. A polynomial has the following expanded form: $$f_{t}=(c_{1}\\cdot x_{1}^{e_{1}^{(1)}}...x_{n}^{e_{n}^{(1)}})\\;+ \\;(c_{2}\\cdot x_{1}^{e_{1}^{(2)}}...x_{n}^{e_{n}^{(2)}})\\;+\\;...$$ We can represent it as a list of terms, each term consisting of a coefficient and a monomial, and each monomial is a list $[e_{1}^{(i)},...,e_{n}^{(i)}]$ of exponents for each $x_{j}$.\nf_t = [ (c1, [e11, e12, ..., e1n]), (c2, [e21, e22, ..., e2n]), ..., ] We multiply $f_{t}\\cdot x_{j} \\mod I_{k}$ by incrementing $j$th exponent in each term of $f_{t}$ (modulo $k$).\nThis is\u0026hellip; less than ideal. First, while the updates themselves are fast, the memory footprint is atrocious, since we must allocate a vector for each term. Second, to simplify the polynomial (when we do the subtraction at each step), we want to combine like terms. To do this, we must iterate by pairs of terms in $f_{t}\\cdot x_{i}$ and $f_{t}\\cdot x_{j}$, which is $O(n^{2})$. Vector comparison is another $O(n)$ factor.\nThe natural two improvements are:\nHash the monomial exponent vectors into single integers for $O(1)$ comparison Use a hashmap $\\text{monomial exponent vector hash }\\mapsto \\text{ coefficient}$ exp_list = [ (hash1, [e11, e12, ..., e1n]), (hash2, [e21, e22, ..., e2n]), ..., ] coeffs_map = { hash1 -\u0026gt; c1, hash2 -\u0026gt; c2, ... } Now the algorithm can do this:\n# input: edge ij, previous-step polynomial f_t new_exps = [] new_coeffs = {} for exp_hash, exp_vec in exp_list: exp_vec_i = copy(exp_vec) exp_vec_i = (exp_vec_i[i] + 1) % k new_hash_i = hash(exp_vec_i) # O(?) exp_vec_j = copy(exp_vec) exp_vec_j = (exp_vec_j[j] + 1) % k new_hash_j = hash(exp_vec_j) new_coeffs[new_hash_i] = coeffs[exp_hash] new_coeffs[new_hash_j] -= coeffs[exp_hash] new_exps.insert( (new_hash_i, exp_vec_i) ) new_exps.insert( (new_hash_j, exp_vec_j) ) exp_list = new_exps coeffs_map = new_coeffs Okay, now assume hash() is constant time. This works nicely because combining like terms occurs naturally as we simply update coefficient values indexed by hashes of the exponent vectors. This doesn\u0026rsquo;t solve the storage problem, but the loop is certainly faster.\nOkay, now un-assume hash() is constant time, because you cannot, in fact, process a length-$n$ vector in $O(1)$. You can do something else, though: make the hash incremental, i.e., homomorphic with respect to coordinate-wise incrementation, so we can compute new_hash_i from exp_hash and i in constant time. The rest of this section is dedicated to this problem.\nThere are two kinds of modifiable hashes that I considered for this: polynomial rolling hash and Zobrist hash. The former hashes a vector $A=[a_{1},...,a_{n}]$ as $$h(A)=\\sum_{i=0}^{n-1}a_{i}\\cdot B^{i} \\mod P,$$ where $B$ and $P$ are pre-determined (prime) constants. This hash is widely used in competitive programming for processing strings, since it has nice properties for when you want to compare prefixes and suffixes. For our purposes, defining $A'$ to be the vector identical to $A$ except in the $j$th coordinate, for which the value is $b\\neq a_{j}$, we write: $$h(A')\\equiv h(A)-a_{j}B^{j}+bB^{j} \\pmod P.$$ So we undo the $j$th coordinate in the original hash and replace it with the desired value $b$, which in our case would be $a_{j}+1 \\mod k$.\nThe Zobrist hash is quite different; it is incremental by design and was invented to store games like chess - when a move is made, the state of the board changes only slightly, and you\u0026rsquo;d not want to recompute the hash from scratch when only two squares change.\nGenerate a $n\\times k$ table of random values, call it $Z$. Define $$h(A)=\\bigoplus_{i=1}^{n}Z[i, a_{i}],$$where $\\oplus$ is bitwise XOR. Now to modify the $j$th coordinate, we again undo the current value and replace it with another:\n$$h(A')=h(A)\\oplus Z[j,a_{j}]\\oplus Z[j,b].$$ Both of these hashes work, but for this task I went with Zobrist, since it doesn\u0026rsquo;t require finding suitable constants, XOR is faster than mod-multiplication, and overall it\u0026rsquo;s just cleaner.\nAn aside about benchmarking When I began this project, I wanted to put together some graphs for testing accuracy and speed. Initially I drew a few up by hand, and tested my results against the symbolic computation done by SageMath. For a graph on more than $10$ vertices, SageMath took too long, though, and I didn\u0026rsquo;t want to generate random graphs and run actually practical coloring algorithms on them to figure out the true $\\chi(G)$. For this reason I turned to two known graph families: the Mycielski and the Turán graphs. The point is that both of them are easily constructible and have a regular structure that makes the chromatic number a fundamental property, without needing any computation.\nThe Mycielski graph $M_{k}$ is specifically designed to have chromatic number $k$, with each successive $M_{k}$ being constructed iteratively from $M_{k-1}$. The special property is that it has no triangle subgraphs, making it quite sparse. The Turán graph $T_{n,r}$ is the opposite - it is the $r$-partite complete graph, having chromatic number $r$ and as many edges as possible.\n3. Eliminating Exponent Vector Storage The Zobrist hash trick allows us to compare monomials\u0026rsquo; exponent vectors via their hashes, as well as update these hashes incrementally. However, the memory footprint remains: to update a hash by incrementing $a_{j}$, we actually have to know the current value, because the value we set is $a_{j}+1$ if $a_{j} \u003c k$ and $0$ otherwise; there is a branch involved. We cannot perform a \u0026ldquo;blind\u0026rdquo; incrementation mod $k$ on the hash, so we must store the entire vector.\nThis was a glaring inefficiency that didn\u0026rsquo;t let me sleep at night for some time. ▲ Problem: Cyclic Incremental Hash Let $A=[a_{1},...,a_{n}]$ be an $n$-vector with $a_{i}\\in\\set{0,...,k-1}$, and let $A'$ be the same vector in all coordinates except $j$, where it is $a'_{j}=a_{j}+1 \\mod k$. Find a hash $h:\\mathbb{Z}_{k}^{n}\\rightarrow \\set{0,1}^{128}$ such that $h(A')=f(h(A),j)$ for some computable function $f$, that is, $h(A')$ can be computed directly knowing only $h(A)$ and $j$. Okay, so the \u0026ldquo;blind\u0026rdquo; wrap-around mod $k$ should point to a cyclic representation of some kind. Suppose $G$ is a group of prime order $p$, so every non-identity element is a generator. If we pick $n$ such elements $g_{1},...,g_{n}$ and define $h(A)=\\prod_{i\\in n}g_{i}^{a_{i}}$, then we will see that we can change the $j$th element to some arbitrary element $b$ with: $$h(A')=h(A)\\cdot g_{j}^{-a_{i}}\\cdot g_{j}^{b}.$$ This idea I found in Bellare et al. (2021)1. The group here would be a multiplicative group mod $p$, where $p$ is be chosen to satisfy $p=2q+1$ for another prime $q$; this ensures $\\mathbb{Z}_{p}^{\\times}$ has prime order. However, this does not yet give us the cyclic effect mod $k$. For that, we can use a trick: choose $p$ to instead satisfy $p\\equiv 1\\pmod k$. Since it no longer has prime order, not every element is a generator, but nevertheless there are $\\varphi(p-1)$ of them2. We need $n$ elements $g_{1},...,g_{n}$ with order $k$, for which we can take a generator $g$ and raise it to power $(p-1)/k$. Once again let $h(A)=\\prod_{i\\in[n]}g_{i}^{a_{i}}$ but now, to increment the $j$th coordinate we simply multiply $h(A')=h(A)\\cdot g_{j}$. This works because we\u0026rsquo;ve made it so $g_{j}^{k}=e$, which gives us the desired modulo effect.\nHowever, there\u0026rsquo;s a significant problem with this approach (see if you can find it). Because we take a product over elements with $g_{i}^{k}=e$, the set of outputs is exactly the elements of the group which satisfy this. So what is the size of our hash\u0026rsquo;s image?\n◆ Lemma: Hash Output Space Let $G(k)=\\set{g\\in\\mathbb{Z}_{p}^{\\times}\\mid g^{k}=e}$. Then $|G(k)|=k$. Proof: $\\mathbb{Z}_{p}^{\\times}$ is a cyclic group, so $\\exists x\\text{ s.t. }\\langle x\\rangle=\\mathbb{Z}_{p}^{\\times}$. Then $x^{(p-1)/k}$ has order $k$, and subsequently $y_{t}:=x^{t(p-1)/k}$ satisfies $y_{t}^{k}=e$ for $t\\in 1,...,k$. This means $|G(k)|\\geq k$. On the other hand, we can note that $\\mathbb{Z}_{p}$ is a field and the elements of $G(k)$ are $k$-th roots of unity in it, in other words, the roots of $x^{k}-1$. A degree $k$ polynomial has at most $k$ roots. Having bounded above and below, we get $|G(k)|=k$. So while the hash function thus constructed satisfies our original requirements, it can only ever output $k$ distinct values\u0026hellip; This is far too few to ever be useful. My solution here was to extend this idea - rather than have only one $k$-cyclic subgroup which hard-caps our outputs to a small number, we embed multiple.\nLet $p$ and $q$ be prime, and $N:=pq$. Recall that $\\mathbb{Z}^{\\times}_{N}\\cong \\mathbb{Z}^{\\times}_{p}\\times \\mathbb{Z}^{\\times}_{q}$ and that each of the groups on the right side are cyclic. Choosing again $p$ such that $|\\mathbb{Z}^{\\times}_{p} |=p-1$ is divisible by $k$, we guarantee the group contains a cyclic subgroup of order $k$ (isomorphic to the cyclic group $C_{k}$). We choose $q$ likewise, with the only added constraint of $p\\neq q$. Applying the isomorphism above, we\u0026rsquo;ve ensured that $C_{k}\\times C_{k}$ is isomorphic to a subgroup of $\\mathbb{Z}_{N}^{\\times}$. If $\\langle g\\rangle=C_{k}$, then $C_{k}^{2}=C_{k} \\times C_{k}=\\langle (g,e), (e,g)\\rangle$. In other words, if we can find the elements corresponding to $(g,e)$ and $(e,g)$ in $\\mathbb{Z}_{N}^{\\times}$, we can add an extra dimension to our hash image!\nLet\u0026rsquo;s do this concretely. The Chinese Remainder theorem gives us $\\mathbb{Z}^{\\times}_{p}\\times \\mathbb{Z}^{\\times}_{q}\\cong \\mathbb{Z}^{\\times}_{N}$, let $\\phi$ be this isomorphism, and suppose we\u0026rsquo;ve found the elements $h_{p}\\in\\mathbb{Z}^{\\times}_{q}$ and $h_{q}\\in\\mathbb{Z}^{\\times}_{q}$ as above, so the $\\text{ord}(h)=k$. The basis $\\set{ (h_{p},e),(e,h_{q})}$ generates $C_{k}^{2}$ and we can lift it to $\\mathbb{Z}^{\\times}_{N}$: define $g_{1}:=\\phi(h_{p},e)$ and $g_{2}:=\\phi(e,h_{q})$. Restating above, we get: $$C_{k}^{2}\\cong \\langle g_{1},g_{2}\\rangle\\leq \\mathbb{Z}^{\\times}_{N}.$$ The point is that the size of $\\text{span}\\set{g_{1},g_{2}}$ in $\\mathbb{Z}^{\\times}_{N}$ is $k^{2}$ and we can find these elements very easily using a CRT solver (essentially the implementation of $\\phi$) for the systems $$\\begin{cases} g_{1}\\equiv h_{p}\\pmod p \\\\ g_{1}\\equiv 1 \\pmod q\\end{cases} \\qquad\\begin{cases} g_{2}\\equiv 1 \\pmod p \\\\ g_{2}\\equiv h_{q}\\pmod q. \\end{cases}$$ Lastly, we choose $s_{1},s_{2}$ from $0,...,k-1$ randomly, and define $w=g_{1}^{s_{1}}g_{2}^{s_{2}}$. Since $g_{i}^{k}=e$, we know also that $w^{k}=e$. Now rather than just one, generate $n$ pairs $(s_{1}^{(i)},s_{2}^{(i)})$, and for each of them let $w_{i}$ be defined likewise. What we have done at this point is construct $w_{1},...,w_{n}\\in \\mathbb{Z}^{\\times}_{N}$ which span the product cyclic subgroup and satisfy $w^{k}=e$, which is the exact condition we wanted. We formalize the hash function for $A=[a_{1},...,a_{n}]$ as $$h(A)=\\prod_{i\\in[n]}w_{i}^{a_{i}}$$ and verify that for $A'$ defined as $A$ but with $j$th coordinate incremented mod $k$, it satisfies $$h(A')=h(A)\\cdot w_{j}.$$ The order of each $w$ is $k$ (or, in rare cases, a divisor of $k$ if $gcd(k,s_{1},s_{2})\u003e1$), so $w^{k}$ cycles back to $e$, and $\\text{span}\\set{w_{1},...,w_{n}}=\\text{span}\\set{g_{1},g_{2}}$ with size $k^{2}$. Thus we have our magic hash!\nAsterisks This yields a hash with image size $k^{2}$ but in practice this is still far too small. However, this method is entirely extendable: rather than use two primes, we use $N:=p_{1}p_{2}...p_{d}$, leverage $\\mathbb{Z}_{p_{1}}^{\\times}\\times ... \\times \\mathbb{Z}_{p_{d}}^{\\times}\\cong \\mathbb{Z}_{N}^{\\times}$ and the generalization $$C_{k}^{d}=\\langle h_{1}\\rangle\\times ...\\times \\langle h_{d}\\rangle \\cong \\langle \\phi(h_{1},e,e,...,e), \\phi(e,h_{2},e,...,e),...\\rangle\\leq \\mathbb{Z}_{N}^{\\times}$$ to make $w_{1},...,w_{n}$ span $d$ dimensions, resulting in the image size $k^{d}$. Since the operations in $\\mathbb{Z}_{N}^{\\times}$ happen modulo $N$, this is the bottleneck - the product of our $d$ distinct special primes must fit in an (128-bit) integer data type. To maximize $d$, we choose the primes to be as small as possible, but on average, we can get $\\approx 18$. Still, this gives us a very sizable output space.\nA key step I\u0026rsquo;ve skipped is producing a generator $g$ of $\\mathbb{Z}_{p}^{\\times}$. There\u0026rsquo;s not a known closed form solution for this problem, but (for any $p$) on average about a third of the elements of $\\mathbb{Z}_{p}^{\\times}$ are generators, so a guess-and-check algorithm is actually quite efficient for this. We know by Lagrange\u0026rsquo;s theorem that if an element $g$ has order $m \u003c n:=p-1=|\\mathbb{Z}_{p}^{\\times}|$, then $m$ is a divisor of $n$. The contrapositive tells us that if for some $m\\mid n,\\; g^{m}=e$, then $g$ is not a generator; thus, we will pick a random element and go through all maximal divisors of $n$ to see if our candidate evaluates to $e$ with that exponent; if it doesn\u0026rsquo;t for any of them, we\u0026rsquo;ll have found our generator.\nwhile True: x = randint(2,p-1) failed = False for q in prime_factors(n): if pow(x, n/q, p) == 1: failed = True break if not failed: return x For prime factorization I will precompute the SPF array (spf[n] = smallest prime factor of n) up to a constant maximum; this needs only to be done once and allows $O(\\log x)$ factorization. I use this method very often in competitive programming.\nNow having found the generator of each $\\mathbb{Z}_{p_{i}}^{\\times}$, we let $h_{i}$ be it\u0026rsquo;s $((p_{i}-1)/k)^{\\text{th}}$ power; observe that each $h_{i}$ has order $k$ and thus $C_{k}\\cong \\langle h_{i}\\rangle \\leq \\mathbb{Z}_{p_{i}}^{\\times}$.\nSummary To recap, what we\u0026rsquo;ve done so far is construct a group $\\mathbb{Z}_{N}^{\\times}$ that contains as many embedded copies of the cyclic group $C_{k}$ in it as we could fit. We know that their product $C_{k}\\times ... \\times C_{k}\\cong \\langle h_{i}\\rangle\\times...\\times\\langle h_{d}\\rangle$ is a group to which there exists an isomorphic subgroup in $\\mathbb{Z}_{N}^{\\times}$ by the Chinese remainder theorem. We find the vectors spanning $C_{k}^{d}$ and lift them to $\\mathbb{Z}_{N}^{\\times}$, and we generate weights which give us the cyclic incrementation property.\nAs a final note, solving a CRT modular system of $d$ equations in the standard way requires $O(d)$ time; as there are $d$ such systems, we\u0026rsquo;re looking at $O(d^{2})$. However, the systems are all vectors of the \u0026ldquo;standard basis\u0026rdquo;: in the $i$th vector, all elements but the $i$th are identity. We can use this to speed up the computation to $O(d)$ in total by doing it in a batch with $O(1)$ per system/vector with $N$ known.\n4. Reduction via Edge Ordering The algorithm can be viewed as a process that begins with an empty graph on $n$ vertices and adds an edge at each step. The revelation of each edge $(ij)$ corresponds to multiplying the previous step\u0026rsquo;s polynomial by $(x_{i}-x_{j})$. The number of terms in this polynomial grows very quickly (more or less exponentially, as you might imagine); the more terms we have at step $t$, the more we will have to process and insert at $t+1$. But not all edges give the same growth. Can we slow the term count growth by revealing edges in a clever order?\nThe answer is yes, and here\u0026rsquo;s a simple example: $f_{t}$ is the polynomial of the partial graph with edges $1,...,t$ (with respect to a particular ordering), and $\\text{T}(f_{t})$ is the number of terms in the polynomial. Notice that the second ordering produces fewer terms at steps 3 and 4. Of course, the final result will be the same in any ordering as the final graph is fixed, but smaller intermediate results means less work at each subsequent step.\nHere\u0026rsquo;s a more noticeable difference: The second ordering processes, in total, 73 fewer terms than the first. Clearly, among all orderings of edges of the graph $G$, there must be an ordering which yields the fewest intermediate terms over the course of execution. So\u0026hellip; what is it?\n▲ Problem: Minimum Cost Edge Ordering For a given graph $G$, we as usual define the polynomial $f=\\prod_{ij\\in E}(x_{i}-x_{j})$ and let $\\text{T}(f)$ denote the number of terms in the expanded \u0026amp; simplified form of the polynomial. If $e_{1}\\prec e_{2}\\prec ...\\prec e_{m}$ is any ordering of edges of $G$, we call $f_{t}^{\\prec}$ the partial polynomial of the first $t$ edges: $\\prod_{ij\\in E \\;:\\; (ij)\\preceq e_{t}}(x_{i}-x_{j})$; it is defined w.r.t. the ordering. Finally, define the cost of the ordering $c(\\prec):=\\sum_{t\\in[m]}\\text{T}(f_{t}^{\\prec})$.\nFind the optimal ordering $\\prec^{\\star}$ such that the cost $c(\\prec^{\\star})$ is minimum. This is a hard problem, I hate to say, and I don\u0026rsquo;t have a solution. But there are some heuristics which can give a pretty good approximation. The main problem is the behavior of the function $\\text{T}(\\cdot)$ - I don\u0026rsquo;t have a simpler formula for the term count than its definition, even a recurrent one. However, observe that each time we introduce an edge containing an endpoint not already in the graph, the term count doubles. This should make sense: in the step $f_{t}\\mapsto x_{i}f_{t}-x_{j}f_{t}$ the terms of our polynomial are duplicated, but a cancellation may occur when we combine like terms. However, if $x_{i}$ or $x_{j}$ is not in $f_{t}$ already, a term from $x_{i}f_{t}$ can never be equal to a term in $x_{j}f_{t}$. Thus, $\\text{T}(f_{t+1})\\leq2\\text{T}(f_{t})$ with equality iff $x_{i}$ or $x_{j}$ are added for the first time. In a way, our task here is to maximize cancellation, which occurs when we link two vertices already connected; that is, close a cycle.\nHere\u0026rsquo;s the key to why the second ordering in the both above examples wins: we close cycles greedily, aiming to close the shortest cycle available at each step, as short cycles provide the largest cancellation effect. When no cycles can be closed immediately, we attempt to predict which vertex reveals would produce potential for shortest cycles in the future. This method allows us to maximize cancellation at each step, and thus reduce the number of intermediate terms.\nLet\u0026rsquo;s denote $G_{t}$ as the partial graph with the first $t$ edges revealed. Which edge should we pick next to maximize the closure of shortest cycles? A working strategy is to compute the distances $d_{t}(u,v)$ for all edges $uv$ remaining not revealed at step $t$ and pick the one that minimizes this distance, thereby closing a $d_{t}(u,v)+1$-long cycle. Naively this would require recomputing the distances in $G_{t}$ between the endpoints of all potential edges at every step; with BFS it comes out to $O((m+n)m^{2})$ where $m$ is the number of edges in $G$. Here, there\u0026rsquo;s a clever trick we can do: observe that if we insert an edge $uv$, the distance between any two other vertices $x$ and $y$ changes in a very predictable way.\nHere we plan to reveal the edge $uv$ and the black solid line is the shortest path between $u$ and $v$ in $G_{t}$ (prior to the insertion). What is the shortest path between $x$ and $y$ in $G_{t+1}$? There are two options: it either does not include the edge $uv$ at all (1) or includes it once (2). In the first case, $d_{t+1}(x,y)=d_{t}(x,y)$, and in the second, the route first takes the segment $x-u$, then the newly added shortcut edge, then the segment $v-y$, so $d_{t+1}(x,y)=d_{t}(x,u)+1+d_{t}(v,y)$. We also get a candidate path by swapping $u$ and $v$ in this expression. Thus, to update distances after the insertion of $uv$ we write: $$d_{t+1}(x,y)=\\min\\set{d_t(x,y),\\;d_{t}(x,u)+1+d_{t}(y,v),\\;d_{t}(y,u)+1+d_{t}(v,x)}.$$ We still need to know the distances to $u$ and $v$, so we will run BFS to get them. The algorithm looks like this and runs in $O((n+m)m)$:\norder = [] future_edges = edges d = {{a,b} : INF for {a,b} in edges} # distances map for future edges while future_edges not empty: if have_finite_distance: # can close a cycle # take shortest distance edge uv = min(future_edges, key=d) future_edges.delete(uv) order.append(uv) u_dists = bfs(u) v_dsists = bfs(v) # update remaining distances for xy in future_edges: d[xy] = min(d[xy], u_dists[x]+1+v_dists[y], u_dists[y]+1+v_dists[x]) if d[xy] \u0026lt; INF: have_finite_distance = True else: # ... We want to defer the revealing of new vertices (revealing edges with a previously unseen endpoint) and prioritize adding edges between vertices already present. But If all distances on future edges are infinite, we can no longer close a cycle, so we must extend our graph to a new vertex. This cannot be done greedily and requires a certain prescience - it\u0026rsquo;s not clear which new vertex will lead to short cycles in the future. A heuristic can be used here: pick the vertex with the most back-edges to the discovered subgraph. The edge will be taken arbitrarily (though this can also be improved):\nfuture_vertices = vertices revealed_neighbors = {v : 0 for v in vertices} # ... else: v = max(future_vertices, key=revealed_neighbors) future_vertices.delete(v) # update futures and pick arbitrary back-edge from v for w in G.neighbors(v): if w not in future_vertices: u = w revealed_neighbors[w]++ order.append(uv) This is not an exact solution to the ordering problem, but it performs decently well for slowing down growth of terms. Unfortunately, since every new vertex reveal doubles the term count with no chance for cancellation, we\u0026rsquo;re still looking at $\\geq 2^{n}$ terms in the expression.\n5. A Faster Mod-Multiplication At this point, the algorithm clears Mycielski $M_{4}$ in a fraction of a second, but struggles to make it past the first ~25 edges of $M_{5}$. A closer look at performance reveals that over a third of the CPU cycles are spent on doing mod-multiplication to update the term hash entries. And no wonder, since my algorithm for doing that up to this point looks like this:\nui128 modmult(ui128 a, ui128 b, ui128 m) { ui128 result = 0; while (b\u0026gt;0) { if (b\u0026amp;1) // result = (result + a) % m; result=(result\u0026gt;=m-a) ? result-(m-a) : reuslt + a //a = 2*a % m; a = (a\u0026gt;=m-a) ? a-(m-a) : 2*a; b \u0026gt;\u0026gt;= 1; } return result; } This is awful and is essentially naive multiplication through addition in $O(\\log b)$. The reason I did this initially was because 128-bit ints are the largest int data type in C++, so I could not cast the intermediate result $a\\cdot b$ (which is up to 256 bits) to a larger type before taking mod. This was my next challenge. ▲ Problem: Max Precision Modular Multiplication Compute $(a\\cdot b \\mod m)$ as fast as possible given that all three integers are up to $B$ bits long, and no larger data type is available. Fortunately for me, some clever people solved this problem back in the 80s. The method I went for is called Montgomery multiplication. I won\u0026rsquo;t go into the full detail; I think the Wikipedia page explains it really well, it\u0026rsquo;s what I used to learn it. The base concept is this: we pick a radix base $r$ to be coprime to $m$, find the special Montgomery form $a*b$ defined in relation to $m$ and $r$, and express our result as $x\\equiv (a*b)r^{-1}\\pmod m$. To find the remainder in constant time, we must use the division operation which is the most expensive arithmetic instruction; what this lets us do is push the division over to $r$ rather than an expression of $a$ or $b$. The twist here is that we pick $r$ to be a power of two, which makes division as simple as bit shifting.\nMontgomery speeds up my modmult by a factor of ~x15, from 0.00414 to 0.00026ms on average for 128-bit ints.\n6. Last ditch efforts The Monty optimization pushes my CPU time spent on modmult down to about 5%; most of the rest is the time spent on the methods of unordered_map - allocation, the []get operator, emplace, and a couple minor ones. My guess is that at this point the fault is in frequent cache misses - the STL hashmap allocates nodes on the heap scattered across memory. I switched to a flat, open addressing map from the GTL library developed by Gregory Popovitch. It actually has a pretty cool parallel hashmap template that breaks the map into 16 submaps and leverages SIMD instructions to speed up search and resizing; however, it turned out that for my use case the standard flat map was faster. I attribute this to my never needing to resize - I reserve sufficient space at initialization.\nOpen addressing gives a nice bonus, but with all these optimizations I could not break Mycielski $M_{5}$ (23 vertices, 71 edges) or Turán $T_{15,3}$ (15 vertices, 75 edges). The space footprint still grows exponentially, and by 30-35 edges in my CPU begins stalling as most time is spent waiting for memory rather than actual computation.\nA strange trick I discovered here: it turned out that the map\u0026lt;ui128, int\u0026gt; was considerably slower than map\u0026lt;ui128, int64_t\u0026gt; - running on $M_{5}$, the first 30 edges took about 20% less time to process. Initially I guessed this is due to memory alignment rules splitting the 20-byte pairs across different 64-byte memory segments, requiring two cache lines to get a single pair; but in hindsight I don\u0026rsquo;t believe this is right - both options should require a full 32 bytes with padding to store. This is something I might investigate further.\nAnother idea I spent considerable time on: one could discard the hashmap entirely and switch to simple vectors. The problem requires us to match terms with equal monomial hashes, so if $x_{i}f_{t}$ and $x_{j}f_{t}$ were both represented by sorted arrays, we could do a zip merge to combine them into one. This would have a benefit over the hashmap since it would give us better locality control and improve the cache hit rate. However, sorting the arrays would take a long time (I tried with radix sort, it\u0026rsquo;s theoretically $O(n)$ but still slower than the hashmap approach), so instead what would be nice to have is a sort order that would be invariant under the action of multiplying each element by $w_{j}$. That is, if $h_{1}\\prec ... \\prec h_{T}$ is the sorted array of hashes of terms of $f_{t}$, then for $x_{i}f_{t}$ we maintain $h_{1}w_{i}\\prec ...\\prec h_{T}w_{i}$ and likewise for $j$. Then we could simply perform a sorted merge. The problem is, such a total order is impossible in a finite group: suppose elements of $G$ are ordered as $g_{1}\\prec ... \\prec g_{n}$ and we scale by $w\\in G\\text{ with }w\\neq e$, then $wg_{1}\\neq g_{1}$ is out of order.\nHowever, I\u0026rsquo;m not entirely convinced. For one, our condition is slightly weaker than this: not all elements can appear as $w$ and they are known in advance. For another, we don\u0026rsquo;t need to have a linear order - we could use a circular one. A circular order on a set $S$ is a trinary relation $[a,b,c]$ that indicates the order in which you\u0026rsquo;d see elements moving along the circle. For instance, if $S$ are hours of the clock, $[1,4,9]$ is true but $[4,8,5]$ is not. One could imagine the list of terms as a circular array (first and last elements are considered adjacent) sorted under a circular ordering, so a sorted merge could work much the same way.\nConclusion Ultimately, there\u0026rsquo;re many things left I could still investigate to improve performance, from high level mathematical representation to hardware optimization. This was an interesting experiment, and though I\u0026rsquo;ve learned some things I had not expected to even touch on, the practical results are\u0026hellip; meager - this method only works for very small graphs. I wouldn\u0026rsquo;t recommend it.\n7. Verdict If you need a graph colored, I advise using something less deranged.\nBellare et al.\u0026#160;\u0026#x21a9;\u0026#xfe0e;\nhttps://crypto.stanford.edu/pbc/notes/numbertheory/gen.html\u0026#160;\u0026#x21a9;\u0026#xfe0e;\n","date":"2026-07-15T00:00:00Z","permalink":"https://saan-volta.github.io/post/graph_col_irl/","title":"Graph Coloring via Rings IRL (II)"},{"content":"This is my final project report as part of CSE 552 Randomized \u0026amp; Approximation Algorithms taught by Dr. Andrea Richa at ASU.\nYour browser does not support embedded PDFs. Download the PDF.\n","date":"2026-05-15T00:00:00Z","permalink":"https://saan-volta.github.io/post/trace-reconstruction/","title":"Trace Reconstruction"},{"content":"Introduction While taking a course on ring theory, I looked up some applications of it to other fields. My favorite one was this connection to graph theory, in particular a characterization of the chromatic number from the perspective of ideal theory and algebraic geometry. In fact, there are two completely separate ways of getting to it, and each provides pretty cool insights.\nBackground ◇ Definition A commutative ring $R$ is a set with two operations: addition and multiplication. As in the integers, both are associative, commutative, and multiplication distributes over addition. $R$ is closed under the operations, and contains an additive identity $0$ and a multiplicative identity $1$. A polynomial ring over a field $F$ with one variable, $F[x]$, is the set of polynomials of $x$ with coefficients in $F$. Likewise, $F[x_{1},...,x_{n}]$ is the set of such multivariate polynomials. An ideal $I\\subseteq R$ of a ring is a subset such that: $\\forall a,b\\in I,\\;a+b\\in I$; $\\forall a\\in I, r\\in R, \\;ar\\in I$; An ideal $I$ is generated by $a\\in R$, written as $I=\\langle a\\rangle$, if $I=\\set{ar\\mid r\\in R}$. Likewise, $\\langle a_{1},...,a_{n}\\rangle=\\set{\\sum_{i\\in[n]}a_{i}r_{i}\\mid r_{i}\\in R}$. Here, $\\set{a_{1},...,a_{n}}$ is the basis of the ideal. For a set of polynomials $f_{1},...,f_{s}\\in F[x_{1},...,x_{n}]$, define the variety $\\mathbf{V}(f_{1},...,f_{s})=\\set{a=(a_{1},...,a_{n})\\in F^{n}\\mid f_{1}(a)=...=f_{s}(a)=0}$, that is, the set of common zeroes of the polynomials. ◆ Proposition: Fact 1 In a polynomial ring, if $I=\\langle f_{1},...,f_{s}\\rangle$ then $\\mathbf{V}(f_{1},...,f_{s})=\\mathbf{V}(I)$. This means that the variety of an ideal is not determined by the chosen basis. The first construction For a graph $G=(V,E)$, consider the multivariate polynomial ring $R=\\mathbb{C}[V]$ (with variables being vertices). For an integer $k$, define $I_{k}=\\langle v^{k}-1\\mid v\\in G\\rangle\\subseteq R$, which is an ideal. We get something interesting when we consider the variety $\\mathbf{V}(I_{k})$: if some point $(a_{1},...,a_{n})\\in\\mathbb{C}^{n}$ is in $\\mathbf{V}(I_{k})$, meaning all $v_{i}^{k}-1$ evaluate to $0$ at that point, or $a_{i}^{k}=1$, then each $a_{i}$ is a $k$-root of unity! In other words, the point assigns to each vertex of $G$ a value in $\\set{1,\\zeta,...,\\zeta^{k-1}}$, of which there are $k$, thus we can interpret $\\mathbf{V}(I_{k})$ to be the set of all $k$-colorings of $G$. This is cool, but not significant yet, since these colorings are not proper.\nNext, we introduce a way to \u0026ldquo;filter\u0026rdquo; these colorings. Let $$f_{G}(v_{1},...,v_{n})=\\prod_{(v_{i}v_{j})\\in E(G)}(v_{i}-v_{j})\\quad \\in \\mathbb{C}[V].$$ Here, we think of the variables as the assigned colors of vertices. It is easy to see that $f_{G}=0$ if and only if for some two vertices which share an edge, the color assignments are equal. Therefore, this polynomial vanishes exactly on the set of improper colorings.\n◆ Theorem: Theorem 1 $G$ is $k$-colorable if and only if $f_{G}\\notin I_{k}$. Proof $(\\Rightarrow )$. $f_{G}\\in I_{k}$ implies $f_{G}$ vanishes on $\\mathbf{V}(I_{k})$; let $a=(a_{1},...,a_{n})$ be a point of the variety, then $0=f_{G}(a)$ implies that for some edge $(v_{i}v_{j})$ we get $a_{i}=a_{j}$. Since $a$ was an arbitrary point corresponding to a $k$-coloring of $G$, all $k$-colorings fail to be proper.\nFor the reverse direction, we will need to invoke Hilbert\u0026rsquo;s Nullstellensatz. ◆ Theorem: Hilbert\u0026#39;s Nullstellensatz. If $f\\in F[x_{1},...,x_{n}]$ vanishes on $\\mathbf{V}(I)$ for some ideal $I$ then $f^{m}\\in I$ for some integer $m$. Proof $(\\Leftarrow )$. $G$ being non-$k$-colorable implies any point of $\\mathbf{V}(I_{k})$, which corresponds to a $k$-coloring, is not proper, so $f_{G}$ vanishes on it. Applying the Nullstellensatz, we know $f_{G}^{m}\\in I_{k}$ for some integer $m$. We see the result follows immediately if $I_{k}$ is a radical ideal. ◇ Definition: Radical ideal An ideal $I\\subset R$ is radical if for any $a$ and integer $m$, $a^{m}\\in I\\Rightarrow a\\in I$. ◆ Proposition $I_{k}$ is a radical ideal. To prove this, a bit of machinery is needed. Feel free to skip this if you are content to take the above as fact. First, we use a lemma that gives an alternative characterization of radical ideals; second, I very cautiously introduce tensor product between two rings. I am by no means an expert on this, but for this proof we only need the most basic idea. ◆ Lemma Ideal $I\\subset R$ is radical if and only if quotient ring $R/I$ satisfies $a^{2}=0\\Rightarrow a=0$ for all its elements $a$ (this is called a reduced ring). A tensor product of two polynomial rings $R[x]$ and $R[y]$ over $R$ essentially combines them into a single ring that incorporates both $x$ and $y$: $R[x]\\otimes_{R}R[y]\\cong R[x,y]$. Moreover, if we have $f(x)\\in R[x],\\;g(y)\\in R[y]$, then $R[x]/\\langle f\\rangle \\otimes_{R}R[y]/\\langle g\\rangle \\cong R[x,y]/\\langle f,g\\rangle$. This can be thought of by considering the latter\u0026rsquo;s basis being composed of the product of the bases $\\set{1,x,...,x^{\\text{deg}(f)-1}}$ and $\\set{1,y,...,y^{\\text{deg}(g)-1}}$.\nProof of prop. Taking the above as fact, we write: $$\\frac{\\mathbb{C}[v_{1},...,v_{n}]}{\\langle v_{1}^{k}-1,...,v_{n}^{k}-1\\rangle}=\\bigotimes_{i\\in[n]}\\frac{\\mathbb{C}[v_{i}]}{\\langle v_{i}^{k}-1\\rangle},$$ where the tensor product is taken over $\\mathbb{C}$. Next we observe that $v^{k}-1=\\prod_{j=0}^{k}(v-\\zeta^{j})$ by factoring the roots in $\\mathbb{C}[v]$. Since this is a PID and for $j\\neq j'$ the polynomials $v-\\zeta^{j}$ and $v-\\zeta^{j'}$ are coprime, $\\langle v-\\zeta^{j}\\rangle$ are all pairwise comaximal, thus the Chinese remainder theorem applies: $$\\mathbb{C}[v]/\\langle v^{k}-1\\rangle\\cong \\prod_{j=0}^{k-1}\\mathbb{C}[v]/\\langle v-\\zeta^{j}\\rangle\\cong \\prod_{j=0}^{k-1}\\mathbb{C}=\\mathbb{C}^{k}$$ where the second isomorphism follows due to $v-\\zeta^{j}$ being linear (consider $f(v)\\mapsto f(\\zeta^{j})$). Now $\\mathbb{C}[V]/I_{k}=\\mathbb{C}^{k}\\otimes_\\mathbb{C} ...\\otimes_\\mathbb{C} \\mathbb{C}^{k}=\\mathbb{C}^{k^{n}}$; this can be thought of again as multiplying the $k$ bases $n$ times. Finally, take $a=(a_{1},...,a_{k^{n}}) \\in\\mathbb{C}^{k^{n}}$ and suppose $a^{2}=0=(0,...,0)$. Since each component $a_{i}^{2}=0$ in the field $\\mathbb{C}$, it must be that $a_{i}=0\\Rightarrow a=0$. Applying the lemma above, we get $I_{k}$ is radical.\nConclusion This approach constructs the set of all $k$-colorings of $G$ as an affine variety of a carefully chosen ideal, and then filters them based on inclusion of a graph polynomial, thus we get $\\chi(G)=\\min\\set{k\\mid f_{G}\\not\\in I_{k}}$. The computer scientist brain, at this point, is asking: how hard is it to check whether $f_{G}$ is contained in $I_{k}$? To do this, we have to compute the Gröbner basis of the ideal - a particularly nice generating set. This is, unfortunately (but predictably), exponentially hard. This technique does not really give us a more optimal algorithm for graph coloring, but it\u0026rsquo;s an interesting perspective on a problem that we usually think of as purely combinatorial.\nThe second construction # todo :D\n","date":"2026-04-22T00:00:00Z","permalink":"https://saan-volta.github.io/post/graph_coloring_in_ring_theory/","title":"Graph Coloring via Rings"},{"content":"Introduction This is a really clean result I quite like about bounding the deviation of the chromatic number of a random graph. It seems like something that would be incredibly difficult to even argue about, but it only takes a mildly clever random process construction combined with a generic deviation bound inequality, and the answer comes out in two lines.\nThe Tricks ◇ Definition: Doob martingale Let $X$ be a random variable with $E[X)]\u003c\\infty$ and let $(\\mathcal{F}_{n})$ be any filtration. Define $(Y_{n})$ by $Y_{n}=E[X|\\mathcal{F}_{n}]$. $(Y_{n})$ is a martingale: $$E[Y_{n+1}|\\mathcal{F}_{n}]=E[E[X|\\mathcal{F}_{n+1}]|\\mathcal{F}_{n}]\\overset{(*)}{=}E[X|\\mathcal{F}_{n}]=Y_{n}$$$(*)$ follows from projection rule, since $\\mathcal{F}_{n}\\subset \\mathcal{F}_{n+1}$. This is also called an exposure martingale, since the filtration \u0026ldquo;exposes\u0026rdquo; information about $X$ sequentially. In this particular case, we will examine two examples below. Let $G\\sim\\mathcal{G}(n,p)$ be an Erdos random graph, that is, a graph on $n$ vertices where each edge has independent probability of appearing $p$. ◇ Definition: Edge exposure martingale Let $m=\\binom{n}{2}$, and fix an arbitrary ordering of possible edges $e_{1},e_{2},...,e_{m}$. Define events $A_{i}=\\mathbb{1}_{\\set{e_{i}\\in G}}$ and let $f$ be a real-valued function defined on $n$-vertex graphs, then: $$X_{i}=E[f(G)\\mid \\sigma(A_{1},...,A_{i})]\\quad\\text{ for }i\\in[0,m]$$ is the edge exposure martingale.\nNote that at $i=0$, there is no information, $X_{0}=E[f(G)]$, and at $i=m$, every edge is revealed, $\\sigma(A_{1},...,A_{m})=\\sigma(G)$, so $X_{m}=f(G)$.\n◇ Definition: Vertex exposure martingale Fix an arbitrary ordering of vertices $v_{1},...,v_{n}$. Then for $i\\in[1,n]$ define $$X_{i}=E\\bigg[f(G)\\mid \\sigma(G[v_{1},...,v_{i}])\\bigg],$$ where $G[S]$ for $S\\subseteq V$ is the induced subgraph of $S$.\nAs above, we can see that $X_{0}=E[f(G)]$ and $X_{n}=f(G)$. Next, to bound the concentration of the chromatic number, we need, well, a concentration bound. ◆ Lemma: Azuma-Hoeffding inequality For martingale $(X_{t})$ with bounded increments $|X_{t+1}-X_{t}|\\leq c_{t}\\text{ a.s.}$, time $T$, and any $\\varepsilon\u003e0$: $$P\\bigg(|X_{T}-X_{0}|\u003e\\varepsilon\\bigg) \\leq 2\\exp\\left(\\frac{-2\\varepsilon^{2}}{\\sum_{t\\leq T}c_{t}^{2}}\\right).$$ Finally, we have all the tools in place. ◆ Theorem: Theorem 1 Let $G\\sim\\mathcal{G}(n,p)$. Then $$|E[\\chi(G)]-\\chi(G)|\\leq \\sqrt{\\frac{n}{2}\\ln n}\\quad \\text{a.a.s.}$$ Let $X_{i}$ be the vertex exposure martingale with $\\chi$ as our function. Check that $$|X_{i+1}-X_{i}|=E[\\chi(G)\\mid \\sigma(G[v_{1},.,,,v_{i},v_{i+1}])]-E[\\chi(G)\\mid\\sigma(G[v_{1},...,v_{i}])]\\leq 1$$ since the inclusion of a single vertex can change the chromatic number by at most $1$ (i.e., it will require a new color). Thus we get our bound, and all that\u0026rsquo;s left is to put Azuma at the wheel: $$P(|E[\\chi(G)]-\\chi(G)|\u003e\\varepsilon)=P(|X_{0}-X_{n}|\u003e\\varepsilon)\\leq 2\\exp\\left(\\frac{-2\\varepsilon^{2}}{n}\\right).$$ Using $\\varepsilon=\\sqrt{(n/2)\\ln n}$ gives us $P(\\cdot \u003e\\varepsilon)\\leq 2\\exp(-\\ln n)=2/n$, which means the statement holds asymptotically almost surely as $n\\rightarrow \\infty$. $\\square$\nConclusion In fact there are results which give a much stricter bound, but they\u0026rsquo;re not as clean as this. I first learned about this during HackMIT hackathon; when we were looking for an open classroom to stay in overnight, the one we wandered into had this theorem written on the chalkboard from, I\u0026rsquo;m assuming, an earlier lecture. So I guess this is now my good luck charm theorem.\n","date":"2026-03-14T00:00:00Z","permalink":"https://saan-volta.github.io/post/chrom_num_rand_graph/","title":"Chromatic Number of a Random Graph"},{"content":"1. The problem (and the solution) This is a puzzle on 3Blue1Brown\u0026rsquo;s talent page posted by Jane Street. It\u0026rsquo;s uncharacteristically easy for JS, being pretty much a textbook ergodic Markov chain problem. Essentially all it takes is to observe this is a symmetric random walk on a finite undirected aperiodic graph (the pentagons and hexagons are the vertices), to which we apply the standard formula for mean recurrence time $E[T_x]=2m/d(x)$, where $m$ is the number of edges.\nIf the problem was not so easy (directed or infinite graph, etc), we would have to build the transition matrix $M$ and solve for the stationary distribution (eigenvector) $\\pi M =\\pi$. This is also not hard, in principle, though we\u0026rsquo;d spent some time computing it if the number of vertices is large. This post is about a way to make it quite a bit simpler still by exploiting the structure of the problem (but really I wanted an excuse to apply group theory).\n2. And the other part What\u0026rsquo;s cool here is that the graph is not only undirected and unweighted, but also highly structurally symmetric. If we begin in the red pentagon in the picture and take two steps to the purple pentagon, it is the same as being in any of the 4 other colored pentagons that are 2 steps away from red.\nAll we need to notice here is that the football graph is \u0026ldquo;radial\u0026rdquo;, consisting of concentric layers around the origin. Here\u0026rsquo;s the trick: call $F=(V,E)$ the graph (for football) and let $\\text{Aut}(F)$ be the automorphism group of the graph. A graph automorphism is a function $f:V\\rightarrow V$ that preserves vertex adjacency, i.e. $\\{v_{1}v_{2}\\}\\in E\\Rightarrow \\{f(v_{1})f(v_{2})\\}\\in E$. In other words, such a function simply relabels the vertices, maintaining the structure of the graph.\n$\\text{Aut}(F)$ consists of all such functions, but for our purposes we only need some of them. We will take $G_{0}=\\text{stab(0)}\\leq \\text{Aut}(F)$ to be the stabilizer subgroup of $0$ (the origin pentagon vertex). What this means is any $f\\in G_{0}$ is an automorphism of the graph that fixes the origin in place. The desired consequence of this is that it also fixes the relative distances of all vertices to the origin, since the adjacencies are preserved.\nFor instance, consider the hexagon vertex between the red, blue, and green pentagon vertices on the image above; call this point $w$. An $f\\in G_{0}$ must fix the origin, so $f(\\text{red})=\\text{red}$, but the preservation of adjacencies means that $f(w)$ is linked to $\\text{red}$ under the automorphism, so $w$ must be mapped to one of the $5$ neighboring vertices around $\\text{red}$. In other words, under such a relabelling, $w$ remains in the same layer.\nWith this, we consider the orbits of the action of $G_{0}$ on $F$: two vertices $v_{1},v_{2}$ are in the same orbit if $\\exists f\\in G_{0}\\text{ s.t. }f(v_{1})=v_{2}$. As we see in the example above, the group action cleanly partitions the vertices of the graph into these orbits based on the \u0026ldquo;layer\u0026rdquo; they\u0026rsquo;re in away from the origin. It\u0026rsquo;s kinda cute that in this graph, due to the radial structure, the group action orbits are literal orbits around the origin vertex.\nWe can finally collapse the graph by factoring out the symmetries and obtain the resulting Markov chain $F/G_{0}$. What we have done, essentially, is reduce the chain on the original graph (the surfaces of the football) to a chain on orbits under radial symmetry.\nThere are in total 8 orbits, with each having 5 vertices, except for 0th and 7th, which have only 1.\n3. Conclusion This reduces our 32x32 transition matrix to an 8x8 one basically for free! The $F/G_{0}$ chain gives us an easy eigenvector calculation (the answer to the puzzle is $E_{0}[T_{0}]=36$ btw). Of course, the initial problem has a small enough state space that it is not difficult at all to solve it directly (we only save on time inputting the matrix), but it is clear that on a large Markov chain a symmetry-exploiting simplification like this can reduce the required compute by orders of magnitude.\n","date":"2026-01-31T00:00:00Z","permalink":"https://saan-volta.github.io/post/andy_the_ant_and_markov_chain_collapse/","title":"Andy the Ant and Markov Chain Collapse"},{"content":"0. What is this? This is intended to be an in-detail walkthrough of how measure-theoretic probability works under the hood. When I took my first course on this topic, I found that there was not a lot of material that actually connected the measure theory \u0026ldquo;machine code\u0026rdquo; to probability problems. I\u0026rsquo;ve put together some explanations and examples that would\u0026rsquo;ve been useful to me when I first learned about it. This assumes knowledge of (very) basic measure theory \u0026amp; probability.\n1. Here\u0026rsquo;s what you have to know ◇ Definition: Measure Space $(\\Omega,\\mathcal{F},\\mu)$ is a measure (probability) space, where:\n$\\Omega$ is a set (of outcomes); $\\mathcal{F}$ is a $\\sigma$-algebra on $\\Omega$, consisting of subsets of $\\Omega$ and satisfying the following: $\\varnothing,\\Omega\\in\\mathcal{F}$; $A\\in\\mathcal{F}\\Rightarrow A^{c}\\in\\mathcal{F}$ (stable under complement); $\\set{A_{n}}\\in\\mathcal{F}\\Rightarrow \\bigcup_{n}A_{n}\\in\\mathcal{F}$ (stable under countable union); In probability, this is the event space. $\\mu:\\mathcal{F}\\rightarrow [0,\\infty]$ is the measure function $\\mu$ satisfies $\\sigma$-additivity, i.e.: $\\mu(\\bigsqcup_{n}A_{n})=\\sum_{n}\\mu(A_{n})$; $\\mu(\\varnothing)=0$; In probability, the measure is written as $P$, has codomain $[0,1]$, and $P(\\Omega)=1$. Comments:\n$\\mathcal{F}$ is stable by (countable) intersection; $P(\\varnothing)=0$ and $P(\\Omega)=1$ mean that something must happen; The trivial (coarsest) $\\sigma$-algebra over $\\Omega$ is $\\set{\\Omega,\\varnothing}$, and the fullest (finest) is $\\mathcal{P}(\\Omega)$ (if $\\Omega$ is countable). ◇ Definition: Random variable A (real-valued) random variable in a probability space $(\\Omega,\\mathcal{F},P)$ is a function $X:\\Omega\\rightarrow \\mathbb{R}$ that is measurable, i.e., satisfies: $$\\forall B\\in\\mathcal{B}(\\mathbb{R})\\quad X^{-1}(B)\\in\\mathcal{F}.$$ where $\\mathcal{B}(\\mathbb{R})$ is the Borel $\\sigma$-algebra of the real line.\nIf you have no measure theory background, $\\mathcal{B}(\\mathbb{R})$ might be hard to conceptualize. It is not the powerset of $\\mathbb{R}$, but it may be useful to think of it that way. The subsets of $\\mathbb{R}$ which do not fall in it require nontrivial and fairly contrived constructions that will never come up in a probability application.\nHere are some useful identities/definitions about expectation: ◇ Definition $E[X\\mid Y]=E[X\\mid\\sigma(Y)]$ (this is just a shorthand) $E[X]=E[E[X\\mid \\mathcal{A}]]$ (law of total expectation) $E[\\mathbb{1}_{A}X]=E[\\mathbb{1}_{A}E[X\\mid\\mathcal{A}]]\\quad \\forall A\\in\\mathcal{A}\\quad$ (def. of conditional expectation) $E[aX+bY]=aE[X]+bE[Y]$ (linearity of expectation) $E[X\\mid \\mathcal{A}]=X$ when $X$ is $\\mathcal{A}$-measurable and $E[X]$ when $X,\\mathcal{A}$ indep. $E[XY\\mid\\mathcal{A}]=XE[Y\\mid\\mathcal{A}]$ when $X$ is $\\mathcal{A}$-measurable 2. The conditioner Let $(\\Omega,\\mathcal{F},P)$ be a probability space and let $A\\in\\mathcal{F}$ be an event. In this segment, we will examine what it means to condition $P(A\\mid \\cdot)$ on a $\\sigma$-algebra.\n● Note I want to emphasize that when we write $P(A)$, what we mean mathematically is $E[\\mathbb{1}_{A}]=\\int_{\\Omega}\\mathbb{1}_{A}dP$, and more explicitly $\\int_{\\Omega}\\mathbb{1}_{A}(\\omega)dP(\\omega)$, where $\\mathbb{1}_{A}(\\cdot)$ is in fact a random variable. Let\u0026rsquo;s look at the easiest case: suppose $B\\in\\mathcal{F}$ is another event, then $P(A\\mid B)$ is just a number. Here, we simply condition on the realization that $B$ occurs.\nWhat if we don\u0026rsquo;t know whether it does? In such a case, we would condition on $\\sigma(B)$, the $\\sigma$-algebra generated by this event. The interesting thing is that $P(A\\mid\\sigma(B))$ is in fact a random variable (a measurable function of $\\omega$). Intuitively, $\\sigma(B)$ gives us some information about $A$, but the information is random and dependent on where $\\omega$ lands. Since $\\sigma(B)=\\set{\\varnothing,B,B^{c},\\Omega}$ has two nontrivial possibilities, the value of $P(A\\mid\\sigma(B))(\\omega)$ depends on whether $\\omega\\in B$ or $\\in B^{c}$: $$P(A\\mid\\sigma(B))=\\mathbb{1}_{B}P(A|B)+\\mathbb{1}_{B^{c}}P(A|B^{c}).$$ This works out nicely because $\\set{B,B^{c}}$ is a partition of the outcome space. We can easily extend this \u0026ndash; let $\\set{B_{n}}$ be a partition, i.e., a collection of disjoint subsets of $\\Omega$ that add up to the whole set. Then: $$P(A\\mid\\sigma\\set{B_{n}})=\\sum\\limits_{n}\\mathbb{1}_{B_{n}}P(A\\mid B_{n}).$$ We extend this further to compute the expectation of a general random variable: $$E[X\\mid\\sigma\\set{B_{n}}]=\\sum\\limits_{n}\\mathbb{1}_{B_{n}} E[X\\mid B_{n}]=\\sum\\limits_{n}\\mathbb{1}_{B_{n}} \\frac{1}{P(B_{n})}\\int_{B_{n}}XdP.$$3. Measurability Let $\\mathcal{H}\\subset \\mathcal{F}$ and $A\\in \\mathcal{F}$. The definition of measurable functions tells us that $A$ being $\\mathcal{H}$-measurable means that $\\forall B\\in\\mathcal{B}(\\mathbb{R}), \\mathbb{1}_{A}^{-1}(B)\\in \\mathcal{H}$. There are four cases, dictated by whether $0$ and $1$ are in $B$. If they both are or aren\u0026rsquo;t, we get that the preimage is respectively $\\Omega$ or $\\varnothing$, which is trivially true as $\\mathcal{H}$ contains them by definition. If only $1\\in B$, then the preimage is $\\set{\\omega\\in A}$, and if $0\\in B$ it is $\\set{\\omega\\notin A}$. Note that it is sufficient for one of these to be in $\\mathcal{H}$, since the other will be contained automatically as complement. Since $\\sigma(A)\\subseteq \\mathcal{H}$, $A$ is thus completely determinable. So we have: $$A\\in\\mathcal{H}\\Leftrightarrow \\mathbb{1}_{A}(\\cdot)\\; \\mathcal{H}\\text{-measurable }\\Leftrightarrow P(A\\mid\\mathcal{H})=E[\\mathbb{1}_{A}\\mid\\mathcal{H}]=\\mathbb{1}_{A}\\in \\set{0,1}\\text{ a.s. }$$Let\u0026rsquo;s now consider that $0\u003c P(A\\mid\\mathcal{H})\u003c1 \\text{ a.s.}$ This is a fairly strong statement about the relationship of $A$ and $\\mathcal{H}$; intuitively it means that the occurrence of $A$ cannot be deterministically established by any event in $\\mathcal{H}$: $$\\forall B\\in\\mathcal{H}\\text{ with }P(B)\u003e0, P(A\\cap B)\\neq 0,1.$$ Define $\\mathcal{H'}=\\mathcal{H}\\lor \\sigma(A)$, the $\\sigma$-algebra generated by including $A$ into $\\mathcal{H}$. What does a $A'\\in\\mathcal{H'}$ look like? Here, $\\set{A,A^{c}}$ partitions the outcomes in two, so $A'$ will consists of the union of some set $B\\in\\mathcal{H}$ intersecting with one part, and some other set $C$ intersecting with the other: $A'=(B\\cap A)\\cup (C\\cap A^{c})\\text{ with }B,C\\in\\mathcal{H}$. We can view the extension by $\\sigma(A)$ as adding one bit of information into the event space.\n![[Pasted image 20251225190318.png]] $A'$ aligns exactly with $B$ in $A$, and with $C$ in $A^{c}$.\nHere\u0026rsquo;s a result that simplifies the form for indeterminate events in $\\mathcal{H}'$: ◆ Proposition Suppose $P(A\\mid\\mathcal{H})\\in(0,1)\\text{ a.s.}$, $\\mathcal{H'}=\\mathcal{H}\\lor\\sigma(A)$, and $A'\\in\\mathcal{H'}$. Then: $$P(A'\\mid\\mathcal{H})\\in (0,1)\\text{ a.s.}\\iff \\exists B\\in\\mathcal{H}\\text{ s.t. }A'=(B\\cap A)\\cup (B^{c}\\cap A^{c}).$$ This gives us a characterization that an event in $\\mathcal{H'}$ has $0\u003c P(A'\\mid\\mathcal{H})\u003c1$, meaning that its occurrence can never be determined from an event in $\\mathcal{H}$, exactly IFF you can find $B$ and $C$ to represent $A'$ with the property $B^{c}=C$. Since $B\\mapsto (B\\cap A)\\cup (B^{c}\\cap A^{c})$ is a bijective map, there are exactly $|\\mathcal{H}|$ such indeterminate events.\n∎ Proof $(\\Rightarrow )$ Given $0\u003c P(A'\\mid\\mathcal{H})\u003c1\\text{ a.s.}$, take $B,C\\in\\mathcal{H}\\text{ s.t. }A'=(B\\cap A)\\cup (C\\cap A^{c})$. Then: $$\\begin{align*} P(A'\\mid\\mathcal{H}) \u0026= P((B\\cap A)\\sqcup (C\\cap A^{c})\\mid\\mathcal{H})\\\\ \u0026= E[\\mathbb{1}_{A}\\mathbb{1}_B\\mid\\mathcal{H}]+E[\\mathbb{1}_{C}\\mathbb{1}_{A^{c}}\\mid\\mathcal{H}]\\\\ \u0026= \\mathbb{1}_{B}P(A\\mid\\mathcal{H})+\\mathbb{1}_{C}P(A^{c}\\mid\\mathcal{H})\\\\ \u0026\\in (0,1). \\end{align*}$$ The $\u003e0$ implies that $B\\cup C=\\Omega$, and $\u003c1$ gives us $B\\cap C=\\varnothing$, thus they are complements. $(\\Leftarrow )$ is easy.\n4. Change of measure and independence Let\u0026rsquo;s examine another angle of the property $0\u003c P(A\\mid\\mathcal{H})\u003c1 \\text{ a.s.}$ Since $\\mathcal{H}$ is an arbitrary sub-$\\sigma$-algebra (not necessarily formed by a nice partition), we can\u0026rsquo;t really tell how the probability is changed by the conditioning.\n◇ Definition: Equivalence of measures Let $\\mu,\\nu:\\mathcal{F}\\rightarrow \\mathbb{R}_{\\geq0}$. They are considered equivalent if their null sets are the same, i.e., $\\forall F\\in\\mathcal{F},\\; \\mu(F)=0\\iff \\nu(F)=0.$ We can construct a measure $Q:\\mathcal{F}\\rightarrow [0,1]$ that is structurally equivalent to $P$, but under which $A$ is independent of $\\mathcal{H}$. For this, we will \u0026ldquo;re-weigh\u0026rdquo; the measures for events in a way that gives us $Q(A\\mid\\mathcal{H})=Q(A)=\\alpha$ for some $\\alpha\\in(0,1)$ of our choosing.\nConsider the function $$f_\\alpha:\\Omega\\rightarrow [0,1]\\text{ with }f_{\\alpha}= \\left(\\alpha \\frac{\\mathbb{1}_{A}}{P(A\\mid\\mathcal{H})} + \\left(1-\\alpha\\right)\\frac{\\mathbb{1}_{A^{c}}}{P(A^{c}\\mid\\mathcal{H})}\\right).$$ What does this function do? When $\\omega$ lands in $A$, we scale the it by the fixed constant $\\alpha$ and normalize by $P(A\\mid\\mathcal{H})$, which is just a \u0026ldquo;fixed\u0026rdquo; function; when it lands outside we likewise scale and normalize by the complements. With this $f$, we can construct our weighted measure of interest: $$\\forall B\\in\\mathcal{F},\\quad Q_\\alpha(B):=\\int_{B}fdP=E_{P}\\left[\\alpha \\frac{\\mathbb{1}_{A\\cap B}}{P(A\\mid\\mathcal{H})} + \\left(1-\\alpha\\right)\\frac{\\mathbb{1}_{A^{c}\\cap B}}{P(A^{c}\\mid\\mathcal{H})}\\right].$$ Equivalently, this means that $f=\\frac{dQ}{dP}$, the Radon-Nikodym derivative.\n◆ Proposition $Q_{\\alpha}$ is a probability measure. Proof. We show that $E_{P}[\\mathbb{1}_{A}/P(A\\mid\\mathcal{H})] =E_{P}[1/P(A\\mid\\mathcal{H})E_{P}[\\mathbb{1}_{A}]]=1$, and the second term is likewise $1-\\alpha$. Thus $Q_{\\alpha}(\\Omega)=\\alpha+1-\\alpha=1$.\n◆ Proposition $Q_{\\alpha}$ is equivalent to $P$. Proof. $(\\Rightarrow )$ Let $N\\in \\mathcal{F}\\text{ with }Q_{\\alpha}(N)=0$. Both terms of $f$ are nonnegative, so by $0\u003c P(A\\mid\\mathcal{H})\u003c1$, each of them must be $0$. $P(A\\cap N)=0\\land P(A^{c}\\cap N)=0\\Rightarrow P(N)=0$. $(\\Leftarrow )$ is trivial.\n◆ Proposition $Q_{\\alpha}(A\\mid\\mathcal{H})=Q_{\\alpha}(A)$. Proof. First: $$Q_{\\alpha}(A)=\\alpha E\\left[\\frac{\\mathbb{1}_{A}}{P(A\\mid\\mathcal{H})}\\right]=\\alpha E\\left[ \\frac{1}{P(A\\mid\\mathcal{H})}E[\\mathbb{1}_{A}\\mid\\mathcal{H}] \\right]=\\alpha.$$ Second, let $B\\in\\mathcal{H}$. Observe that $E_{P}\\left[\\alpha \\frac{\\mathbb{1}_{A\\cap B}}{P(A\\mid\\mathcal{H})}\\right] = \\alpha E\\left[\\frac{\\mathbb{1}_{B}}{P(A\\mid\\mathcal{H})}E[P(A\\mid\\mathcal{H})] \\right]=\\alpha P(B)$ by law of total expectation; and the second term similarly evaluates to $(1-\\alpha)P(B)$. Thus $Q_{\\alpha}(B)=P(B)$. Third, $Q_{\\alpha}(A\\cap B)=\\alpha P(B)$, since the first term evaluates as above, and the second is $0$ since $A^{c}\\cap A\\cap B=\\varnothing$. Finally, $Q(A\\mid B)= Q(A\\cap B)/Q(B) =\\alpha P(B)/P(B)=\\alpha$ by definition.\nThis construction gives us the following:\n$Q(A)=Q(A\\mid B)=\\alpha\\in(0,1)\\;\\forall B\\in\\mathcal{H}$, so $A$ is independent of $\\mathcal{H}$ under $Q$; $Q(B)=P(B)\\;\\forall B\\in\\mathcal{H}$, so all events in $\\mathcal{H}$ are invariant under this scaling; All events that happen a.s. or a.s. never are maintained (equality of measures). ","date":"2025-12-30T00:00:00Z","permalink":"https://saan-volta.github.io/post/how_to_measure/","title":"How to Measure"},{"content":"Intro: This is a write-up of my solution to the Chess challenge from the HackMIT_2025 entry CTF contest. This challenge was the highlight of the contest for me, but not by design. The actual intended solution was fairly unremarkable and dull. What made this fun, however, was the fact that it did not work. I\u0026rsquo;m serious, the challenge was broken. It took them about a week to review the complaints and drop a patch.\nThe reason the organizers took so long to double-check this challenge and find the bug was because they saw that one person has solved it \u0026ndash; me, as it happens. The authors designed the solution to be a long clue chase that step by step led the players to decrypt the flag in the last part of the message, but an error made somewhere in the middle of the process made that flag unreachable. So, instead of following a trail of hints that led nowhere, I ended up cracking the encryption scheme itself and reconstructing the flag byte by byte. This solve won me first blood on a broken challenge, secured the leaderboard, and even got me a cheating accusation, which may have been worth more.\nDescription Solution: In this challenge, we are given a ciphertext, as well as a chessboard .png in mate position that contains a hidden chess game PGN. We will not be using that; rather, we observe that the last 64 bytes of the 383-byte text are unique and determined by the userID1. It is assumed that the userID is hashed to produce the flag, with the bytes of the output being i.i.d. and uniform. We create 30 GitHub accounts, log in through them, and collect the associated ciphertexts.\nSince only the last 64 bytes ever change, we predict it is a 1-to-1 cipher that individually maps each byte of the plaintext to a byte of the ciphertext. Then $\\exists K$ with $pt_{i}[j]\\oplus K[j] =ct_{i}[j]$ for all byte indices $j \\in [0,63]$ for all userIDs $i\\in [0,30]$.2 The challenge is reduced to solving a many-time pad.\nWe observe further that for any fixed $j\\in[63]$, for all $i_{1},i_{2}\\in [30]$, $$ct_{i_{1}}[j] \\oplus ct_{i_{2}}[j]=pt_{i_{1}}[j]\\oplus pt_{i_{2}}[j]\\in [\\text{0x00, 0x0f}] \\cup [\\text{0x50,0x5f}].$$This is the distribution of XORed bytes: Let $H=[\\text{0x00, 0x0f}]\\cup [\\text{0x50, 0x5f}]$ be a subgroup of $([\\text{0x00, 0xff}],\\oplus)$. Then for each fixed index $j$, $pt_{i}[j] \\in g_{j}H$, the coset associated with that index formed by \u0026ldquo;shifting\u0026rdquo; $H$. This limits the set of values from which $pt_{i}[j]$ bytes are drawn to 32. Next we identify the specific set, and show that it is the same for all $j$, and in fact smaller than the full coset.\nIn other challenges, the flag has length of 32 bytes, or 64 hex digits. We claim that in this chall, the generated flag is viewed as a string directly, and then encoded into bytes. As a result, each byte is the encoding of a hex digit, of which there are only 16. Consider the distribution of pairwise XORs of the byte encodings of hex digits:\nThis empirically shows that for all $j$, the set of potential bytes is the encodings of the hex digits. Let this set be called $S$.\nFor the last step, we construct a XOR-SAT problem, finding valid values of $K$ under the constraints: $$K[j]\\in \\bigcap^{n}_{i}\\{ct_{i}[j]\\oplus s|s\\in S\\}.$$ As more non-repeating ciphertexts are given, the intersection tightens. A solver like z3 is able to efficiently find valid keys, but how much data is required for strong constraints? We can estimate the number of ciphertexts needed to converge to a manually searchable keyspace.3\nI will drop indices to consider a single a.b.f byte of the key $k$. Let: $$ \\begin{align*} \u0026A_{n}=\\bigcap_{i}^{n}\\{ct_{i}\\oplus s | s \\in S\\} \\\\ \u0026B_{n}=\\{a\\oplus k | a\\in A_{n}\\}=\\bigcap_{i}^{n}\\{ pt_{i}\\oplus s|s\\in S \\}\\\\ \\end{align*} $$ Observe that, given the solvable key exists, $0\\in B_{n}$, since $pt_{i}\\in S$. We will have solved the problem when only $0$ remains in $B_{n}$, and all the other values have been eliminated. Define the random variable that gives us the smallest satisfying $n$: $$X=\\inf\\{n|B_{n}=\\{0\\} \\}$$We will estimate $P(X\u003en)$, the tail probability. Consider a fixed $y$. What is $P(y\\in B_{n})$, the probability that $y$ is not eliminated in $n$ ciphertexts? Since we assume the bytes of the plaintexts to be independent, then $P(y\\in B_{n})=P(y\\in B_{1})^{n}$. Observe following about the event $\\{y\\in B_{1} \\}$: $$ \\{y\\in B_{1}\\} \\iff \\{\\exists s \\in S, pt=y\\oplus s \\} \\iff \\{ pt\\in y\\oplus S= \\{y\\oplus s |s\\in S\\} \\}. $$ Recall that $pt\\in S$. Then: $P(pt\\in y\\oplus S)= \\frac{|S\\cap y\\oplus S|}{|S|}$. For convenience, define $f_{\\oplus}$, the \u0026ldquo;autocorrelation\u0026rdquo; function between $S$ and the XOR-shifted $S$ by $f_{\\oplus}(y)= |S\\cap y\\oplus S|$. Then for all $y$, $$P(y\\in B_{1})= \\frac{{f_{\\oplus}(y)}}{|S|}, \\text{ and } P(y\\in B_{n})= \\left( \\frac{{f_{\\oplus}(y)}}{|S|} \\right) ^{n}.$$Finally: $$ \\begin{align*} P(X\u003en) \u0026= P \\left( \\bigcup_{y\\neq 0} \\{y\\in B_{n}\\} \\right) \\quad \\text{some y is not eliminated}\\\\ \u0026\\leq \\sum\\limits_{y\\neq 0} P(y\\in B_{n}) \\qquad \\text{ by sigma-additivity}\\\\ \u0026= \\sum\\limits_{y\\neq0} \\left( \\frac{{f_{\\oplus}(y)}}{|S|} \\right)^{n} \\\\\u0026= \\frac{255}{|S|^{n}} \\sum\\limits_{y\\neq0}f_{\\oplus}(y)^{n} \\end{align*} $$ The outcome depends on the structure of $f_\\oplus$, which is in turn determined by the contents of S. A fun consequence of $S\\subset gH$ is that $f_{\\oplus}(y)\u003e0 \\iff y\\in H$. The proof is an exercise for the reader. 4 In practice, this means the distribution of $f_{\\oplus}$ is heavily skewed towards 0, giving us a relatively small value in the sum.\n$$ \\begin{align*} E[X]\u0026=\\sum\\limits_{n=0}^{\\infty} P(X\u003en)\\\\ \u0026= P(X\u003e0)+P(X\u003e1) + \\sum\\limits_{n=2}^{\\infty} P(X\u003en)\\\\ \u0026\\leq 2+ \\sum\\limits_{n=2}^{\\infty} \\sum\\limits_{y\\in H\\setminus0} (f_\\oplus(y)|S|^{-1})^{n}\\\\ \u0026= 2+\\sum\\limits_{y\\in H\\setminus0} \\sum\\limits_{n=2}^{\\infty} (f_\\oplus(y)|S|^{-1})^{n}\\\\ \u0026= 2+\\sum\\limits_{y\\in H\\setminus0} \\frac{(f_\\oplus(y)|S|^{-1})^{2}}{1-f_\\oplus(y)|S|^{-1}} \\qquad \\text{geometric series}\\\\ \u0026=2+ \\frac{31}{|S|^{2}} \\sum\\limits_{y\\in H\\setminus 0} \\frac{f_{\\oplus}(y)^{2}}{ 1-f_\\oplus(y)|S|^{-1}} \\\\ \u0026\\approx 44.57 \\quad \\text{ciphertexts needed to reduce keyspace to 1.} \\end{align*} $$I used 30 ciphertexts to solve for the key, with $P(X\u003e30)\\leq 0.0387$. This is the probability an a.b.f. key byte fails to converge to a single value. Since all bytes are independent, the number of bytes that fail to converge out of 64 follows a binomial distribution $\\text{Bin}(64, 0.0387)$, and has $E[\\cdot]\\approx 2.48$. In my actual case, 4 bytes failed to converge (with 2 candidate bytes each), giving 16 possible keys.\nFrom there it is trivial to compute the flag by taking XOR with the key.\nThe hint for this is in the webpage\u0026rsquo;s ciphertext retrieval endpoint.\u0026#160;\u0026#x21a9;\u0026#xfe0e;\nThis makes the assumption that the encryption scheme can be expressed as a XOR against some consistent implicit key. It would not be true in, for example, a permutation cipher. I suspected the moves in the chess game were used as the encryption key, which would satisfy this. The next observation effectively confirmed it, as the common key was being cancelled out in the XOR.\u0026#160;\u0026#x21a9;\u0026#xfe0e;\nI did the math post factum. At the time of solving, I simply generated new cts until it was few enough keys to brute force, but I was curious what the theoretical expected num of cts was, so I went back to solve for it afterwards.\u0026#160;\u0026#x21a9;\u0026#xfe0e;\nHow it feels to write this 🔥✍️\u0026#160;\u0026#x21a9;\u0026#xfe0e;\n","date":"2025-09-30T00:00:00Z","permalink":"https://saan-volta.github.io/post/hackmit_chess/","title":"HackMIT_2025 Chess"}]