The Compatibility Atlas:
A Convex‑Geometric Index for LoRA Libraries

Sushaan Kandukoori¹, Aarit Atreja², Devom Brahmbhatt³, Avaneesh Parvathareddy⁴, Vladimir Filkov⁵

¹sushaankandukoori@gmail.com ²aaritatreja@gmail.com ³devom.hb@gmail.com
⁴avaneesh.parvathareddy@gmail.com ⁵vfilkov@ucdavis.edu

probe merges
\(N_d\) designs, once

fitted task models
\(\{\widehat q_t\}_{t=1}^{n}\)

convex minimax
\(\min_{c}\max_{t\in T}\widehat q_t(c)\)

verdict + surrogate certificate
\((c^\star,\lambda^\star,\text{cores})\)

Reusable atlas	Result
Mistral index: 16 adapters, 136 calibration merges	0.96 Pearson on 120 off-design triples
Mistral index: 50 adapters, 1,275 calibration merges	0.95 AUROC on 1,096 pair/triple queries
Local Qwen screen, 20% validation budget	45.5% recall, 77.6% precision, 73% compute saved

The atlas pipeline and its main results: probe merges are scored once per task, and every later subset query is a convex minimax over the fitted models, returning a surrogate-certified verdict with task and adapter cores of order at most \(d{+}1\).

Abstract

LoRA adapters merge into one model at low cost, but a fifty-adapter library already yields more than two million five-way subsets, too many for exhaustive held-out validation. We fix the base model, merge rule, allowed merge weights, evaluation prompts, and the tokens on which the loss is scored, then fit a quadratic model of each task’s loss in the merge weights. Each query names adapters to combine and tasks to retain; a small convex optimization over the fitted models returns the verdict and a surrogate certificate for it, with no further evaluation of the merged model; certifying the deployed loss itself additionally requires uniform approximation envelopes, which we measure on finite designs rather than establish over the whole chart. The per-library record of these verdicts is the Compatibility Atlas. By Helly’s theorem, any rejection of the fitted models is forced by at most \(d{+}1\) tasks; by Carathéodory’s theorem, an accepted merge places its free weight on at most \(d{+}1\) adapters, where \(d\) is the number of free merge weights; and for \(d\ge2\), checking every pair at a fixed threshold is not enough to decide every triple. One fit on sixteen adapters, using the loss on answer tokens, predicts the measured loss of 120 off-design three-adapter merges at Pearson correlation 0.96 (bootstrap interval \([0.75,0.98]\)); under linear merging, one fit on fifty screens 1,096 pair and triple merges at AUROC 0.95. On a separate library of fifty adapters and four merge methods, the locally fitted score recovers 45.5% of the merges that pass held-out retention validation at 78% precision, using 27% of the exhaustive validation cost.

The method

Compatibility Atlas. Given calibration losses for a small design of merge coefficients, fit one convex quadratic \(\widehat q_t(c)\) per task, once. A query \((T,U,\ell)\) names the tasks to retain, the adapters allowed into the merge, and a per-adapter weight floor; answer it by solving

\[\min_{c\in C(U,\ell)}\ \max_{t\in T}\ \widehat q_t(c).\]

Return merge weights \(c^\star\), a surrogate accept/reject certificate, a small blocking task core if rejected, and a sparse adapter core if accepted. No merged model is evaluated at query time.

Given a library of LoRA adapters trained on the same base, exhaustively validating every candidate merge on held-out data is expensive, since a fifty-adapter library already has \(\binom{50}{5}=2{,}118{,}760\) five-way candidates. Our running example is a Qwen2.5-0.5B-Instruct base with a Hugging Face PEFT library of fifty LoRA adapters spanning instruction following, code generation, GSM8K-style math, SQL writing, summarization, refusal training, and multi-turn chat. Many merge operators exist, among them weight averaging, task arithmetic, TIES (trim, elect sign, and merge), and variants of DARE (drop and rescale), and all of them assume the subset has already been chosen. We ask which subsets are worth validating.

A concrete query from this library asks whether the math, SQL, and summarization adapters can be merged, each with weight at least 0.2, so that the math and SQL tasks keep their single-adapter performance.

Every certificate in this paper is a surrogate certificate, a guarantee about the fitted convex models; upgrading it to a deployed-loss certificate, a guarantee about the model that is actually served, requires uniform approximation envelopes over the query chart, which we measure on finite designs but do not establish uniformly. The per-library record of verdicts is the Compatibility Atlas.

Our contributions:

A surrogate-certified convex screen. Sion duality gives every verdict a checkable primal–dual certificate, and supplying approximation envelopes upgrades it to a sound accept/reject/inconclusive test for the deployed loss.
Two-sided finite cores. Rejection is forced by at most \(d{+}1\) protected tasks (Helly) and an accepted merge places its free weight on at most \(d{+}1\) adapters (Carathéodory), the two cores forming a discrete saddle; a shared rank \(r\) lowers both orders to \(r{+}1\).
Pairwise impossibility. For \(d\ge2\) no thresholded pairwise compatibility graph decides every triple; the disagreements occur in practice.
Evidence at library scale. We validate atlas indexes fitted once at sixteen to fifty adapters, a four-operator screening run, and deployed coefficient interventions.

Compatibility queries over tasks and adapters

A query fixes the adapters \(U\subseteq[m]\) allowed in the merge, the tasks \(T\subseteq[n]\) it must retain, and a coefficient floor \(\ell\in\mathbb{R}^m_{\ge0}\) supported on \(U\) with \(\|\ell\|_1\le1\); the free mass is \(s_\ell:=1-\|\ell\|_1\). The merge chart is the simplex on \(U\) raised above the floor, \(C(U,\ell)=\{\ell+s_\ell z:z\in\Delta_U\}\subseteq\mathbb{R}^m\), a compact convex chart of intrinsic affine dimension \(|U|{-}1\) when \(s_\ell>0\); when \(s_\ell=0\) it is the single point \(\ell\).

For a protected-task subset \(S\subseteq[n]\), define the minimax retention score \[\rho_S(C)=\min_{c\in C}\max_{t\in S}q_t(c).\] The subset is compatible at threshold \(\tau\) iff \(\rho_S(C)\le\tau\).

Theorem 1 (Primal-dual certificate). For compact convex \(C\) and continuous convex quadratic task models \(q_t\), \[\rho_S(C)=\max_{\lambda\in\Delta_S}\min_{c\in C}\sum_{t\in S}\lambda_tq_t(c),\] where \(\Delta_S\) is the probability simplex on \(S\) (Sion, 1958; Rockafellar, 1970). Thus any primal \(c\in C\) and dual \(\lambda\in\Delta_S\) give \(L(\lambda):=\min_{z\in C}\sum_t\lambda_tq_t(z)\ \le\ \rho_S(C)\ \le\ U(c):=\max_tq_t(c)\). For any threshold \(\tau\), \(U(c)\le\tau\) certifies compatibility at level \(\tau\), \(L(\lambda)>\tau\) certifies incompatibility at level \(\tau\), and \(U(c)-L(\lambda)\) certifies optimality of \(\rho_S(C)\).

Theorem 2 (Finite cores, one- and two-sided). Let \(C\subset\mathbb{R}^d\) be compact and convex. (i) Fixed chart. If \(S\) is incompatible at threshold \(r\), some \(T\subseteq S\) with \(|T|\le d{+}1\) is already incompatible at \(r\) (Helly, 1923); the full score is realized by a sparse core, \[\rho_S(C)=\max_{T\subseteq S,\ |T|\le d+1}\rho_T(C),\] and the order \(d{+}1\) is tight. Suppose further that \(q_t(c)=\tilde q_t(Rc)\) with \(\tilde q_t\) convex, and consider the two-sided score \(\rho(T;U,\ell)\). (ii) Task core. Applying Helly in the latent image gives \(A^\star\subseteq T\), \(|A^\star|\le k(C(U,\ell)){+}1\), with \(\rho(A^\star;U,\ell)=\rho(T;U,\ell)\); this needs only a compact convex chart, so it holds for a router or box as well. (iii) Adapter core. On the chart \(C(U,\ell)\) with \(s_\ell>0\), some minimizer \(c^\star\) has residual support \(V^\star=\{i:c^\star_i>\ell_i\}\) of size at most \(k_U{+}1\), and restricting the free mass to \(V^\star\) preserves the score; at zero floor the merge therefore uses at most \(k_U{+}1\) adapters. (iv) Discrete saddle. For \(s_\ell>0\), over nonempty \(A\subseteq T\) and \(V\subseteq U\) of size at most \(k_U{+}1\), \(\max_A\min_V\rho(A;V,\ell)=\min_V\max_A\rho(A;V,\ell)=\rho(T;U,\ell)\). Since \(k_U\le d\) every core has order at most \(d{+}1\).

Theorem 3 (No pairwise surrogate on a fixed chart). There is no universally exact representation of multiway adapter compatibility by a pairwise mergeability graph. For \(d\ge2\), two libraries can have identical pairwise mergeability graphs on the same adapters but different three-way compatibility. Hence clique completion of a pairwise graph is incomplete for local LoRA-library merging.

protected tasks \(S\): task 1 task 2 task 3 threshold \(\tau\): 0.40

The minimax program on the symmetric tight instance (\(r=2\): three tasks, adapter atoms at the vertices of a regular simplex). Drag the primal point \(c\); the open marker is the minimizer \(c^\star\), and the dual \(\lambda^\star\) is read from its active tasks. \(U(c)\le\tau\) certifies compatibility at level \(\tau\), \(L(\lambda)>\tau\) certifies incompatibility at level \(\tau\), and \(U(c)-L(\lambda)\) certifies optimality. Both orders are simultaneously tight: a symmetric instance with \(m=n=r{+}1\) and adapter atoms at the vertices of a regular \(r\)-simplex forces each core to use all \(r{+}1\) indices.

Effective dimension and the atlas as an index

If the task models share an \(r\)-dimensional representation, both core bounds improve from \(d{+}1\) to \(r{+}1\), and the order is tight; the minimal such \(r\) is exactly the rank \(r_{\mathrm{exact}}(C)\) of an intrinsic curvature matrix computed in the chart’s own coordinates, and a factorization holding only within a uniform error \(\delta\) moves every score by at most \(2\delta\). As a computable diagnostic, the mean-Hessian projector residual certifies an upper bound on the spectral effective dimension \(\sigma\text{-}\mathrm{adim}_\epsilon\), the smallest rank at which any projector’s strict per-task residual falls below \(\epsilon\).

On a shared \(m\)-adapter simplex with design step \(\tfrac12\), the poised quadratic design (poised: the losses measured at its points determine the quadratic’s coefficients uniquely) consists of all single adapters and all equal-weight pairs; this is Scheffé’s quadratic simplex-lattice design \(\{m,2\}\), the classical saturated design for quadratic response surfaces on a mixture simplex. At \(d=m{-}1\) this gives \(N_d=m+\binom{m}{2}\) design merges, so building the fifty-adapter atlas takes one pass over 50 singles and 1,225 equal-weight pairs. Every later query is a convex minimax over the fitted quadratics; on a laptop CPU core the released solver answers one in 13 to 46 milliseconds on the \(m{=}8\) charts and at a median of 0.46 seconds for a five-way query at \(d{=}49\).

Eigenvalue spectrum of the mean Hessian on two five-adapter sub-libraries, log scale. — Eigenvalue spectrum of the mean Hessian \(\bar H\) on the two \(m{=}5\), \(d{=}4\) sub-libraries (log scale). TinyLlama (chat and creative adapters) concentrates 82.8% of \(\mathrm{tr}\,\bar H\) in the top eigenvalue and certifies \(\sigma\text{-}\mathrm{adim}_{0.20}\le 2\); the Qwen five-adapter sample concentrates 70.4% and 91.3% in the top one and two eigenvalues, certifying \(\sigma\text{-}\mathrm{adim}_{0.10}\le 3\) and \(\sigma\text{-}\mathrm{adim}_{0.20}\le 2\).

Experiments

Exhaustive validation is expensive at \(m{=}16\): the \(\binom{16}{3}{=}560\) triples cost hours to validate while one 136-point build scores them all and every higher order, with fidelity 0.96 on 120 independently evaluated off-design triples and screening AUROC 0.99.

Three panels: predicted versus measured answer loss on off-design merges, worst-member accuracy change at the atlas coefficient, and off-design fidelity against design redundancy. — **The index fitted once on the answer loss.** (a) Predicted versus measured answer loss on off-design merges: one 136-point build at \(m{=}16\) predicts all 120 off-design triples (triangles) at Pearson 0.96, pairs on the design (circles) at 0.98. (b) Worst-member accuracy at the atlas coefficient minus equal weights: on three of the four triples where equal weights leave a member below 0.6 accuracy the coefficient improves that member (+0.07 mean), and it changes little elsewhere. (c) Off-design fidelity rises with design redundancy.

The same construction extends to thirty and fifty adapters on the same Mistral-7B base. Each build fits the singles-and-pairs design once (465 and 1,275 design merges) and answers every subset query on its faces; validation covers every pair plus sampled triples, 443 and 1,096 subsets with every member scored. At the 70% retention threshold the screen gives AUROC 0.94 at \(m{=}30\) and 0.95 at \(m{=}50\) (bootstrap intervals \([0.87,0.99]\) and \([0.89,0.98]\); triples alone 0.93 and 0.94), and pooled fidelity on off-design triples is 0.86 and 0.85.

The released public panel totals 7,784 evaluated pair/triple records: Qwen2.5 \(m{=}50\) balanced/circulant slices plus full pair/triple enumeration for TinyLlama \(m{=}12\) and Mistral, Phi-2, Pythia-410M, and Pythia-1B \(m{=}10\) panels.

merge operator:

passes held-out validation fails held-out validation

Screening on the Qwen \(m{=}50\) panel: each point is one of the 1,000 pair or triple queries evaluated per operator, placed by its fitted score and its held-out score, both relative to the retention rule. On the Qwen \(m{=}50\) run, whose calibration prompts are generic instructions, the atlas AUROC is linear 0.89, magnitude pruning 0.85, TIES 0.79, and DARE-TIES 0.76. Feasibility labels are finite-design labels: a positive query has at least one evaluated coefficient that passes the retention rule on held-out prompts, and a negative label does not rule out an unevaluated coefficient elsewhere on the chart.

Four panels: atlas AUROC per operator with bootstrap ranges, AUPRC against the feasible base rate, recall across validation budgets, and the atlas against low-cost baselines on triples. — The Qwen \(m{=}50\) panel (4,000 pair and triple events, four predeclared slices). (a) Atlas AUROC per operator with cluster-bootstrap stability ranges. (b) AUPRC per operator against the feasible base rate. (c) Recall of top-\(k\) screening across validation budgets. (d) The atlas versus the low-cost baselines on triples.

At a 20% validation budget (full validation \(\approx 33\) MI300X-hours), validating the atlas’s top-\(k\) recovers 45.5% of the subsets that pass held-out validation at 77.6% precision and saves 73% of the GPU-hours including the \(\sim\)2 s per-query screening cost (adapter-resampling bootstrap: recall \([34.2,59.6]\%\), precision \([60.8,91.3]\%\)).

Observed feasibility rate within certificate-margin buckets per panel, with Wilson intervals; the curves increase with margin. — Observed feasibility rate within six certificate-margin buckets per panel, for linear merges, with 95% Wilson intervals and bucket medians on the horizontal axis. The broadly increasing curves show that a larger certificate margin predicts a higher held-out feasibility rate; buckets hold 166 to 167 events on Qwen and 27 to 48 on the smaller panels.

Atlas screening outperforms the label-free low-cost screens on Qwen, at per-operator AUROC 0.76 to 0.89 versus 0.64 to 0.73 for a single-adapter calibration-gain floor and 0.63 to 0.81 for leave-one-slice-out logistic regression, and cosine baselines on raw \((B,A)\) and on \(\Delta W{=}BA\) trail it by at least 25 AUROC points.

Limitations

Every certificate we report is a surrogate certificate: exact for the fitted task models, and silent about the deployed loss until envelopes are supplied. Decisions about the deployed loss additionally require uniform approximation envelopes over the continuous chart; these remain unestimated. Finite-design versions are evaluated on six dense charts, while the larger panels’ certificates remain statements about the fitted models, and the deployed tests of coefficient selection cover sixteen queries in total.

Code and data

The screening runner, the analysis scripts, and every released result are at github.com/sushaan-k/lora-compatibility-atlas. Running verify.py there recomputes the paper’s headline numbers from the released records and checks them against the values printed here.

Citation

@misc{kandukoori2026compatibilityatlas,
  title  = {The Compatibility Atlas: A Convex-Geometric
            Index for {LoRA} Libraries},
  author = {Kandukoori, Sushaan and Atreja, Aarit and
            Brahmbhatt, Devom and Parvathareddy, Avaneesh
            and Filkov, Vladimir},
  year   = {2026},
  url    = {https://github.com/sushaan-k/lora-compatibility-atlas}
}