API reference

Campaign

Campaign: the closed loop.

Campaign is DENIA’s user-facing object. It follows the ask/tell pattern of sequential experimental design:

1. Campaign.from_file(...) - agentic (or heuristic) ingestion of the spreadsheet as-is.

2. recommend(candidates, batch_size) - rank a candidate pool and return the next batch of experiments with predictions, uncertainties, and acquisition scores appended.

3. tell(new_results) - append measured results; the surrogate is refit on the enlarged history at the next recommendation.

The candidate pool can be an explicit dataframe (rows = runnable experiments), or generated by perturbing/recombining the observed design space.

Schema and ingestion

Campaign schema: a machine-actionable description of an experimental table.

The schema is the contract between the agentic ingestion layer and the classical active-learning core. Whether it is produced by an LLM agent (denia.agents.SchemaAgent) or by heuristic inference (denia.ingest.infer_schema()), downstream components only ever see this structure.

class denia.schema.ColumnRole(value)

Role of a column in an experimental campaign.

class denia.schema.ColumnSpec(name: str, role: ColumnRole, unit: str | None = None, partner: str | None = None, maximize: bool = True, notes: str = '')

Specification of a single column.

class denia.schema.CampaignSchema(columns: List[ColumnSpec] = <factory>, provenance: str = 'heuristic')

Full schema of an experimental campaign table.

composition_pairs() → List[tuple]

(category_column, fraction_column) pairs, e.g. (‘Cation 1’, ‘Cation 1 mol%’).

Ingestion: load a spreadsheet as-is and infer a campaign schema.

DENIA’s first design commitment is that the user’s file is never reformatted. The table is loaded in its native shape and a CampaignSchema is inferred on top of it - by an LLM agent when available (denia.agents.SchemaAgent), or by the deterministic heuristics below, which also serve as the agent’s verifiable baseline.

denia.ingest.load_table(path: str | Path, sheet: str | None = None) → DataFrame

Load an .xlsx/.csv experimental table without altering its structure.

For Excel files with several sheets, the largest sheet is assumed to be the data sheet unless sheet is given (reference lists and column descriptions typically live in the smaller sheets).

denia.ingest.infer_schema(df: DataFrame, target: str | List[str] | None = None) → CampaignSchema

Deterministic, dependency-free schema inference.

Heuristics encoded here:

• columns whose name matches metadata patterns (ids, references) -> METADATA

• user-specified target (or yield-like names) -> TARGET

• conversion/selectivity-like names -> AUXILIARY

• object-dtype columns -> CATEGORY_INPUT

• numeric columns named like "<Category> mol%" and sharing a prefix with a category column -> FRACTION_INPUT paired to it

• remaining numeric columns -> NUMERIC_INPUT

Agents

Agentic layer: LLM agents at the edges of the discovery loop.

DENIA’s architecture puts agents where language understanding adds value and keeps the experiment-selection core classical and reproducible:

• SchemaAgent - reads arbitrary column headers/sample rows and returns a machine-actionable CampaignSchema (roles, units, category/fraction pairings, optimization direction).

• ReporterAgent - turns a recommendation batch into a natural-language briefing an experimentalist can act on.

All agents degrade gracefully: without the anthropic package or an ANTHROPIC_API_KEY, DENIA falls back to deterministic heuristics (denia.ingest.infer_schema()) and template-based reports, so the full loop runs offline. Agent output is always validated against the dataframe before use; the LLM proposes, the code verifies.

class denia.agents.SchemaAgent(model: str = 'claude-sonnet-4-6')

Infer a campaign schema from raw headers and sample rows via an LLM.

The agent receives only column names, dtypes, and a few sample values - never the full dataset - and must return strict JSON mapping each column to a role. The result is validated column-by-column against the dataframe; any invalid assignment falls back to the heuristic inference for that column. This “propose-verify” pattern is what makes an LLM safe to place inside a scientific pipeline.

class denia.agents.ReporterAgent(model: str = 'claude-sonnet-4-6')

Produce a natural-language briefing for a recommendation batch.

Featurization

Featurization: from a raw experimental table to a model-ready matrix.

Driven entirely by the CampaignSchema:

• (category, fraction) pairs - e.g. ('Cation 1', 'Cation 1 mol%') - are merged into a sparse composition vector: one feature per distinct species, valued by its fraction, summed across slots. This makes ‘Mn 9.2 % in slot 1’ and ‘Mn 9.2 % in slot 3’ identical, as chemistry demands.

• unpaired categorical inputs (e.g. preparation method) are one-hot encoded.

• numeric inputs are median-imputed and standardized.

The featurizer is deliberately transparent (feature names are preserved) so recommendations remain explainable back to the original spreadsheet columns.

Models and acquisition

Surrogates and acquisition functions - the classical, reproducible core.

Two surrogates cover the practical range of tabular campaigns:

• EnsembleSurrogate (default) - random-forest ensemble; uncertainty from across-tree dispersion. Robust to mixed/sparse features and scales to thousands of rows (the regime of literature-compiled datasets like OCM).

• GPSurrogate - Gaussian process (Matérn 5/2); preferable for small in-house campaigns (tens to a few hundred rows).

Acquisition: expected improvement (EI) and upper confidence bound (UCB), plus greedy batch selection with a diversity penalty so a recommended batch does not collapse onto near-duplicates of the same candidate.

class denia.models.EnsembleSurrogate(n_estimators: int = 200, random_state: int = 0)

Random-forest surrogate with across-tree predictive dispersion.

class denia.models.GPSurrogate(random_state: int = 0)

Gaussian-process surrogate (Matérn 5/2 + noise) for small campaigns.

denia.models.select_batch(scores: ndarray, X: ndarray, batch_size: int, diversity: float = 0.3) → ndarray

Greedy batch selection with a squared-exponential diversity penalty.

After each pick, candidate scores are down-weighted by proximity to the already-selected points (feature-space distance), preventing batches of near-duplicates - the practical failure mode of naive top-k selection on literature datasets full of replicated conditions.

Objectives, generation, persistence

Multi-objective optimization: Pareto fronts and scalarized acquisition.

Real campaigns rarely optimize one number: yield vs selectivity, performance vs cost, strength vs weight. DENIA handles multiple targets with per-target surrogates and ParEGO-style random Chebyshev scalarization of per-target expected improvements, so each slot of a recommended batch probes a different trade-off region of the Pareto front.

denia.objectives.pareto_mask(Y: ndarray, maximize: Sequence[bool]) → ndarray

Boolean mask of non-dominated rows of Y (n_points x n_objectives).

denia.objectives.chebyshev_scalarize(scores: ndarray, weights: ndarray, rho: float = 0.05) → ndarray

Augmented Chebyshev scalarization of per-objective scores (higher=better).

scores: (n_candidates, n_objectives), already normalized to comparable scale.

denia.objectives.normalize_scores(scores: ndarray) → ndarray

Min-max normalize each objective’s score column to [0, 1].

denia.objectives.pareto_front(history: DataFrame, targets: List[str], maximize: Sequence[bool]) → DataFrame

Non-dominated subset of the campaign history over the given targets.

Constraints and candidate generation.

A recommendation engine is only useful if it can propose experiments a lab can actually run. CandidateGenerator builds a candidate pool directly from the campaign’s observed design space - recombining categorical choices and perturbing/resampling numeric conditions - subject to practical constraints:

• numeric bounds per column,

• allowed/forbidden category values,

• fixed columns (e.g. your reactor’s fixed pressure),

• mixture normalization: composition fractions rescaled to a total (e.g. mol% summing to 100).

This turns DENIA from “rank my list” into “suggest what to try next” without requiring the user to enumerate a pool.

class denia.generate.Constraints(bounds: 'Dict[str, Tuple[float, float]]'=<factory>, allowed: 'Dict[str, Sequence]'=<factory>, forbidden: 'Dict[str, Sequence]'=<factory>, fixed: 'Dict[str, object]'=<factory>, mixture_total: 'Optional[float]' = None)

class denia.generate.CandidateGenerator(schema: CampaignSchema, constraints: Constraints | None = None, random_state: int = 0)

Generate candidate experiments from the observed design space.

Strategy per candidate: pick a random history row as a template, then mutate it - resample categorical slots from observed values, jitter numeric conditions within observed (or constrained) ranges, and renormalize mixtures. Simple, transparent, and guaranteed to stay in a region the featurizer understands.

Campaign persistence: one portable file per campaign.

campaign.save("my_project.denia") writes a single zip archive containing the full history (CSV), the schema (JSON), and campaign metadata; Campaign.load("my_project.denia") restores it. This is what turns DENIA from a one-shot recommender into an ongoing assistant: recommend, run the experiments, tell the results, save - and pick the campaign up next week, next month, or on a colleague’s machine.

The format is deliberately transparent: unzip it and you get your data back as plain CSV and JSON. No lock-in, ever.

Insights

Insights: what drives the target?

drivers() answers the question every experimentalist asks right after “what should I run next?”: which variables actually control my outcome?

A random forest is fit on the campaign’s featurized history and its impurity importances are reported at two levels:

• feature level - individual species and settings (Cation::Mn, Preparation::Impregnation, Temperature, K), and

• column level - importances summed back to the original spreadsheet columns/groups, so the answer is phrased in the user’s own vocabulary.

These are descriptive, not causal: they say what the model relies on given the (possibly biased) history, which is exactly the right caveat to show a lab before they reorganize their program around it.

denia.insights.drivers(history: DataFrame, schema, target: str | None = None, n_estimators: int = 300, random_state: int = 0) → Tuple[DataFrame, DataFrame]

Return (feature_level, column_level) importance tables for a target.

Both tables have columns name and importance and sum to 1.

Benchmark

Retrospective active-learning benchmark.

The flagship validation for a next-experiment engine on literature data: hide the dataset, seed the campaign with a few random experiments, and let DENIA sequentially “run” experiments by revealing rows from the hidden pool. The figure of merit is the best target value found as a function of the number of experiments, compared against random selection - i.e., how much experimental budget the decision layer saves.

class denia.benchmark.RetrospectiveResult(n_experiments: 'np.ndarray', best_denia: 'np.ndarray', best_denia_std: 'np.ndarray', best_random: 'np.ndarray', best_random_std: 'np.ndarray', oracle: 'float', target: 'str', curves_denia: 'List[np.ndarray]' = <factory>, curves_random: 'List[np.ndarray]' = <factory>)

savings_at(fraction_of_oracle: float = 0.9) → dict

Experiments needed to reach a fraction of the oracle optimum.

denia.benchmark.plot_retrospective(result: RetrospectiveResult, path: str | None = None)

Discovery curve: best-found target vs experimental budget.