NIAID-BRCs AI Codeathon 2.0 · September 16–18, 2026 · Argonne National Laboratory

Interview-Driven Workflow Construction & Repair

Autonomous Galaxy Workflow Synthesis for Emerging Pathogen Research

Transforming conversational research requests into peer-reviewed, reproducible Galaxy workflows.
Marius van den Beek (@mvdbeek) John Chilton (@jmchilton) Dave Rogers (@NoopDog) Dejian Zhao (@dzhaobio) Scott Cain (@scottcain)
Press SPACE or → to advance · O for overview · S for speaker notes
The Challenge 0:45 - 1:35

Unstructured Intent vs. Deterministic Execution

Input from the Researcher
Natural Language Request
"I have Illumina paired-end RNA-seq of A549 lung cells infected with Ebola virus vs mock infection across 24 and 48 hours. I need transcript quantification with Salmon and differential expression with DESeq2."
Conversational shorthand, omitted reference genomes, missing contrast designs, zero workflow syntax.
The Universal Hazard: In computational biology, an unverified workflow that runs but silently corrupts statistical outputs or discards candidate genes is dangerous. Pipelines require mechanical verification.
Why Unconstrained Generation Fails
  • Tool Version Drift: Models cannot derive exact software versions (Salmon 1.10.1, DESeq2 2.11.40) or dependency hashes from pretraining memory.
  • Broken Collection Wiring: Conflating individual files with paired-end sample cohorts (list:paired) aborts automated batch execution.
  • Silent Statistical Hazards: Setting invalid design formulas or blindly stripping headers discards significant differential expression results.
  • Zero Execution Feedback: Unvalidated scripts fail silently or crash without actionable diagnostics.
The Codeathon Commitment
Replace unconstrained prompt generation with deterministic compilation, live registry discovery, and automated execution oracles.
Foundations · The Foundry Pattern 1:35 - 2:25

The Foundry Pattern: Compiling Knowledge into Skills

Grounding in Certified Science: The Intergalactic Workflow Commission (IWC) curates over 130 peer-reviewed, production Galaxy workflows. The Foundry compiles agent skills against this certified baseline rather than generating workflows in a vacuum.
01 · Source of Truth
Knowledge Base
Markdown notes, controlled registries, typed frontmatter, and upstream tool specifications.
02 · Action Spec
The Mold
Typed manifest of references plus procedural skeleton. Defines load policies, placement, and oracles.
03 · Compiler
The Cast
Deterministic compiler. Resolves links, imports schemas, and generates frozen skills.
04 · Verified Output
Skill + Provenance
Frozen agent skill bundle accompanied by complete build log and verification hashes.
1. Upstream Authority: Ground against IWC's 130+ peer-reviewed workflows. 2. Deterministic Tools: Tools execute deterministic work; models translate. 3. Executable Rails: Static validation gates precede execution.
Pipeline Catalog 2:25 - 3:15

The Foundry Pipeline Ecosystem

The Foundry provides purpose-built pipelines across distinct scientific entry points:
Natural Language Intake
INTERVIEW → GALAXY
Converts conversational researcher interviews into concrete, validated gxformat2 workflows. Executed in Case Study 1.
Literature Extraction
PAPER → GALAXY
Extracts computational protocols from papers and XML supplements into executable workflows. Executed in Case Studies 2 and 3.
Cross-Language Porting
NEXTFLOW → GALAXY
Translates Nextflow DSL2 source trees and process definitions directly into native Galaxy workflows.
Standard Interchange
CWL → GALAXY
Maps Common Workflow Language workflows and CommandLineTools into structured Galaxy workflows.
Conversational Editing
UPDATE-INTERVIEW
Applies reviewable, step-anchored change-sets to existing Galaxy workflows through interactive interviews.
Quality & Standards
IWC MATURATION
Applies the 130-workflow IWC publication checklist, static linting, and automated test coverage.
Pipeline Journey · INTERVIEW → GALAXY 3:15 - 4:15

The 12-Phase Synthesis Journey

Select any phase below to inspect declared Molds, input requirements, output artifacts, and validation oracles:
Phase 1: Interview Normalization interview-to-freeform-summary
Inputs: Conversational text file (researcher prompt or Biostars question)
Outputs: freeform-summary.md
Oracle / Gate: Schema validation and entity extraction (identifies host, pathogen, assay, and experimental design)
Filters out conversational noise, resolves abbreviations, and normalizes intent into structured prose with organism and sequencing metadata.
Case Study 1 · INTERVIEW → GALAXY (Gemini 3.8 Flash) 4:15 - 5:20

Ebola Virus Host RNA-seq: Verified Execution

12 / 12
Phases Completed
100%
Planemo Pass Rate
5 / 5
Jobs Green in Docker
Live API
usegalaxy.org Pass
Original Intake Prompt (Gemini 3.8 Flash)
"I have Illumina paired-end RNA-seq of A549 lung cells infected with Ebola virus vs mock infection across 24 and 48 hours. I need transcript quantification with Salmon and differential expression with DESeq2."
Resolved Selections & Toolchain
• Inputs: reads_paired_collection (list:paired) + human GENCODE transcriptome + tx2gene.tabular.
• Design Matrix: 2×2×2 factorial matrix (WT/KO, Mock/Ebola, 24h/48h) in sample_metadata.tabular.
• Tool Pins: Salmon 1.10.1+galaxy5 (29e12b90949d) → DESeq2 2.11.40.8+galaxy3 (b060944b3989).
Autonomous Triage: An initial 2-sample test failed DESeq2 dispersion fitting. The triage Mold diagnosed the degrees-of-freedom deficit from stderr, generated a 4-sample fixture, set primary contrast to ~ Condition, and re-executed to a green pass.
# Planemo Execution Summary: ebola-rnaseq
$ planemo test --biocontainers galaxy-workflow.gxwf.yml
[galaxy.workflow.scheduling] Scheduling 6 workflow steps...
[galaxy.jobs.runners] Job 1 (salmon_quant): OK (exit 0)
[galaxy.jobs.runners] Job 2 (salmon_quant): OK (exit 0)
[galaxy.jobs.runners] Job 3 (salmon_quant): OK (exit 0)
[galaxy.jobs.runners] Job 4 (salmon_quant): OK (exit 0)
[galaxy.jobs.runners] Job 5 (deseq2): OK (exit 0)
Passed: 1, Failed: 0, Errors: 0 (100% pass)
Verdict: galaxy-workflow.gxwf.yml_0: passed
Case Study 2 · PAPER → GALAXY (Claude Opus) 5:20 - 6:25

Candida auris: 26-Step Autonomous Loop & Biological Validation

26
Loop Iterations
27
Concrete Steps
172
Test Assertions
Live API
usegalaxy.org Pass
Source Literature & Protocol Discovery (Claude Opus)
Source: Santana et al. 2023 (Science, PMID 37769084). Main text omitted computational methods.
Extracted Input: PMC JATS XML full text + 60-page PDF supplement (NIHMS2004453).
Pipeline Selection & Toolchain Pins
• Scope Choice: Selected Pipeline A (clinical RNA-seq) over Pipeline B (T-DNA insertion mapping).
• Sequencing: BioProject PRJNA904261 (NextSeq 2000 2×50bp, stranded total RNA Ribo-Zero Plus).
• Tool Pins: FastQC → Cutadapt (Phred 20) → RNA STAR → featureCounts (-s 2) → DESeq2 (2 contrasts against AR0382, no shrinkage).
Static Validation Oracle: 100% Green
gxwf validate: 20 validated / 7 skipped / 0 failed.
gxwf validate-tests: 172 assertions passed with zero structural errors.
Biological Ground Truth Cross-Check
Independent statistical testing verified that the subsetted fixture preserves the primary biological finding:
Contrastlog2FCpadj
tnSWI1 vs AR0382−6.932.1e-31
AR0387 vs AR0382−7.954.1e-18
SCF1 down-regulation is preserved across both contrasts with unambiguous statistical significance.
Case Study 3 · Cross-Model Comparison (Scott Cain) 6:25 - 7:25

Opus vs. Sonnet on the Same Paper

12 / 12
Phases Completed
9 vs 26
Loop Iterations
3 UDTs
User Tools Authored
Live API
usegalaxy.org Executed
Reproduction Setup (Claude 3.5 Sonnet)
Scott Cain re-executed the paper-to-galaxy pipeline against the same C. auris paper to test cross-model reproducibility. All 12 pipeline phases completed end-to-end.
Strategy Contrast: Tool Search vs. Tool Authoring
• Opus Strategy (26 loops): Exhaustively searched the Tool Shed, mapped conditional XML parameter trees, and pinned strict standard wrappers.
• Sonnet Strategy (9 loops): When wrappers were missing or complex, Sonnet authored 3 containerized User-Defined Tools (class: GalaxyUserTool) to bridge the steps.
• Server Deployment: Invoked live on usegalaxy.org via automated API scripts.
Upstream Galaxy Hardening Receipts
The Sonnet run recorded 25 feedback entries, uncovering tool ID version stripping in Galaxy export routines and dynamic tool UUID resolution gaps.
Comparison Takeaway
• Opus is the deep explorer: Spends iterations finding exact community wrappers and detecting statistical subtleties.
• Sonnet is the pragmatic builder: Compresses drafting into fewer iterations by authoring custom tools, pushing directly to live server APIs.
• Both models successfully synthesized functional workflows from the 60-page supplement.
Findings · Grounded in Execution 7:25 - 8:15

Four Empirical Edge Traps Discovered During Execution

Click each discovery below to inspect how unverified generation fails in practice:
1. The DESeq2 Header Trap (Biological Destruction)
Symptom: DESeq2 produces two tabular files: deseq_out (no header) and normalized_counts (has header).
Root Cause: The certified IWC exemplar rnaseq-de applies header_lines: '1' to its downstream Filter1 steps. However, that workflow explicitly prepends a synthetic header beforehand via tp_text_file_with_recurring_lines → tp_cat. Copying header_lines: '1' without the prepender silently drops row 1.
Scientific Impact: Row 1 of deseq_out is the most significantly dysregulated gene. In Santana et al. (Science 2023), row 1 is SCF1 itself. Naively copying the IWC pattern deletes the primary discovery of the paper from the final output table.
The Fix: Documented the invariant, set header_lines: '0' across all Filter1 steps, and preserved the rationale in galaxy-workflow-draft.gxwf.yml.
Upstream Contributions 8:15 - 9:05

Upstream Hardening Across Repositories

Over 30 upstream commits across 4 repositories from Wednesday to Friday:
foundry 7c0c715d Add conversion-run dashboard and the run record it reads (6,900+ LOC TS)
foundry 5aac370f Merge pull request #565: Introduce experimental Workflow Brief contract
foundry 157510fe Merge pull request #555: Reuse an already-pinned wrapper within same draft
foundry 7d704248 Merge pull request #557: Declare galaxy-tool-cache in advance-draft-step
foundry 12f246ac Enforce strict tool-state validation in draft loop
galaxy 50c5420f Add MCP discovery: setup page, welcome card, and permanent links (#23507)
galaxy f2d6d400 Fix Tool Shed searches repeating the last page (#23482)
galaxy 731d4e2e Require subworkflow completion before completing its parent
galaxy 5929be86 Add depot resolver for prebuilt Singularity images
galaxy 700e0146 Require top-level container field for user-defined tools (UDT)
tools 7f2ece95 galaxy-tool-util-ts: compact ToolFetchError on ParsedTool decode failure
Deliverables & Tooling 9:05 - 9:40

Deterministic Run Dashboards & Live Publication

New CLI Tool: foundry-build run-dashboard
Engineered during the hackathon (commit 7c0c715d, 6,900+ lines of TypeScript):
  • Parses foundry-run.yml records and verifies artifact checksums against cast metadata.
  • Renders self-contained, offline dashboard.html files without network dependencies.
  • Audits phase progression, carried requirement ledgers, and execution test outcomes.
Live Publication: Both case study dashboards, documentation, and complete artifact bundles are published at:
https://niaid-brc-codeathons.github.io/interview-driven-workflows/
Dashboard Architecture
  • Phase Progression: Status indicators for static validation and execution stages.
  • Artifact Lineage: Direct links to verified gxformat2 workflows and test specifications.
  • Planemo Execution Records: Stored stdout and stderr diagnostics.
  • Responsive Presentation: Operates offline and respects system dark mode.
Summary · NIAID-BRCs AI Codeathon 2.0 9:40 - 10:00

Principles for Autonomous Scientific Workflows

1. The Foundry Pattern
Maintain source knowledge in typed Knowledge Bases. Cast frozen skills with explicit provenance records. Eliminate unconstrained prompts.
2. Empirical Grounding
Query live registries via MCP. Apply IDF token weighting. Automatically request user clarification when input specifications conflict.
3. Deterministic Oracles
gxwf validate checks static structure; planemo test executes containers. Automated feedback repairs statistical failures.
Next Steps: Broadening BRC pathogen coverage, automated BioContainers testing in CI, and contributing verified workflows to the Intergalactic Workflow Commission (IWC).
Marius van den Beek (@mvdbeek) John Chilton (@jmchilton) Dave Rogers (@NoopDog) Dejian Zhao (@dzhaobio) Scott Cain (@scottcain)
Thank you! Questions?