conversion run
paper-to-galaxy — 2026-09-16 16:09 to 2026-09-18 16:41, 61.8 MB on disk.
| phase | artifact | file | kind | presence | size | modified | copies |
|---|---|---|---|---|---|---|---|
| phase 1 | freeform-summary | freeform-summary.md | markdown | present | 15.1 KB | 2026-09-16 16:09 | — |
| phase 2 | freeform-galaxy-interface | freeform-galaxy-interface.md | markdown | present | 14.5 KB | 2026-09-16 16:15 | — |
| phase 2 | open-requirements-ledger | open-requirements.ledger.yml | yaml | present | 71.1 KB | 2026-09-17 21:06 | — |
| phase 3 | freeform-galaxy-data-flow | freeform-galaxy-data-flow.md | markdown | present | 28.9 KB | 2026-09-16 16:25 | — |
| phase 4 | iwc-comparison-notes | iwc-comparison-notes.md | markdown | present | 24.0 KB | 2026-09-16 16:38 | — |
| phase 4 | iwc-exemplar-gxformat2 | iwc-exemplar.gxwf.yml | yaml | present | 20.0 KB | 2026-09-16 16:36 | — |
| phase 5 | galaxy-workflow-draft | galaxy-workflow-draft.gxwf.yml | yaml | present | 103.9 KB | 2026-09-16 19:36 | — |
| phase 6 | galaxy-workflow | galaxy-workflow.gxwf.yml | yaml | present | 42.6 KB | 2026-09-17 14:19 | — |
| phase 7 | test-data-refs | test-data-refs.json | json | present | 29.2 KB | 2026-09-16 19:56 | — |
| phase 8 | galaxy-test-plan | galaxy-test-plan.yml | yaml | present | 71.6 KB | 2026-09-16 21:16 | — |
| phase 9 | galaxy-workflow-test | galaxy-workflow.gxwf-tests.yml | yaml | present | 32.3 KB | 2026-09-16 21:41 | — |
| phase 10 | galaxy-workflow-validation-result | galaxy-workflow-validation-result.json | json | present | 20.1 KB | 2026-09-17 22:31 | — |
| phase 11 | workflow-test-result | workflow-test-result.json | json | missing | — | — | — |
| phase 12 | workflow-debug-report | workflow-debug-report.md | markdown | not-yet-due | — | — | — |
| phase — | foundry-feedback-ledger | foundry-feedback.ledger.yml | yaml | present | 169.9 KB | 2026-09-17 21:05 | — |
| phase — | foundry-run-manifest | foundry-run.yml | yaml | optional-absent | — | — | — |
Runtime artifact initialized by the harness ([[foundry-feedback-ledger]]).
run:
pipeline: paper-to-galaxy
run_slug: auris-scf1
status: running
phases:
- { n: 1, kind: mold, skill: summarize-paper, status: done, feedback_checked: true }
- { n: 2, kind: mold, skill: freeform-summary-to-galaxy-interface, status: done, feedback_checked: true }
- { n: 3, kind: mold, skill: freeform-summary-to-galaxy-data-flow, status: done, feedback_checked: true }
- { n: 4, kind: mold, skill: compare-against-iwc-exemplar, status: done, feedback_checked: true }
- { n: 5, kind: mold, skill: freeform-summary-to-galaxy-template, status: done, feedback_checked: true }
- { n: 6, kind: mold, skill: advance-galaxy-draft-step, status: done, feedback_checked: true, iterations: 26 }
- { n: 7, kind: branch, pattern: test-data-resolution, status: done, feedback_checked: true, selected: paper-to-test-data }
- { n: 8, kind: mold, skill: freeform-summary-to-galaxy-test-plan, status: done, feedback_checked: true }
- { n: 9, kind: mold, skill: implement-galaxy-workflow-test, status: done, feedback_checked: true }
- { n: 10, kind: mold, skill: validate-galaxy-workflow, status: done, feedback_checked: true }
- { n: 11, kind: mold, skill: run-workflow-test, status: running }
- { n: 12, kind: mold, skill: debug-galaxy-workflow-output, status: pending }
entries:
- id: summarize-paper-declares-no-freeform-summary-shape
raised_by: summarize-paper
observed_in:
mold: summarize-paper
path: content/molds/summarize-paper/index.md
revision: 2
content_hash: ac601516a655a5a97bf9cc6d865839622efd509a295f99410c9e47e3952f269d
foundry_head: 63a3f9cf9c97a637fe1628d298e524f35709e289
subject:
kind: mold
label: summarize-paper
locator: content/molds/summarize-paper/index.md
content_hash: ac601516a655a5a97bf9cc6d865839622efd509a295f99410c9e47e3952f269d
kind: gap
severity: major
what: >-
The Mold produces the shared freeform-summary handoff but specifies no shape for it.
The procedure says only "a free-form Markdown summary capturing the workflow's steps,
tools, parameters, and sample/reference-data leads", the cast bundle packages no
template, example, or reference, and the runtime notes forbid reading Foundry source
at runtime. Four downstream Molds consume this artifact
(freeform-summary-to-galaxy-interface, -data-flow, -template, -test-plan), so the
section structure they can rely on was guessed at, not settled by the instructions.
expected: >-
Package a freeform-summary skeleton or worked example in the cast bundle, or state in
the procedure the minimum sections every consumer can assume (source identity,
candidate workflow scope, per-step tool/parameter table, reference data, sample data
accessions, assumptions, open questions). interview-to-freeform-summary emits the
same artifact id and should share whatever contract is written.
evidence: >-
Bundle carries SKILL.md plus _feedback/_provenance/_verify only; refs is empty and
"Load Upfront"/"Load On Demand" are both "None declared". Output section names the
filename and format but no structure.
status: open
issue: null
- id: summarize-paper-silent-on-supplementary-methods-retrieval
raised_by: summarize-paper
observed_in:
mold: summarize-paper
path: content/molds/summarize-paper/index.md
revision: 2
content_hash: ac601516a655a5a97bf9cc6d865839622efd509a295f99410c9e47e3952f269d
foundry_head: 63a3f9cf9c97a637fe1628d298e524f35709e289
subject:
kind: mold
label: summarize-paper
locator: content/molds/summarize-paper/index.md
content_hash: ac601516a655a5a97bf9cc6d865839622efd509a295f99410c9e47e3952f269d
kind: gap
severity: major
what: >-
The Mold's entire job is to extract methods from a paper, but it declares no required
tools and describes no procedure for actually obtaining the text. For this run the
article's main text contained no Methods section at all: every computational detail
lived only in a supplementary PDF. The publisher page returned HTTP 403, the article
is not in the Europe PMC open-access set, and a single fetch of the PMC article page
yielded a tool list with no parameters and no pipeline. The retrieval route that
worked was improvised and is not described anywhere in the bundle.
expected: >-
State in the procedure that Science/Nature-style papers carry their Methods in
supplementary materials and that the main text alone is not a sufficient source, and
package a reference covering the retrieval routes worth trying in order (NCBI eutils
efetch db=pmc for JATS full text, the supplementary-material file list inside that
XML, Europe PMC, preprint and institutional-repository copies), plus the instruction
to report inaccessible Methods as an open question rather than summarize around them.
evidence: >-
Main-text JATS body ended at a bare "Supplementary Material" label; the full methods,
tool parameters, and reference-assembly accession came only from the supplement PDF
listed in that XML. Artifact open question 10 records the same for the next reader.
status: open
issue: null
- id: testability-note-silent-on-sample-sheet-test-fixtures
raised_by: freeform-summary-to-galaxy-interface
observed_in:
mold: freeform-summary-to-galaxy-interface
path: content/molds/freeform-summary-to-galaxy-interface/index.md
revision: 3
content_hash: de0cb02532205ad18f4aa1ab2881cac725a3191b448468a8a11d0663ff1ec97d
foundry_head: 63a3f9cf9c97a637fe1628d298e524f35709e289
subject:
kind: research
label: galaxy-workflow-testability-design
locator: content/research/galaxy-workflow-testability-design/index.md
content_hash: 81b01801ec3d14810d570dd2d7815da2bb11e41a562af9182c29fd2974f4b855
kind: gap
severity: major
what: >-
Section 5 ("Design inputs with fixtures in mind") instructs the designer to match workflow
input collection types to realistic fixture shapes and to choose labels readable as test
`job:` keys, and its evidence covers `list`, `list:paired`, data, string, boolean and int
inputs. It says nothing about the `sample_sheet` family, even though the sibling note
galaxy-sample-sheet-collections is packaged in the same bundle and is what pushes the
designer toward that shape. Nothing in the bundle states whether a `sample_sheet:paired`
workflow input — element identifiers plus per-row typed `columns` plus collection-level
`column_definitions` — can be expressed in a Planemo/IWC `-tests.yml` job block at all. This
run's most consequential interface decision, the primary reads input, had to be made without
knowing whether the same run's own test phase can express it.
expected: >-
Add a rule and evidence to section 5 covering sample_sheet-family inputs: either a corpus or
Planemo-schema citation showing the job-block fixture syntax for `column_definitions` and
per-row `columns`, or an explicit statement that no such fixture form exists yet and that a
sample_sheet input therefore trades testability for metadata carriage. Either answer settles
the decision; the absence of both leaves it a guess. The sample_sheet note's "Edges to flag"
list would be the natural second home for the same statement.
evidence: >-
Bundle packages galaxy-sample-sheet-collections (which documents the workflow YAML
`column_definitions` form) and galaxy-workflow-testability-design (which documents fixture
design) with no overlap on this point; the note that would cover it,
iwc-test-data-conventions, is cross-referenced from section 5 but is not packaged. The
interface brief at <run>/freeform-galaxy-interface.md settles on `sample_sheet:paired` and
carries open-requirements entry `sample-sheet-input-test-fixture-expressibility` plus a named
`list:paired` fallback precisely because the question could not be answered.
status: open
issue: null
- id: paper-to-galaxy-path-has-no-reference-data-owner
raised_by: freeform-summary-to-galaxy-interface
observed_in:
mold: freeform-summary-to-galaxy-interface
path: content/molds/freeform-summary-to-galaxy-interface/index.md
revision: 3
content_hash: de0cb02532205ad18f4aa1ab2881cac725a3191b448468a8a11d0663ff1ec97d
foundry_head: 63a3f9cf9c97a637fe1628d298e524f35709e289
subject:
kind: pipeline
label: paper-to-galaxy
locator: content/pipelines/paper-to-galaxy/index.md
content_hash: null
kind: gap
severity: major
what: >-
The Nextflow path has a dedicated Mold for deciding the Galaxy-side shape of external
reference data (nextflow-summary-to-galaxy-reference-data, listed among the producers of the
open-requirements ledger). The paper path has no equivalent, and this run's phase roster
contains no reference-data phase. The source paper nonetheless pins a specific non-model
assembly (GCA_002759435.2, C. auris B8441) and requires an annotation file, so the decision
between a built-in index, a data-table string, and a portable history dataset had to be made
somewhere. The interface Mold made it, unprompted: nothing in its procedure or references
assigns reference-data shape to the interface tier.
expected: >-
Either add a reference-data phase to the paper-to-galaxy pipeline (a freeform sibling of
nextflow-summary-to-galaxy-reference-data, since the input is a narrative accession rather
than an igenomes-style parameter), or state explicitly in
freeform-summary-to-galaxy-interface that reference-data delivery shape is its call and
package the guidance that decision needs. Today the decision is unowned, which means it gets
made by whichever Mold notices, with no shared basis.
evidence: >-
Run roster phases 1-12 in this ledger contain no `*-to-galaxy-reference-data` step. The
interface brief settles the genome as a history fasta on portability grounds and carries
open-requirements entry `reference-genome-delivery-shape-unverified` recording that no
built-in-index check was performed, because no phase owns performing one. The subject's
content hash is null: this Mold's runtime notes forbid reading Foundry source, so the
pipeline note could not be hashed from inside the run.
status: open
issue: null
- id: interface-mold-gives-no-rule-for-parameter-exposure
raised_by: freeform-summary-to-galaxy-interface
observed_in:
mold: freeform-summary-to-galaxy-interface
path: content/molds/freeform-summary-to-galaxy-interface/index.md
revision: 3
content_hash: de0cb02532205ad18f4aa1ab2881cac725a3191b448468a8a11d0663ff1ec97d
foundry_head: 63a3f9cf9c97a637fe1628d298e524f35709e289
subject:
kind: mold
label: freeform-summary-to-galaxy-interface
locator: content/molds/freeform-summary-to-galaxy-interface/index.md
content_hash: de0cb02532205ad18f4aa1ab2881cac725a3191b448468a8a11d0663ff1ec97d
kind: gap
severity: minor
what: >-
The procedure names inputs, outputs, labels, collection shapes and checkpoints as the things
to settle, but gives no rule for which source-stated parameters become exposed typed workflow
inputs and which are baked into step defaults. The only nearby guidance is one line in the
packaged testability note ("Keep typed parameters explicit when tests need to set them"),
which answers the test-facing half and not the design half. For a paper source the
distinction is sharp and recurring: a value the paper pins is settled and can be baked, while
a value inferred from a kit name or an accession list is exactly what a reviewer must be able
to see and change. That rule was invented here, not read.
expected: >-
State the rule in the procedure: expose as a typed workflow parameter any value the source
leaves unstated or that this brief infers, so the inference is visible and overridable, plus
any value a test must set; bake values the source pins. Applies equally to the Nextflow and
CWL interface Molds, whose sources pin parameters far more often than a paper does.
evidence: >-
Interface brief section 2.3 has to justify its own exposure policy from scratch: strandedness
exposed because inferred, Cutadapt `-q 20` and RNA STAR defaults baked because stated,
significance thresholds exposed because stated but test-relevant. Three different
justifications for one decision class, none of them sourced from the bundle.
status: open
issue: null
- id: design-briefs-duplicate-open-questions-into-ledger-entries
raised_by: freeform-summary-to-galaxy-interface
observed_in:
mold: freeform-summary-to-galaxy-interface
path: content/molds/freeform-summary-to-galaxy-interface/index.md
revision: 3
content_hash: de0cb02532205ad18f4aa1ab2881cac725a3191b448468a8a11d0663ff1ec97d
foundry_head: 63a3f9cf9c97a637fe1628d298e524f35709e289
subject:
kind: research
label: open-requirements-ledger
locator: content/research/open-requirements-ledger/index.md
content_hash: e1d5e3a4d6d650af5b2efc4703fe6f79bb4d0cc12e6cd14e841c5b91df122ea0
kind: friction
severity: minor
what: >-
The Mold's output contract requires an "open questions" section in the Markdown brief and a
ledger of open entries, with no rule for which destination an unresolved choice belongs in.
Every unresolved item in this run was genuinely both an obligation a later Mold must
discharge and a thing a human reviewer should see, so all ten were written twice, in two
different shapes, and cross-referenced by entry id by hand to stop them drifting. Confirming
a real-run instance of something the note already lists under Open work
("Reconcile the design-tier briefs' free-text 'open questions' sections with the ledger");
filed as corroboration rather than as a new finding.
expected: >-
Pick one source of truth and say so. The cheapest version that would have helped here: make
the ledger authoritative and specify the brief's open-questions section as a rendered index
of the entries it raised, one line each keyed by entry id, rather than an independently
authored list.
evidence: >-
Interface brief section 5 carries ten numbered open questions, each ending in the entry id it
mirrors; the ledger carries the same ten as structured entries. The duplication is manual and
nothing checks it.
status: open
issue: null
- id: data-flow-mold-packages-pattern-mocs-without-their-recipe-pages
raised_by: freeform-summary-to-galaxy-data-flow
observed_in:
mold: freeform-summary-to-galaxy-data-flow
path: content/molds/freeform-summary-to-galaxy-data-flow/index.md
revision: 3
content_hash: 22613db4e710f58b29bfd2d6fe1d49f167afe451e09afefb2efd755b4a6e4da1
foundry_head: 63a3f9cf9c97a637fe1628d298e524f35709e289
subject:
kind: mold
label: freeform-summary-to-galaxy-data-flow
locator: content/molds/freeform-summary-to-galaxy-data-flow/index.md
content_hash: 22613db4e710f58b29bfd2d6fe1d49f167afe451e09afefb2efd755b4a6e4da1
kind: gap
severity: major
what: >-
All four packaged pattern references are MOC index pages. Each is a list of wiki-links to
operation and recipe pages — sync-collections-by-identifier, collection-cleanup-after-
mapover-failure, tabular-to-collection-by-row, tabular-filter-by-column-value and roughly
thirty others — and none of those pages is in the bundle, while the Mold's runtime notes
forbid reading Foundry source at runtime. The collection MOC states its own role plainly:
"the operation and recipe pages are the actionable references". So the Mold packages the
index and withholds the content it indexes. Every collection-idiom choice in this run's
brief was made from a one-line MOC description, with no corpus-observed recipe available to
check the shape, the built-in tool id, or the failure modes against.
expected: >-
Package the recipe and operation pages the MOCs name, or at least the subset a data-flow
brief can act on (the Cleanup, Identifiers, Structural Reshape and Bridges sections of
galaxy-collection-patterns and the Bridges section of galaxy-tabular-patterns). If the full
leaf set is too large for a bundle, say so in the Mold and state that MOC entries are naming
hints only, so the brief records idiom selections as unverified rather than as
corpus-grounded. Applies identically to the Nextflow and CWL data-flow Molds, which package
the same MOCs. The refs manifest marks these `evidence: corpus-observed`, which as packaged
is true of the map but not of anything the runtime can actually read.
evidence: >-
Bundle references/patterns holds exactly four files, all `pattern_kind: moc`. The brief at
<run>/freeform-galaxy-data-flow.md selects the identifier-sync idiom for its central
decision and has to state in its confidence section that the selection rests on a one-line
index entry.
status: open
issue: null
- id: data-flow-contract-silent-on-contradicting-the-interface-brief
raised_by: freeform-summary-to-galaxy-data-flow
observed_in:
mold: freeform-summary-to-galaxy-data-flow
path: content/molds/freeform-summary-to-galaxy-data-flow/index.md
revision: 3
content_hash: 22613db4e710f58b29bfd2d6fe1d49f167afe451e09afefb2efd755b4a6e4da1
foundry_head: 63a3f9cf9c97a637fe1628d298e524f35709e289
subject:
kind: research
label: galaxy-data-flow-draft-contract
locator: content/research/galaxy-data-flow-draft-contract/index.md
content_hash: f8f75f5a4ff750bff5f682847466492dd8a9910b95b5078373567f84ce16fb54
kind: gap
severity: major
what: >-
The contract partitions ownership between data-flow, template and step implementation, and
the Mold declares the interface brief as an input that "pins inputs, outputs, and labels".
Neither says what to do when the data-flow analysis shows a pinned interface decision is not
achievable under Galaxy semantics. That happened three times in this run: FastQC mapped over
a sample_sheet:paired input fans out per read direction, so two outputs the interface
declared as flat lists are nested; the outer axis of every mapped output is sample_sheet-
shaped, not the declared `list`; and a linear fold-change parameter is declared against a
DESeq2 column reported in log2. Whether the data-flow brief may correct the interface, must
defer to it, or must only record the conflict is not stated anywhere in the bundle. The
route taken here — ledger each one and state the correction in the brief — was invented.
expected: >-
State the rule in the contract's Boundary section: the data-flow draft may contradict an
interface decision when Galaxy collection or map-over semantics make it unachievable, and
must record each contradiction as an open-requirements entry naming the interface decision
it invalidates, so the interface brief and the template cannot silently diverge. Worth
pairing with a note in the open-requirements research page, whose `supersedes` mechanism
covers a refuted justification on an existing entry but has nothing for a refuted claim in
a sibling brief that was never ledgered.
evidence: >-
Interface brief section 3 declares outputs 1-8 as `list`/`list:paired` and input 7 as a
linear fold change; the data-flow brief section 7 shows all three are unachievable as
written and raises `fastqc-per-read-fanout-not-a-flat-list`,
`mapped-outputs-carry-sample-sheet-outer-axis` and
`fold-change-threshold-linear-vs-deseq2-log2fc` in the open-requirements ledger.
status: open
issue: null
- id: sample-sheet-note-omits-sample-sheet-to-tabular-output-columns
raised_by: freeform-summary-to-galaxy-data-flow
observed_in:
mold: freeform-summary-to-galaxy-data-flow
path: content/molds/freeform-summary-to-galaxy-data-flow/index.md
revision: 3
content_hash: 22613db4e710f58b29bfd2d6fe1d49f167afe451e09afefb2efd755b4a6e4da1
foundry_head: 63a3f9cf9c97a637fe1628d298e524f35709e289
subject:
kind: research
label: galaxy-sample-sheet-collections
locator: content/research/galaxy-sample-sheet-collections/index.md
content_hash: 98736fdcd0c4f4698af4e0cd4c539b33e3d0db872de1e10487556b21033030ec
kind: gap
severity: major
what: >-
The note's closing guidance is that carry-forward of sample-sheet metadata past map-over
"must be explicit (re-attaching metadata via `__SAMPLE_SHEET_TO_TABULAR__` or rules DSL
`add_column_from_sample_sheet_index`)", and the note is packaged into this Mold precisely to
drive that re-attachment. But it describes the tool in one clause — "iterates and tab-joins
for downstream tabular consumers" — and never states its output columns. Whether the element
identifier is emitted as a column is the single fact the re-attachment turns on, because the
identifier is the only key that survives map-over and therefore the only possible join key
back to a downstream collection. The note recommends the mechanism without supplying what is
needed to wire it.
expected: >-
State the tool's output schema in the "Tool-side access" section: whether the element
identifier is emitted, in which column, and how the remaining columns order relative to
`column_definitions`. Do the same for the rules-DSL alternative — which axis
`add_column_from_sample_sheet_index` reads and whether it can apply to a collection that has
already lost its `column_definitions`, since that is the case a reader arrives with. The
sources list already cites lib/galaxy/tools/sample_sheet_to_tabular.xml, so the fact is one
line away from where it is needed.
evidence: >-
This run's central data-flow decision (<run>/freeform-galaxy-data-flow.md section 4) is an
identifier-keyed split that joins this tool's output to the featureCounts collection. It had
to ship with open-requirements entry `sample-sheet-to-tabular-identifier-column-unverified`
and a named substitute node, because the join key could not be confirmed from the bundle and
the Mold's runtime notes forbid reading Galaxy or Foundry source.
status: open
issue: null
- id: iwc-corpus-path-hard-coded-with-no-override
raised_by: compare-against-iwc-exemplar
observed_in:
mold: compare-against-iwc-exemplar
path: content/molds/compare-against-iwc-exemplar/index.md
revision: 10
content_hash: 5b198e9b87cf554cb01aceaa14a5aaa3ab24c6a35afc35dab7d09ce154cc3a77
foundry_head: 63a3f9cf9c97a637fe1628d298e524f35709e289
subject:
kind: mold
label: compare-against-iwc-exemplar
locator: content/molds/compare-against-iwc-exemplar/index.md
content_hash: 5b198e9b87cf554cb01aceaa14a5aaa3ab24c6a35afc35dab7d09ce154cc3a77
kind: defect
severity: blocker
what: >-
The procedure's first and only prerequisite step is "Clone or pull and merge the IWC corpus
(https://github.com/galaxyproject/iwc) to `~/.foundry/iwc`". That path is hard-coded, and the
Mold names no environment variable, no configuration key, and no fallback for a machine where
it cannot be created. On this machine it cannot: `mkdir ~/.foundry` fails with `Operation not
permitted`, with and without the Bash sandbox. The Mold is the corpus-first check of every
Galaxy-targeting pipeline, so an unsatisfiable corpus location makes the whole phase
unrunnable, and the only reason this run produced a comparison at all is that the harness
supplied a clone at a different path out of band. Nothing in the bundle describes that route,
and an unattended run has no way to discover it.
expected: >-
Read the corpus location from an environment variable with `~/.foundry/iwc` as the default —
the same shape `OMC_STATE_DIR` already has elsewhere in this toolchain — and state in the
procedure that a caller-supplied corpus path is honoured and pinned rather than pulled.
Two smaller corrections belong with it. State that when the corpus is supplied rather than
cloned, the Mold must not `git pull` or write inside it, since a caller-owned checkout may be
shared or read-only. And require the corpus commit to be recorded in `iwc-comparison-notes`
as provenance in every case, not only the supplied one: a structural comparison is a claim
about a moving corpus, and without the commit no later reader can tell whether a divergence
this Mold reported has since been closed upstream. The output artifact description says
nothing about provenance today.
evidence: >-
`mkdir ~/.foundry` returned `Operation not permitted` on darwin 25.5.0 under both the
sandboxed and unsandboxed Bash tool. The phase ran against a caller-supplied shallow clone
pinned at galaxyproject/iwc `fe41a79`, recorded in <run>/iwc-comparison-notes.md under
"Corpus provenance" on this phase's own initiative, since no packaged instruction asks for it.
status: open
issue: null
- id: iwc-exemplar-artifact-assumes-a-single-nearest-exemplar
raised_by: compare-against-iwc-exemplar
observed_in:
mold: compare-against-iwc-exemplar
path: content/molds/compare-against-iwc-exemplar/index.md
revision: 10
content_hash: 5b198e9b87cf554cb01aceaa14a5aaa3ab24c6a35afc35dab7d09ce154cc3a77
foundry_head: 63a3f9cf9c97a637fe1628d298e524f35709e289
subject:
kind: mold
label: compare-against-iwc-exemplar
locator: content/molds/compare-against-iwc-exemplar/index.md
content_hash: 5b198e9b87cf554cb01aceaa14a5aaa3ab24c6a35afc35dab7d09ce154cc3a77
kind: gap
severity: major
what: >-
The Mold speaks of "nearest IWC exemplar(s)" in its summary, its procedure and its confidence
table, but the gxformat2 artifact is specified strictly in the singular — one declared
filename, "the nearest exemplar's relevant subgraph", "Once the nearest exemplar is chosen
(High or Medium confidence), convert it". Nothing says what to emit when the honest answer is
several exemplars covering disjoint parts of the subject. That is what happened here and it
was not an edge case: IWC publishes the subject's journey as two workflows joined at the
count-table boundary, so one exemplar covers the map-over head and a different one covers the
differential-expression tail, and a third workflow in another domain was the only corpus
source for one tool's output shape. Whether that is one file with several YAML documents,
several files, or a single forced choice was guessed at.
expected: >-
State the multi-exemplar case in the "Nearest exemplar (gxformat2) view" section and settle
the encoding. The shape that worked here and is worth specifying: one file at the declared
filename, one YAML document per exemplar, each document headed by the abstract IWC workflow
ID it came from, the steps covered, the steps dropped, and its own confidence level — so a
Low-confidence cross-domain citation cannot be mistaken for a domain exemplar, which the
section already warns about in prose but gives no structural way to express. Worth pairing
with a sentence in the Feature Hierarchy noting that a corpus which splits the subject's
scope across workflows is itself a first-class structural finding for the template tier, not
a retrieval failure.
evidence: >-
Ranking produced transcriptomics/rnaseq-de/rnaseq-de-filtering-plotting at High for the DE
tail, transcriptomics/rnaseq-pe/rnaseq-pe at Medium for the map-over head, and
epigenetics/cutandrun/cutandrun as tool-level-only evidence for the Cutadapt output shape.
<run>/iwc-exemplar.gxwf.yml carries all three as three YAML documents at the one declared
filename, an encoding invented in this run.
status: open
issue: null
- id: bounded-subgraph-artifact-has-no-stated-validity-contract
raised_by: compare-against-iwc-exemplar
observed_in:
mold: compare-against-iwc-exemplar
path: content/molds/compare-against-iwc-exemplar/index.md
revision: 10
content_hash: 5b198e9b87cf554cb01aceaa14a5aaa3ab24c6a35afc35dab7d09ce154cc3a77
foundry_head: 63a3f9cf9c97a637fe1628d298e524f35709e289
subject:
kind: mold
label: compare-against-iwc-exemplar
locator: content/molds/compare-against-iwc-exemplar/index.md
content_hash: 5b198e9b87cf554cb01aceaa14a5aaa3ab24c6a35afc35dab7d09ce154cc3a77
kind: gap
severity: major
what: >-
The artifact is described as a "Cleaned gxformat2 conversion (via convert --to format2
--compact) of the nearest IWC exemplar's relevant subgraph", and separately as "bounded to
the relevant subgraph, not the whole workflow". Those two requirements cannot both hold
literally. `convert` emits the whole workflow; bounding it to a subgraph is hand surgery that
necessarily leaves dangling `source:` references to removed steps and outputs, so the result
is well-formed YAML but not a loadable gxformat2 workflow. The Mold never says which property
matters, and the downstream consumer is named only as something that "pattern-matches
against" the file — which does not distinguish a human-read reference from a parsed one. This
run resolved it by treating the artifact as a reading aid, eliding non-structural tool_state
and marking every elision inline, but a template Mold that tried to load the file would
fail, and nothing warned it.
expected: >-
State the contract in the "Nearest exemplar (gxformat2) view" section: the artifact is a
bounded reading aid, not a runnable or validatable workflow; dangling references to elided
steps are expected; and elisions must be marked in place so a reader can tell a bound from
an absence. If instead the file is meant to stay loadable, say that and specify how — keep
every transitively referenced step, or rewrite dropped sources to workflow inputs. Either
answer is workable; the absence of both means each run invents its own and the downstream
Mold cannot rely on either. A one-line statement in the artifact description would settle it,
since that is where a consumer looks.
evidence: >-
<run>/iwc-exemplar.gxwf.yml drops the visualization tail of one exemplar and roughly
two-thirds of the other, leaving references to steps that are no longer present — for
example the second document's `outputs[Counts Table].outputSource: _unlabeled_step_25/...`,
whose producing step was dropped — and requiring several step `in:` entries to be commented
out rather than resolved. It parses as three valid YAML documents and would not load as
gxformat2. The convert CLI reference packaged in this bundle documents no subsetting option,
so the bounding is necessarily out-of-band.
status: open
issue: null
- id: gxwf-version-flag-reports-stale-version
raised_by: compare-against-iwc-exemplar
observed_in:
mold: compare-against-iwc-exemplar
path: content/molds/compare-against-iwc-exemplar/index.md
revision: 10
content_hash: 5b198e9b87cf554cb01aceaa14a5aaa3ab24c6a35afc35dab7d09ce154cc3a77
foundry_head: 63a3f9cf9c97a637fe1628d298e524f35709e289
subject:
kind: related-project
label: "@galaxy-tool-util/cli — gxwf --version"
locator: https://github.com/jmchilton/galaxy-tool-util-ts
kind: defect
severity: minor
what: >-
`gxwf --version` prints `1.0.0` regardless of the installed package version. The version
actually installed here is `@galaxy-tool-util/cli@1.10.1`, confirmed by `npm ls -g`. Every
Foundry Mold that requires gxwf pins a package version in `_required_tools.json` — this one
pins `^1.8.1` — and none of those pins can be checked at runtime, because the only version
the CLI reports is a constant that satisfies no pin and matches no release. The packaged
availability check works around this by grepping `--help` for a subcommand name, which
detects presence but says nothing about version.
expected: >-
Have `gxwf --version` report the package version from package.json. Until it does, a Mold
that needs version-sensitive behaviour has no runtime signal, and a run that records its tool
versions as provenance records a number that is always `1.0.0`.
evidence: >-
`gxwf --version` → `1.0.0`; `npm ls -g @galaxy-tool-util/cli` → `@galaxy-tool-util/cli@1.10.1`;
this Mold's `_required_tools.json` pins `package_version: ^1.8.1` with
`availability_check: gxwf --help | grep -q draft-validate`. Conversions in this phase
succeeded, so the defect is in version reporting only, not in the tool's behaviour.
status: open
issue: null
- id: draft-format-note-silent-on-step-addressing-by-label
raised_by: freeform-summary-to-galaxy-template
observed_in:
mold: freeform-summary-to-galaxy-template
path: content/molds/freeform-summary-to-galaxy-template/index.md
revision: 6
content_hash: 0a95f8c04465d48131ace523e0b27af53f1ade6e525a9173c5ff1e936ddc3366
foundry_head: 63a3f9cf9c97a637fe1628d298e524f35709e289
subject:
kind: research
label: galaxy-workflow-draft-format
locator: content/research/galaxy-workflow-draft-format/index.md
content_hash: 93f3b231dd063bf059c550f18b2200c61909af0da9408e0d43351116e886bf7b
kind: gap
severity: major
what: >-
The note never says which step field a connection's `source:` resolves against. Its only
example uses the map form, where the step key and its identity are the same string, so the
question cannot arise there. In the list form a step may carry both `id` and `label`, and
`gxwf draft-validate` resolves `source:` against the LABEL when one is present — an `id`
that differs from the label is not addressable at all.
expected: >-
State the rule in the "Relaxations vs. gxformat2" section: a labelled step is addressed by
its label, so in list form `id` and `label` must be the same string (which is what the IWC
corpus does), or the label must be omitted. Adding a list-form example alongside the
existing map-form sketch would carry the rule by demonstration.
evidence: >-
A first draft of 27 steps, each with a snake_case `id` and a separate human `label`, with
every `source:` written against the id. `gxwf draft-validate` returned 47 topology errors,
all of the form `references unknown step "<id>"`, while reporting the same steps in its
diagnostic paths under their labels. Renaming every `id` to match its `label` and rewriting
all 47 references cleared it to `draft valid` with no other change. Neither the note nor the
packaged `draft-validate` command reference mentions the distinction.
status: open
issue: null
- id: draft-format-note-example-writes-format-as-a-scalar
raised_by: freeform-summary-to-galaxy-template
observed_in:
mold: freeform-summary-to-galaxy-template
path: content/molds/freeform-summary-to-galaxy-template/index.md
revision: 6
content_hash: 0a95f8c04465d48131ace523e0b27af53f1ade6e525a9173c5ff1e936ddc3366
foundry_head: 63a3f9cf9c97a637fe1628d298e524f35709e289
subject:
kind: research
label: galaxy-workflow-draft-format
locator: content/research/galaxy-workflow-draft-format/index.md
content_hash: 93f3b231dd063bf059c550f18b2200c61909af0da9408e0d43351116e886bf7b
kind: defect
severity: major
what: >-
The note's "Example (sketch)" declares a workflow input as `format: fastqsanger.gz` — a bare
scalar. The draft schema the same Mold packages types `format` as `null | ReadonlyArray<string>`,
so the sketch is not valid against the contract it illustrates. An author who copies the
sketch, as it invites, gets a structure error.
expected: >-
Write `format:` as a list in the sketch (`format: [fastqsanger.gz]` or the block form), and
say in the relaxations section that `format` is a list even for a single value. If the
scalar form is in fact accepted by some other consumer, say which and why the draft schema
rejects it.
evidence: >-
The draft was authored with `format: fastqsanger.gz`, `format: fasta` and `format: gtf`, all
copied in form from the note's sketch. `gxwf draft-validate` returned one structure error
against the whole `inputs` array; converting all three to single-element lists cleared it.
Confirmed independently against the packaged
`references/schemas/galaxy-workflow-draft.schema.json`, where the input variants type
`format` as an array of strings only.
status: open
issue: null
- id: draft-format-tiers-conflate-a-named-tool-with-a-named-tool-id
raised_by: freeform-summary-to-galaxy-template
observed_in:
mold: freeform-summary-to-galaxy-template
path: content/molds/freeform-summary-to-galaxy-template/index.md
revision: 6
content_hash: 0a95f8c04465d48131ace523e0b27af53f1ade6e525a9173c5ff1e936ddc3366
foundry_head: 63a3f9cf9c97a637fe1628d298e524f35709e289
subject:
kind: research
label: galaxy-workflow-draft-format
locator: content/research/galaxy-workflow-draft-format/index.md
content_hash: 93f3b231dd063bf059c550f18b2200c61909af0da9408e0d43351116e886bf7b
kind: gap
severity: major
what: >-
The Identity-pinned tier admits "a source summary that names a specific `tool_id` with
evidence", and the Mold's source-tendency paragraph relaxes that to "a free-form source that
does name a specific tool/version with evidence hardens to the matching tier". Two
paragraphs earlier the same note forbids pinning "on plausibility". A paper naming "FastQC"
names a piece of software, not a Galaxy `tool_id`; reading the tier rule literally licenses
writing a Tool Shed path from memory, which is exactly the plausibility pin the note
forbids. The note never distinguishes the two kinds of naming, and they come apart on every
free-form source.
expected: >-
Say explicitly that Identity-pinned requires a concrete `tool_id` STRING from evidence — a
corpus workflow, a pattern page's worked example, or a source that quotes the Galaxy tool id
— and that a source naming a tool by its software name, with no corpus or pattern hit for
its wrapper, is Deferred with the software name recorded in `_plan_context`. The
source-tendency paragraph should be reworded to match, since as written it points the other
way.
evidence: >-
All six tools in this run's source are named by the paper. Five had a corpus-confirmed
`tool_id` from the phase-4 exemplar and were Identity-pinned. FastQC had none: the nearest
exemplar uses `iuc/falco` instead, so the only route to a FastQC `tool_id` was recall. That
step was Deferred, against the plainest reading of the source-tendency sentence, on the
strength of the plausibility prohibition. The two rules gave opposite answers for the same
step and the note offers nothing to break the tie.
status: open
issue: null
- id: draft-format-has-no-way-to-mark-a-region-provisional
raised_by: freeform-summary-to-galaxy-template
observed_in:
mold: freeform-summary-to-galaxy-template
path: content/molds/freeform-summary-to-galaxy-template/index.md
revision: 6
content_hash: 0a95f8c04465d48131ace523e0b27af53f1ade6e525a9173c5ff1e936ddc3366
foundry_head: 63a3f9cf9c97a637fe1628d298e524f35709e289
subject:
kind: research
label: galaxy-workflow-draft-format
locator: content/research/galaxy-workflow-draft-format/index.md
content_hash: 93f3b231dd063bf059c550f18b2200c61909af0da9408e0d43351116e886bf7b
kind: gap
severity: minor
what: >-
The draft superset can mark a STEP as unresolved (`TODO`, `_plan_*`) but has no way to mark
a REGION as provisional, or to record the named alternative a later phase should swap in.
The `_plan_*` family is per-step and explicitly wrapper-tier, so a multi-step topology
choice that is settled-but-unprecedented has nowhere durable to live.
expected: >-
Decide a home for it and add it to the note — a workflow-level annotation beside the
`topology_repair` budget the open-requirements note already wants moved into the draft, or
an explicit statement that a titled `comments:` frame plus an open-requirements entry IS the
intended mechanism. Either answer is fine; the absence of one means each template Mold run
invents its own.
evidence: >-
Phase 4 instructed this phase to build the sample-sheet condition split "as a delimited
region with the phase-2 alternative one deletion away". Thirteen steps, no corpus precedent,
settled topology. The delimitation was expressed three ways, none of them contractual: YAML
comments (lost on any round-trip through Galaxy), a titled `comments:` frame (schema-legal
and durable, but the packaged `galaxy-workflow-comments` note describes frames as narrative
stage annotation, not risk marking), and a `type: markdown` comment holding the four-step
swap procedure as prose. A reader of the draft alone has no typed signal that the region is
provisional.
status: open
issue: null
- id: template-mold-packages-pattern-mocs-without-their-recipe-pages
raised_by: freeform-summary-to-galaxy-template
observed_in:
mold: freeform-summary-to-galaxy-template
path: content/molds/freeform-summary-to-galaxy-template/index.md
revision: 6
content_hash: 0a95f8c04465d48131ace523e0b27af53f1ade6e525a9173c5ff1e936ddc3366
foundry_head: 63a3f9cf9c97a637fe1628d298e524f35709e289
subject:
kind: mold
label: freeform-summary-to-galaxy-template
locator: content/molds/freeform-summary-to-galaxy-template/index.md
content_hash: 0a95f8c04465d48131ace523e0b27af53f1ade6e525a9173c5ff1e936ddc3366
kind: gap
severity: major
what: >-
The Mold packages four pattern references and all four are MOC index pages. Each names the
recipe pages that carry the actual tool ids, port names and worked wiring, and none of those
pages is in the bundle. The runtime notes forbid reading Foundry source, so a recipe named
in a packaged MOC is unreachable at runtime — the reference resolves to a one-line
description and a dead wiki-link.
expected: >-
Add the recipe pages the template actually reaches for to the manifest, or state in the
Mold's procedure that the MOCs are for naming an idiom only and that port names and tool ids
must come from the exemplar artifact or from `discover-shed-tool`. The first is better; the
second at least stops the bundle promising what it cannot deliver. The data-flow Mold has
the same defect, filed separately as
`data-flow-mold-packages-pattern-mocs-without-their-recipe-pages` — different manifest, same
correction, so fixing one does not fix the other.
evidence: >-
Three steps in this draft implement idioms the packaged MOCs name and cannot describe.
`sync-collections-by-identifier` (collection MOC) is the pattern behind the three
`__FILTER_FROM_FILE__` steps, but with no recipe page its input and output port names stayed
`TODO_` sentinels. `tabular-filter-by-column-value` and `tabular-cut-and-reorder-columns`
(tabular MOC) are the `Filter1` and `Cut1` steps; `Filter1`'s ports were recoverable only
because the phase-4 exemplar happened to contain a worked instance, and `Cut1`'s were not
recoverable at all and are recorded as assumed-by-convention in `_plan_context`. The phase-3
brief flagged the same limitation prospectively in its section 9 evidence-class caveat; this
is the concrete cost at the template tier.
status: open
issue: null
- id: gxwf-drops-connectedvalue-under-format2-state-key
raised_by: advance-galaxy-draft-step
observed_in:
mold: advance-galaxy-draft-step
path: content/molds/advance-galaxy-draft-step/index.md
revision: 4
content_hash: c92d452f69a7b40bcab5fc8e63b03e1e2abf98f2a636abc4c93ad7f3c1d03082
foundry_head: 63a3f9cf9c97a637fe1628d298e524f35709e289
subject:
kind: related-project
label: "@galaxy-tool-util/cli — gxwf tool-state validation (state: vs tool_state:)"
locator: https://github.com/jmchilton/galaxy-tool-util-ts
kind: defect
severity: major
what: >-
`gxwf validate` (and `draft-validate --concrete`) drops a
`component_value: {__class__: ConnectedValue}` from a step's tool state before validating
it when the state is written under the format2 key `state:`, but honours it under
`tool_state:`. The same bytes under the two keys therefore get opposite verdicts. Any
conditional parameter case whose generated `workflow_step_linked` branch marks
`component_value` as required — every non-text case — then fails as
"component_value: is missing", and the anyOf fallback reports the misleading
"select_param_type: Expected \"text\", actual \"float\"" alongside it.
expected: >-
Treat `state:` and `tool_state:` identically in the tool-state validation path, so a
connected value placeholder survives to validation under both keys. Failing that, the
generated `workflow_step_linked` schema should not require `component_value` for a
parameter that a step connection supplies, which is exactly the relaxation that
distinguishes it from `workflow_step`. A one-line note in the `draft-validate` /
`validate` docs would not be enough: the failure names a required field the author
deliberately connected, which reads as an authoring error rather than a tool bug.
evidence: >-
gxwf 1.10.1. Minimal one-step workflow around
`iuc/compose_text_param/compose_text_param@0.1.1`, second repeat component in the float
case, `component_value: {__class__: ConnectedValue}`, connected via
`components_1|param_type|component_value`. Under `state:` the tool-state check fails with
the five diagnostics above; byte-identical content under `tool_state:` reports
`tool_state: OK`. Failure does not depend on the connection being present, on
`__index__`, or on `__current_case__`; substituting a literal float under `state:` passes,
which is the wrong workaround to be nudged toward — it leaves a shadow default behind a
connected parameter. The IWC exemplar this run compares against
(`transcriptomics/rnaseq-de/rnaseq-de-filtering-plotting` at fe41a79) passes its three
compose_text_param steps precisely because `gxwf convert --to format2` emits `tool_state:`.
status: open
issue: null
- id: advance-draft-mold-needs-a-tool-cache-it-never-declares
raised_by: advance-galaxy-draft-step
observed_in:
mold: advance-galaxy-draft-step
path: content/molds/advance-galaxy-draft-step/index.md
revision: 4
content_hash: c92d452f69a7b40bcab5fc8e63b03e1e2abf98f2a636abc4c93ad7f3c1d03082
foundry_head: 63a3f9cf9c97a637fe1628d298e524f35709e289
subject:
kind: mold
label: advance-galaxy-draft-step
locator: content/molds/advance-galaxy-draft-step/index.md
content_hash: c92d452f69a7b40bcab5fc8e63b03e1e2abf98f2a636abc4c93ad7f3c1d03082
kind: gap
severity: major
what: >-
The procedure names `galaxy-tool-cache list` as the way to resolve a stock tool's version,
and the per-step validator it mandates only produces a real verdict when given
`--cache-dir` with the step's tool cached. Neither the cache nor the `galaxy-tool-cache`
binary appears anywhere in the bundle's contract: `_required_tools.json` declares `gxwf`
alone, derived from the three `gxwf` commands the Mold cites, and no step of the procedure
says to populate a cache. Without it `draft-validate --concrete` reports
`skip_tool_not_found` for every step and the iteration's green is vacuous.
expected: >-
Declare `galaxy-tool-cache` as a required tool of this Mold (it ships in the same
`@galaxy-tool-util/cli` package, so it costs no new install), and make cache population an
explicit move in the procedure — `galaxy-tool-cache add <tool_id> --tool-version <v>` after
wrapper resolution, with the resulting `--cache-dir` passed to `draft-validate --concrete`.
Citing the command as a CLI reference rather than in prose would also let the required-tools
deriver pick it up, which is why it is missing today.
evidence: >-
Iteration 1 of this run. `galaxy-tool-cache` is not on PATH after the documented
`npm install -g @galaxy-tool-util/cli` (it exists in the package's bin directory but only
`gxwf` was linked), and its default cache root `~/.galaxy/tool_info_cache` is not writable
in this sandbox, so both had to be worked around before any concrete verdict was possible.
Until the cache was populated by hand, `draft-validate --concrete` reported
`Tool state: 0 ok, 0 fail, 1 skip` — which the procedure's "on green, return" would have
accepted.
status: open
issue: null
- id: implement-step-mold-says-state-without-saying-which-key
raised_by: advance-galaxy-draft-step
observed_in:
mold: advance-galaxy-draft-step
path: content/molds/advance-galaxy-draft-step/index.md
revision: 4
content_hash: c92d452f69a7b40bcab5fc8e63b03e1e2abf98f2a636abc4c93ad7f3c1d03082
foundry_head: 63a3f9cf9c97a637fe1628d298e524f35709e289
subject:
kind: mold
label: implement-galaxy-tool-step
locator: content/molds/implement-galaxy-tool-step/index.md
content_hash: 197be2d9b27741c3f258092f8392654369d1ca234d7fad492387ff69d656292b
kind: gap
severity: major
what: >-
The procedure says to "shape the step's `state` against
`input_schemas.workflow_step_linked`" and speaks of `state` throughout, but the
galaxy-workflow-draft schema it packages admits both `state` and `tool_state` on a step and
the Mold never says which to write. The two are not interchangeable in practice: under
gxwf 1.10.1 a connected non-text parameter validates under `tool_state:` and fails under
`state:` (filed as `gxwf-drops-connectedvalue-under-format2-state-key`), so following the
Mold's own wording is what produces the red verdict.
expected: >-
Name one key as the authoring convention and say it once — `tool_state:`, which is what
`gxwf convert --to format2` emits and what every converted IWC exemplar a run compares
against will therefore show — and note that a step mixing the two conventions within one
draft is a readability cost, not a correctness one. This is worth fixing independently of
the gxwf defect: the ambiguity is in the instruction, and it will still be there after the
validator is corrected.
status: open
issue: null
- id: gxwf-cannot-decode-collection-outputs-of-builtin-collection-operations
raised_by: advance-galaxy-draft-step
observed_in:
mold: advance-galaxy-draft-step
path: content/molds/advance-galaxy-draft-step/index.md
revision: 4
content_hash: c92d452f69a7b40bcab5fc8e63b03e1e2abf98f2a636abc4c93ad7f3c1d03082
foundry_head: 63a3f9cf9c97a637fe1628d298e524f35709e289
subject:
kind: related-project
label: "@galaxy-tool-util/cli — gxwf tool fetch/decode for built-in collection operations"
locator: https://github.com/jmchilton/galaxy-tool-util-ts
kind: defect
severity: major
what: >-
`gxwf draft-validate --concrete` cannot bring the built-in `__FLATTEN__` into its tool
cache. The fetch is reported as `toolshed fetch failed ... for __FLATTEN__`, but the
accompanying dump is a decode failure against the tool schema, not a transport error:
`["outputs"][0]` is decoded as the collection-output branch and `["structure"]` `is
missing`. The tool's collection output carries no `structure`, and the schema requires
one. The step is then reported `skip_tool_not_found`, so its tool state is never
validated — and no amount of cache priming can fix it, because the tool cannot be
decoded into the cache in the first place. The same shape is likely to hit the other
collection-operation built-ins (`__UNZIP_COLLECTION__`, `__FILTER_FROM_FILE__`,
`__APPLY_RULES__`), which are exactly the steps a Galaxy draft uses for collection
plumbing.
expected: >-
Make `structure` optional on the collection-output branch of the tool schema (or supply a
default for tools that declare a collection output without one), so built-in collection
operations decode and cache like any other tool. Separately, distinguish a transport
failure from a decode failure in the message: "toolshed fetch failed" sent this run
looking at network and cache-priming for a fault that was neither.
evidence: >-
`gxwf draft-validate <run>/galaxy-workflow-draft.gxwf.yml --concrete --cache-dir <cache>`
at iteration 2, gxwf 1.10.1. Verdict line `Tool state: 2 ok, 0 fail, 1 skip`, with
`0 (__FLATTEN__) [skip_tool_not_found] __FLATTEN__ not in cache (fetch failed)`. The two
compose_text_param steps validated from the same cache in the same run, so the cache path
and network were both working. The decode dump is also ~4 KB of expanded structural type
text printed ahead of the verdict, which buries the one line that names the cause.
status: open
issue: null
- id: advance-draft-mold-reresolves-a-wrapper-already-pinned-in-the-draft
raised_by: advance-galaxy-draft-step
observed_in:
mold: advance-galaxy-draft-step
path: content/molds/advance-galaxy-draft-step/index.md
revision: 4
content_hash: c92d452f69a7b40bcab5fc8e63b03e1e2abf98f2a636abc4c93ad7f3c1d03082
foundry_head: 63a3f9cf9c97a637fe1628d298e524f35709e289
subject:
kind: mold
label: advance-galaxy-draft-step
locator: content/molds/advance-galaxy-draft-step/index.md
content_hash: c92d452f69a7b40bcab5fc8e63b03e1e2abf98f2a636abc4c93ad7f3c1d03082
kind: friction
severity: minor
what: >-
The procedure's resolve-then-summarize sequence is written as if each iteration met its
wrapper for the first time. It has no notion of a wrapper already resolved earlier in the
same run: this draft uses one wrapper at five steps, and the identity-pinned branch still
directs the iteration to confirm the pin via discover-shed-tool and then invoke
summarize-galaxy-tool, both of which a sibling step's already-concrete
`tool_id` + `tool_version` pair has settled. The Mold never describes what this iteration
actually did, which was to take the sibling's pin and read the summary already in the
cache.
expected: >-
Add a short third case to the resolve step: when another step of this same draft is
already concrete on the same `tool_id`, adopt its `tool_version` and skip discovery,
re-using the cached summary rather than re-summarizing. Say plainly that this is the
intended move, so an iteration that takes it is following the procedure rather than
departing from it. The Mold should also state that a per-run resolved-wrapper reuse is
safe precisely because the pin is recorded in the artifact, not in operator memory.
evidence: >-
Iteration 2 of this run, step `Build first-contrast row predicate`. Iteration 1 had
already resolved `iuc/compose_text_param` to 0.1.1 against the live Tool Shed for
`Build adjusted p-value predicate`, and written the pin into the draft. Three more steps
of the same draft await the same wrapper.
status: open
issue: null
- id: advance-draft-mold-cites-draft-format-note-it-does-not-package
raised_by: advance-galaxy-draft-step
observed_in:
mold: advance-galaxy-draft-step
path: content/molds/advance-galaxy-draft-step/index.md
revision: 4
content_hash: c92d452f69a7b40bcab5fc8e63b03e1e2abf98f2a636abc4c93ad7f3c1d03082
foundry_head: 63a3f9cf9c97a637fe1628d298e524f35709e289
subject:
kind: mold
label: advance-galaxy-draft-step
locator: content/molds/advance-galaxy-draft-step/index.md
content_hash: c92d452f69a7b40bcab5fc8e63b03e1e2abf98f2a636abc4c93ad7f3c1d03082
kind: gap
severity: major
what: >-
The procedure's resolve step branches on the draft tiers and sends the reader to
`galaxy-workflow-draft-format` for them, but the Mold does not package that note: the
bundle's references are three gxwf CLI pages, `galaxy-tool-job-failure-reference`,
`open-requirements-ledger`, and the tool-summary and draft JSON Schemas. The Runtime
Notes then forbid reading Foundry source files at runtime, so the one document the
procedure names for the distinction it asks the iteration to make is unreachable from
inside the bundle. The draft JSON Schema does not carry the tiers — they are prose.
expected: >-
Package `content/research/galaxy-workflow-draft-format/index.md` in this Mold's
references, as `freeform-summary-to-galaxy-template` already does. It is the note this
Mold's central branch depends on, and the per-step loop reads a draft written against it
on every iteration. If the intent is that the tiers be inferable from the draft alone,
say that instead and drop the cross-reference.
evidence: >-
Iteration 3 of this run, step `Build log2 fold-change predicate`. The tier vocabulary was
taken from the draft's own `doc:` strings (`Tier: Identity-pinned.` / `Tier: Resolved.`,
a convention iteration 1 introduced) rather than from any packaged contract, and the same
gap left the fate of the step's `_plan_context` provenance undecided — see
`advance-draft-mold-silent-on-plan-provenance-at-concretion`.
status: open
issue: null
- id: advance-draft-mold-silent-on-plan-provenance-at-concretion
raised_by: advance-galaxy-draft-step
observed_in:
mold: advance-galaxy-draft-step
path: content/molds/advance-galaxy-draft-step/index.md
revision: 4
content_hash: c92d452f69a7b40bcab5fc8e63b03e1e2abf98f2a636abc4c93ad7f3c1d03082
foundry_head: 63a3f9cf9c97a637fe1628d298e524f35709e289
subject:
kind: mold
label: advance-galaxy-draft-step
locator: content/molds/advance-galaxy-draft-step/index.md
content_hash: c92d452f69a7b40bcab5fc8e63b03e1e2abf98f2a636abc4c93ad7f3c1d03082
kind: gap
severity: minor
what: >-
Concretizing a step forces its `_plan_*` fields to be deleted — `draft-next-step` counts
any surviving `_plan_*` as remaining work, so a finished step that keeps one is selected
again on the next iteration and the harness loop cannot terminate. Those fields are also
where the corpus provenance lives: the step implemented here carried a `_plan_context`
citing the exemplar workflow and step that fixes its component text. The Mold says only
that the implement phase "resolves the chosen step's remaining `TODO_*` / `_plan_*`
slots", and never says whether that provenance should be carried into `doc:`, dropped,
or recorded elsewhere. This run has been keeping it in a YAML comment above the step, a
convention it invented, and nothing states whether `draft-extract` preserves comments
when it re-serializes the concrete workflow (this iteration did not test that).
expected: >-
Say in the implement step what becomes of a concretized step's planning provenance: fold
the load-bearing part of `_plan_context` into the step's `doc:` (which survives
extraction and is visible to a Galaxy user), and drop the rest. State plainly that
`_plan_*` must not survive concretion, and why — `draft-next-step` would re-select the
step forever. If YAML comments are in fact preserved by `draft-extract`, say so and
sanction the comment form; if they are not, say that too, so a run does not park
provenance somewhere that silently disappears at loop endstate.
evidence: >-
Iteration 3, step `Build log2 fold-change predicate`. Its `_plan_context` cited
`transcriptomics/rnaseq-de` at corpus fe41a79, step `_unlabeled_step_10`, as the source
of the `abs(c3)>` component text; the step's post-implementation `doc:` and the YAML
comment above it were written by hand to retain that, with no instruction either way.
`gxwf draft-next-step` lists every `_plan_*` field of the selected step under `work`,
which is what makes their removal mandatory rather than stylistic.
status: open
issue: null
- id: advance-draft-mold-treats-a-skipped-tool-state-as-green
raised_by: advance-galaxy-draft-step
observed_in:
mold: advance-galaxy-draft-step
path: content/molds/advance-galaxy-draft-step/index.md
revision: 4
content_hash: c92d452f69a7b40bcab5fc8e63b03e1e2abf98f2a636abc4c93ad7f3c1d03082
foundry_head: 63a3f9cf9c97a637fe1628d298e524f35709e289
subject:
kind: mold
label: advance-galaxy-draft-step
locator: content/molds/advance-galaxy-draft-step/index.md
content_hash: c92d452f69a7b40bcab5fc8e63b03e1e2abf98f2a636abc4c93ad7f3c1d03082
kind: gap
severity: major
what: >-
The validate step is written as a two-valued gate — "On green, return; on red, route per
the failure-routing rules" — but `draft-validate --concrete` reports three tool-state
outcomes, `ok`, `fail`, and `skip`. A step reported `skip` had its tool state checked
against nothing at all, yet the run prints `Concrete: OK` and the procedure's green branch
accepts it. The Mold never says which of the two branches a skip belongs to, that a
skipped step's state is unvalidated, or that some skips are permanent and cannot be
cleared by priming the cache. An iteration reading only this Mold would return green on a
draft whose collection-plumbing steps have never been validated, and no later phase would
know which steps those were.
expected: >-
Make the accept criterion three-valued in the validate step: `fail` routes per the
existing rules; `skip` is neither green nor red but an explicit hole — say that the
iteration must name each skipped step in its report and hand-review that step's `state`
against the wrapper, because nothing else will. Distinguish a skip that cache priming
would clear (tool simply not yet added) from one that priming cannot clear (the tool
cannot be decoded into the cache at all), and say that repeatedly re-priming the latter is
wasted work. Carry the list of skipped steps into the loop endstate so terminal validation
inherits it rather than rediscovering it.
evidence: >-
Iteration 4 of this run. `gxwf draft-validate <run>/galaxy-workflow-draft.gxwf.yml
--concrete --cache-dir <cache>` returned `Concrete: OK` with
`Tool state: 4 ok, 0 fail, 1 skip`, the skip being `0 (__FLATTEN__)
[skip_tool_not_found]`. That step's `state` has gone unvalidated for four consecutive
iterations and is accepted as green every time. Only a convention carried in this run's
own harness brief — not anything in the Mold bundle — told the iteration to treat the skip
as a permanent hole needing review by eye rather than as a cache-priming task to retry.
status: open
issue: null
- id: gxwf-linked-step-schema-rejects-the-keys-its-own-converter-emits
raised_by: advance-galaxy-draft-step
observed_in:
mold: advance-galaxy-draft-step
path: content/molds/advance-galaxy-draft-step/index.md
revision: 4
content_hash: c92d452f69a7b40bcab5fc8e63b03e1e2abf98f2a636abc4c93ad7f3c1d03082
foundry_head: 63a3f9cf9c97a637fe1628d298e524f35709e289
subject:
kind: related-project
label: "@galaxy-tool-util/cli — generated input_schemas.workflow_step_linked vs gxwf convert --to format2"
locator: https://github.com/jmchilton/galaxy-tool-util-ts
kind: defect
severity: major
what: >-
Three surfaces of the same CLI disagree about whether format2 `tool_state` may carry the
`__`-prefixed bookkeeping keys Galaxy writes into conditionals and repeats. For
`iuc/map_param_value/map_param_value` 0.2.0, `galaxy-tool-cache summarize` generates
`input_schemas.workflow_step_linked` with `additionalProperties: false` on every
conditional branch object (allowing only `type`, `input_param`, `mappings`) and on every
`mappings` item (allowing only `from`, `to`). That schema rejects `__current_case__` and
`__index__`. But `gxwf convert --to format2` of a real Galaxy workflow emits exactly those
keys, and `gxwf draft-validate --concrete` accepts a state block containing them. The
generated schema is therefore stricter than both the converter that produces format2 and
the validator that checks it — and it is the one surface a Mold is told to author against.
implement-galaxy-tool-step step 2 says to "shape the step's `state` against
`input_schemas.workflow_step_linked`"; following that literally produces a state block
that diverges from what Galaxy round-trips, and no gate reports the divergence, because
the validator is the permissive one.
expected: >-
Make the three surfaces agree, and say which one is normative. Either the generated
linked-step schema should admit the `__`-prefixed bookkeeping keys the converter emits
(`__current_case__`, `__index__`, and the top-level `__page__` /
`__rerun_remap_job_id__`), or `gxwf convert --to format2` should strip them and the
validator should reject them. Until then, document in the schema-generation output which
form is canonical for authoring, so a Mold binding a step against the published schema
gets the same answer as the validator and the converter.
evidence: >-
Iteration 6 of this run, gxwf 1.10.1, step `Get featureCounts strandedness parameter`.
Four state shapes were probed against `gxwf draft-validate <run>/galaxy-workflow-draft.gxwf.yml
--concrete --cache-dir <cache>`: with the bookkeeping keys, without them, with the
connected `input_param` omitted from the state block, and with the whole block moved from
`tool_state:` to `state:`. All four returned `Tool state: 6 ok, 0 fail, 1 skip` — the
validator discriminates none of them. `gxwf convert --to format2` of the corpus workflow
`transcriptomics/rnaseq-pe/rnaseq-pe` at IWC fe41a79 emits, for the same step,
`input_param_type: {type: text, __current_case__: 0, input_param: {__class__:
ConnectedValue}, mappings: [{__index__: 0, from: ..., to: ...}, ...]}` and
`unmapped: {on_unmapped: fail, __current_case__: 1}` — every key the generated schema
forbids. The round-trip form was taken as authoritative for this step.
status: open
issue: null
- id: shed-search-normalization-misses-abbreviation-expansion
raised_by: discover-shed-tool
observed_in:
mold: discover-shed-tool
path: content/molds/discover-shed-tool/index.md
revision: 5
content_hash: 05c99cbac2ec6a4d130b06204c15d0e76cf4fd0e00cdc8164dd606dac72809e2
foundry_head: 63a3f9cf9c97a637fe1628d298e524f35709e289
subject:
kind: mold
label: discover-shed-tool
locator: content/molds/discover-shed-tool/index.md
content_hash: 05c99cbac2ec6a4d130b06204c15d0e76cf4fd0e00cdc8164dd606dac72809e2
kind: gap
severity: major
what: >-
The procedure's query-normalization recipe for a tool-id-shaped need is to strip any
`owner/` prefix, split on `_` / `-` into space-separated words, and also try the bare
significant word — and it states that "a `miss` is only honest after the name variants
have been tried". Every variant that recipe generates fails for
`iuc/map_param_value/map_param_value`, a tool that is published, current, and pinned by
the IWC corpus. The recipe does not cover the one transform that works: expanding an
abbreviated word in the id to the word the human tool name actually uses. Galaxy tool ids
abbreviate routinely (`param`, `val`, `seq`, `align`, `qc`, `col`), so this is a recurring
shape, not a one-off. An iteration following the packaged recipe literally would have
declared `miss` and fallen through to author-galaxy-tool-wrapper — authoring a new wrapper
for a tool that already exists, which is the most expensive possible wrong answer this
Mold can produce.
expected: >-
Add abbreviation expansion to the §1 normalization list, with the common Galaxy
abbreviations spelled out, and make the honest-miss condition explicit that it includes
expanded variants. Better still, invert the last resort: before returning `miss`, search
the significant words with the id token dropped entirely (here `parameter value` alone
ranks the target first), and state that a `miss` is only honest when a name-shaped query
has been tried, not merely an id-shaped one.
evidence: >-
Iteration 6 of this run, gxwf 1.10.1, resolving the wrapper for step
`Get featureCounts strandedness parameter`. `gxwf tool-search` returned zero hits for
`map param value` (the recipe's underscore split), zero for the same query scoped
`--owner iuc`, and zero for the raw token `map_param_value`. The bare significant word
`map` returned only unrelated tools (`tdrmapper`, `multi_fasta_glimmerhmm`,
`glimmerhmm_predict`). Expanding `param` to `parameter` found it immediately and first:
`map parameter value` scores 42.1 at rank 1, and `parameter value` scores 29.7 at rank 1.
The tool's human name is "Map parameter value".
status: open
issue: null
- id: stock-tool-version-resolution-is-circular-in-the-builtin-branch
raised_by: advance-galaxy-draft-step
observed_in:
mold: advance-galaxy-draft-step
path: content/molds/advance-galaxy-draft-step/index.md
revision: 4
content_hash: c92d452f69a7b40bcab5fc8e63b03e1e2abf98f2a636abc4c93ad7f3c1d03082
foundry_head: 63a3f9cf9c97a637fe1628d298e524f35709e289
subject:
kind: mold
label: advance-galaxy-draft-step
locator: content/molds/advance-galaxy-draft-step/index.md
content_hash: c92d452f69a7b40bcab5fc8e63b03e1e2abf98f2a636abc4c93ad7f3c1d03082
kind: gap
severity: major
what: >-
Procedure step 2's built-in/stock branch offers exactly two ways to get a stock tool's
version — "read it from a populated cache via `galaxy-tool-cache list` or take a known pin
from the step plan" — and then forbids the only remaining move: "never hand-guess a stock
version". Both offered sources presuppose the version is already known. For a stock tool
meeting the run for the first time, with `tool_version: TODO` in the draft and no step-plan
pin, the cache is empty of it precisely because populating the cache requires an `add`
that takes `--tool-version`. The branch is circular, and its stated prohibition closes the
only exit. There is no third route to fall back on: the Tool Shed publishes no version
listing for a bare stock id at all.
expected: >-
Say that a bare-id probe is the sanctioned route for a stock tool and why it is not a
guess: `galaxy-tool-cache add <id> --tool-version <v>` either returns a summary whose own
`id` and `version` fields confirm the pin, or fails loudly with a 404 from
`/api/tools/<id>/versions/<v>`. There is no silent-wrong outcome, so the probe is
self-verifying and the prohibition should be narrowed to "never record an unconfirmed
version" rather than "never try one". Name the starting probe — Galaxy's built-in
collection operations ship at `1.0.0` — and require that the confirming summary be read
back before the version is written into the draft.
evidence: >-
Iteration 7 of phase 6 in this run, gxwf/galaxy-tool-cache 1.10.1, resolving
`__SAMPLE_SHEET_TO_TABULAR__` for step `Project sample sheet to tabular`. `galaxy-tool-cache
list` held only the two Tool Shed wrappers earlier iterations had cached. The step plan
pinned identity but not version (`tool_version: TODO`). Bare `add __SAMPLE_SHEET_TO_TABULAR__`
failed twice over — TRS versions 500, then `/versions/_default_` 404 — and reported
"Failed to fetch tool", which reads as "not available" rather than "version unresolved".
`add __SAMPLE_SHEET_TO_TABULAR__ --tool-version 1.0.0` succeeded immediately and returned
the full summary; `--tool-version 0.1.0` 404'd. `GET /api/tools/__SAMPLE_SHEET_TO_TABULAR__/versions`
(the non-TRS listing) returns "No route for" — checked directly, so no listing fallback
exists. The cached summary then let `draft-validate --concrete` validate the step's state
for real (7 ok, 0 fail, 1 skip), rather than skipping it.
status: open
issue: null
- id: tool-cache-records-a-shed-path-for-a-bare-stock-tool-id
raised_by: advance-galaxy-draft-step
observed_in:
mold: advance-galaxy-draft-step
path: content/molds/advance-galaxy-draft-step/index.md
revision: 4
content_hash: c92d452f69a7b40bcab5fc8e63b03e1e2abf98f2a636abc4c93ad7f3c1d03082
foundry_head: 63a3f9cf9c97a637fe1628d298e524f35709e289
subject:
kind: related-project
label: "galaxy-tool-util-ts / galaxy-tool-cache (@galaxy-tool-util/cli 1.10.1)"
locator: https://github.com/jmchilton/galaxy-tool-util-ts
kind: defect
severity: minor
what: >-
When `galaxy-tool-cache add` caches a stock Galaxy tool by its bare id, it writes the id
into `index.json` with a Tool Shed repository path glued on the front, producing
`toolshed.g2.bx.psu.edu/repos/__SAMPLE_SHEET_TO_TABULAR__` — an identifier that names
nothing: there is no such shed repository, and no workflow may legally carry that tool id.
The cached summary body alongside it is correct (`"id": "__SAMPLE_SHEET_TO_TABULAR__"`), so
the damage is confined to the index, but the index is what `galaxy-tool-cache list` prints.
That matters because `list` is the surface the advance-galaxy-draft-step procedure directs
an author to read a stock tool's version off, and what it shows there cannot be pasted into
a draft.
expected: >-
Preserve a bare stock tool id verbatim in the cache index, as the summary body already
does. A tool id with no `owner/repo` path is not a shed-relative name and should not have
`toolshed.g2.bx.psu.edu/repos/` prefixed to it.
evidence: >-
Iteration 7 of phase 6 in this run. After `galaxy-tool-cache add __SAMPLE_SHEET_TO_TABULAR__
--tool-version 1.0.0 --cache-dir <run-scratch>/gxwf-cache`, `index.json` records
`"tool_id": "toolshed.g2.bx.psu.edu/repos/__SAMPLE_SHEET_TO_TABULAR__"` against
`"source_url": "https://toolshed.g2.bx.psu.edu/api/tools/__SAMPLE_SHEET_TO_TABULAR__/versions/1.0.0"`,
while the summary file it points at carries `"id": "__SAMPLE_SHEET_TO_TABULAR__"`.
`galaxy-tool-cache list` prints the prefixed form. Lookup at validation time is evidently
keyed off the summary body rather than the index, since `draft-validate --concrete`
resolved the step against the cache and reported it ok.
status: open
issue: null
- id: collection-output-decode-is-a-flat-vs-nested-shape-mismatch-not-a-missing-field
raised_by: advance-galaxy-draft-step
observed_in:
mold: advance-galaxy-draft-step
path: content/molds/advance-galaxy-draft-step/index.md
revision: 4
content_hash: c92d452f69a7b40bcab5fc8e63b03e1e2abf98f2a636abc4c93ad7f3c1d03082
foundry_head: 63a3f9cf9c97a637fe1628d298e524f35709e289
subject:
kind: related-project
label: "@galaxy-tool-util/cli — ParsedTool collection-output decoding vs the Tool Shed tool API"
locator: https://github.com/jmchilton/galaxy-tool-util-ts
kind: defect
severity: major
what: >-
This CORRECTS the root cause and the scope recorded in
`gxwf-cannot-decode-collection-outputs-of-builtin-collection-operations`, which is filed against
the same decode failure. Two things in that entry are wrong.
(1) The collection output does NOT "carry no `structure`". The Tool Shed serializes a collection
output FLAT — `collection_type`, `collection_type_source`, `collection_type_from_rules`,
`structured_like` and `discover_datasets` all sit at the top level of the output object — while
the decoder expects exactly those five fields nested inside a `structure` object. Every field the
decoder wants is present in the payload; only the nesting differs. The correction that entry
proposes — make `structure` optional, or default it — would therefore decode cutadapt's
`out_pairs` with a NULL collection type when the API plainly said `"collection_type": "paired"`.
That is worse than the current loud failure: downstream shape reasoning would silently lose the
one fact the output exists to carry.
(2) It is not a built-in phenomenon. The trigger is a `<collection>` output, wherever it occurs.
`toolshed.g2.bx.psu.edu/repos/lparsons/cutadapt/cutadapt` — a mainstream IUC-maintained wrapper
pinned by eight workflows in IWC at fe41a79 — fails identically at every version tried, because
its paired-collection branch declares `<collection name="out_pairs" type="paired">`. So the
blast radius is not "the collection-operation built-ins a draft uses for plumbing"; it is every
tool with a collection output, which includes a large share of the wrappers real Galaxy
workflows are built from. Any such step is permanently unvalidatable by
`draft-validate --concrete`, and no cache priming can help.
expected: >-
Decode the flat form the Tool Shed actually emits: read `collection_type`,
`collection_type_source`, `collection_type_from_rules`, `structured_like` and
`discover_datasets` from the output object itself when no `structure` key is present, and
populate `structure` from them. Do not make `structure` optional or default it to nulls — that
discards `collection_type`, which is the only thing a caller needs the branch for. If the
nested form is the intended contract, then the Tool Shed's
`/api/tools/<trs-id>/versions/<v>` serializer is the side that must change, and the decoder
should say which shape it received rather than printing the expected type.
evidence: >-
Iteration 8 of phase 6 in this run, gxwf/galaxy-tool-cache 1.10.1, resolving the Cutadapt
wrapper for step `Quality-trim reads`. `galaxy-tool-cache add
toolshed.g2.bx.psu.edu/repos/lparsons/cutadapt/cutadapt --tool-version 5.2+galaxy2` fails with
`toolshed fetch failed ... for lparsons~cutadapt~cutadapt` and the same
`["outputs"][0] ... ["structure"] is missing` dump as `__FLATTEN__`; 5.2+galaxy0, 4.9+galaxy1
and 3.7+galaxy0 fail identically, so it is not version-specific. Fetching the same payload
directly — `GET https://toolshed.g2.bx.psu.edu/api/tools/lparsons~cutadapt~cutadapt/versions/5.2+galaxy2`,
200 — shows `outputs[0]` is `{"name": "out_pairs", "type": "collection", "collection_type":
"paired", "collection_type_source": null, "collection_type_from_rules": null,
"structured_like": null, "discover_datasets": ..., "hidden": ..., "label": ...}` with no
`structure` key and nothing missing from it. `outputs[1]` (`split_output`) has the same shape;
the thirteen `data` outputs decode fine. The eight IWC workflows at fe41a79 that pin this
wrapper all pin 5.2+galaxy2 at changeset f6168dd17f82. Net effect on this run:
`draft-validate --concrete` reports `7 ok, 0 fail, 2 skip`, both skips being collection-output
tools, and the Cutadapt step's tool state had to be bound by hand against the wrapper XML and a
`gxwf convert --to format2` round-trip of a corpus `.ga`.
status: open
issue: null
- id: tool-search-default-page-returns-every-hit-three-times
raised_by: advance-galaxy-draft-step
observed_in:
mold: advance-galaxy-draft-step
path: content/molds/advance-galaxy-draft-step/index.md
revision: 4
content_hash: c92d452f69a7b40bcab5fc8e63b03e1e2abf98f2a636abc4c93ad7f3c1d03082
foundry_head: 63a3f9cf9c97a637fe1628d298e524f35709e289
subject:
kind: related-project
label: "@galaxy-tool-util/cli — gxwf tool-search result de-duplication"
locator: https://github.com/jmchilton/galaxy-tool-util-ts
kind: defect
severity: minor
what: >-
With no `--max-results`, `gxwf tool-search` returns every hit three times — in the table
rendering and in `--json` alike. Passing any explicit `--max-results` returns distinct hits, so
the duplication is confined to the default page size. It is not cosmetic for a discovery
procedure that triages by counting and comparing candidates: the default view of a search makes
a field of twenty wrappers look like sixty, and the triage rule "multiple plausible hits ... →
weak" reads a repeated single candidate as a cluster.
expected: >-
De-duplicate on `(repoName, repoOwnerUsername, toolId)` before returning, on the default page
size as well as an explicit one — or, if the repetition encodes distinct revisions, surface the
revision that distinguishes the rows instead of emitting rows that are byte-identical.
evidence: >-
Iteration 8 of phase 6 in this run, gxwf 1.10.1. `gxwf tool-search "cutadapt" --json` returns 50
`trsToolId` values of which 20 are distinct, each repeated three times; the table rendering of
the same query shows the same triples. `--max-results 5`, `--max-results 10` and
`--max-results 20` each return exactly N values, all distinct.
status: open
issue: null
- id: implement-step-mold-has-no-binding-path-when-no-tool-summary-exists
raised_by: advance-galaxy-draft-step
observed_in:
mold: advance-galaxy-draft-step
path: content/molds/advance-galaxy-draft-step/index.md
revision: 4
content_hash: c92d452f69a7b40bcab5fc8e63b03e1e2abf98f2a636abc4c93ad7f3c1d03082
foundry_head: 63a3f9cf9c97a637fe1628d298e524f35709e289
subject:
kind: mold
label: implement-galaxy-tool-step
locator: content/molds/implement-galaxy-tool-step/index.md
content_hash: 197be2d9b27741c3f258092f8392654369d1ca234d7fad492387ff69d656292b
kind: gap
severity: major
what: >-
Both Molds assume a tool summary always exists. advance-galaxy-draft-step's sequence step 3 is
an unconditional "Invoke summarize-galaxy-tool on the resolved wrapper", and
implement-galaxy-tool-step's sequence step 2 is an unconditional "Read the galaxy-tool-summary
manifest". Neither says what to do when the summary cannot be produced at all. That is not a
hypothetical: summarize-galaxy-tool can only summarize what `galaxy-tool-cache add` could
decode, and a wrapper with a collection output cannot be decoded (see
`collection-output-decode-is-a-flat-vs-nested-shape-mismatch-not-a-missing-field`). The Mold's
one nearby escape hatch does not cover it either — "If `input_schemas` is `null`, consult
`warnings[]`" presupposes a manifest with a warnings array, and here there is no manifest.
The result is an author improvising the most consequential part of a step, its tool state, with
no stated evidence standard, on the first genuinely complex wrapper of the run.
expected: >-
Give implement-galaxy-tool-step an explicit no-summary branch that names the fallback evidence
in priority order and requires the step to record which one it used: (1) the wrapper XML at the
pinned changeset, which is authoritative for parameter names, defaults and output filters;
(2) `gxwf convert --to format2` over a corpus `.ga` that pins the same version, which is
authoritative for the round-trip state shape; (3) nothing else. summarize-galaxy-tool already
sanctions raw XML as supporting evidence ("Optional raw XML source for ambiguity checks"), so
the material exists — it is the implement Mold that never mentions it. The branch should also
require the step to carry a visible marker that its state was hand-bound and is therefore
unchecked by `draft-validate`, since the verdict line will show it as a skip and a skip is
indistinguishable from an absent step in the counts.
evidence: >-
Iteration 8 of phase 6 in this run, step `Quality-trim reads`, wrapper
`lparsons/cutadapt/cutadapt` 5.2+galaxy2 at changeset f6168dd17f82. `galaxy-tool-cache add`
failed at four different versions, so summarize-galaxy-tool could not run and no
`galaxy-tool-summary.json` was ever produced. The step's `tool_state` — a `library`
conditional at `__current_case__: 2` with six adapter repeats, plus `output_selector`, which
gates the promoted `report` output through an XML `<filter>` — was bound entirely by hand from
the tools-iuc wrapper XML at the pinned version and from `gxwf convert --to format2` over
`epigenetics/cutandrun/cutandrun.ga` and
`VGP-assembly-v2/post-curation-processing/Post_Curation.ga` at fe41a79. That route was
inferred, not instructed. `draft-validate --concrete` then reported `7 ok, 0 fail, 2 skip` with
this step among the skips, so nothing in the run checks the binding.
status: open
issue: null
- id: discovery-schema-cannot-express-an-unresolved-alternate
raised_by: discover-shed-tool
observed_in:
mold: discover-shed-tool
path: content/molds/discover-shed-tool/index.md
revision: 5
content_hash: 05c99cbac2ec6a4d130b06204c15d0e76cf4fd0e00cdc8164dd606dac72809e2
foundry_head: 63a3f9cf9c97a637fe1628d298e524f35709e289
subject:
kind: schema
label: galaxy-tool-discovery
locator: package://@galaxy-foundry/gxwf-foundry#galaxyToolDiscoverySchema
content_hash: c0940fc0265e8fd25955f4380f03e088b62d25155fe98e7f051a5a3957d2840c
kind: gap
severity: minor
what: >-
`alternates[]` reuses the full `ToolCandidate` shape, which requires `version` and
`changeset_revision` (both `minLength: 1`) plus a numeric `score`. There is no way to record
a candidate the discovery deliberately did NOT pin. The procedure asks for exactly that —
"multiple plausible hits ... → `weak` with the leading candidate plus alternates", and the
surrounding Molds hand this skill named alternatives in a step's `_plan_context` — but an
alternate is only representable after it has been fully resolved to a changeset. So an author
recording a runner-up faces two bad options: spend Tool Shed calls pinning a wrapper being
rejected, or write placeholder strings. The second validates green. `"changeset_revision":
"unresolved"` and `"score": 0` pass `validate-galaxy-tool-discovery` without a murmur, in a
field the schema itself documents as "Selected Tool Shed Mercurial changeset revision for
reproducible gxformat2 tool_shed_repository pinning" and one documented as "Higher is better".
A machine-read pin contract should not be able to carry a fabricated pin.
expected: >-
Let an alternate be unresolved. Either make `version` and `changeset_revision` nullable on
`ToolCandidate` and require them non-null only for the selected `candidate` (the existing
`allOf` if/then on `status` is already the place to say so), or split a lighter
`AlternateCandidate` shape carrying identity plus rationale and no pin fields. Add a
`match_fields` enum value for a candidate surfaced from an upstream plan hint rather than from
the lexical index — `matched_terms` already anticipates this case ("Empty only when the
candidate came from a non-lexical hint") but `match_fields` has no corresponding value, so a
plan-hint alternate has to claim lexical evidence it does not have or leave the array empty.
evidence: >-
Iteration 9 of phase 6 in this run, step `Read quality report`. The step was Deferred and its
`_plan_context` named `iuc/falco` as the corpus-observed alternative to `devteam/fastqc`,
asking that it be recorded if not taken. Recording it cost three extra Tool Shed calls
(`tool-search falco`, `tool-versions`, `tool-revisions`) for a wrapper being rejected, and the
score needed its own query because `iuc~falco~falco` does not appear in the `fastqc` result set
at all. The first artifact written instead used `"changeset_revision": "unresolved"` and
`"score": 0`; `foundry validate-galaxy-tool-discovery` reported `galaxy-tool-pin.json: valid`.
It was corrected by hand afterwards, not by the gate. Notably the schema ref's own
`verification` line in the cast provenance is "Run discover-shed-tool against known FastQC,
ambiguous BWA-style, and no-hit queries and validate each emitted recommendation" — this is
that FastQC query, and the alternates path is what it does not exercise.
status: open
issue: null
- id: no-authority-for-scalar-value-encoding-in-tool-state
raised_by: advance-galaxy-draft-step
observed_in:
mold: advance-galaxy-draft-step
path: content/molds/advance-galaxy-draft-step/index.md
revision: 4
content_hash: c92d452f69a7b40bcab5fc8e63b03e1e2abf98f2a636abc4c93ad7f3c1d03082
foundry_head: 63a3f9cf9c97a637fe1628d298e524f35709e289
subject:
kind: mold
label: implement-galaxy-tool-step
locator: content/molds/implement-galaxy-tool-step/index.md
content_hash: 197be2d9b27741c3f258092f8392654369d1ca234d7fad492387ff69d656292b
kind: gap
severity: minor
what: >-
Nothing the Mold names settles how a scalar parameter's VALUE is encoded in the step's
state block, and the two authorities an author can reach disagree. Procedure step 2 sends
the author to `parsed_tool` for ports and datatypes and to
`input_schemas.workflow_step_linked` for the state shape. For `Filter1`'s `header_lines`,
`parsed_tool` declares `parameter_type: gx_integer`, `type: integer`, `value: 0` — read
literally that says write the YAML integer `0`. Every real Galaxy workflow writes the
string `'0'`. This is a different axis from the two disagreements already filed here:
`implement-step-mold-says-state-without-saying-which-key` is about WHICH block key, and
`gxwf-linked-step-schema-rejects-the-keys-its-own-converter-emits` is about `__`-prefixed
bookkeeping keys. This one is about the scalar leaf value, and it is unaddressed by either
correction. The Mold's guidance is not merely silent — following the one authority it
names produces the form the corpus never uses.
expected: >-
Say in procedure step 2 that a wrapper's declared parameter type fixes the SEMANTICS of a
state value, not its serialization, and that the authoring form for scalar leaves is what
`gxwf convert --to format2` emits from a real workflow — strings for integer, float, and
select parameters alike, because Galaxy's `tool_state` is JSON-string-encoded at the
source. That is the same authority the run already had to fall back on for conditional
and repeat bookkeeping, so naming it once covers all three axes. If the Foundry would
rather not carry that rule, say instead that either encoding is accepted and that the
choice is a consistency matter within a draft — but say which, because an author who
reads only `parsed_tool` currently has no way to find out and no gate that will tell them.
evidence: >-
Phase 6 iteration 10 of this run, step `Select first-contrast samples` (`Filter1` 1.1.1).
The cached summary declares `header_lines` as `gx_integer` with `value: 0`. A survey of
all 21 `Filter1` states in the pinned IWC corpus found the value serialized as a string in
every one ('0' x14, '1' x7) and as an integer in none; `gxwf convert --to format2` of
`transcriptomics/rnaseq-de/rnaseq-de-filtering-plotting` likewise emits `header_lines: "1"`.
`gxwf draft-validate --concrete` was then run twice over otherwise byte-identical drafts,
once with `header_lines: '0'` and once with `header_lines: 0`, against a cache holding the
real `Filter1` summary. Both returned `draft valid` / `Concrete: OK` /
`Tool state: 9 ok, 0 fail, 2 skip` — identical verdicts, so the gate decides nothing here
and an author cannot resolve the question by trying it. The binding was settled by corpus
frequency alone, which is exactly the guess the Mold should have removed.
status: open
issue: null
- id: tool-summary-whens-order-is-the-current-case-index-and-nothing-says-so
raised_by: advance-galaxy-draft-step
observed_in:
mold: advance-galaxy-draft-step
path: content/molds/advance-galaxy-draft-step/index.md
revision: 4
content_hash: c92d452f69a7b40bcab5fc8e63b03e1e2abf98f2a636abc4c93ad7f3c1d03082
foundry_head: 63a3f9cf9c97a637fe1628d298e524f35709e289
subject:
kind: schema
label: galaxy-tool-summary conditional whens
locator: package://@galaxy-foundry/gxwf-foundry#galaxyToolSummarySchema
content_hash: bf77d0735410fa0028bc9bea08edaff3016580508bf27851fdcae65d038f2d94
kind: gap
severity: major
what: >-
A conditional's `__current_case__` is an INDEX, and nothing in the packaged summary
schema, the Mold, or any packaged note says what it indexes. The summary gives each
conditional a `test_parameter.options` array and a `whens` array with a `discriminator`
per element. The index that `__current_case__` must carry is the position in `whens`
(i.e. `<when>` document order), NOT the position of the matching value in `options`. The
two are not interchangeable and this run hit a live divergence: in
iuc/rgrnastar/rna_star 2.7.11b+galaxy1, `refGenomeSource[history].GTFconditional` lists
its options as `without-gtf, with-gtf` but its whens as `with-gtf, without-gtf`, so
`with-gtf` is case 0 while the dropdown shows it second. An author reading the field an
author would naturally read gets 1. The bundle offers no way to know which array is
authoritative, so the rule had to be re-established from outside the bundle -- by
round-tripping a corpus `.ga` that happens to use the same tool and reading the wrapper
XML's `<when>` order to confirm the correspondence.
expected: >-
State once, in the summary schema's own description of `whens` (and echo it in
implement-galaxy-tool-step's state-authoring guidance), that `whens` is emitted in
`<when>` document order and that a conditional's `__current_case__` is the zero-based
index into that array -- explicitly warning that it may differ from the option order,
with a one-line example. Better still, emit the index on each `whens` element so the
author never has to count, which also makes the contract checkable rather than
conventional. This is a third axis of the same family already filed here:
`implement-step-mold-says-state-without-saying-which-key` is about which block key,
`no-authority-for-scalar-value-encoding-in-tool-state` is about the scalar leaf value,
and this one is about the bookkeeping key's VALUE. The `__index__` key on repeat entries
has the same problem and the same fix.
status: open
issue: null
- id: tool-summary-drops-output-filter-expressions
raised_by: advance-galaxy-draft-step
observed_in:
mold: advance-galaxy-draft-step
path: content/molds/advance-galaxy-draft-step/index.md
revision: 4
content_hash: c92d452f69a7b40bcab5fc8e63b03e1e2abf98f2a636abc4c93ad7f3c1d03082
foundry_head: 63a3f9cf9c97a637fe1628d298e524f35709e289
subject:
kind: schema
label: galaxy-tool-summary parsed_tool.outputs
locator: package://@galaxy-foundry/gxwf-foundry#galaxyToolSummarySchema
content_hash: bf77d0735410fa0028bc9bea08edaff3016580508bf27851fdcae65d038f2d94
kind: gap
severity: major
what: >-
`parsed_tool.outputs` carries name, label, hidden, type, format, format_source,
metadata_source, discover_datasets, from_work_dir and precreate_directory -- but not the
output's `<filter>` expression. A Galaxy output filter is what decides whether a declared
output EXISTS for a given parameter branch, so the summary can enumerate ten outputs for
a tool that will produce four, with nothing marking the difference. That is not cosmetic
for this Mold: a step's `out:` list and every workflow output wired to it are only valid
if the chosen tool state keeps those outputs alive, and the summary is the artifact
procedure step 3 produces expressly so step 4 can bind the step against it. Concretely,
this run's STAR step had to establish that `output_log` and `mapped_reads` survive
`quantMode: '-'` and `outWigType: None`, because its plan makes a suppressed `output_log`
a hard failure -- and the summary cannot answer that question at all. The answer
(`reads_per_gene` and `transcriptome_mapped_reads` are filtered on quantMode; the two
promoted outputs carry no filter) came only from reading the wrapper XML. Iteration 8 hit
the same wall on lparsons/cutadapt, where the `report` and `out_pairs` filters are what
keep two promoted outputs alive.
expected: >-
Carry the raw `<filter>` expression string on each output in `parsed_tool.outputs` (a
nullable `filter` field is enough; the Mold does not need it evaluated, only visible),
and have implement-galaxy-tool-step's procedure say that before finalizing a step's
`out:` list the author must check each promoted output's filter against the state just
bound. Without the field the correct instruction is unfollowable, because the evidence is
not in the artifact the procedure hands the author.
status: open
issue: null
- id: no-reachable-wrapper-xml-at-a-pinned-changeset
raised_by: advance-galaxy-draft-step
observed_in:
mold: advance-galaxy-draft-step
path: content/molds/advance-galaxy-draft-step/index.md
revision: 4
content_hash: c92d452f69a7b40bcab5fc8e63b03e1e2abf98f2a636abc4c93ad7f3c1d03082
foundry_head: 63a3f9cf9c97a637fe1628d298e524f35709e289
subject:
kind: related-project
label: "Galaxy Tool Shed + galaxy-tool-cache: raw wrapper source retrieval"
locator: https://toolshed.g2.bx.psu.edu
kind: gap
severity: major
what: >-
Several authoring paths depend on reading a wrapper's XML at the pinned changeset -- the
correction already filed as `implement-step-mold-has-no-binding-path-when-no-tool-summary-exists`
names it as fallback evidence "authoritative for parameter names, defaults and output
filters", and it is the only source for output filters at all (see
`tool-summary-drops-output-filter-expressions`). No declared tool can fetch it. The
summary manifest sets `artifacts.raw_tool_source_path: null` for a toolshed-sourced tool;
the Tool Shed's `repos/<owner>/<repo>/raw-file/<changeset>/<path>` endpoint answers 403;
and `/api/tools/<trs-id>/versions/<v>/raw_tool_source` answers 404 with "No route". The
only thing that worked this run was guessing the upstream repository layout from the
repository record's `remote_repository_url` and fetching the file from GitHub at the
default branch -- where the filename was `rg_rnaStar.xml`, not the `<repo>/<tool_id>.xml`
the shed path implies, so the first two guesses 404'd. That fetch is also UNPINNED: it
happened to match the pinned version here only because tools-iuc main still carried
@TOOL_VERSION@ 2.7.11b / @VERSION_SUFFIX@ 1, which a later iteration on a lagging wrapper
cannot count on.
expected: >-
Give `galaxy-tool-cache add` an option to retain the raw tool source it already
downloads, and populate `artifacts.raw_tool_source_path` for toolshed sources rather than
nulling it -- the bytes are in hand at fetch time, so this is retention, not new
retrieval. Failing that, the Tool Shed should expose a supported raw-source route per
(tool id, version) so the pin and the source agree. Until one of those exists, any Mold
instruction of the form "bind against the wrapper XML at the pinned version" names an
artifact the run has no sanctioned way to obtain, and authors will keep reaching an
unpinned copy by guesswork.
status: open
issue: null
- id: draft-format-sentinel-hint-cannot-express-a-conditional-nested-port
raised_by: advance-galaxy-draft-step
observed_in:
mold: advance-galaxy-draft-step
path: content/molds/advance-galaxy-draft-step/index.md
revision: 4
content_hash: c92d452f69a7b40bcab5fc8e63b03e1e2abf98f2a636abc4c93ad7f3c1d03082
foundry_head: 63a3f9cf9c97a637fe1628d298e524f35709e289
subject:
kind: research
label: galaxy-workflow-draft-format
locator: content/research/galaxy-workflow-draft-format/index.md
content_hash: 93f3b231dd063bf059c550f18b2200c61909af0da9408e0d43351116e886bf7b
kind: gap
severity: major
what: >-
The `TODO_<port>` input sentinel carries the port's semantic hint as a FLAT identifier, so
it cannot express a port that lives inside a conditional -- and it reads as though no
qualification were needed. `__FILTER_FROM_FILE__` has one top-level `input` and a
`filter_source` that exists only inside the `how` conditional, but the template wrote both
as siblings, `TODO_input` and `TODO_filter_source`. The concrete `in:` key for the second
is `how|filter_source`; a literal reading of the sentinel yields a bare `filter_source:`,
which connects nothing. `gxwf draft-next-step` then restates the flat hint verbatim --
"assign the real wrapper input port name (semantic hint: 'filter_source')" -- so the loop's
own work list repeats the wrong shape at the moment the author acts on it. The note does
define qualified keys elsewhere (concrete steps in the same draft carry
`select_data|rep_factorName_0|rep_factorLevel_0|countsFile`), so the shape is expressible;
what is missing is any statement that the sentinel's hint is a NAME rather than a PATH,
and that concretion may have to qualify it.
expected: >-
State in the sentinel section that a `TODO_<port>` hint names a parameter, not its address,
and that the concrete `in:` key must be the tool's full parameter path -- `|`-qualified
through every enclosing conditional and repeat. Better still, let the sentinel carry the
path it already knows when the template resolves a nested port, so the hint and the answer
have the same shape. A worked conditional-nested example next to the existing flat one
would carry the rule by demonstration, as the map-form/list-form correction did.
evidence: >-
Established from the raw TRS payload `GET /api/tools/__FILTER_FROM_FILE__/versions/1.1.0`
(200), where `filter_source` appears only under `how.whens[*].parameters` and never at the
top level, and from `mgnify-amplicon-pipeline-v5-rrna-prediction` at IWC fe41a79 through
`gxwf convert --to format2`, which writes `- id: how|filter_source`. The cost of the gap is
that NOTHING catches the literal reading: this tool's outputs are collections, so it fails
the cache decode and reports `skip_tool_not_found`, and a probe run of the same draft with
the unqualified `filter_source:` key returned a byte-identical verdict to the correct one
-- `draft valid`, `Concrete: OK`, `Tool state: 16 ok, 0 fail, 3 skip`. A silently
disconnected input survives every gate in the run.
status: open
issue: null
- id: no-evidence-route-for-an-output-content-property-the-declaration-cannot-carry
raised_by: advance-galaxy-draft-step
observed_in:
mold: advance-galaxy-draft-step
path: content/molds/advance-galaxy-draft-step/index.md
revision: 4
content_hash: c92d452f69a7b40bcab5fc8e63b03e1e2abf98f2a636abc4c93ad7f3c1d03082
foundry_head: 63a3f9cf9c97a637fe1628d298e524f35709e289
subject:
kind: mold
label: implement-galaxy-tool-step
locator: content/molds/implement-galaxy-tool-step/index.md
content_hash: 197be2d9b27741c3f258092f8392654369d1ca234d7fad492387ff69d656292b
kind: gap
severity: major
what: >-
Some step bindings depend on a property of an UPSTREAM output's CONTENT -- does this
tabular output carry a header row, is the element identifier emitted as a column -- and no
artifact the Mold names can answer that class of question. `parsed_tool.outputs` carries
name, label, type, format and discovery rules; it carries nothing about the rows inside the
file, and it never could, because the property is a fact about the wrapper's script rather
than about its declaration. The Mold has no instruction for the case, so the author either
guesses or invents a method. This is not the same wall as
`implement-step-mold-has-no-binding-path-when-no-tool-summary-exists`: there the summary is
merely absent, and the fallback list that entry proposes -- wrapper XML at the pinned
changeset, then a corpus round-trip, then "nothing else" -- would still not answer this,
because the XML's `<param>`/`<data>` declarations are silent on it and the round-trip
actively misleads. The cost is silent: a wrong `header_lines` drops the first data row of
every filtered table, or passes a header into a numeric comparison, and either way the
workflow emits a plausible result.
expected: >-
Add to the binding procedure a named evidence route for output CONTENT properties, distinct
from the one for parameter shape, and require the step to record which source settled it:
the wrapper's `<test>` blocks -- `assert_contents` on the output in question is a positive
statement about what the file actually contains, and the absence of a header assertion on
one output beside its presence on a sibling is itself evidence -- and, when the wrapper is
open-source, its script's write call. Say explicitly that a corpus round-trip is NOT
authority for this class: a corpus workflow's binding is evidence about the table IT filters,
which may be several steps removed from the producer, and copying it across is exactly the
error this route exists to prevent. Related but separable: this run's phase-5 Mold recorded
the question as "closable by a summarize-galaxy-tool pass", which is false for the same
reason -- the pass it names cannot see the property.
evidence: >-
Phase 6 iteration 21 of this run, closing open-requirement
`deseq2-result-table-header-presence-unverified` for `iuc/deseq2/deseq2` @ 2.11.40.8+galaxy4
(changeset 05f9e54d7e81), which gates `header_lines` on the four downstream `Filter1` steps.
The tool summary route was unavailable at all (the cache add fails on the collection-output
decode). What settled it was outside every sanctioned source: `deseq2.R` writes the result
table with `write.table(..., col.names = FALSE)` while writing the normalized counts one
screen up with `col.names = NA` -- one tool, opposite answers for two tabular outputs, so no
tool-level generalization is available either; and `deseq2.xml`'s test asserts `deseq_out`
content starting at a data row with `has_n_lines`, while the `vst_out` and `counts_out`
assertions in the same test block DO assert a sample-name header line. The round-trip is the
trap: `gxwf convert --to format2` over IWC
`transcriptomics/rnaseq-de/rnaseq-de-filtering-plotting` at fe41a79 shows BOTH its `Filter1`
steps binding `header_lines: "1"`, which reads as a direct answer and is not one -- that
workflow manufactures a header with `tp_text_file_with_recurring_lines` + `tp_sed_tool` and
concatenates it onto the DESeq2 output with `tp_cat` before filtering. Copying its binding
into a workflow that filters `deseq_out` directly would have silently discarded the most
significant gene from every result table.
status: open
issue: null
- id: draft-format-cannot-express-a-cross-step-invariant
raised_by: advance-galaxy-draft-step
observed_in:
mold: advance-galaxy-draft-step
path: content/molds/advance-galaxy-draft-step/index.md
revision: 4
content_hash: c92d452f69a7b40bcab5fc8e63b03e1e2abf98f2a636abc4c93ad7f3c1d03082
foundry_head: 63a3f9cf9c97a637fe1628d298e524f35709e289
subject:
kind: research
label: galaxy-workflow-draft-format
locator: content/research/galaxy-workflow-draft-format/index.md
content_hash: 93f3b231dd063bf059c550f18b2200c61909af0da9408e0d43351116e886bf7b
kind: gap
severity: major
what: >-
Every annotation the draft format offers is scoped to ONE step (`TODO_*`, `_plan_*`, the
tier vocabulary), so an invariant that binds two or more steps has nowhere to live except
prose inside one of them -- and `advance-galaxy-draft-step` advances exactly one step per
invocation, so it is also structurally unable to check one. The step's own `_plan_state`
said it outright: "Bind both steps together so the factor name, level ordering, header flag
and output_selector cannot drift apart between the two contrasts." That is an instruction
with no mechanism behind it. The author is the only enforcement, and only if they happen to
read a sibling step's plan prose while implementing a different step.
expected: >-
Give the format a typed, workflow-level way to declare that named steps must agree on named
state paths -- a sibling of the `comments:` frame, or a `_plan_invariant` block naming the
step ids and the `|`-qualified paths -- and have `draft-validate` check it once every named
step is concrete. It is a cheap check: the paths are already addressable, and this run's
loop concretized the two coupled steps 1 iteration apart. Failing that, state in the note
that cross-step couplings are out of scope for the draft and must be carried as
open-requirements entries, so a Mold run stops inventing its own mechanism.
evidence: >-
Two couplings in one workflow, both invisible to every gate. (1) The two DESeq2 nodes must
stay key-for-key identical apart from one counts collection and one factor level; this was
settled only by hand-diffing the two steps' parsed `in:`/`out:`/`tool_state` trees (5 `in:`
keys, 3 `out:` ids, 27 state leaves). (2) Worse, `advanced_options.lfc_shrinkage_type: none`
on BOTH DESeq2 nodes determines the result table's column count, and two DIFFERENT steps
bake the resulting indices as opaque string literals -- `"c7<"` and `"abs(c3)>"` in two
`compose_text_param` bridges. Any other shrinkage value drops the `stat` column, moves padj
from c7 to c6, and `"c7<"` then names a column that does not exist. A three-step coupling
expressed as a magic number in a text field. Both DESeq2 steps additionally report
`skip_tool_not_found` (their `split_output` collection output fails the cache decode), so
not even the tool-state gate looks at them.
status: open
issue: null
- id: a-required-port-missing-from-in-carries-no-sentinel-and-no-gate
raised_by: advance-galaxy-draft-step
observed_in:
mold: advance-galaxy-draft-step
path: content/molds/advance-galaxy-draft-step/index.md
revision: 4
content_hash: c92d452f69a7b40bcab5fc8e63b03e1e2abf98f2a636abc4c93ad7f3c1d03082
foundry_head: 63a3f9cf9c97a637fe1628d298e524f35709e289
subject:
kind: research
label: galaxy-workflow-draft-format
locator: content/research/galaxy-workflow-draft-format/index.md
content_hash: 93f3b231dd063bf059c550f18b2200c61909af0da9408e0d43351116e886bf7b
kind: gap
severity: major
what: >-
A `TODO_<port>` sentinel marks a port whose NAME is unresolved, but there is no way to mark
a port that is simply ABSENT from `in:` -- and an absent port is invisible to
`gxwf draft-next-step`, whose `work` list enumerates sentinels and `_plan_*` fields and so
reports nothing at all. The obligation survives only as prose. Distinct from
`draft-format-sentinel-hint-cannot-express-a-conditional-nested-port`, which is about a
sentinel whose hint has the wrong SHAPE; here there is no sentinel to misread.
expected: >-
Require in the note that every port the step's plan calls for appears in `in:` -- as a
`TODO_<port>` sentinel while unresolved -- so that "the plan named it" and "the work list
reports it" cannot come apart. A cheap partial check for `draft-next-step`: when a step's
`_plan_*` prose names a sibling step as its model, flag an `in:` key set that is a strict
subset of that sibling's.
evidence: >-
`Differential expression: second contrast vs reference` declared three `in:` keys; the
already-concrete sibling it was told to copy declared five. The two missing ones were
`select_data|rep_factorName_0|rep_factorLevel_{0,1}|factorLevel`. `draft-next-step` reported
only `TODO[tool_version]` plus four `_plan_*` strings; the sole trace of the two missing
connections was the sentence "As for the first contrast, with `Counts for the second
contrast level` at level index 0". Left unwired, both factor levels would fall back to the
wrapper's empty default -- blank contrast titles on the diagnostic plots, and the silent
re-opening of resolved entry `deseq2-factor-level-names-not-parameterized`, whose closure
asserts that no level literal is baked into any step. Nothing would have failed: this run
already measured that a disconnected input validates byte-identically green, and this step
reports `skip_tool_not_found` besides.
status: open
issue: null
- id: ledger-resolved-entries-are-never-read-back-at-binding-time
raised_by: advance-galaxy-draft-step
observed_in:
mold: advance-galaxy-draft-step
path: content/molds/advance-galaxy-draft-step/index.md
revision: 4
content_hash: c92d452f69a7b40bcab5fc8e63b03e1e2abf98f2a636abc4c93ad7f3c1d03082
foundry_head: 63a3f9cf9c97a637fe1628d298e524f35709e289
subject:
kind: mold
label: advance-galaxy-draft-step
locator: content/molds/advance-galaxy-draft-step/index.md
content_hash: c92d452f69a7b40bcab5fc8e63b03e1e2abf98f2a636abc4c93ad7f3c1d03082
kind: gap
severity: major
what: >-
The open-requirements ledger is the only artifact that carries a decision from one
iteration of the per-step loop to a later one, but the Mold never reads it as an INPUT to
binding. Its `Inputs` section declares the ledger carries "the run's open, resolved, and
surrendered entries", and then the procedure's single ledger touchpoint is step 5, AFTER
implement: "Inspect the open-requirements-ledger for a new `open` blocking entry ...
appended against this step." New, open, post-hoc. A decision settled five iterations
earlier lives in a `resolved` entry -- whose `note` field is where the ledger note itself
says the closure reasoning goes -- and no procedure step ever routes an author back to it
before they bind state. The packaged `open-requirements-ledger` note does not close the
gap either: its "how downstream reads it" paragraph covers only
repair-galaxy-draft-topology reading OPEN blocking entries, and its one line about
resolved entries ("Resolving is not deleting -- a resolved entry stays in the ledger as
the audit trail") frames them as provenance, not as an input.
expected: >-
Add a read to the procedure BEFORE implement: select the ledger entries whose `step` names
the step being concretized -- regardless of status -- and treat a `resolved` entry's
`note` as a binding already settled for this step, to be copied rather than re-derived.
Entries carry a `step` field precisely so this lookup is a filter, not a search. Say
explicitly that a resolved entry is authority at binding time and not merely an audit
trail, and say the converse too: a Mold that settles a binding for a step it is not
currently implementing must record it in that entry's `note`, because the draft itself has
nowhere to put it (the sibling steps' `_plan_state` prose is deleted at their own
concretion).
evidence: >-
This iteration concretized `Filter with p-adj threshold: first contrast`. Its whole
remaining decision was `header_lines`, and the answer -- `'0'`, because the pinned
iuc/deseq2 wrapper writes `deseq_out` with `col.names = FALSE` -- had been established
two iterations earlier and recorded ONLY in the `note` of the now-`resolved` entry
`deseq2-result-table-header-presence-unverified`. Following the procedure literally, that
note is never opened: step 5 filters for new open entries and runs after the binding is
already written. The default an author reaches for instead is the corpus, and the corpus
is wrong here -- `rnaseq-de-filtering-plotting` binds `header_lines: "1"` because it
MANUFACTURES a header (tp_text_file_with_recurring_lines -> tp_sed -> tp_cat) before
filtering, which this workflow omits. Binding "1" against the headerless `deseq_out`
discards row 1 of a table sorted by padj: the most significant gene, which for this
paper is *SCF1*, the finding the workflow exists to reproduce. No error, no warning, and
`draft-validate --concrete` returns identical green for '0' and '1'. Three more steps
(the remaining significance filters) must copy that same value, each from an iteration
that will not be told to look.
status: open
issue: null
- id: draft-extract-destroys-yaml-comments
raised_by: advance-galaxy-draft-step
observed_in:
mold: advance-galaxy-draft-step
path: content/molds/advance-galaxy-draft-step/index.md
revision: 4
content_hash: c92d452f69a7b40bcab5fc8e63b03e1e2abf98f2a636abc4c93ad7f3c1d03082
foundry_head: 63a3f9cf9c97a637fe1628d298e524f35709e289
subject:
kind: cli-command
label: gxwf draft-extract
locator: content/cli/gxwf/draft-extract.md
content_hash: f8e9350c96ce005ed5e38deefff6c993824c951784f17f79f7e60d1ad664d8d7
kind: gap
severity: major
what: >-
`gxwf draft-extract` re-serializes the workflow from a parsed data model, so every YAML
`#` comment in the draft is destroyed. The note describes the command as three subtractive
operations — drop drafty steps, strip `_plan_*`, promote `class` — and its Output and
Gotchas sections say nothing about comments. Nothing in this Mold's bundle does either.
The loss is total and silent: it is not reported in `--report-json` (which counts only
dropped steps, dropped outputs and rewritten inputs) and the extracted file validates
clean, so no gate in the pipeline can see it. It matters because a per-step draft loop has
nowhere else to put the reasoning: `_plan_*` fields MUST be deleted at concretion or
`draft-next-step` re-selects the step forever, and the gxformat2 `doc:` field is
user-facing prose, not a place for a corpus citation or a cross-step warning. A run that
parks that material in comments above each step — as this one did, and as the concrete
steps of a template naturally invite — loses all of it at the exact moment the artifact
becomes the one downstream Molds consume. The two surfaces that DO survive are `doc:` and
the gxformat2 `comments:` block (frames/markdown), and the note names neither as the place
to put anything load-bearing.
expected: >-
State in the note's Output section that `draft-extract` is a re-serialization, not a
textual edit, and that YAML comments do not survive it; add it to Gotchas next to the
existing "this is a transformation, not a validator" warning. Name the two surfaces that
do survive — step `doc:` and the gxformat2 `comments:` block — and say that anything a
later reader must not lose belongs there before the loop reaches endstate. Ideally
`--report-json` should also count discarded comment lines, so the loss is at least
visible in the sidecar; failing that, the note is the only place a run can learn it, and
it has to say so. (The upstream fix — a comment-preserving round-trip — is a separate,
larger ask; the note must describe today's behaviour either way.)
evidence: >-
Iteration 26 of this run, at loop endstate, gxwf 1.10.1. `gxwf draft-extract
<run>/galaxy-workflow-draft.gxwf.yml -o <run>/galaxy-workflow.gxwf.yml --report-json
<report>` exited 0 with `0 steps dropped, 0 outputs dropped, 0 input rewrites;
class_after=GalaxyWorkflow`. Parsing both files and comparing: `steps`, `inputs`,
`outputs` and the `comments:` block are deep-equal, and the only structural difference is
`class`. The textual difference is 850 comment lines / 62,565 characters present in the
draft and 0 in the extract — 106,359 bytes down to 41,862, about 59% of the file. What was
in them: the per-step corpus citations this loop moved out of `_plan_*` at concretion, and
the two cross-step invariant warnings this run established by hand — that
`lfc_shrinkage_type: none` on both DESeq2 nodes is what makes the `c7<` and `abs(c3)>`
predicates in four other steps refer to the right columns, and that `header_lines: '0'` on
the four significance filters must NOT be the corpus's `'1'`. Neither invariant is
expressible in the draft schema (see `draft-format-cannot-express-a-cross-step-invariant`)
and neither is checked by any gate, so the comment was the only record, and the extract is
where it stopped existing.
status: open
issue: null
- id: paper-to-test-data-declares-no-test-data-refs-shape
raised_by: paper-to-test-data
observed_in:
mold: paper-to-test-data
path: content/molds/paper-to-test-data/index.md
revision: 2
content_hash: 863d7b503a77fa63449899f124d1669e181fdfd9a72f81f516a219cdc89cb6bb
foundry_head: 63a3f9cf9c97a637fe1628d298e524f35709e289
subject:
kind: mold
label: paper-to-test-data
locator: content/molds/paper-to-test-data/index.md
content_hash: 863d7b503a77fa63449899f124d1669e181fdfd9a72f81f516a219cdc89cb6bb
kind: gap
severity: major
what: >-
The Mold emits `test-data-refs.json` and specifies no shape for it. The whole procedure is
one sentence — "derive concrete workflow test inputs and expected outputs — resolvable
URLs, file shapes, and expected hashes — emitted as `test-data-refs`" — and the bundle
packages no schema, template, or worked example (`refs: []`, "Load Upfront" and "Load On
Demand" both "None declared"). The whole JSON structure was invented at runtime: how to key
an input against a workflow input label, how to express a collection input's elements and
per-row metadata, where expected outputs live and how to separate an assertion the data
supports from one it does not. Two downstream Molds in this pipeline consume the artifact
(freeform-summary-to-galaxy-test-plan at phase 8, implement-galaxy-workflow-test at phase
9) and neither can rely on any particular key existing. This is the same defect class
already filed as `summarize-paper-declares-no-freeform-summary-shape`, at the other end of
the same pipeline.
expected: >-
Package a `test-data-refs` skeleton or worked example, or state in the procedure the
minimum keys every consumer can assume. At minimum: one entry per workflow input keyed by
the input's label, carrying source URL, hash, datatype and (for collections) collection
type, element identifiers and per-element metadata; expected outputs separated from
inputs; and an explicit place to record which expected outputs the resolved data can
actually produce versus which it cannot. The three sibling test-data Molds
(`nextflow-to-test-data`, `cwl-to-test-data`, `find-test-data`) emit the same artifact id
and should share whatever contract is written.
evidence: >-
Bundle carries SKILL.md plus _feedback/_provenance/_verify only. The Output section names
the filename, the format (`json`) and a one-line description, and nothing else. The
artifact this run wrote (<run>/test-data-refs.json) has seventeen top-level keys, none of
which were prescribed.
status: open
issue: null
- id: paper-to-test-data-silent-on-subsetting-deposited-data
raised_by: paper-to-test-data
observed_in:
mold: paper-to-test-data
path: content/molds/paper-to-test-data/index.md
revision: 2
content_hash: 863d7b503a77fa63449899f124d1669e181fdfd9a72f81f516a219cdc89cb6bb
foundry_head: 63a3f9cf9c97a637fe1628d298e524f35709e289
subject:
kind: mold
label: paper-to-test-data
locator: content/molds/paper-to-test-data/index.md
content_hash: 863d7b503a77fa63449899f124d1669e181fdfd9a72f81f516a219cdc89cb6bb
kind: gap
severity: major
what: >-
The Mold is silent on subsetting, which is the central problem of deriving test data from a
paper. A paper's deposited data is essentially always orders of magnitude too large to be a
workflow test fixture — this run's six SRA runs are 22–30 M read pairs each, ~6.5 GB — so
the Mold's real task is not "resolve URLs" but "resolve URLs AND decide a subset AND
establish whether that subset still supports the paper's claims". The procedure's phrasing
("resolvable URLs, file shapes, and expected hashes") reads as though the deposited data is
used as-is. Nothing tells the runtime to subset, how to choose a depth, how to keep the
recipe deterministic, or how to decide whether the result is a fixture that reproduces the
paper's finding or one that only exercises the workflow's shape. That last decision is
exactly what the Foundry's own fixture rule makes mandatory — "a fixture must also be able
to produce the outcome the scenario bound to it claims" (AGENTS.md) — and the Mold never
routes to it. The subsetting strategy, the depth, the evidence standard and the tiering of
assertions by whether the data supports them were all invented here.
expected: >-
Add a subsetting step to the procedure and state the decision it has to reach. Name the
common strategies and when each applies (deterministic head subset; alignment-based region
targeting; downsampling to a fixed seed), require the recipe be reproducible and the
subset be hashed on its UNCOMPRESSED bytes, and require the output artifact to state
explicitly whether the subset reproduces the paper's result or is shape-only — citing the
fixture rule so the runtime knows an unmarked shape-only fixture is a defect and not a
shortcut. Relatedly, "Required Tools: None declared. Procedure should not assume external
CLIs are present" sits in tension with the same procedure demanding "expected hashes": a
hash of a subsetted fixture cannot be produced without tooling, so either the tool
expectation or the hash expectation should be stated honestly.
evidence: >-
This run reached a defensible answer only by going well outside anything the Mold
describes: measuring SCF1-matching reads per 1 M-read block to show a head subset carries
no positional bias, then aligning 200,000-pair subsets of all six runs to the pinned
reference to measure that SCF1 retains 414/391 fragments in AR0382 against 1–4 in the two
comparison conditions. Without that work the honest answer would have been "shape-only,
probably"; the Mold gave no reason to do it and no standard to judge it against.
status: open
issue: null
- id: galaxy-test-staging-drops-sample-sheet-column-definitions
raised_by: paper-to-test-data
observed_in:
mold: paper-to-test-data
path: content/molds/paper-to-test-data/index.md
revision: 2
content_hash: 863d7b503a77fa63449899f124d1669e181fdfd9a72f81f516a219cdc89cb6bb
foundry_head: 63a3f9cf9c97a637fe1628d298e524f35709e289
subject:
kind: related-project
label: Galaxy — test-data staging for sample_sheet collections (galaxy.tool_util.cwl.util / galaxy.tool_util.client.staging)
locator: https://github.com/galaxyproject/galaxy
kind: gap
severity: major
what: >-
A sample_sheet collection staged from a Planemo/gxwf test `job:` block never carries
collection-level `column_definitions`, although the API it posts to accepts them.
`galactic_job_json`'s `replacement_collection()` passes only `rows` and `name` for a
sample_sheet collection type, and `StagingInterface`'s `create_collection_func` has no
`column_definitions` parameter to pass — while `CreateNewCollectionPayload` declares the
field and `SampleSheetDatasetCollectionType.generate_elements` reads it. The result is
that a workflow input declaring `column_definitions` is, under test, fed a collection that
has none. Per-element `columns` survive, so most workflows still behave, and the two
`column_definitions_compatible()` call sites are both in `DataCollectionToolParameter`
option-building (UI dropdown filtering) which a test bypasses by supplying the HDCA by id
— so the divergence is silent rather than caught. It becomes visible at
`__SAMPLE_SHEET_TO_TABULAR__`, whose header line is emitted only
`#if $include_headers and $input.collection.column_definitions`: a workflow using that
tool with headers on produces a header in the UI and no header under test, and no gate
reports the difference.
expected: >-
Accept `column_definitions` on a `Collection` entry in a test job block and thread it
through `galactic_job_json` → `create_collection_func` → the collections API, alongside
`rows`. Then a sample sheet staged for a test is the same object a user builds, and
`validate_row` actually validates the fixture's rows against the declared columns instead
of short-circuiting. Failing that, Galaxy should say plainly that test-staged sample
sheets are column-definition-free, so workflow authors know that any behaviour gated on
`column_definitions` is untestable.
evidence: >-
Read at galaxyproject/galaxy dev while resolving open-requirements entry
`sample-sheet-input-test-fixture-expressibility` for this run:
lib/galaxy/tool_util/cwl/util.py `replacement_collection()` (sample_sheet branch sets only
`kwds["rows"]`); lib/galaxy/tool_util/client/staging.py `create_collection_func` signature
`(element_identifiers, collection_type, rows=None, name=None)`;
lib/galaxy/schema/schema.py `CreateNewCollectionPayload.column_definitions`;
lib/galaxy/model/dataset_collections/types/sample_sheet.py `generate_elements`;
lib/galaxy/model/dataset_collections/types/sample_sheet_util.py `validate_row` and
`column_definitions_compatible`; lib/galaxy/tools/sample_sheet_to_tabular.xml. This run's
workflow is unaffected only because its `Project sample sheet to tabular` step sets
`include_headers: false`. This entry also supplies part of the evidence asked for by
`testability-note-silent-on-sample-sheet-test-fixtures`, which remains open on its own
subject.
status: open
issue: null
- id: galaxy-unit-test-documents-a-sample-sheet-rows-shape-the-api-rejects
raised_by: paper-to-test-data
observed_in:
mold: paper-to-test-data
path: content/molds/paper-to-test-data/index.md
revision: 2
content_hash: 863d7b503a77fa63449899f124d1669e181fdfd9a72f81f516a219cdc89cb6bb
foundry_head: 63a3f9cf9c97a637fe1628d298e524f35709e289
subject:
kind: related-project
label: Galaxy — test/unit/tool_util/test_cwl_util.py sample_sheet rows tests
locator: https://github.com/galaxyproject/galaxy
kind: defect
severity: minor
what: >-
`test_galactic_job_json_sample_sheet_collection_with_rows` asserts that `rows` round-trips
as `{"el1": {"condition": "treatment"}, "el2": {"condition": "control"}}` — a mapping of
column name to value. The real contract is a POSITIONAL LIST per row: `validate_row` in
lib/galaxy/model/dataset_collections/types/sample_sheet_util.py rejects on
`len(row) != len(column_definitions)` and then zips `row` against `column_definitions` in
order. The unit test never catches this because it mocks `collection_create_func`, so
nothing downstream of `galactic_job_json` is exercised. These unit tests are the most
discoverable documentation of the job-block `rows` syntax, and the shape they document
will not validate whenever the target collection has column definitions.
expected: >-
Change the two `rows` unit tests to the positional-list form
(`{"el1": ["treatment"], "el2": ["control"]}`), matching
lib/galaxy_test/workflow/collection_semantics_cat_sample_sheet.gxwf-tests.yml which
already uses lists (`rows: {el1: [], el2: []}`). Better, add an integration test that
stages a sample sheet WITH column definitions through the real API, which would have
caught the divergence and would also cover the gap filed as
`galaxy-test-staging-drops-sample-sheet-column-definitions`.
evidence: >-
Read at galaxyproject/galaxy dev: test/unit/tool_util/test_cwl_util.py lines ~192–219
versus `validate_row` in
lib/galaxy/model/dataset_collections/types/sample_sheet_util.py. Encountered while
deriving the job block now recorded in <run>/test-data-refs.json; the dict form was
adopted from the unit test first and corrected only after reading the validator.
status: open
issue: null
- id: test-plan-mold-ignores-available-concrete-workflow
raised_by: freeform-summary-to-galaxy-test-plan
observed_in:
mold: freeform-summary-to-galaxy-test-plan
path: content/molds/freeform-summary-to-galaxy-test-plan/index.md
revision: 2
content_hash: 9c1d56e625a5dd26cb7a82082ac8f4eeb7db287a626bf00e73457c8dd491ad6d
foundry_head: 63a3f9cf9c97a637fe1628d298e524f35709e289
subject:
kind: mold
label: freeform-summary-to-galaxy-test-plan
locator: content/molds/freeform-summary-to-galaxy-test-plan/index.md
content_hash: 9c1d56e625a5dd26cb7a82082ac8f4eeb7db287a626bf00e73457c8dd491ad6d
kind: gap
severity: major
what: >-
The procedure's "Labels and fixtures are assumed, not bound" section instructs the Mold to
bind assertions to interface-brief labels with `label_status: assumed` and
`workflow.label_source: interface-brief`, and to record fixtures as `storage: unresolved` with
`location: null`, on the stated premise that the concrete workflow and the resolved test-data
refs "exist in the harness run-state by the time the plan is authored, but they are reconciled
downstream rather than here". In this run both were supplied as phase-8 inputs and both were
settled: `galaxy-workflow.gxwf.yml` (27 steps, 9 inputs, 16 outputs) carries the real labels,
and `test-data-refs.json` carries resolved URLs, md5s and a measured verdict. Following the
instruction would have meant writing `assumed` over labels read byte-for-byte from the
workflow and `unresolved` over fixtures with pinned NCBI URLs and md5s — discarding verified
information and handing implement-galaxy-workflow-test a reconciliation job already done. The
Mold was deliberately disobeyed on both counts, and the deviation had to be argued inside the
artifact rather than settled by the procedure.
expected: >-
Make the premise conditional rather than absolute. State that when a concrete workflow or a
resolved test-data-refs artifact is available to the invocation, the plan binds to it and
records `label_status: resolved` / `label_source: draft` and real fixture storage; the
`assumed` / `unresolved` path is the fallback for the template-era case the section describes.
Declaring `galaxy-workflow-gxformat2` and `test-data-refs` as optional consumed artifacts
would make that explicit in the bundle instead of leaving it to the runtime to notice.
evidence: >-
<run>/galaxy-test-plan.yml `workflow.notes` records the deviation and its reasoning; every
`label_status` in the plan is `resolved` and `workflow.label_source` is `draft`. The interface
brief had itself drifted from the draft (open requirement
`interface-brief-output-and-parameter-surface-drifted-from-draft`), so binding to the brief
would have produced labels that do not exist in the workflow under test.
status: open
issue: null
- id: test-plan-schema-has-no-evidence-class-for-run-measured-assertions
raised_by: freeform-summary-to-galaxy-test-plan
observed_in:
mold: freeform-summary-to-galaxy-test-plan
path: content/molds/freeform-summary-to-galaxy-test-plan/index.md
revision: 2
content_hash: 9c1d56e625a5dd26cb7a82082ac8f4eeb7db287a626bf00e73457c8dd491ad6d
foundry_head: 63a3f9cf9c97a637fe1628d298e524f35709e289
subject:
kind: schema
label: galaxy-workflow-test-plan
locator: package://@galaxy-foundry/gxwf-foundry#galaxyWorkflowTestPlanSchema
content_hash: 325e9cf1bb5e075fa818dc68868647de95389ca8df8581ee8cf1e2be61196e37
kind: gap
severity: minor
what: >-
`AssertionIntent.evidence` is a two-value enum, `test-evidence` or `intent`, described as
"whether this assertion was translated from upstream test evidence or synthesized from
intent". A third case is routine on the paper and interview paths and occurred throughout this
run: an assertion neither translated from an upstream fixture nor merely synthesized, but
MEASURED against the run's own resolved test data before the plan was written. The same gap
exists at plan level, where `source.derived_from` has the same two values. Forced to pick,
every such assertion is recorded `evidence: intent`, so a reviewer filtering on the field sees
a uniformly speculative plan and systematically undercounts its grounding. `confidence: high`
is the only signal left, and it means something different.
expected: >-
Add a third value — `run-measured` (or `fixture-measured`) — to `AssertionIntent.evidence` and
to `source.derived_from`, meaning the value was obtained by measuring the run's own resolved
fixtures rather than read from upstream tests or inferred from intent. Nothing else in the
schema needs to change, and the existing two values keep their meaning.
evidence: >-
<run>/galaxy-test-plan.yml records the same complaint twice because the field cannot carry it:
`source.notes` argues that `derived_from: intent` understates the plan's grounding, and
warnings[] `fixture-measured-assertions-lack-an-evidence-class` states the consequence.
Roughly twenty assertions in the plan rest on direct measurement of the resolved fixtures by
phase 7 (per-sample SCF1 fragment counts, strandedness, mapping rate, annotation gene count)
or by phase 8 (DESeq2 log2 fold change and adjusted p-value per contrast), and all of them are
filed as `intent`.
status: open
issue: null
- id: an-interrupted-phase-leaves-its-declared-artifact-written-but-unvalidated
raised_by: freeform-summary-to-galaxy-test-plan
observed_in:
mold: freeform-summary-to-galaxy-test-plan
path: content/molds/freeform-summary-to-galaxy-test-plan/index.md
revision: 2
content_hash: 9c1d56e625a5dd26cb7a82082ac8f4eeb7db287a626bf00e73457c8dd491ad6d
foundry_head: 63a3f9cf9c97a637fe1628d298e524f35709e289
subject:
kind: research
label: foundry-feedback-ledger
locator: content/research/foundry-feedback-ledger/index.md
content_hash: null
kind: gap
severity: major
what: >-
A Mold that writes one large artifact and validates it at the end has a window in which the
artifact exists on disk and nothing records whether it conforms to its schema. This run
entered that window: phase 8's session was terminated after writing a 67 KB
`galaxy-test-plan.yml` and before running `foundry validate-galaxy-workflow-test-plan`, and
before its ledger pass. The run-lifecycle section addresses only run status — it says a hard
interruption leaves the run `running`, "which is distinguishable from success without a
recovery write" — and says nothing about the phase's declared OUTPUT. So `status: running` on
a phase is ambiguous in a way that matters: the artifact may be absent, present and valid,
present and invalid, or present and half-written, and the four look identical from the ledger.
A resuming agent, or a next phase that simply reads the artifact because it is there, has
nothing telling it the verify step never ran.
expected: >-
State in the run-lifecycle section that a phase left `running` may have written its declared
output artifacts in an UNVALIDATED state, and that whoever resumes or succeeds it must re-run
that phase's declared validation before treating those artifacts as input. Stronger: give the
phase row a field the skill writes when its own validation passes — `artifacts_validated:
true`, the exact analogue of `feedback_checked` and justified by the same argument the note
already makes for it, that a run which looked and found nothing must be distinguishable from
one that never looked.
evidence: >-
The terminated phase-8 session left `<run>/galaxy-test-plan.yml` complete and, as it turned
out, schema-valid, but nothing on disk said so; the finishing session had to re-run the
validator to find out. The same interruption also left the artifact citing a feedback entry id
(`test-plan-mold-ignores-available-concrete-workflow`) that did not exist in this ledger,
because the ledger pass is likewise an end-of-phase step — the artifact and the ledger were
inconsistent with each other and only the ledger's `running` status hinted at it.
status: open
issue: null
- id: cast-validation-fallback-resolves-an-unpinned-published-validator
raised_by: freeform-summary-to-galaxy-test-plan
observed_in:
mold: freeform-summary-to-galaxy-test-plan
path: content/molds/freeform-summary-to-galaxy-test-plan/index.md
revision: 2
content_hash: 9c1d56e625a5dd26cb7a82082ac8f4eeb7db287a626bf00e73457c8dd491ad6d
foundry_head: 63a3f9cf9c97a637fe1628d298e524f35709e289
subject:
kind: implementation
label: cast-mold caster — generated Validation section
locator: packages/build-cli/src/commands/cast-mold.ts
content_hash: null
kind: friction
severity: minor
what: >-
The caster emits a Validation instruction of the form "run `foundry <cmd> <artifact>` from
`@galaxy-foundry/gxwf-foundry`; if the command is not on PATH, run `npx --package
@galaxy-foundry/gxwf-foundry foundry <cmd> <artifact>`". The fallback names no version, so it
resolves whatever `latest` is on the registry at runtime, while the bundle already carries the
schema it was cast against, verbatim, under `references/schemas/`. Those two can disagree, and
when they do the fallback returns a confident green verdict against a contract the skill is
not bound by. In this run the fallback was the only available route — the checkout had no
installed dependencies and no package manager on PATH — and the published 0.1.2 schema had to
be diffed against the bundled copy by hand to know the verdict meant anything, which the
runtime notes ("do not read Foundry source files at runtime") discourage in the first place.
expected: >-
Record the validator package version in `_verify.json` at cast time and emit it in the
fallback (`npx --package @galaxy-foundry/gxwf-foundry@<version> ...`), so the fallback
validates against the same contract the bundle carries. Alternatively, or additionally, say in
the generated Validation line that `references/schemas/<name>.schema.json` in the bundle is the
binding contract and that a fallback verdict is only meaningful if the two agree.
evidence: >-
`_verify.json` for this Mold carries `validator_bin: foundry` and args, with no package
version anywhere in the bundle. The published schema happened to be byte-identical to
`references/schemas/galaxy-workflow-test-plan.schema.json` here, so the verdict stands, but
nothing in the bundle establishes that and nothing would have flagged it had they diverged.
status: open
issue: null
- id: tests-format-schema-rejects-two-shapes-the-iwc-corpus-uses
raised_by: implement-galaxy-workflow-test
observed_in:
mold: implement-galaxy-workflow-test
path: content/molds/implement-galaxy-workflow-test/index.md
revision: 8
content_hash: 966c486ccda2eb1e0f06afa674c5d6b85a7e7bfcc9b1545309e3ef8b1016f931
foundry_head: 63a3f9cf9c97a637fe1628d298e524f35709e289
subject:
kind: related-project
label: galaxy-tool-util-ts — tests-format schema / gxwf validate-tests
locator: https://github.com/jmchilton/galaxy-tool-util-ts
kind: defect
severity: major
what: >-
The tests-format schema rejects two shapes that production IWC workflow tests use and that
Galaxy accepts at run time. (1) The inner element of a `list:paired` job input written as
`class: Collection` + `type: paired` — the `Collection` `$def` is `additionalProperties:
false` with only `collection_type`, so `type` is an unknown property and the whole job input
fails `oneOf`. (2) A nested-collection output assertion whose outer `element_tests` entry
carries `elements:` without a `class: Collection` discriminator — the schema's `if/then` on
`class` routes it to the dataset-element model, which has no `elements` property. Run over
the pinned IWC corpus (fe41a79) with `gxwf validate-tests` from @galaxy-tool-util/cli 1.10.1,
31 of 122 committed `*-tests.yml` files fail, and the two clusters above account for the bulk
of them. This makes the static gate unusable as a conformance check against the corpus the
Foundry treats as normative, and it silently invalidates the corpus-derived recipes the
Mold's own packaged notes teach.
expected: >-
Accept `type` as an alias for `collection_type` on a nested `Collection` in a job block, and
allow `elements:` on a collection element assertion without requiring an explicit
`class: Collection`, matching what the Galaxy job-block loader and the test-format runner
actually accept. If the strict form is deliberate, the schema should say so and Galaxy's
Pydantic models (galaxyproject/galaxy, the source these are generated from) plus the IWC
corpus should be migrated together, rather than leaving a validator that fails a quarter of
the published corpus.
evidence: >-
Probed directly. A two-file probe differing only in `type: paired` versus
`collection_type: paired` gives 48 schema errors versus OK. Removing `class: Collection` from
one outer `element_tests` entry of this run's own test file gives
`/0/outputs/Trimmed reads/element_tests/AR0382_A: must NOT have additional properties`.
Corpus sweep at fe41a79: 91 pass, 31 fail; failures cluster on `/0/job/<label>/class`
(e.g. sars-cov-2-pe-illumina-wgs-variant-calling, three VGP Hi-C workflows,
generic-variant-calling-wgs-pe) and on `/0/outputs/<label>/element_tests`
(e.g. scrna-seq-fastq-to-matrix-10x-cellplex, metagenomic-raw-reads-amr-analysis).
status: open
issue: null
- id: iwc-test-data-conventions-teaches-a-nesting-shape-the-schema-gate-rejects
raised_by: implement-galaxy-workflow-test
observed_in:
mold: implement-galaxy-workflow-test
path: content/molds/implement-galaxy-workflow-test/index.md
revision: 8
content_hash: 966c486ccda2eb1e0f06afa674c5d6b85a7e7bfcc9b1545309e3ef8b1016f931
foundry_head: 63a3f9cf9c97a637fe1628d298e524f35709e289
subject:
kind: research
label: iwc-test-data-conventions
locator: content/research/iwc-test-data-conventions/index.md
content_hash: 1921e939703444604440e0768caca6e6b834a73dcfa964f450fa081143e008d3
kind: defect
severity: major
what: >-
Section 2e states normatively "Note: outer `collection_type: list:paired`, inner
`type: paired` (not `collection_type:`)", and section 2f's nested-output example omits the
`class: Collection` discriminator on the outer `element_tests` entry. Both forms are faithful
transcriptions of the IWC corpus and both are rejected by `references/schemas/
tests-format.schema.json`, which the same Mold packages and names as the gate its output must
pass. An agent that follows the note produces a file that fails the Mold's own step 4, with
48 unhelpful `oneOf` errors pointing at the whole job input rather than at the offending key.
The note is the only packaged guidance on these two shapes, so there is nothing else to fall
back to.
expected: >-
Give both shapes in each section, marked: the corpus form (`type: paired`; bare `elements:`)
and the schema-valid form (`collection_type: paired`; `class: Collection` then `elements:`),
with a one-line note that the validator accepts only the latter and a pointer to the upstream
entry `tests-format-schema-rejects-two-shapes-the-iwc-corpus-uses`. Until upstream converges,
a Foundry-authored test file should use the schema-valid form, and the note should say so
rather than leaving the reader to discover it from the validator.
evidence: >-
This run authored the reads input from section 2e verbatim; `gxwf validate-tests` 1.10.1
returned 48 errors. Switching the single key `type` to `collection_type` returned OK. The
same substitution is needed for the section 2f output form, confirmed by removing
`class: Collection` from one element of the finished file.
status: open
issue: null
- id: asserts-idioms-nested-element-tests-recipe-omits-the-class-discriminator
raised_by: implement-galaxy-workflow-test
observed_in:
mold: implement-galaxy-workflow-test
path: content/molds/implement-galaxy-workflow-test/index.md
revision: 8
content_hash: 966c486ccda2eb1e0f06afa674c5d6b85a7e7bfcc9b1545309e3ef8b1016f931
foundry_head: 63a3f9cf9c97a637fe1628d298e524f35709e289
subject:
kind: research
label: planemo-asserts-idioms
locator: content/research/planemo-asserts-idioms/index.md
content_hash: 81b8feeccd891642572e587a494d1e4d3a1b369a19d92947626230494fd92beb
kind: gap
severity: minor
what: >-
Section 5 is the note an agent reaches for when writing collection-output assertions, and its
nested-collection rule — "outer `element_tests:` keyed by outer identifier; inner `elements:`
(note plural, no `_tests` suffix on the inner)" — is incomplete in the one way that matters
to the gate: the outer entry must also carry `class: Collection`, or the schema routes it to
the dataset-element model and rejects `elements` as an additional property. The section also
does not mention that an `asserts:` mapping cannot repeat a key, so any output needing two
`has_text` probes has to use the list form (`- that: has_text`) — which this run needed on
five outputs and which the section's own examples never show.
expected: >-
Add `class: Collection` to the nested example in section 5 and say why it is required. Add a
line to section 4 or 9 stating that repeated assertion families require the `- that: <family>`
list form, since the dict form silently loses all but the last occurrence in YAML.
evidence: >-
`Trimmed reads` in <run>/galaxy-workflow.gxwf-tests.yml is a `list:paired` output needing the
nested form; without `class: Collection` on the outer entry `gxwf validate-tests` reports
`must NOT have additional properties` at `/0/outputs/Trimmed reads/element_tests/AR0382_A`.
Four of this run's outputs carry between four and six assertions of which two or more share a
family, which the dict form cannot express.
status: open
issue: null
- id: tonative-shape-sniff-skips-normalization-on-list-form-format2
raised_by: validate-galaxy-workflow
observed_in:
mold: validate-galaxy-workflow
path: content/molds/validate-galaxy-workflow/index.md
revision: 5
content_hash: 74e3743fc376f33c1eb37832ab4869c327d48f87e015c7a38597cf4c4ca939cd
foundry_head: 79bf5c3ab98eec3a67948d8bdebb9dba7d8a2359
subject:
kind: related-project
label: "@galaxy-tool-util/schema - toNative normalization guard (_isNormalizedFormat2)"
locator: https://github.com/jmchilton/galaxy-tool-util-ts
kind: defect
severity: major
what: >-
`toNative` mistakes a raw format2 workflow for an already-normalized one whenever `inputs:` and
`steps:` are written in list form, skips normalization, and then aborts with an uncaught
`TypeError: step.in is not iterable` as soon as a step's `in:` uses the mapping form gxformat2
equally permits. `_isNormalizedFormat2` decides on three top-level facts that say nothing about
per-step shape - `class === "GalaxyWorkflow"`, `Array.isArray(inputs)`, `Array.isArray(steps)` -
so `normalizedFormat2`, whose `normalizeStepIn`/`normalizeStepOut` handle both spellings
correctly, is never called and `_extractConnections` iterates a mapping. The defect is in
`toNative`, not in the connection validator that surfaced it: `gxwf convert --to native` and
`ensureNative` crash on the same input with no connection flag involved. For this run the
consequence is that `gxwf validate --connections` - the only static gate covering connection
types, collection algebra and map-over - could not be run at all.
expected: >-
Normalize unconditionally. `normalizedFormat2` is idempotent and performs plain object shaping
rather than a schema decode, so the guard bought nothing; teaching it to inspect every step's
`in`/`out` would cost more than normalizing and would leave the same class of bug waiting on the
next field. Whatever the shape of the fix, a validator crashing on its own supported input
format gives a harness no way to distinguish a broken tool from a broken workflow. Submitted
upstream with a regression test as jmchilton/galaxy-tool-util-ts#179.
evidence: >-
First seen on gxwf 1.10.1 in this run's phase 10; reproduced unchanged on 1.12.0 (both
`@galaxy-tool-util/cli` and `@galaxy-tool-util/schema`), the current release as of 2026-09-17,
so it is not a stale-CLI artifact. Trigger is one corner of the shape matrix, established on
20-line workflows with a single `Filter1@1.1.1` step: list `inputs:` + list `steps:` + mapping
`in:` crashes; the same workflow with list-form `in:`, with map-form `inputs:`, or with map-form
`steps:` converts cleanly, because those spellings fail the sniff and get normalized. The
earlier claim in this entry that the flag breaks on every format2 workflow with a tool step was
too broad - it breaks on the dialect this Foundry emits. `gxwf convert <run>/galaxy-workflow.gxwf.yml
--to native` crashes identically with no `--connections`, at `runConvert` in
`cli/dist/commands/convert.js:83`. Upstream's own suite never crosses the guard: every existing
`toNative` test writes `inputs`/`steps` in map form. With the guard removed, all 4999
`packages/schema` tests pass and the three new list-form cases go from crash to green. Workaround
available to the Foundry without waiting on a release: emit `in:` in list form
(`- id: <port>` / `source: <ref>`).
status: filed
issue: https://github.com/jmchilton/galaxy-tool-util-ts/pull/179
- id: gxwf-strict-encoding-demands-the-state-key-its-validator-mishandles
raised_by: validate-galaxy-workflow
observed_in:
mold: validate-galaxy-workflow
path: content/molds/validate-galaxy-workflow/index.md
revision: 5
content_hash: 74e3743fc376f33c1eb37832ab4869c327d48f87e015c7a38597cf4c4ca939cd
foundry_head: 79bf5c3ab98eec3a67948d8bdebb9dba7d8a2359
subject:
kind: related-project
label: "@galaxy-tool-util/cli — gxwf validate --strict-encoding vs the tool-state validator"
locator: https://github.com/jmchilton/galaxy-tool-util-ts
kind: defect
severity: major
what: >-
gxwf's two validation paths demand opposite tool-state keys, so no format2 workflow can satisfy
both. `--strict-encoding` rejects `tool_state:` on every step with `uses "tool_state" instead
of "state" (format2 should use "state")` and exits 2, while the tool-state validator silently
drops a `{__class__: ConnectedValue}` placeholder under `state:` and honours it only under
`tool_state:` (this ledger's `gxwf-drops-connectedvalue-under-format2-state-key`). Moving to the
key `--strict-encoding` wants reintroduces that bug; staying on the key that works means
`--strict` can never be part of a Foundry gate. `gxwf convert --to format2` emits `tool_state:`,
so gxwf's own converter produces output its own `--strict-encoding` rejects.
expected: >-
Reconcile the two paths as one decision. If `state:` is the canonical format2 key, fix the
tool-state validator to honour ConnectedValue under it first, then keep the strict-encoding
diagnostic. If `tool_state:` is to remain accepted, `--strict-encoding` should not flag it, and
`gxwf convert --to format2` should emit whichever key the strict path blesses. Until then the
diagnostic actively steers authors toward the broken key.
evidence: >-
gxwf 1.10.1. `gxwf validate <run>/galaxy-workflow.gxwf.yml --json --strict-structure
--strict-encoding --cache-dir <cache>` exits 2 and emits the message for all 26 steps that
carry tool state (step 0 has none); the same file under default strictness reports zero
encoding errors and zero structure errors. The workflow uses `tool_state:` on 26 steps because
phase 6 established at iteration level that `state:` breaks the three
`iuc/compose_text_param/compose_text_param@0.1.1` steps. Companion to, not a duplicate of,
`gxwf-drops-connectedvalue-under-format2-state-key`: that entry is the validator dropping the
placeholder, this one is the strict gate requiring the key that triggers it. A maintainer
fixing one need not touch the other; triage may merge them into a single upstream issue.
status: open
issue: null
- id: gxwf-validate-json-output-is-not-machine-parseable
raised_by: validate-galaxy-workflow
observed_in:
mold: validate-galaxy-workflow
path: content/molds/validate-galaxy-workflow/index.md
revision: 5
content_hash: 74e3743fc376f33c1eb37832ab4869c327d48f87e015c7a38597cf4c4ca939cd
foundry_head: 79bf5c3ab98eec3a67948d8bdebb9dba7d8a2359
subject:
kind: related-project
label: "@galaxy-tool-util/cli — gxwf validate --json stdout contract"
locator: https://github.com/jmchilton/galaxy-tool-util-ts
kind: defect
severity: major
what: >-
`gxwf validate --json` does not put JSON, and only JSON, on stdout, so the documented
machine interface cannot be consumed by parsing stdout. Two separate breaches. (1) Every
uncached tool that fails to decode prints a multi-line `toolshed fetch failed (...) for <id>:`
block to stdout ahead of the JSON document - the full Effect schema type, roughly 55 lines for
seven such tools - so `JSON.parse(stdout)` throws and the report has to be located by scanning
backwards for a line that is exactly `{`. (2) Adding any strict flag makes gxwf abandon JSON
entirely: exit 2, plain-text diagnostics on stderr, and stdout completely empty, although
`--json` was passed.
expected: >-
With `--json`, write the report and nothing else to stdout, and route fetch/decode diagnostics
to stderr. Carry the strict-mode findings inside the JSON report (the schema already has
`structure_errors` and `encoding_errors` arrays for exactly this) and keep emitting it on the
strict failure path, signalling the verdict through the exit code rather than by withholding
the document. A harness cannot classify a failure it cannot parse.
evidence: >-
gxwf 1.10.1. Terminal validation of <run>/galaxy-workflow.gxwf.yml: stdout begins with the
decode-failure text for `__FLATTEN__` and continues for six more tools before the report; the
JSON body starts at line 57 of 273. The same command plus `--strict-structure
--strict-encoding` returns exit 2 with 0 bytes on stdout and the 26 encoding messages on
stderr. The packaged CLI reference states "JSON output should be treated as the preferred
cast-skill interface", which is the contract being broken.
status: open
issue: null
- id: gxwf-validate-never-checks-in-key-names
raised_by: validate-galaxy-workflow
observed_in:
mold: validate-galaxy-workflow
path: content/molds/validate-galaxy-workflow/index.md
revision: 5
content_hash: 74e3743fc376f33c1eb37832ab4869c327d48f87e015c7a38597cf4c4ca939cd
foundry_head: 79bf5c3ab98eec3a67948d8bdebb9dba7d8a2359
subject:
kind: related-project
label: "@galaxy-tool-util/cli — gxwf validate tool-state / step-input name checking"
locator: https://github.com/jmchilton/galaxy-tool-util-ts
kind: gap
severity: major
what: >-
`gxwf validate` never checks that a step's `in:` keys name real parameters of the pinned tool,
even when that tool is fully cached and its state validates. A step wiring a connection to a
port the tool does not have is reported `tool_state: OK` and counted in the validated total.
The same permissiveness runs the other way inside `tool_state:`: an unknown extra parameter is
accepted, and a required parameter that is simply absent is accepted. What the validator
actually checks is the type and value of the parameters that happen to be present. No
strictness flag changes this - not `--strict-structure`, not `--strict-state`, not `--strict`.
This matters most exactly where Galaxy's own syntax is easiest to get wrong: a conditional's
nested port must be qualified (`how|filter_source`), and the unqualified spelling is silently
accepted.
expected: >-
When the step's tool is resolved from the cache, check each `in:` key against the tool's input
tree - including conditional qualification (`cond|param`), repeat indexing (`name_<n>|param`)
and sections - and report an unknown port as an error, or at minimum under `--strict-state`.
Report a missing required parameter the same way. A validator that reports `20 validated` while
never having looked at a single port name overstates its own coverage to any harness reading
the summary.
evidence: >-
gxwf 1.10.1, measured directly. A minimal format2 workflow with a single `Filter1@1.1.1` step,
the tool present in the cache, and `in: {not_a_real_tool_input: tbl}` reports `Structural
validation: OK`, `tool_state: OK`, `Tool state: 1 validated, 0 skipped`, identically under
`--strict-structure --strict-state`. On <run>/galaxy-workflow.gxwf.yml, adding
`bogus_param_probe: "xyz"` to a validated Filter1 step leaves the summary at 20 ok / 0 fail /
7 skip, and deleting the required `cond` parameter also leaves it at 20 / 0 / 7; by contrast
`header_lines: "notanint"` on the same step does fail, which is what the check does cover.
This run's three `__FILTER_FROM_FILE__` steps depend on the qualified `how|filter_source`
spelling and nothing in the toolchain would have caught the unqualified one.
status: open
issue: null
- id: validate-mold-terminal-pass-has-no-cache-and-no-skip-vocabulary
raised_by: validate-galaxy-workflow
observed_in:
mold: validate-galaxy-workflow
path: content/molds/validate-galaxy-workflow/index.md
revision: 5
content_hash: 74e3743fc376f33c1eb37832ab4869c327d48f87e015c7a38597cf4c4ca939cd
foundry_head: 79bf5c3ab98eec3a67948d8bdebb9dba7d8a2359
subject:
kind: mold
label: validate-galaxy-workflow
locator: content/molds/validate-galaxy-workflow/index.md
content_hash: 74e3743fc376f33c1eb37832ab4869c327d48f87e015c7a38597cf4c4ca939cd
kind: gap
severity: major
what: >-
The Mold owns the run's last automated gate and its procedure never mentions the tool cache,
never mentions skips, and gives the artifact a three-valued status - `pass`, `fail`,
`not-run` - with no value for the outcome this gate actually produces. Two consequences, both
hit in this run. (1) The invocation the procedure implies, `gxwf validate <file> --json`, omits
`--cache-dir` and returns `Tool state: 0 validated, 27 skipped` with exit code 0: a green that
checked nothing, on the terminal gate, with no warning anywhere in the bundle. (2) When the
cache is supplied, seven steps still skip permanently, and the Mold offers no way to say so -
`pass` overstates it, `not-run` understates it - and no route to discharge a skip, although one
exists and is cheap. The Mold's one instruction on this, "A `not-run` status is never reported
as a pass", guards the case that cannot happen quietly and not the one that can.
expected: >-
Three changes to the procedure. Declare the tool cache as part of the contract and make
`--cache-dir` explicit in the invocation, stating that omitting it turns every tool step into
a skip and still exits 0. Make the artifact's status carry coverage: either a fourth value for
a pass with unvalidated steps, or a required `validated`/`skipped` split alongside `status`,
so a downstream reader cannot cite the green without the number. And give the skip a discharge
route rather than leaving it to eyeball review: a skipped step's tool can be fetched from a
Galaxy instance at `GET /api/tools/<tool_id>?io_details=true` and its `in:` keys and tool_state
parameter names checked against the real input tree, which is what this phase did by hand and
what the procedure nowhere describes. Sibling of
`advance-draft-mold-needs-a-tool-cache-it-never-declares` and
`advance-draft-mold-treats-a-skipped-tool-state-as-green`, which found the same two holes in
the per-step loop Mold; this is the terminal gate, where they cost more.
evidence: >-
This run, phase 10. Only the harness brief - not anything in the cast bundle - carried the
`--cache-dir` requirement and the expected skip list. With the cache, `gxwf validate
<run>/galaxy-workflow.gxwf.yml --json --cache-dir <cache>` returns `ok: 20, fail: 0, skip: 7`
and exit 0; phase 6 iteration 26 recorded that the same command without it returns `0
validated, 27 skipped`, also exit 0. The seven skips are `__FLATTEN__`, `lparsons/cutadapt`,
three `__FILTER_FROM_FILE__` and two `iuc/deseq2`, none clearable by priming the cache (see
`collection-output-decode-is-a-flat-vs-nested-shape-mismatch-not-a-missing-field`). All seven
were discharged here against the live Galaxy tool API with zero mismatches, in one pass.
status: open
issue: null
- id: validate-cli-note-recommends-a-flag-that-crashes-and-omits-the-cache-trap
raised_by: validate-galaxy-workflow
observed_in:
mold: validate-galaxy-workflow
path: content/molds/validate-galaxy-workflow/index.md
revision: 5
content_hash: 74e3743fc376f33c1eb37832ab4869c327d48f87e015c7a38597cf4c4ca939cd
foundry_head: 79bf5c3ab98eec3a67948d8bdebb9dba7d8a2359
subject:
kind: cli-command
label: gxwf validate
locator: content/cli/gxwf/validate.md
content_hash: 1bf4b4c50e9987f63e65ad36f6deefe6e399fe009280d4bee8533d6c451acf1e
kind: defect
severity: major
what: >-
The note's Gotchas section warns about the one flag that weakens validation visibly and is
silent on the two ways it fails invisibly. It says `--no-tool-state` weakens validation, but
never says that omitting `--cache-dir` has the same effect and worse - every tool step becomes
a skip and the command still exits 0. `--cache-dir` appears only as a bare option line, "Tool
cache directory", with nothing about what happens without it. Separately, the note actively
recommends a flag that cannot run: "Use `--connections` when tool cache metadata is available
and data-shape compatibility matters, especially around collections and map-over", and its
Examples block lists `gxwf validate workflow.gxwf.yml --json --connections --strict` - a
command that at 1.10.1 crashes on any format2 workflow with a tool step, and would abandon
JSON even if it did not.
expected: >-
Add a Gotchas line making the cache trap explicit - without `--cache-dir` pointing at a
populated cache, `validate` reports every tool step as skipped and still exits 0, so the split
must be read rather than the exit code. Mark `--connections` as not working at 1.10.1 with a
pointer to the upstream defect, and drop or flag the `--connections --strict` example so the
note stops recommending a crash. Record which gxwf version the page was verified against; the
page carries none today, and `gxwf --version` self-reports 1.0.0 regardless.
evidence: >-
gxwf 1.10.1, this run's phase 10. Following the note's own recommendation was the first thing
attempted and it exited 1 with `TypeError: step.in is not iterable` and no report; see
`tonative-shape-sniff-skips-normalization-on-list-form-format2`. The `--strict` half of the same
example returns exit 2 with empty stdout; see
`gxwf-validate-json-output-is-not-machine-parseable`. The cache trap is the failure phase 6
iteration 26 hit on its first terminal validate: `0 validated, 27 skipped`, exit 0.
status: open
issue: null
Runtime artifact initialized by the harness ([[foundry-run-manifest]]).
not on disk.
Reviewable Markdown brief: abstract operations, collection map/reduce choices, shape-changing placeholder steps, unresolved Galaxy tool needs, confidence, open questions.
# Galaxy data-flow brief — *C. auris* Scf1 RNA-seq differential expression
Source handoffs: `freeform-summary.md` (Santana et al. 2023, *Science* 381:1461–1467; DOI 10.1126/science.adf8972) and `freeform-galaxy-interface.md` (phase 2).
This brief owns the Galaxy-shaped abstract DAG: nodes, edges, collection map/reduce choices, shape-changing placeholder transformations, and unresolved tool needs. It is **not** gxformat2, pins no Tool Shed tool ids or versions, and settles no step parameters. Built-in Galaxy collection operations are named as *candidates* (`__SAMPLE_SHEET_TO_TABULAR__`, `__FILTER_FROM_FILE__`) because the shape argument is unintelligible without them; the template and step-implementation Molds confirm or replace each one.
Consumers: `compare-against-iwc-exemplar`, `freeform-summary-to-galaxy-template`, `freeform-summary-to-galaxy-test-plan`.
---
## 1. What this phase settled
**Settled.** The per-sample `condition` metadata reaches DESeq2's factor levels by an identifier-keyed split of the counts collection, driven by a tabular projection of the sample sheet. §4 gives the path, the rejected alternatives, and the one residual verification. This closes open-requirements entry `sample-sheet-condition-to-deseq2-factor-wiring`.
**Deliberately left open.** The DESeq2 node's arity — one run over a three-level factor versus two runs over two-level factors (`deseq2-contrast-realization-unsettled`) — is not settled here, and §4.4 shows why it does not need to be: both realizations consume the *same* per-level counts collections, so the entire upstream wiring is realization-independent. What would settle it is wrapper evidence this phase has no route to: an IWC exemplar at phase 4, or a `summarize-galaxy-tool` pass on the chosen DESeq2 wrapper.
**Newly surfaced.** Three places where the interface brief's declared shapes are not achievable as written under Galaxy map-over semantics (§7), plus two parameterization gaps (§7.3, §5.4). All five are ledgered.
---
## 2. Abstract operation graph
```
[in 1] RNA-seq reads (collection, sample_sheet:paired, column_definitions: condition, replicate)
│
├─ map over each fastq ──► (A) qc_raw_reads ────────────────► [out 1] text summary (nested — §7.1)
│ [out 2] HTML report
│
├─ map over each row ───► (B) trim_reads ───────────────────► [out 3] trimming report
│ │ [out 4] trimmed reads
│ ▼
│ (C) align_reads ◄── [in 2] genome FASTA
│ │ ◄── [in 3] annotation GTF
│ ├──────────────────────────────► [out 5] BAM
│ ├──────────────────────────────► [out 6] mapping summary
│ ▼
│ (D) count_features ◄── [in 3] annotation GTF
│ │ ◄── [in 5] strandedness
│ ├──────────────────────────────► [out 7] gene counts per sample
│ └──────────────────────────────► [out 8] assignment summary
│ │
└─ (E) sample_metadata_table ────────┐ │ (counts collection, keyed by element identifier)
(__SAMPLE_SHEET_TO_TABULAR__) │ │
▼ │
(F₁..F₃) select_level_L ×3 ◄── [in 4] reference condition level (+ §7.3)
│ │
▼ ▼
(G₁..G₃) counts_for_level_L ×3 (__FILTER_FROM_FILE__)
│
▼ reduce: collection → multiple=true data input, one per level
(H) differential_expression (DESeq2)
├──────────► [out 9] normalized counts
├──────────► [out 10] results: tnSWI1 vs AR0382
├──────────► [out 11] results: AR0387 vs AR0382
├──────────► [out 14] diagnostic plots
▼
(I₁, I₂) filter_significant ×2 ◄── [in 6] padj, [in 7] fold change (§7.3)
├──────────► [out 12] significant: tnSWI1 vs AR0382
└──────────► [out 13] significant: AR0387 vs AR0382
```
### 2.1 Nodes
| Node | Abstract operation | Input shape | Output shape | Execution | Confidence |
|---|---|---|---|---|---|
| A `qc_raw_reads` | read quality report | `sample_sheet:paired` fastq.gz | nested collection of txt + html, one element pair per sample | map-over, **one job per fastq** (2 per sample, 12 total) | high on the operation, medium on the output shape (§7.1) |
| B `trim_reads` | quality-trim reads at Phred 20, no adapter | `sample_sheet:paired` fastq.gz | paired-per-sample fastq.gz + per-sample txt report | map-over, one job per row (6 jobs) | high on the operation, medium on whether the pair survives as one inner collection (§7.2) |
| C `align_reads` | splice-aware alignment, index built at run time | paired fastq per sample + genome FASTA + GTF | `bam` per sample + `Log.final.out` per sample | map-over, one job per row (6 jobs); FASTA and GTF broadcast to every job | high |
| D `count_features` | per-gene read counting, stranded | `bam` per sample + GTF + strandedness | 2-column counts tabular per sample + summary tabular per sample | map-over, one job per row (6 jobs) | high |
| E `sample_metadata_table` | project sample-sheet column metadata to tabular | `sample_sheet:paired` (the **workflow input**, not any mapped output) | one tabular: element identifier, condition, replicate | single job, no map-over | medium — mechanism named in the packaged sample-sheet note, output columns unverified (§4.3) |
| F₁–F₃ `select_level_L` | keep rows whose condition equals level L, project the identifier column | tabular from E | one identifier-list tabular per level | 3 single jobs | high on the operation, low on how L is supplied (§7.3) |
| G₁–G₃ `counts_for_level_L` | filter the counts collection to the identifiers of level L | counts collection (from D) + identifier list (from F_L) | counts sub-collection, 2 elements each | 3 single jobs | medium-high |
| H `differential_expression` | negative-binomial DE across a one-factor, three-level design | one counts sub-collection per factor level | result table(s), normalized counts, diagnostic plots | **reduction** — collections consumed by `multiple=true` data inputs | high on the design, **open** on node arity (`deseq2-contrast-realization-unsettled`) |
| I₁, I₂ `filter_significant` | threshold the result table on adjusted p and effect size | one DESeq2 result tabular | filtered tabular | 2 single jobs | high on the operation, medium on the expression (§7.3) |
No node exists for any step the paper does not name. There is no MultiQC, no post-trim QC, no deduplication, no rRNA filter, and no merged count matrix — consistent with the interface brief's closed tool set.
### 2.2 Edges
| Edge | Shape before → after | Evidence |
|---|---|---|
| in 1 → A | `sample_sheet:paired` → per-fastq fan-out | Galaxy map-over: FastQC takes one dataset, so the inner `paired` axis also fans out |
| in 1 → B | `sample_sheet:paired` → per-row jobs | a paired-aware trimmer consumes the inner `paired` element whole |
| B → C | trimmed pair → alignment job | stated tool order in the supplement |
| in 2, in 3 → C | `data` broadcast into every mapped job | no map-over: a `data` input wired to a mapped step is broadcast |
| C → D | `bam` per sample → counting job | stated tool order |
| in 3 → D | `data` broadcast | featureCounts requires the same annotation as the aligner |
| in 1 → E | `sample_sheet:paired` → tabular | §4; the only edge in the workflow that reads `column_definitions` |
| E → F_L | tabular → tabular | row filter on the condition column |
| D, F_L → G_L | (collection, identifier list) → sub-collection | identifier-keyed membership filter |
| G_L → H | collection → reduced multi-data port | sample_sheet-family collections reduce into `multiple=true` data inputs |
| H → I_c | tabular → tabular | the paper's stated significance criteria |
---
## 3. Collection map/reduce decisions
**One map-over region, one reduction point.** Everything from FastQC through featureCounts is a single map-over region over the sample axis established by workflow input 1. DESeq2 is the workflow's only reduction. That is the whole of the collection story except for the split in §4.
**Element identifiers are the spine.** `AR0382_A`, `AR0382_B`, `AR0387_A`, `AR0387_B`, `AR0382_tnSWI1_A`, `AR0382_tnSWI1_B` enter at input 1 and must be unchanged at node D, because (a) the interface keys its checkpoint assertions by element identifier, and (b) §4's split matches identifiers between the metadata table and the counts collection. No node in the map-over region may rename, sort, or reshape the outer axis. If a step-implementation choice would relabel elements, that is a topology defect, not a cosmetic one.
**No collection cleanup node.** `collection-cleanup-after-mapover-failure` (filter empty/failed elements) is *not* warranted here: every step produces exactly one output per input, no step is conditional, no step can legitimately produce an empty element, and a failed element means a real failure that should surface rather than be filtered away. Six elements, dense, homogeneous. Recorded so a later Mold does not add the idiom reflexively.
**No fan-in / concatenate node.** The interface deliberately exposes no merged count matrix, because the Galaxy DESeq2 tool consumes per-sample count files. A `tabular-concatenate-collection-to-table` bridge would be invented method. The only fan-in in the design is DESeq2's own reduction of the per-level collections.
**The outer axis is nominally `sample_sheet`, not `list`.** A tool mapped over a `sample_sheet`-family collection produces a `sample_sheet`-shaped output *without* `column_definitions` (packaged note `galaxy-sample-sheet-collections`, "Mapping rules"). Behaviourally that is a list — it maps, reduces, and filters like one — but its declared collection type is not the string `list`, which matters to anything that type-checks, including test assertions. The interface's output table calls outputs 1–8 `list`/`list:paired`. Ledgered: `mapped-outputs-carry-sample-sheet-outer-axis`.
---
## 4. The condition factor: how it reaches DESeq2 (settled)
This is the decision phase 2 handed down and the reason this phase exists.
### 4.1 The problem, precisely
The `condition` and `replicate` values live in `column_definitions` / per-row `columns` on workflow input 1. Galaxy does not propagate either through map-over. By node D the counts collection carries element identifiers and nothing else. DESeq2 needs its samples grouped by factor level — one set of count files per level — so the grouping has to be reconstructed from a source that still has it.
The metadata survives in exactly one place: **the workflow input itself**, which is still addressable as a node input no matter how far downstream the consumer sits. Every workable route starts there.
### 4.2 The settled route
```
in 1 (sample_sheet:paired, metadata intact)
└─► E __SAMPLE_SHEET_TO_TABULAR__ → sample metadata table
(element identifier | condition | replicate)
└─► F_L filter rows: condition == L ; project the identifier column
→ identifier list for level L (×3: AR0382, AR0387, tnSWI1)
└─► G_L __FILTER_FROM_FILE__(counts from D, identifier list)
→ counts sub-collection for level L (2 elements)
└─► H DESeq2 factor-level port L
```
Three properties make this the right route rather than merely a workable one:
1. **It joins on element identifiers**, which is the one key Galaxy guarantees across the map-over region — the identifier-keyed wiring the packaged sample-sheet note prescribes over "inventing parallel parameter inputs".
2. **It touches the map-over region not at all.** Nodes A–D are unchanged, promoted outputs 1–8 keep their identifier space, and the split is a side branch off the workflow input.
3. **It is the `sync-collections-by-identifier` idiom** from the packaged collection-pattern MOC (membership sync: derive identifiers from one source, filter a sibling collection), used for its intended purpose rather than adapted.
### 4.3 The one residual
The packaged note documents `__SAMPLE_SHEET_TO_TABULAR__` as iterating elements and tab-joining "for downstream tabular consumers", but does not state its output columns — specifically whether the **element identifier is emitted as a column**. The identifier is the join key for the entire split; if the tool emits only the `column_definitions` values, node E produces a table whose rows cannot be attributed to collection elements and F/G collapse.
This is a bounded, checkable question rather than a design unknown, and it does not change the route — only the implementation of node E. If the identifier is absent, node E is replaced by an Apply Rules projection over the same input collection (identifier column plus a metadata column), and everything from F onward is unchanged. Ledgered: `sample-sheet-to-tabular-identifier-column-unverified`.
### 4.4 Why this is independent of the DESeq2 realization
Both candidate realizations of `deseq2-contrast-realization-unsettled` consume the same three per-level counts collections:
- **One run, three-level factor** — H has three factor-level ports (G₁, G₂, G₃) and emits both contrasts against the reference level.
- **Two runs, two-level factors** — H splits into H₁ (G_AR0382, G_tnSWI1) and H₂ (G_AR0382, G_AR0387); the reference-level collection feeds both.
Nodes E, F₁–F₃, G₁–G₃ are identical either way, as is the map-over region. The choice changes only how many DESeq2 nodes exist and which level ports they carry, so it can be deferred to wrapper evidence without holding up the template's spine. The interface's output surface (two result tables, two filtered tables) is satisfied under both.
### 4.5 Why this is robust to the interface's own fallback
Phase 2 named a fallback: replace input 1 with `list:paired` reads plus a `data` input `Sample metadata table` (sample_id, condition, replicate). Under that fallback, **node E disappears and nothing else changes** — the user-supplied table lands where E's output lands, and F/G/H are untouched. So if `sample-sheet-input-test-fixture-expressibility` resolves against the sample sheet at the test-plan phase, the cost is one node and an interface edit, not a redesign. That is the main reason to route the metadata through a tabular intermediate instead of a sample-sheet-native mechanism.
### 4.6 Rejected alternatives
| Alternative | Why rejected |
|---|---|
| **Apply Rules `add_column_from_sample_sheet_index` on the counts collection** | The rule reads sample-sheet column metadata, and the counts collection has none — by node D the `column_definitions` are gone. Applied to input 1 instead, it degenerates to the settled route with a different node E. |
| **Filter the counts collection by element-identifier regex** | The identifiers encode the condition, so a regex looks free. It is a trap: `AR0382_tnSWI1_A` contains the substring `AR0382`, so an unanchored match for the reference level silently captures the mutant samples and corrupts the contrast. An anchored `^AR0382_[AB]$` works but hard-binds the workflow to this study's naming convention, which is exactly what the sample-sheet input was chosen to avoid. |
| **Split the reads collection by condition up front and run three parallel map-over regions** | Triples every step, and replaces the six promoted per-sample output collections with three per-condition ones — breaking the interface's checkpoint identifier space and its assertion design. |
| **Three condition-scoped `list:paired` workflow inputs** | Already rejected at interface time for hard-coding the three-level design into the public API; nothing in the data-flow analysis reopens it. |
| **A parameter input carrying conditions in parallel with the reads** | The failure mode the packaged sample-sheet note names explicitly: parallel parameter inputs have no guaranteed correspondence with collection elements, so the sample↔condition binding becomes positional and silently wrong on reorder. |
---
## 5. Shape-changing and placeholder transformations
| # | Transformation | Where | Necessity | Note |
|---|---|---|---|---|
| 5.1 | sample-sheet → tabular bridge | node E | required | The only read of `column_definitions` in the workflow. §4.3. |
| 5.2 | tabular row filter + column projection | F₁–F₃ | required | One per factor level. Trivially implementable; the open part is where the level string comes from (§7.3). |
| 5.3 | collection membership filter by identifier file | G₁–G₃ | required | `__FILTER_FROM_FILE__` emits *two* collections (matching and non-matching); only the matching branch is consumed. The template should not promote the discarded branch. |
| 5.4 | log₂ conversion or threshold restatement before I₁/I₂ | I₁, I₂ | conditional | DESeq2 reports log₂ fold change; workflow input 7 is a **linear** fold change (default 2.0). The filter must compare `abs(log2FoldChange) > log2(threshold)`. §7.3, ledgered. |
| 5.5 | collection flatten after FastQC | after A | conditional | Needed only if the interface's flat `list` shape for outputs 1–2 is kept rather than corrected. §7.1 recommends correcting the interface instead. |
| 5.6 | re-pair trimmed reads | after B | conditional | Needed only if the chosen trimmer emits R1 and R2 as two parallel collections rather than one paired-inner collection. §7.2, ledgered. |
None of these is an invented analysis step: 5.1–5.3 are plumbing the Galaxy collection model forces, and 5.4–5.6 are shape repairs, not method. Nothing here adds a computation the paper does not describe.
---
## 6. Unresolved tool needs
Handoff units for `discover-shed-tool` / the template Mold. Input and output shapes are given because that is what discovery needs; no tool id, owner, or version is asserted.
| Need | Abstract role | In → out | Candidate class | Confidence |
|---|---|---|---|---|
| read QC report | A | `fastqsanger.gz` → `txt` + `html` | FastQC, named by the paper | high |
| quality trimming, paired-aware, Phred 20, no adapter | B | paired `fastqsanger.gz` → paired `fastqsanger.gz` + `txt` | Cutadapt, named by the paper | high |
| splice-aware aligner with run-time index build | C | paired fastq + `fasta` + `gtf` → `bam` + `txt` | RNA STAR, named by the paper | high |
| stranded feature counting | D | `bam` + `gtf` + strandedness → `tabular` ×2 | featureCounts, named by the paper | high |
| sample-sheet → tabular | E | `sample_sheet:paired` → `tabular` | built-in `__SAMPLE_SHEET_TO_TABULAR__` | medium — §4.3 |
| tabular row filter / column cut | F | `tabular` → `tabular` | stock text-manipulation tools | high |
| collection filter by identifier file | G | collection + `tabular` → collection ×2 | built-in `__FILTER_FROM_FILE__` | medium-high |
| differential expression from per-sample counts | H | counts collections per level → `tabular` + `pdf` | DESeq2, named by the paper | high on the tool, open on arity |
| significance filter with a log₂ comparison | I | `tabular` + 2 params → `tabular` | stock filter tool, if its expression language supports the comparison in §5.4 | medium |
No need on this list is expected to require `author-galaxy-tool-wrapper`. Every named tool is a long-standing IUC wrapper and every plumbing node is a Galaxy built-in — which is the expected shape for a workflow the authors state they ran on usegalaxy.org.
---
## 7. Where the interface brief's declared shapes do not hold
These are corrections this phase owes upstream, not preferences. Each is ledgered so the template does not quietly implement one reading while the test plan asserts the other.
### 7.1 FastQC outputs are nested, not flat lists
FastQC consumes one dataset. Mapped over a `sample_sheet:paired` collection it therefore fans out over the **inner** axis too: 12 jobs, and outputs shaped one pair of reports per sample, not six flat elements. The interface declares outputs 1 and 2 as `list`.
Two ways out. **Promote the nested collection** (recommended): honest about what the workflow computes, keeps per-read-direction reports — which is what a reader wants from raw-read QC — and preserves identifiers. Or **flatten** with an explicit node, which satisfies the declared `list` but rewrites the identifier space to something like `AR0382_A_forward`, doubling the identifier vocabulary the tests key on for no analytical gain. Ledger: `fastqc-per-read-fanout-not-a-flat-list`.
### 7.2 Trimmed reads may arrive as two parallel collections
The interface declares `Trimmed reads` as `list:paired`. Whether the trimmer emits one paired-inner collection per sample or two parallel single-ended collections (R1, R2) is wrapper-dependent and not knowable in this phase. If it is the latter, the design needs a re-pair node (§5.6) before node C, or node C must take two parallel collection inputs in dot-product. Ledger: `trimmed-reads-paired-reassembly-conditional`.
### 7.3 Two parameter gaps the wiring exposes
- **Factor level names.** Nodes F₁–F₃ each need a literal condition value. The interface exposes only `Reference condition level` (default `AR0382`). The two contrast levels — `AR0387`, `tnSWI1` — have no parameter and would otherwise be baked into two filter steps, which re-hard-codes into the *steps* the design the sample-sheet input kept out of the *interface*. Ledger: `deseq2-factor-level-names-not-parameterized`.
- **Fold-change units.** Input 7 is `Minimum absolute fold change`, default `2.0`, linear. DESeq2 emits log₂. Either the filter expression converts, or input 7 is restated as a log₂ threshold (default `1.0`) with its label changed. Silently comparing `|log2FoldChange| > 2.0` would apply a 4-fold cut and quietly fail to reproduce the paper's gene list. Ledger: `fold-change-threshold-linear-vs-deseq2-log2fc`.
---
## 8. Data-flow evidence bearing on entries this phase did not close
Recorded here rather than by editing another Mold's ledger entries.
- **`reference-genome-delivery-shape-unverified`.** The history-FASTA choice means RNA STAR builds its index **inside each of the six mapped jobs** — the design has no separate index-build node, because the standard RNA STAR wrapper with a history reference has no separate index step to wire. Six redundant index builds of a ~12.5 Mb fungal genome is cheap in absolute terms, so this does not argue against the portability decision; it does mean a built-in index, if one exists, would change job count but not workflow topology. Whoever revisits the entry should know the topology is insensitive to it.
- **`featurecounts-annotation-source-unnamed`.** Input 3 is consumed **twice** (nodes C and D), so the annotation choice is load-bearing for splice junctions as well as counting. If the chosen source is GFF3, a conversion node appears between input 3 and both consumers — a shape change, not just a datatype edit.
- **`rnaseq-strandedness-inferred-from-kit-name`.** The empirical check is already wired: node D's summary output (interface output 8) is promoted, so the check needs no new node and no new data.
- **`galaxy-tool-versions-unpinnable-from-source`** and **`cutadapt-adapter-and-length-filter-unstated`** have no data-flow consequence; the graph is identical under any resolution.
---
## 9. Confidence
| Claim | Evidence class | Confidence |
|---|---|---|
| Node order A→B→C→D→H→I | stated tool chain in the supplement | high |
| Single map-over region over the sample axis, single reduction at DESeq2 | Galaxy collection semantics + the interface's input shape | high |
| Condition reaches DESeq2 by identifier-keyed split (§4.2) | packaged sample-sheet note (map-over does not propagate metadata) + collection-pattern MOC (`sync-collections-by-identifier`) | high on the shape, medium on node E's implementation |
| `__SAMPLE_SHEET_TO_TABULAR__` emits the element identifier | not stated in any packaged reference | low — §4.3 |
| `__FILTER_FROM_FILE__` is the right collection filter | named by the collection-pattern MOC's identifier section, one-line description only | medium-high |
| FastQC fans out per fastq (§7.1) | Galaxy map-over semantics on a single-dataset input | high |
| Trimmer output pairing (§7.2) | wrapper-dependent, no evidence available in this phase | low |
| DESeq2 node arity | wrapper-dependent, no evidence available in this phase | open, not estimated |
| No cleanup node needed | dense homogeneous collection, no conditional steps | high |
| Fold-change unit mismatch (§7.3) | DESeq2 reports log₂ by definition; interface parameter is linear | high |
Evidence-class caveat: the packaged pattern references in this bundle are **MOC index pages**, not the recipe pages they name. Every idiom selection in §3–§5 was made from a one-line MOC description, with no corpus recipe available to check tool ids or worked wiring against. Confidence above is stated accordingly, and phase 4's exemplar comparison is the natural place to correct it.
---
## 10. Open questions
Rendered as an index of the open-requirements ledger, one line per entry, so the ledger stays the single source of truth. Entry ids are the join key.
Inherited and still open:
1. `featurecounts-annotation-source-unnamed` — annotation unnamed; consumed twice (§8).
2. `rnaseq-strandedness-inferred-from-kit-name` — inference; check already wired (§8).
3. `sample-sheet-input-test-fixture-expressibility` — test-plan phase; this design absorbs either answer at the cost of one node (§4.5).
4. `deseq2-contrast-realization-unsettled` — node arity only; upstream wiring is realization-independent (§4.4).
5. `reference-genome-delivery-shape-unverified` — topology-insensitive (§8).
6. `cutadapt-adapter-and-length-filter-unstated` — no data-flow consequence.
7. `galaxy-tool-versions-unpinnable-from-source` — no data-flow consequence; expected to be surrendered at the terminal.
8. `tnbcy1-contrast-not-carried` (dropped) — unchanged.
9. `pipeline-b-tdna-mapping-not-carried` (dropped) — unchanged.
Closed by this phase:
10. `sample-sheet-condition-to-deseq2-factor-wiring` — resolved; §4.
Raised by this phase:
11. `sample-sheet-to-tabular-identifier-column-unverified` — §4.3.
12. `deseq2-factor-level-names-not-parameterized` — §7.3.
13. `fold-change-threshold-linear-vs-deseq2-log2fc` — §7.3.
14. `fastqc-per-read-fanout-not-a-flat-list` — §7.1.
15. `trimmed-reads-paired-reassembly-conditional` — §7.2.
16. `mapped-outputs-carry-sample-sheet-outer-axis` — §3.
---
## 11. Handoff notes
**To `compare-against-iwc-exemplar` (phase 4).** The two things worth diffing against the corpus are (a) how published RNA-seq workflows get a per-sample factor into DESeq2 — if IWC has a worked sample-sheet or metadata-table split, it likely settles §4.3 and possibly §7.3 at once — and (b) the DESeq2 node arity behind multiple contrasts, which settles `deseq2-contrast-realization-unsettled`. Both are shape questions an exemplar answers directly. The rest of the spine (FastQC → trim → align → count) is conventional enough that structural divergence there would be surprising.
**To `freeform-summary-to-galaxy-template` (phase 5).** Build the map-over region A–D and the split E→F→G as separate template regions; they connect only at node G's collection input. Nodes F and G are three near-identical steps each — do not collapse them into a single templated step, because DESeq2's factor-level ports are structurally distinct. Leave node H's arity as a deferred step until phase 4 reports. Do not promote the discarded branch of any `__FILTER_FROM_FILE__` (§5.3).
**To `freeform-summary-to-galaxy-test-plan` (phase 8).** Three things here change what tests can assert: the collection type of outputs 1–8 is `sample_sheet`-family, not `list` (§3); FastQC's output identifier space depends on how §7.1 resolves; and the strongest deterministic checkpoint remains output 7, keyed by gene id, which is also the input to the whole §4 split — an assertion there covers the map-over region and the identifier spine in one.
Reviewable Markdown brief: Galaxy workflow inputs, outputs, labels, collection shapes, checkpoint outputs, source-summary provenance, confidence, open questions.
# Galaxy workflow interface brief — *C. auris* Scf1 RNA-seq differential expression
Source handoff: `freeform-summary.md` (Santana et al. 2023, *Science* 381:1461–1467; DOI 10.1126/science.adf8972; PMID 37769084). All computational detail in that summary comes from the supplementary Materials and Methods, not the main text.
This brief is a design handoff. It is not a gxformat2 skeleton and pins no tool ids or versions. Consumers: `freeform-summary-to-galaxy-data-flow`, `compare-against-iwc-exemplar`, `freeform-summary-to-galaxy-template`, `freeform-summary-to-galaxy-test-plan`.
---
## 1. Scope
**In scope — Pipeline A, bulk RNA-seq differential expression.** FastQC → Cutadapt → RNA STAR → featureCounts → DESeq2 → significance filter, as named in the supplement. The authors state this ran on the Galaxy public server at usegalaxy.org, so the source tool chain is already a Galaxy tool chain.
**Out of scope — Pipeline B, AtMT T-DNA insertion-site mapping.** Excluded by harness decision before this phase. Recorded in the open-requirements ledger as `pipeline-b-tdna-mapping-not-carried` (`kind: dropped`) with its cited reasons, so the cut is legible to every later Mold rather than living only in this prose.
**Not ledgered.** The paper's other computational work — BLAST homology search, UniProt domain annotation, FungiDB/CGOB synteny, AlphaFold2/ColabFold + Foldseek structure prediction, Fiji + CellProfiler image analysis, FlowJo, R statistics — consumes no sequencing reads and was never candidate workflow content. It is not recorded as dropped workflow work.
**Tool set is closed to what the paper names.** No MultiQC, no aggregation or reporting step, no read-deduplication, no rRNA filter. The summary names six steps; adding a seventh would be inventing method.
---
## 2. Workflow inputs
Labels are the public API: Planemo and IWC tests address inputs by label, so these are chosen to read as test `job:` keys and are treated as breaking changes if renamed.
| # | Label | Galaxy shape | Format / type | Provenance | Confidence |
|---|-------|--------------|---------------|------------|-----------|
| 1 | `RNA-seq reads (sample sheet)` | collection, `sample_sheet:paired` | `fastqsanger.gz` | PRJNA904261 / SRP409192: 6 runs, 2 × 50 bp paired-end, NextSeq 2000 | high on paired-end shape; medium on the `sample_sheet` variant (see §2.1) |
| 2 | `Reference genome FASTA` | data | `fasta` | *C. auris* B8441, NCBI assembly GCA_002759435.2 (`Cand_auris_B8441_V2`), named in the supplement | high on the accession; medium on the delivery shape (see §2.2) |
| 3 | `Gene annotation GTF` | data | `gtf` | **Not named in the paper.** featureCounts and RNA STAR splice-junction input both require it | low — source unresolved |
| 4 | `Reference condition level` | parameter, `text` | default `AR0382` | The paper's two contrasts are both *versus* the parent AR0382 | high |
| 5 | `featureCounts strandedness` | parameter, `text` | default `reverse`; allowed `unstranded` / `forward` / `reverse` | Inferred from the library kit name (Illumina Stranded Total RNA Prep with Ribo-Zero Plus), **not stated** | low — inference only |
| 6 | `Adjusted p-value threshold` | parameter, `float` | default `0.05` | Stated: adjusted p-value < 0.05 | high |
| 7 | `Minimum absolute fold change` | parameter, `float` | default `2.0` | Stated: \|fold change\| > 2 | high |
### 2.1 Why `sample_sheet:paired` for the reads
The six runs are three conditions × two biological replicates. DESeq2 needs a per-sample condition assignment, and that assignment is per-sample typed metadata attached to paired fastq — which is exactly the `sample_sheet:paired` shape (`sample_sheet` outermost, inner `paired`).
```yaml
inputs:
RNA-seq reads (sample sheet):
type: collection
collection_type: sample_sheet:paired
column_definitions:
- {type: string, name: condition, optional: false,
restrictions: [AR0382, AR0387, tnSWI1]}
- {type: string, name: replicate, optional: false,
restrictions: [A, B]}
```
Element identifiers are the deposited sample names and must survive map-over and every collection reshape, because collection tests key assertions by element identifier:
`AR0382_A`, `AR0382_B`, `AR0387_A`, `AR0387_B`, `AR0382_tnSWI1_A`, `AR0382_tnSWI1_B`
Run-to-sample binding, for fixture work downstream: SRR22376032 → `AR0382_A`, SRR22376031 → `AR0382_B`, SRR22376030 → `AR0387_A`, SRR22376029 → `AR0387_B`, SRR22376028 → `AR0382_tnSWI1_A`, SRR22376027 → `AR0382_tnSWI1_B`.
Two consequences the data-flow Mold owns, both ledgered rather than assumed away:
- **Column metadata does not propagate through map-over.** A tool mapped over a `sample_sheet` produces a `sample_sheet`-shaped output *without* `column_definitions`. So the `condition` column is readable at the input and is gone by the time featureCounts has produced counts. Getting condition back — to split counts per factor level for DESeq2 — has to be an explicit step (`__SAMPLE_SHEET_TO_TABULAR__` plus a filter/split, or the rules DSL). Ledger: `sample-sheet-condition-to-deseq2-factor-wiring`.
- **Fallback, if that wiring proves unbuildable.** Replace input 1 with a `list:paired` collection labeled `RNA-seq reads` plus a `data` input `Sample metadata table` (tabular: sample_id, condition, replicate), and split on that table. This is named here so the fallback is a recorded alternative, not an improvisation. The same ledger entry carries it.
Rejected alternative: three condition-scoped `list:paired` inputs (one per condition), which wires to DESeq2's factor-level repeats trivially but hard-codes the three-level design into the interface and makes the workflow unusable for any other sample set.
### 2.2 Reference data delivery
*C. auris* B8441 is not a Galaxy mainstream reference. This brief settles the genome as a **history dataset (fasta)** rather than a built-in index or data-table string, on portability grounds: a remote-URL fixture is resolvable by a test on any server, a CVMFS index is not. RNA STAR then builds its index at run time from that FASTA plus the annotation.
Whether usegalaxy.org actually carries a built-in `GCA_002759435.2` index was **not checked in this phase** — no phase of this run owns reference-data shape (this run's roster has no `*-to-galaxy-reference-data` Mold). Ledgered as `reference-genome-delivery-shape-unverified` so the data-flow Mold can revisit it with evidence.
Note that the annotation choice is load-bearing and unresolved: NCBI RefSeq GFF and FungiDB GFF for the same assembly differ in gene ID space and in attribute keys (`gene_id` vs `ID`), which changes both featureCounts' `-g` attribute and every downstream gene identifier — including whether `B9J08_001458` (SCF1) appears under that name at all. The interface fixes the *datatype* as `gtf`; if the chosen source is GFF3, either the input datatype or a conversion step changes. Ledger: `featurecounts-annotation-source-unnamed`.
### 2.3 Parameter surface
Exposed as typed workflow parameters: the two significance thresholds (stated by the paper, and the workflow's headline criteria), the reference condition level (needed to make a three-level factor explicit), and strandedness (inferred, so a reviewer must be able to see and change it).
Baked into steps, not exposed: Cutadapt quality cutoff `-q 20` (pinned by the paper), RNA STAR defaults ("default parameters", pinned by the paper). No adapter sequence is exposed because the paper names none — the Cutadapt step is read as quality trimming only. Ledger: `cutadapt-adapter-and-length-filter-unstated`.
---
## 3. Workflow outputs
Every output below is a top-level `outputs:` entry with the label as its public name and an explicit `outputSource`. Roles: **checkpoint** = promoted because it is the best deterministic or structural assertion target; **user-facing** = promoted for the user, weak assertion expected.
| # | Label | Shape | Format | Role | Assertion intent |
|---|-------|-------|--------|------|------------------|
| 1 | `FastQC raw reads: text summary` | collection (`list`) | `txt` | checkpoint | per-sample module PASS/WARN/FAIL lines, total-sequence counts by regex |
| 2 | `FastQC raw reads: HTML report` | collection (`list`) | `html` | user-facing | existence / size |
| 3 | `Cutadapt trimming report` | collection (`list`) | `txt` | checkpoint | reads processed / written counts; deterministic given fixed input |
| 4 | `Trimmed reads` | collection (`list:paired`) | `fastqsanger.gz` | user-facing | size only |
| 5 | `STAR alignments (BAM)` | collection (`list`) | `bam` | user-facing | size only — binary |
| 6 | `STAR mapping summary` | collection (`list`) | `txt` | checkpoint | `Log.final.out`: input read count, uniquely-mapped % — the text stats behind the binary |
| 7 | `Gene counts per sample` | collection (`list`) | `tabular` | checkpoint | strongest checkpoint in the workflow: exact per-gene integer counts, addressable by gene id |
| 8 | `featureCounts assignment summary` | collection (`list`) | `tabular` | checkpoint | Assigned / Unassigned\_NoFeatures / Unassigned\_Ambiguous totals — also the direct read-out of whether strandedness was set right |
| 9 | `DESeq2 normalized counts` | data | `tabular` | checkpoint | supports the paper's sanity check that SCF1 sits in the top 2.5% of AR0382 expression |
| 10 | `DESeq2 results: tnSWI1 vs AR0382` | data | `tabular` | checkpoint | full result table; SCF1 = `B9J08_001458` present, negative log2FC |
| 11 | `DESeq2 results: AR0387 vs AR0382` | data | `tabular` | checkpoint | full result table; SCF1 negative log2FC, ~29-fold per the paper |
| 12 | `Significant genes: tnSWI1 vs AR0382` | data | `tabular` | checkpoint | filtered at inputs 6 and 7; SCF1 is the top row by significance |
| 13 | `Significant genes: AR0387 vs AR0382` | data | `tabular` | checkpoint | filtered; SCF1 the most down-regulated gene |
| 14 | `DESeq2 diagnostic plots` | data | `pdf` | user-facing | existence / size |
Design notes:
- Outputs 5 and 6 are a deliberate pair, and so are 1 and 2: where the natural output is binary or a rendered report, the sibling text/stats artifact is promoted alongside it so the workflow has something assertable behind the thing users want to look at.
- Output 7 is the single most valuable checkpoint. It is deterministic given a fixed reference, annotation, and strandedness, it is addressable per gene id, and it fails loudly if any of those three is wrong — which is precisely where this reconstruction's largest unknowns sit.
- **Two contrasts, two result outputs.** The paper reports tnSWI1 vs AR0382 and AR0387 vs AR0382. This brief fixes the *output surface* at two labeled result tables and two labeled filtered tables. Whether those come from one DESeq2 run over a three-level factor or two runs over two-level factors is a data-flow/wrapper decision, not an interface one. Ledger: `deseq2-contrast-realization-unsettled`.
- No merged count matrix is exposed: the Galaxy DESeq2 tool consumes per-sample count files directly, so a merge step would be invented rather than translated.
- No output uses `hide` or `delete_intermediate_datasets`, and no promoted checkpoint relies on `rename` for its identity.
---
## 4. Provenance and confidence
| Decision | Evidence class | Confidence |
|---|---|---|
| Six-step tool chain (FastQC, Cutadapt, RNA STAR, featureCounts, DESeq2, filter) | stated in the supplement | high |
| Paired-end, 2 × 50 bp, six runs, three conditions, n = 2 | deposited metadata (PRJNA904261) | high |
| Element identifiers = deposited sample names | deposited metadata | high |
| Genome = GCA_002759435.2 (B8441) | stated in the supplement | high |
| Significance thresholds (\|FC\| > 2, padj < 0.05) | stated in the supplement | high |
| Cutadapt = quality trimming at Phred 20, no adapter | stated cutoff; no-adapter reading is inference | medium |
| Strandedness = reverse | inference from kit name only | low |
| One-factor / three-level / n = 2 DESeq2 design | inferred from the deposited runs; no design formula in the paper | medium |
| Annotation GTF source | absent from the paper | none — open |
| Tool versions | absent from the paper for every Galaxy step | none — open |
| `sample_sheet:paired` input variant | design inference from Galaxy collection semantics | medium |
| Reference genome as history FASTA | design inference on portability grounds | medium |
**Version fidelity is not claimable.** The paper versions only its non-Galaxy software (R 4.0.3, DescTools 0.99.49, survminer 0.4.9, Fiji 1.52, CellProfiler 3.1.9). Any workflow built from this brief pins its own tool versions and reproduces the authors' *method*, never their exact software stack. This should be stated on the workflow itself, not just here. Ledger: `galaxy-tool-versions-unpinnable-from-source`.
---
## 5. Open questions
Each of these is also an open-requirements ledger entry; the ledger is the machine-readable form and the entry id is given so the two do not drift.
1. **Which annotation?** NCBI RefSeq GFF for GCA_002759435.2, or FungiDB's B8441 GFF. The choice changes gene IDs, counts, and the featureCounts `-g` attribute. Largest single gap in reproducing this analysis. → `featurecounts-annotation-source-unnamed`
2. **Is the library really reverse-stranded?** Only the kit name supports it. Output 8 is the empirical check: a wrong setting shows up as a large `Unassigned_NoFeatures` fraction. → `rnaseq-strandedness-inferred-from-kit-name`
3. **How does per-sample `condition` reach DESeq2's factor levels**, given that sample-sheet column metadata does not propagate through map-over? Fallback named in §2.1. → `sample-sheet-condition-to-deseq2-factor-wiring`
4. **Can a `sample_sheet:paired` workflow input be expressed as a test fixture** in a Planemo/IWC `-tests.yml` job block? If not, the §2.1 fallback becomes mandatory and this interface changes. → `sample-sheet-input-test-fixture-expressibility`
5. **One three-level DESeq2 run or two two-level runs** to produce the two contrasts? → `deseq2-contrast-realization-unsettled`
6. **Does usegalaxy.org carry a built-in B8441 index?** Not checked; genome settled provisionally as a history FASTA. → `reference-genome-delivery-shape-unverified`
7. **Cutadapt adapter sequence and minimum-length filter** are both unstated. → `cutadapt-adapter-and-length-filter-unstated`
8. **No tool versions for any Galaxy step.** → `galaxy-tool-versions-unpinnable-from-source`
9. **tnBCY1 vs AR0382 (Fig. S2) cannot be built** — no tnBCY1 runs were deposited. Recorded as dropped source work, not as a gap the pipeline might yet close. → `tnbcy1-contrast-not-carried`
10. **Pipeline B is not carried.** → `pipeline-b-tdna-mapping-not-carried`
Methods, tools, sample data, references, and workflow intent extracted from a primary paper, normalized into the shared free-form source summary handoff.
# Free-form source summary — Santana et al. 2023, *Science*: Scf1, a *Candida auris*-specific adhesin
## Source
- **Title:** A *Candida auris*-specific adhesin, Scf1, governs surface association, colonization, and virulence
- **Authors:** Santana DJ, Anku JAE, Zhao G, Zarnowski R, Johnson CJ, Hautau H, Visser ND, Ibrahim AS, Andes D, Nett JE, Singh S, O'Meara TR
- **Journal:** *Science* 381(6665):1461–1467, 29 September 2023
- **DOI:** 10.1126/science.adf8972 · **PMID:** 37769084 · **PMCID:** PMC11235122
- **What was read:** full main text (PMC JATS full-text XML) **and** the complete supplementary Materials and Methods (NIHMS2004453-supplement-2.pdf, 60 pp, including Tables S1–S5). The main text carries no Methods section; every computational detail below comes from the supplement.
## Workflow intent
The paper is a wet-lab genetics/cell-biology study. Its computational content is two small, self-contained, **short-read Illumina** analyses, both of which the authors state were run **on the Galaxy public server at usegalaxy.org**. That makes this paper an unusually direct Galaxy-workflow reconstruction target: the tool chain named in the Methods is already a Galaxy tool chain.
The two analyses are independent and have different shapes:
- **Pipeline A — bulk RNA-seq differential expression.** Identify genes dysregulated in a low-adhesion insertional mutant (`tnSWI1`) and in a naturally low-adhesion clinical isolate (AR0387), each versus the high-adhesion parent AR0382. This is what surfaced *SCF1* (B9J08_001458) as the top dysregulated ORF (Fig. 1D, Fig. S2, Fig. S5A).
- **Pipeline B — T-DNA insertion-site mapping from WGS.** Locate *Agrobacterium tumefaciens*-mediated transformation (AtMT) transgene integration sites in insertional mutants by soft-clip extraction: align reads to the T-DNA plasmid, pull the clipped genomic flanks, re-align those to the host genome.
Pipeline A is the better primary target (standard Galaxy tools end to end, 6 public paired-end runs, two clean contrasts). Pipeline B is fully specified in the Methods but has one non-Galaxy step (see Open questions).
---
## Pipeline A — RNA-seq differential expression
### Steps, tools, parameters (all as stated in the supplement)
| # | Step | Tool named in paper | Parameters stated |
|---|------|--------------------|-------------------|
| 1 | Read QC | **FastQC** | none stated ("read quality was assessed") |
| 2 | Quality trimming | **Cutadapt** | "Phred cutoff score of 20"; no adapter sequence given |
| 3 | Splice-aware alignment | **RNA STAR** | "default parameters" |
| 4 | Feature quantification | **featureCounts** | none stated |
| 5 | Differential expression | **DESeq2** | none stated |
| 6 | Significance filter | (DESeq2 output filter) | \|fold change\| > 2 **and** adjusted p-value < 0.05 |
Platform: "Galaxy web platform public server at usegalaxy.org". No tool versions are given for any of the six steps.
### Library / sequencing facts
- Sequencing by SeqCenter (Pittsburgh, PA).
- Library prep: Illumina **Stranded Total RNA Prep with Ribo-Zero Plus**, 10 bp IDT for Illumina indices. *Stranded* — the strandedness setting matters for featureCounts and is not spelled out beyond the kit name.
- Instrument: **NextSeq 2000**, **2 × 50 bp paired-end**.
- Input material: RNA from cells grown to mid-exponential phase in YPD at 30 °C (formamide extraction, RNeasy cleanup, DNase-treated).
### Reference data
- **Genome:** *C. auris* **B8441**, NCBI assembly **GCA_002759435.2** (`Cand_auris_B8441_V2`). Same assembly is used as the reference throughout the paper (also for primer design and for Pipeline B).
- **Annotation:** not stated. featureCounts requires a GTF/GFF; the paper never names one. FungiDB and the Candida Gene Order Browser (CGOB) are cited, but only for synteny inspection of the *SCF1* locus, not as the counting annotation.
### Sample data (public, resolvable — strong test-data leads)
**BioProject PRJNA904261** (SRA study SRP409192), RNA-Seq, TRANSCRIPTOMIC, RANDOM selection, PAIRED, Illumina NextSeq 2000, submitted by University of Michigan. Six runs, 2 biological replicates × 3 conditions, ~22–30 M spots each (~2.2–3.0 Gbp per run):
| Run | Experiment | Sample | Condition |
|-----|-----------|--------|-----------|
| SRR22376032 | SRX18346538 | AR0382_A | parent AR0382 (clade I, high adhesion), rep A |
| SRR22376031 | SRX18346539 | AR0382_B | parent AR0382, rep B |
| SRR22376030 | SRX18346540 | AR0387_A | AR0387 (clade I, low adhesion), rep A |
| SRR22376029 | SRX18346541 | AR0387_B | AR0387, rep B |
| SRR22376028 | SRX18346542 | AR0382_tnSWI1_A | AR0382 tnSWI1 (B9J08_003460) insertional mutant, rep A |
| SRR22376027 | SRX18346543 | AR0382_tnSWI1_B | AR0382 tnSWI1, rep B |
Taxon 498019 (*Candidozyma auris*, formerly *Candida auris*). BioSamples SAMN31835905–SAMN31835910.
### Contrasts and expected biological result (useful as workflow assertions)
- **tnSWI1 vs AR0382:** *SCF1* (B9J08_001458) is the strongest and most significantly dysregulated gene (down). No significant dysregulation of the ALS or IFF/HYR adhesin families.
- **AR0387 vs AR0382:** *SCF1* is the most down-regulated gene; ~29-fold expression difference between the two wild-type isolates. Little transcriptome overlap with tnSWI1.
- A third comparison, tnBCY1 (B9J08_002818) vs AR0382 (Fig. S2), is described in the text — *SCF1* strongly down, *IFF4109* not — but **no tnBCY1 runs exist in PRJNA904261**. Only three conditions were deposited.
- Background fact usable as a sanity check: *SCF1* expression in AR0382 sits in the top 2.5% of all genes.
---
## Pipeline B — AtMT T-DNA insertion-site mapping (WGS)
### Steps, tools, parameters
| # | Step | Tool named in paper | Parameters stated |
|---|------|--------------------|-------------------|
| 1 | Read QC | **FastQC** | none stated |
| 2 | Quality trimming | **Trimmomatic** | "Phred quality cutoff of 20"; trimming mode (SLIDINGWINDOW / TRAILING / LEADING) not stated |
| 3 | Align to T-DNA plasmid | **BWA-MEM** | reference = *linearized* pTO128 (pPZP-NAT); **minimum seed length = 50**, **band width = 2** |
| 4 | Extract soft-clipped flanks | **`extractSoftClipped` from SE-MEI** (https://github.com/dpryan79/SE-MEI) | none stated; operates on the aligned BAM |
| 5 | Align flanks to host genome | **BWA-MEM** | **default configuration**; reference = *C. auris* B8441, GCA_002759435.2 |
| 6 | Confirmation | Sanger sequencing of each locus | wet-lab, out of workflow scope |
Platform: also stated as run on usegalaxy.org (the Galaxy statement covers the sequencing-data analysis in this section).
The unusual parameters in step 3 (seed 50, band width 2) are deliberate: they force long exact matches to the T-DNA and suppress gapped extension, so the read portion that is host genomic DNA stays soft-clipped rather than being forced into the plasmid alignment. Any reimplementation must preserve them.
### Library / sequencing facts
- Sequencing by SeqCenter; library prep based on the **Illumina Nextera** kit.
- Instrument: **NextSeq 550**, **2 × 150 bp paired-end**.
- Input: genomic DNA (PCA extraction) from AtMT mutants of interest.
### Sample data
**BioProject PRJNA904262** (SRA study SRP409193), WGS, GENOMIC, RANDOM, PAIRED, Illumina NextSeq 550. Two runs — these are **pools**, not single mutants:
| Run | Experiment | Sample | Spots | Avg length |
|-----|-----------|--------|-------|-----------|
| SRR22376034 | SRX18346544 | Pool_A | 16,373,182 | 294 (2 × ~147) |
| SRR22376033 | SRX18346545 | Pool_B | 19,413,010 | 293 (2 × ~147) |
BioSamples SAMN31836173–SAMN31836174.
### Reference data
- **Host genome:** same, GCA_002759435.2 (B8441).
- **T-DNA plasmid:** pTO128 (pPZP-NAT), carried by *A. tumefaciens* EHA105 strain `At pTO131`. Cited to reference (52) (the prior AtMT method paper), **not** to a public accession. The linearized sequence is a required input with no resolvable public URL found in the paper.
---
## Other computational / analytical methods in the paper (not sequencing workflows)
Recorded for completeness; none of these are plausible Galaxy workflow steps, and none are the workflow this pipeline should build.
- **Homology search:** BLAST of full-length Scf1 and of its N-terminal domain, E-value cutoff 0.05; hits limited to low-complexity repeats discarded. Reciprocal cross-BLAST of Scf1 vs *S. cerevisiae* and Flo11 vs *C. auris* (no significant homology at E ≤ 0.05).
- **Domain annotation:** UniProt automatic annotations (Scf1 = UniProt **A0A2H1A319**, N-terminal domain length 228 aa, 7.9% Arg, 14.5% Arg+Lys — Table S2).
- **Synteny:** FungiDB and Candida Gene Order Browser (CGOB) annotations for the *SCF1* locus.
- **Structure prediction:** **AlphaFold2 via ColabFold** on the N-terminal domain; **Foldseek** structure search (returned Fibronectin-III-fold proteins across domains of life, including FLO11 homologs).
- **Allelic comparison (Table S1):** Scf1 N-terminal domain % identity and tandem-repeat counts across clade representatives B8441 (I, reference), B11220 (II), B11221 (III), B11243 (IV), IFRC2087 (V), and *C. haemulonii* B11899 (60.9% NT identity). No variant-calling method is described for this table.
- **Primer design:** NCBI Primer-BLAST against GCA_002759435.2 (*C. auris*) or GCA_002926055.1 (*C. haemulonii* B11899).
- **Image analysis:** Fiji/ImageJ **1.52** custom macro (brightfield edge-detection segmentation → binary mask) → **CellProfiler 3.1.9** cell counting; z-factor assay QC (mean 0.7167, acceptance 0.5 < z < 1.0), mutant hit calling at z-score < −3 across 2,560 arrayed insertional mutants.
- **Flow cytometry analysis:** FlowJo v10.8.2; plate-reader analysis in Gen5 3.12.
- **Statistics:** R **4.0.3**, DescTools **0.99.49** (ANOVA), survminer **0.4.9** (survival). Student's t-test; one-way ANOVA with Tukey's or Dunnett's post hoc; Pearson correlation; Mantel–Haenszel log-rank with Benjamini–Hochberg correction for survival. Source data provided as Data S1 (`NIHMS2004453-supplement-Data_1_Source_Data.xlsx`).
## Strains and other identifiers worth carrying forward
- Key genes: **SCF1 = B9J08_001458** (the paper's discovery), **IFF4109 = B9J08_004109**, **SWI1 = B9J08_003460**, **BCY1 = B9J08_002818**, **ALS4112 = B9J08_004112**, **IFF4892 = B9J08_004892**. *C. haemulonii* SCF1 homolog = **CXQ85_003100**. AR0381 SCF1 allele = **CJI96_0001187**.
- Twelve ALS/IFF-HYR adhesin deletions in AR0382: B9J08_002582, _004498, _004112 (ALS); _004100, _004109, _004098, _004110, _001531, _004892, _001155, _004451, _000675 (IFF/HYR).
- Isolates: AR0382 (= B11109, clade I, high adhesion), AR0387 (= B8441, clade I, low adhesion — the reference-assembly strain), AR0381 (= B11220, clade II, low adhesion), plus AR0383–AR0390, AR0931, AR0932, AR1097, AR1099–AR1105, Chicago-1..4 from the CDC/FDA Antibiotic Resistant Isolate Bank. 19 *C. albicans* isolates from ATCC; *C. haemulonii* AR0395 (B10441).
## Assumptions carried forward
1. **The Galaxy-relevant workflow is Pipeline A (RNA-seq DE), with Pipeline B as a second candidate.** The paper contains no other analysis that consumes sequencing reads. Everything else is wet-lab or desktop software.
2. **The DESeq2 design is a simple one-factor, three-level condition model** (AR0382 / AR0387 / tnSWI1) with n = 2, yielding two pairwise contrasts against AR0382. The paper states the contrasts and the significance thresholds but never writes out a design formula; n = 2 is inferred from the six deposited runs, not stated in the Methods.
3. **Strandedness:** the Ribo-Zero Plus *Stranded* Total RNA kit is reverse-stranded (dUTP) in Illumina's standard chemistry, so featureCounts would run reverse-stranded. The paper does not say this; it is inferred from the kit name.
4. **The B8441 reference is used for RNA-seq alignment even for AR0382/AR0387 samples**, i.e. reads from a different clade-I isolate are mapped to the B8441 assembly. This is what the Methods say; it is not a transcription error.
5. **"Cutadapt with a Phred cutoff score of 20"** is read as quality trimming (`-q 20`), not adapter trimming — no adapter sequence is given anywhere.
6. Tool versions for the Galaxy steps are unknowable from the paper; any Galaxy workflow built from this must pin its own versions and cannot claim to reproduce the authors' exact versions.
## Open questions
1. **featureCounts annotation is unspecified.** No GTF/GFF source is named for the B8441 assembly. A workflow must choose one (NCBI RefSeq GFF for GCA_002759435.2, or FungiDB's B8441 GFF) and that choice will shift gene IDs and counts. This is the single largest gap for reproducing Pipeline A.
2. **No tool versions for any Galaxy step** (FastQC, Cutadapt, RNA STAR, featureCounts, DESeq2, Trimmomatic, BWA-MEM). Only non-Galaxy software is versioned (R 4.0.3, DescTools 0.99.49, survminer 0.4.9, Fiji 1.52, CellProfiler 3.1.9, FlowJo 10.8.2, Gen5 3.12).
3. **Trimmomatic trimming mode is unstated** for Pipeline B — "Phred quality cutoff of 20" could be SLIDINGWINDOW, TRAILING, or AVGQUAL. Minimum-length filtering is likewise unstated for both pipelines.
4. **`extractSoftClipped` (SE-MEI) has no known Galaxy Tool Shed wrapper.** Pipeline B cannot be assembled from stock Galaxy tools as written. Either a wrapper must be authored, or the step must be substituted (e.g. samtools/awk soft-clip extraction, or a clip-aware structural-variant caller) — a substitution changes the method and should be flagged as such, not presented as a faithful port.
5. **The pTO128 (pPZP-NAT) plasmid sequence has no public accession in this paper.** Pipeline B's step-3 reference is unresolvable from the publication alone; it would have to come from the cited AtMT method paper (ref. 52), from Addgene, or by request. This is a hard test-data blocker for Pipeline B.
6. **PRJNA904262 contains two pooled samples, not per-mutant runs.** The Methods describe insertion-site identification "from mutants of interest", but the deposited data are `Pool_A` / `Pool_B`. How pools map to the individual mutants (tnSWI1, tnBCY1) — barcoding, demultiplexing, or post-hoc assignment — is not described. Any per-mutant claim from this data needs a step the paper does not document.
7. **No tnBCY1 RNA-seq runs are public** even though Fig. S2 reports its transcriptome. A workflow test that tries to reproduce Fig. S2 has no input data.
8. **Fig. 2D's cross-isolate SCF1 expression** (r = 0.87, p = 8.4 × 10⁻⁸, 23 isolates) cannot come from RNA-seq — only 3 conditions were sequenced. It is almost certainly the RT-qPCR assay (ACT1-normalized, primers oTO1251/oTO1252), but the supplement does not explicitly tie the figure to the method. Not a sequencing-workflow concern, but do not model it as one.
9. **No variant-calling pipeline is described** for the "206 coding SNPs between AR0382 and AR0387" claim or for Table S1's allelic comparison. If a downstream design brief wants comparative genomics, it is inventing a step this paper does not contain.
10. **Access note:** the Methods exist only in the supplementary PDF. The PMC full-text XML (retrievable via NCBI eutils `efetch db=pmc`) carries the main text but stops at "Supplementary Material"; the publisher page (science.org) returns 403 and the article is not in the Europe PMC open-access set. Anyone re-deriving this summary must fetch `NIHMS2004453-supplement-2.pdf` specifically.
Reviewable Galaxy workflow test plan (see [[galaxy-workflow-test-plan]]): synthesized test cases with job inputs, expected outputs, assertion intent, fixture provenance, label assumptions, unresolved mappings, and omissions.
# galaxy-test-plan — run auris-scf1 (paper-to-galaxy, phase 8)
# Produced by freeform-summary-to-galaxy-test-plan.
# Reviewable handoff for implement-galaxy-workflow-test. NOT a tests-format file.
plan_version: "1"
source:
kind: freeform
name: "Santana DJ et al. 2023, Science 381(6665):1461-1467 (PMID 37769084, doi 10.1126/science.adf8972) — Pipeline A, bulk RNA-seq differential expression"
derived_from: intent
notes: >-
There is no upstream test evidence: the source is a paper, not a Nextflow or CWL pipeline with
its own fixtures, so every assertion below is synthesized rather than translated. That said,
`derived_from: intent` understates this plan's grounding. Phase 7 (paper-to-test-data) resolved
the fixtures and then MEASURED the workflow's central claim against them — six 200,000-read-pair
head subsets aligned to GCA_002759435.2 with bwa-mem and counted over the NCBI GTF's exon
features. SCF1 (B9J08_001458) carries 414/391 fragments in the two AR0382 replicates against 2/1
in AR0387 and 4/3 in tnSWI1. Assertions resting on that measurement are marked `confidence: high`
even though the schema forces `evidence: intent`, because the only two enum values are
`test-evidence` (upstream test fixtures, which do not exist here) and `intent`. See
warnings[] `no-evidence-class-for-fixture-measured`.
Phase 7's measurement used bwa-mem plus a strand-agnostic exon counter, NOT RNA STAR plus
featureCounts with -s 2. It establishes that the signal survives subsetting; it does not predict
the workflow's exact integers. No assertion in this plan pins an exact count. Count-derived
assertions are expressed as thresholds with a stated margin.
THE DIFFERENTIAL-EXPRESSION NUMBERS IN THIS PLAN COME FROM A DIFFERENT IMPLEMENTATION THAN THE
WORKFLOW RUNS. Phase 7 measured counts only. The statistics quoted below — SCF1's log2 fold
change and adjusted p-value per contrast, and the rank figures in omissions[] and unresolved[] —
were measured during phase 8 with pydeseq2 over a strand-aware count matrix built from the same
six fixture BAMs, as two separate two-level n=2 analyses with no LFC shrinkage, mirroring the
workflow's two DESeq2 nodes. pydeseq2 is a faithful reimplementation, NOT the R DESeq2 the Galaxy
wrapper runs, and its input matrix came from bwa-mem rather than RNA STAR plus featureCounts -s
2. Exact values WILL differ. Measured: tnSWI1 vs AR0382 log2FC -6.93, padj 2.1e-31; AR0387 vs
AR0382 log2FC -7.95, padj 4.1e-18. Read those as evidence that a threshold has margin, never as
values to assert. What survives the implementation difference is margin — 31 and 18 orders of
magnitude on padj against a 0.05 cut, and roughly 5 and 6 log2 units of headroom over the plan's
`<= -2` floor. What does not survive it is rank or any exact value, which is why neither is
asserted anywhere. See warnings[] `deseq2-statistics-measured-with-pydeseq2-not-r`.
workflow:
title: "C. auris Scf1 RNA-seq differential expression (Santana et al. 2023)"
label_source: draft
notes: >-
DELIBERATE DEVIATION FROM THE MOLD DEFAULT. freeform-summary-to-galaxy-test-plan instructs that
labels come from the interface brief and be recorded `assumed` / `label_source: interface-brief`,
on the premise that the concrete workflow does not exist yet. In this run it does: the harness
supplied `galaxy-workflow.gxwf.yml` (27 steps, 9 inputs, 16 outputs, produced by phase 6) as a
phase-8 input, and every input and output label below was read from it and matches byte-for-byte.
Recording them as `assumed` would discard verified information and hand
implement-galaxy-workflow-test a reconciliation job that is already done. Filed as
`test-plan-mold-ignores-available-concrete-workflow` in foundry-feedback.ledger.yml.
The interface brief has itself drifted from the draft (open requirement
`interface-brief-output-and-parameter-surface-drifted-from-draft`), which is a second reason to
bind to the workflow rather than the brief.
Labels are the public API of this workflow. Four of them embed condition level names
(`... tnSWI1 vs AR0382`) that are NOT derived from the `First/Second contrast condition level`
parameters; changing those parameters makes the labels stale without breaking anything, and both
test cases below therefore leave the three level parameters at their defaults.
test_cases:
- id: scf1-both-contrasts-200k
doc: >-
End-to-end run over all six deposited runs of PRJNA904261, subset to 200,000 read pairs each,
against the NCBI GCA_002759435.2 reference and annotation. Exercises the whole DAG — flatten,
FastQC, Cutadapt, RNA STAR, featureCounts, the sample-sheet condition split, both DESeq2
reductions, both two-link significance chains — and asserts the paper's central biological
result: SCF1 (B9J08_001458) is significantly down-regulated in BOTH contrasts against the
AR0382 reference level. This is the plan's primary case; it is the only one that makes
biological claims.
derived_from: intent
provenance: >-
Synthesized from the paper's Fig. 1D / Fig. S5A result statement and its |fold change| > 2 and
adjusted p-value < 0.05 significance criteria (freeform-summary.md §"Contrasts and expected
biological result"), then grounded on phase 7's direct measurement of the resolved fixtures
(test-data-refs.json `verdict`, `expected_outputs.safe`, `expected_outputs.biological`).
job_inputs:
- workflow_label: RNA-seq reads (sample sheet)
label_status: resolved
description: >-
Six paired-end runs, 2 x 50 bp NextSeq 2000, each subset to the first 200,000 read pairs.
Element identifiers are the deposited ENA library_name values and are load-bearing: the
condition split joins the featureCounts collection back to the sample sheet on them.
Identifiers, in sample-sheet row order — AR0382_A (SRR22376032), AR0382_B (SRR22376031),
AR0387_A (SRR22376030), AR0387_B (SRR22376029), AR0382_tnSWI1_A (SRR22376028),
AR0382_tnSWI1_B (SRR22376027).
collection_shape: sample_sheet:paired
datatype: fastqsanger.gz
fixture:
storage: unresolved
location: null
checksum: null
provenance: >-
GENERATED AND HASHED BUT NOT HOSTED — this is the one blocking gap in the plan (open
requirement `test-fixtures-not-hosted`; test-data-refs.json `unresolved`
`fixture-hosting-not-done`). Regenerate deterministically by streaming each ENA FASTQ and
taking `head -n 800000`, then verify each against the per-file `md5_uncompressed` in
test-data-refs.json. Hash the UNCOMPRESSED .fastq: gzip output is not byte-stable across
gzip versions, which is why phase 7 pinned uncompressed md5s and why a `hashes:` block on
the .gz fixtures cannot be authored from what phase 7 supplies. ~61 MB compressed for the
12 files, so the IWC idiom applies: host on Zenodo and reference by URL, substituting for
`<FIXTURE_BASE>` in test-data-refs.json `planemo_test_job_block`.
The `rows:` shape is settled and has a trap: it is a mapping of element identifier to a
POSITIONAL LIST of column values ordered to match `column_definitions` — `AR0382_A:
[AR0382, A]`. Galaxy's own non-paired unit tests in test_cwl_util.py write `rows` as a
dict; that form never reaches `validate_row()` in those tests and will NOT validate here.
Staging never passes `column_definitions`, which is harmless only because this workflow's
`Project sample sheet to tabular` step sets `include_headers: false`. Do NOT set
`decompress: true` on the reads — the input declares fastqsanger.gz and the .gz must
survive.
- workflow_label: Reference genome FASTA
label_status: resolved
description: >-
C. auris B8441, NCBI assembly GCA_002759435.2 (Cand_auris_B8441_V2), used whole. At 12.4 Mb
it needs no subsetting, so the gene ID space stays exactly the real one and RNA STAR builds
its own index inside each of the six mapped jobs.
collection_shape: null
datatype: fasta
fixture:
storage: remote-url
location: https://ftp.ncbi.nlm.nih.gov/genomes/all/GCA/002/759/435/GCA_002759435.2_Cand_auris_B8441_V2/GCA_002759435.2_Cand_auris_B8441_V2_genomic.fna.gz
checksum: MD5:a008b270d3aaa04736a8bb7daf3f6dd5
provenance: >-
Pinned NCBI FTP URL, staged with `decompress: true`. Needs no hosting. The md5 is of the
COMPRESSED file as served; see unresolved `hash-vs-decompress-ordering` before pairing it
with a `hashes:` block on a `decompress: true` input.
- workflow_label: Gene annotation GTF
label_status: resolved
description: >-
NCBI's own GTF for the exact accession the paper names. The paper names no annotation at
all, so this choice is load-bearing for both consumers (RNA STAR splice junctions,
featureCounts gene assignment) and determines whether SCF1 appears as B9J08_001458 at all.
Phase 7 downloaded and inspected it: true GTF (#gtf-version 2.2), `gene_id` values ARE the
paper's locus tags, 5586 distinct gene_id values, SCF1 resolves as B9J08_001458, so the
workflow's `gff_feature_attribute: gene_id` and `gff_feature_type: exon` bindings are
correct as they stand.
collection_shape: null
datatype: gtf
fixture:
storage: remote-url
location: https://ftp.ncbi.nlm.nih.gov/genomes/all/GCA/002/759/435/GCA_002759435.2_Cand_auris_B8441_V2/GCA_002759435.2_Cand_auris_B8441_V2_genomic.gtf.gz
checksum: MD5:6e5b9528d48c0a8fc2c8588e6eeea929
provenance: >-
Pinned NCBI FTP URL, staged with `decompress: true`. Closes open requirement
`featurecounts-annotation-source-unnamed`. Same `hashes:` caveat as the FASTA.
- workflow_label: Reference condition level
label_status: resolved
description: The condition value both contrasts are taken against; must match a `condition` value in the sample sheet exactly. Left at the workflow default.
collection_shape: null
datatype: null
fixture:
storage: null
location: null
checksum: null
provenance: "Workflow default `AR0382`, the paper's high-adhesion parent and the reference level of both contrasts."
- workflow_label: First contrast condition level
label_status: resolved
description: Condition value for the first contrast. Left at the workflow default, because four output labels hard-code this level name and are not derived from this parameter.
collection_shape: null
datatype: null
fixture:
storage: null
location: null
checksum: null
provenance: "Workflow default `tnSWI1`, the AR0382 tnSWI1 insertional mutant."
- workflow_label: Second contrast condition level
label_status: resolved
description: Condition value for the second contrast. Left at the workflow default, same label-staleness reason as the first.
collection_shape: null
datatype: null
fixture:
storage: null
location: null
checksum: null
provenance: "Workflow default `AR0387`, the low-adhesion clinical isolate."
- workflow_label: Strandedness
label_status: resolved
description: >-
Left at the workflow default `stranded - reverse`. No longer an inference from the library
kit name: phase 7 measured it. Of R1 reads unambiguously overlapping a single annotated
gene, 98.4% / 98.4% / 98.2% (AR0382_A / AR0387_A / tnSWI1_A) are antisense to the gene,
which is reverse-stranded. Closes open requirement
`rnaseq-strandedness-inferred-from-kit-name`.
collection_shape: null
datatype: null
fixture:
storage: null
location: null
checksum: null
provenance: "Workflow default, confirmed by measurement in test-data-refs.json `inputs.Strandedness.evidence`."
- workflow_label: Adjusted p-value threshold
label_status: resolved
description: Left at the workflow default 0.05, which is the paper's stated criterion. Applied to DESeq2 column c7.
collection_shape: null
datatype: null
fixture:
storage: null
location: null
checksum: null
provenance: "Stated by the paper (adjusted p-value < 0.05)."
- workflow_label: log2 fold change threshold
label_status: resolved
description: >-
Left at the workflow default 1.0. A log2 FC of 1 is exactly the paper's |fold change| > 2
criterion; the filter compares the raw c3 column with no conversion, so a linear 2.0 here
would silently apply a 4-fold cut.
collection_shape: null
datatype: null
fixture:
storage: null
location: null
checksum: null
provenance: "Paper's |fold change| > 2, expressed in log2 units per open requirement `fold-change-threshold-linear-vs-deseq2-log2fc`."
expected_outputs:
- workflow_label: "FastQC raw reads: text summary"
label_status: resolved
description: >-
Twelve elements, not six — the reads are flattened per read direction before QC, so
identifiers are <sample>_forward / <sample>_reverse. The assertable text checkpoint behind
the HTML binary, and the cheapest proof that the flatten side-branch fanned out correctly
and that the fixture is the depth it claims to be.
output_kind: collection
collection_shape: list
assertion_intent:
- family: has_text_matching
intent: >-
Each element's fastqc_data.txt reports exactly 200,000 sequences, proving the fixture
is the declared head subset and that no read loss happened upstream of QC. Apply to all
twelve elements via element_tests.
expected_value: 'Total Sequences\s+200000'
tolerance: null
element_identifier: null
evidence: intent
confidence: high
- family: has_text
intent: "Each element carries the Basic Statistics module, i.e. FastQC actually parsed the file rather than erroring into an empty report."
expected_value: ">>Basic Statistics"
tolerance: null
element_identifier: null
evidence: intent
confidence: high
- family: has_text
intent: >-
The twelve element identifiers are addressed explicitly through element_tests, which is
itself the cardinality and identifier-space assertion: AR0382_A_forward,
AR0382_A_reverse, AR0382_B_forward, AR0382_B_reverse, AR0387_A_forward,
AR0387_A_reverse, AR0387_B_forward, AR0387_B_reverse, AR0382_tnSWI1_A_forward,
AR0382_tnSWI1_A_reverse, AR0382_tnSWI1_B_forward, AR0382_tnSWI1_B_reverse. Do NOT add
an `attributes: {collection_type: list}` assertion here — see unresolved
`promoted-collection-type-string-unverified`.
expected_value: 12
tolerance: null
element_identifier: null
evidence: intent
confidence: high
- workflow_label: Cutadapt trimming report
label_status: resolved
description: >-
Six elements on the sample axis. Deterministic text with exact input counts, so it is a
strong and cheap checkpoint that map-over produced one job per sample row and that the
paired shape survived into Cutadapt.
output_kind: collection
collection_shape: list
assertion_intent:
- family: has_text_matching
intent: "Each of the six reports states that 200,000 read pairs were processed — proving the paired-collection map-over kept both mates together on the six-element sample axis."
expected_value: 'Total read pairs processed:\s+200,000'
tolerance: null
element_identifier: null
evidence: intent
confidence: high
- family: has_text
intent: >-
Element_tests address the six sample identifiers explicitly (AR0382_A, AR0382_B,
AR0387_A, AR0387_B, AR0382_tnSWI1_A, AR0382_tnSWI1_B), asserting that the identifier
space did NOT pick up the _forward/_reverse suffixes of the flatten side-branch.
expected_value: 6
tolerance: null
element_identifier: null
evidence: intent
confidence: high
- workflow_label: Trimmed reads
label_status: resolved
description: >-
Six paired elements. Asserted only for shape: that Cutadapt under
`library.type: paired_collection` emitted one paired-inner collection per sample, so no
re-pair node is needed ahead of RNA STAR. Content is not asserted — FASTQ quality strings
are read-id dependent and the downstream counts table is the stronger checkpoint.
output_kind: collection
collection_shape: list:paired
assertion_intent:
- family: has_size
intent: >-
Each of the six outer elements has non-empty `forward` and `reverse` inner elements
(nested element_tests: outer keyed by sample identifier, inner using `elements:`).
Catches the failure mode where the paired structure collapses or one mate is dropped.
expected_value: null
tolerance:
kind: none
magnitude: null
rationale: "Min-only bound; trimmed output size depends on adapter content and is not predicted here."
element_identifier: null
evidence: intent
confidence: medium
- workflow_label: STAR mapping summary
label_status: resolved
description: >-
Log.final.out per sample — the assertable text behind the binary BAM, which is why the BAM
itself is an intentional omission. Proves the six STAR jobs ran and that the history-FASTA
index build produced a usable genome.
output_kind: collection
collection_shape: list
assertion_intent:
- family: has_text
intent: "The log carries the uniquely-mapped-reads stanza at all, i.e. STAR completed rather than dying in genomeGenerate."
expected_value: "Uniquely mapped reads %"
tolerance: null
element_identifier: null
evidence: intent
confidence: high
- family: has_line_matching
intent: >-
Uniquely mapped reads % is at least 80% in every element. Phase 7 measured 198.2-198.6k
of 200k R1 reads primary-mapping with bwa-mem (>99%); STAR against the same reference
will be lower but comfortably above 80%, so the threshold carries a wide margin. NOTE
the full-line anchoring convention recorded in warnings[].
expected_value: '\s*Uniquely mapped reads % \|\s+(?:8[0-9]|9[0-9]|100)\.[0-9]+%'
tolerance: null
element_identifier: null
evidence: intent
confidence: medium
- workflow_label: Gene counts per sample
label_status: resolved
description: >-
The strongest deterministic checkpoint in the workflow: exact per-gene integer counts,
addressable by gene id, keyed by the six sample element identifiers — and the only place
the SCF1 signal can be asserted independently of DESeq2. featureCounts on a fixed
reference, annotation and strandedness is deterministic, so the IWC comparison notes are
right that an existence-only probe here would be the smell the anti-patterns note names.
This plan is stricter than the corpus default here without pinning exact integers, which
phase 7 cannot supply because it measured with a different aligner.
output_kind: collection
collection_shape: list
assertion_intent:
- family: has_n_columns
intent: "featureCounts `format: tabdel_short` emitted the two-column gene-id/count table the DESeq2 per-level ports consume."
expected_value: 2
tolerance: null
element_identifier: null
evidence: intent
confidence: high
- family: has_n_lines
intent: >-
One row per gene in the annotation. Phase 7 counted exactly 5586 distinct gene_id
values in the NCBI GTF. The ±1 allowance is for the header row: the DESeq2 steps bind
`header: true` on their counts inputs, implying the wrapper emits one, but that was not
read directly from the featureCounts wrapper in this run.
expected_value: 5586
tolerance:
kind: delta
magnitude: 1
rationale: "Header-row presence on featureCounts output_short is inferred from the DESeq2 steps' `header: true` binding, not verified."
element_identifier: null
evidence: intent
confidence: medium
- family: has_line_matching
intent: >-
THE ANNOTATION-SPACE GUARD. At least 5000 rows are a bare B9J08_ six-digit locus tag
plus an integer count. This is the assertion that fails loudly if the annotation input
is swapped for the FungiDB GFF3 (keyed on `ID`, different gene ID space) or if
`gff_feature_attribute` drifts off `gene_id` — failure modes whose natural symptom is a
complete, plausible counts table of the WRONG identifiers rather than an error.
expected_value: 'B9J08_[0-9]{6}\t[0-9]+'
tolerance: null
element_identifier: null
evidence: intent
confidence: high
- family: has_line_matching
intent: "SCF1 is present as a row in every one of the six count tables, with an integer count."
expected_value: 'B9J08_001458\t[0-9]+'
tolerance: null
element_identifier: null
evidence: intent
confidence: high
- family: has_line_matching
intent: >-
SCF1 is highly expressed in the AR0382 replicates — at least 100 fragments. Phase 7
measured 414 and 391, so the threshold sits about 4x below the measurement; it is a
threshold, not the exact count, because phase 7 counted with bwa-mem plus a
strand-agnostic counter rather than STAR plus featureCounts -s 2.
expected_value: 'B9J08_001458\t[0-9]{3,}'
tolerance: null
element_identifier: AR0382_A
evidence: intent
confidence: medium
- family: has_line_matching
intent: "Same SCF1 >= 100 threshold on the second AR0382 replicate (measured 391)."
expected_value: 'B9J08_001458\t[0-9]{3,}'
tolerance: null
element_identifier: AR0382_B
evidence: intent
confidence: medium
- family: has_line_matching
intent: >-
SCF1 is at most 99 fragments in AR0387_A. Measured 2, so the threshold sits ~25x above
the measurement. Together with the two AR0382 assertions this pins the paper's central
separation at the counts table, upstream of DESeq2 and independent of any statistical
model.
expected_value: 'B9J08_001458\t[0-9]{1,2}'
tolerance: null
element_identifier: AR0387_A
evidence: intent
confidence: medium
- family: has_line_matching
intent: "SCF1 <= 99 in AR0387_B (measured 1)."
expected_value: 'B9J08_001458\t[0-9]{1,2}'
tolerance: null
element_identifier: AR0387_B
evidence: intent
confidence: medium
- family: has_line_matching
intent: "SCF1 <= 99 in AR0382_tnSWI1_A (measured 4)."
expected_value: 'B9J08_001458\t[0-9]{1,2}'
tolerance: null
element_identifier: AR0382_tnSWI1_A
evidence: intent
confidence: medium
- family: has_line_matching
intent: "SCF1 <= 99 in AR0382_tnSWI1_B (measured 3)."
expected_value: 'B9J08_001458\t[0-9]{1,2}'
tolerance: null
element_identifier: AR0382_tnSWI1_B
evidence: intent
confidence: medium
- workflow_label: featureCounts assignment summary
label_status: resolved
description: >-
Assigned / Unassigned_NoFeatures / Unassigned_Ambiguous totals per sample — and the direct
read-out of whether the Strandedness parameter reached featureCounts as the right -s value.
The two assertions below are a paired strandedness regression detector: with -s 2 (correct)
Assigned is ~180,000 and NoFeatures ~20,000; with -s 1 (inverted) the two swap, and each
assertion fails on its own digit-count bound.
output_kind: collection
collection_shape: list
assertion_intent:
- family: has_line_matching
intent: >-
Assigned is between 150,000 and 199,999 fragments, i.e. >=75% of the ~198k fragments
entering featureCounts. Phase 7 measured 90-91% of primary-mapped R1 falling inside an
annotated exon (178.9k-181.4k assigned). Under an inverted -s 1 the Assigned total
collapses to five digits and this fails.
expected_value: 'Assigned\t1[5-9][0-9]{4}'
tolerance: null
element_identifier: null
evidence: intent
confidence: medium
- family: has_line_matching
intent: >-
Unassigned_NoFeatures is below 100,000, i.e. under half the fragments. Under an
inverted -s 1 setting NoFeatures balloons to ~180,000 — six digits — and this fails.
This is the empirical strandedness check the workflow input's own doc names.
expected_value: 'Unassigned_NoFeatures\t[0-9]{1,5}'
tolerance: null
element_identifier: null
evidence: intent
confidence: medium
- workflow_label: "DESeq2 results: tnSWI1 vs AR0382"
label_status: resolved
description: >-
Full raw result table for the first contrast, a single dataset (the workflow's only
reduction happens here). HEADERLESS — deseq2.R writes it with `col.names = FALSE` — which
is what makes `header_lines: '0'` correct on the two downstream Filter1 steps. The column
contract is c1 GeneID, c2 baseMean, c3 log2FoldChange, c4 lfcSE, c5 stat, c6 pvalue,
c7 padj, and it holds ONLY while `lfc_shrinkage_type: none`.
output_kind: dataset
collection_shape: null
assertion_intent:
- family: has_n_columns
intent: >-
CROSS-STEP INVARIANT 1 DETECTOR. Seven columns. `get_result_output_columns()` drops the
`stat` column under any shrinkage, which moves padj from c7 to c6 and makes the
`c7<0.05` predicate — built in a different step — filter on nothing. If anyone changes
`lfc_shrinkage_type` off `none` on either DESeq2 node, this assertion fails. Because
the table is headerless, has_n_columns reads a data row here, which is exactly what is
wanted.
expected_value: 7
tolerance: null
element_identifier: null
evidence: intent
confidence: high
- family: not_has_text
intent: >-
HEADER-ASYMMETRY ASSERTION. The word `baseMean` does not appear anywhere in this table.
deseq_out is written with `col.names = FALSE`, so it carries no header row; this is the
premise `header_lines: '0'` on the significance filters depends on, and the gene ID
space (B9J08_ locus tags) guarantees the token cannot occur as data. If a wrapper
upgrade ever starts emitting a header, this fails before the filters silently start
passing a header line into a numeric comparison. The SAME assertion inverted applies to
the normalized-counts tables below, which DO carry a header.
expected_value: baseMean
tolerance: null
element_identifier: null
evidence: intent
confidence: high
- family: has_line_matching
intent: "SCF1 is present as a full seven-field row in the raw result table."
expected_value: 'B9J08_001458(\t[^\t]*){6}'
tolerance: null
element_identifier: null
evidence: intent
confidence: high
- family: has_line_matching
intent: >-
SCF1's log2FoldChange (c3) is negative — down in tnSWI1 relative to AR0382, the
direction the paper reports. Magnitude is asserted only on the filtered table below,
and the paper's ~29-fold figure is NOT asserted anywhere (see omissions).
expected_value: 'B9J08_001458\t[^\t]+\t-[0-9.]+(\t[^\t]*){4}'
tolerance: null
element_identifier: null
evidence: intent
confidence: high
- workflow_label: "DESeq2 results: AR0387 vs AR0382"
label_status: resolved
description: >-
Full raw result table for the second contrast. Same shape, same header asymmetry, same
column contract as the first. Its normalized counts are not interchangeable with the first
contrast's — size factors are estimated over a different sample set.
output_kind: dataset
collection_shape: null
assertion_intent:
- family: has_n_columns
intent: "CROSS-STEP INVARIANT 1 DETECTOR on the second DESeq2 node. The invariant must hold on BOTH nodes, so it is asserted on both raw tables."
expected_value: 7
tolerance: null
element_identifier: null
evidence: intent
confidence: high
- family: not_has_text
intent: "Headerless, as for the first contrast — the premise of `header_lines: '0'` on this contrast's two significance filters."
expected_value: baseMean
tolerance: null
element_identifier: null
evidence: intent
confidence: high
- family: has_line_matching
intent: "SCF1 is present as a full seven-field row."
expected_value: 'B9J08_001458(\t[^\t]*){6}'
tolerance: null
element_identifier: null
evidence: intent
confidence: high
- family: has_line_matching
intent: "SCF1's log2FoldChange is negative — down in AR0387 relative to AR0382, the paper's headline between-isolate result."
expected_value: 'B9J08_001458\t[^\t]+\t-[0-9.]+(\t[^\t]*){4}'
tolerance: null
element_identifier: null
evidence: intent
confidence: high
- workflow_label: "DESeq2 normalized counts: tnSWI1 vs AR0382"
label_status: resolved
description: >-
Normalized counts for the first contrast's four samples. THE OPPOSITE HALF OF THE HEADER
ASYMMETRY: deseq2.R writes this table with `col.names = NA`, so it DOES carry a header row
— padded with a leading blank field so the header is the same width as the data rows. That
is why sample-name text assertions are valid here and invalid on deseq_out. This output is
also the only place the condition split's correctness becomes visible at workflow level,
since every step of the split is hidden.
output_kind: dataset
collection_shape: null
assertion_intent:
- family: has_line_matching
intent: >-
CROSS-STEP INVARIANT 2 DETECTOR (condition-split half). SCF1's row carries EXACTLY four
numeric sample fields. If `header_lines` on `Select first-contrast samples` were ever
changed from '0' to '1', Filter1 would pass row 1 of the sample metadata table
(AR0382_A) through unconditionally regardless of the `c2=='tnSWI1'` predicate, that
sample would enter the first-contrast counts sub-collection as well as the
reference-level one, and this contrast would run on five sample columns instead of
four. Asserting on a DATA row rather than on the header sidesteps the open question of
whether the padded header counts as four or five fields.
expected_value: 'B9J08_001458(\t[0-9.eE+-]+){4}'
tolerance: null
element_identifier: null
evidence: intent
confidence: medium
- family: has_text
intent: >-
The header names the four samples this contrast should see. Assert each of AR0382_A,
AR0382_B, AR0382_tnSWI1_A and AR0382_tnSWI1_B as a separate has_text. This is the
membership half of the condition-split check — the workflow's join key is the element
identifier, and this is the only workflow-level output that shows which identifiers
actually reached the reduction.
expected_value: AR0382_tnSWI1_A
tolerance: null
element_identifier: null
evidence: intent
confidence: medium
- family: not_has_text
intent: >-
No AR0387 sample leaked into the first contrast. `AR0387` cannot occur as data — gene
ids are B9J08_ locus tags — so this is an unambiguous negative on the split.
expected_value: AR0387
tolerance: null
element_identifier: null
evidence: intent
confidence: high
- workflow_label: "DESeq2 normalized counts: AR0387 vs AR0382"
label_status: resolved
description: >-
Normalized counts for the second contrast's four samples. Headered, same as the first
contrast, and the same condition-split visibility — here against a leak of the tnSWI1
samples.
output_kind: dataset
collection_shape: null
assertion_intent:
- family: has_line_matching
intent: "CROSS-STEP INVARIANT 2 DETECTOR on the second contrast's split: SCF1's row carries exactly four numeric sample fields."
expected_value: 'B9J08_001458(\t[0-9.eE+-]+){4}'
tolerance: null
element_identifier: null
evidence: intent
confidence: medium
- family: has_text
intent: "The header names the four samples this contrast should see: AR0382_A, AR0382_B, AR0387_A and AR0387_B, each as a separate has_text."
expected_value: AR0387_A
tolerance: null
element_identifier: null
evidence: intent
confidence: medium
- family: not_has_text
intent: "No tnSWI1 sample leaked into the second contrast. `tnSWI1` cannot occur as data."
expected_value: tnSWI1
tolerance: null
element_identifier: null
evidence: intent
confidence: high
- workflow_label: "Significant genes: tnSWI1 vs AR0382"
label_status: resolved
description: >-
Endpoint of the first contrast's two-link chain — rows of the raw table that cleared BOTH
adjusted p-value < 0.05 (c7) and |log2 fold change| > 1 (abs(c3)). Membership in this table
IS the paper's significance criterion, which is why the biological claim is asserted here
rather than by re-deriving thresholds from the raw table.
output_kind: dataset
collection_shape: null
assertion_intent:
- family: has_n_columns
intent: "Seven columns — the filter chain preserved the raw table's shape rather than reshaping it, and invariant 1 still holds at the chain's endpoint."
expected_value: 7
tolerance: null
element_identifier: null
evidence: intent
confidence: high
- family: has_line_matching
intent: >-
THE PAPER'S CENTRAL CLAIM, FIRST CONTRAST. SCF1 appears in the significant-genes table,
which by construction means padj < 0.05 AND |log2FC| > 1. Phase 7 measured 414/391
fragments in AR0382 against 4/3 in tnSWI1 — a separation DESeq2 will call significant
at n=2 with a wide margin.
expected_value: 'B9J08_001458(\t[^\t]*){6}'
tolerance: null
element_identifier: null
evidence: intent
confidence: high
- family: has_line_matching
intent: >-
SCF1's log2FoldChange is at most -2, i.e. at least 4-fold down. This is a threshold,
deliberately far below the ~270-fold raw separation phase 7 measured and deliberately
NOT the paper's ~29-fold figure, which this plan does not assert anywhere. The regex
encodes "negative with an integer part of 2 or more". MEASURED for this contrast:
log2FC -6.93 at padj 2.1e-31, so the -2 floor clears by ~5 log2 units and the 0.05 cut
by 31 orders of magnitude. The measured value is NOT the assertion: it comes from
pydeseq2 over bwa-mem counts, not from the wrapper's R DESeq2 over featureCounts -s 2
output, and will differ in the last digits at least (warnings[]
`deseq2-statistics-measured-with-pydeseq2-not-r`). The margin is what makes the
threshold safe across that difference.
expected_value: 'B9J08_001458\t[^\t]+\t-(?:[2-9]|[1-9][0-9]+)\.[0-9]+(\t[^\t]*){4}'
tolerance: null
element_identifier: null
evidence: intent
confidence: high
- workflow_label: "Significant genes: AR0387 vs AR0382"
label_status: resolved
description: >-
Endpoint of the second contrast's chain. The paper's headline between-isolate result: SCF1
the most down-regulated gene between AR0382 and the low-adhesion clinical isolate AR0387.
Measurement backs "most down-regulated" HERE — SCF1 ranks 1st by |log2FC| among the 35
genes clearing both filters, and 1st among the 9 down-regulated ones — but not in the first
contrast, where it ranks 2nd of 63 (2nd of 52 down-regulated). No rank is asserted on
either table; see omissions[] for why, and warnings[] for the implementation caveat on
those figures.
output_kind: dataset
collection_shape: null
assertion_intent:
- family: has_n_columns
intent: "Seven columns at the second chain's endpoint."
expected_value: 7
tolerance: null
element_identifier: null
evidence: intent
confidence: high
- family: has_line_matching
intent: >-
THE PAPER'S CENTRAL CLAIM, SECOND CONTRAST. SCF1 cleared both filters. Phase 7 measured
414/391 against 2/1 — the widest separation in the fixture.
expected_value: 'B9J08_001458(\t[^\t]*){6}'
tolerance: null
element_identifier: null
evidence: intent
confidence: high
- family: has_line_matching
intent: >-
SCF1's log2FoldChange is at most -2. Same threshold and same refusal to assert the
paper's ~29-fold magnitude. MEASURED for this contrast: log2FC -7.95 at padj 4.1e-18,
clearing the -2 floor by ~6 log2 units and the 0.05 cut by 18 orders of magnitude. Same
implementation caveat as the first contrast — pydeseq2 over bwa-mem counts, not the
wrapper's R DESeq2 — so the margin is asserted and the value is not.
expected_value: 'B9J08_001458\t[^\t]+\t-(?:[2-9]|[1-9][0-9]+)\.[0-9]+(\t[^\t]*){4}'
tolerance: null
element_identifier: null
evidence: intent
confidence: high
- workflow_label: "DESeq2 diagnostic plots: tnSWI1 vs AR0382"
label_status: resolved
description: >-
Multi-page PDF (dispersion estimates, PCA, MA plot, sample distance heatmap). Deliberately
weak check: the PDF embeds a creation timestamp and the plots are the user-facing view of
numbers already asserted exactly on the tables above.
output_kind: dataset
collection_shape: null
assertion_intent:
- family: has_size
intent: >-
Non-trivially sized PDF (min only, ~10 KB), catching the "R rendered an empty device"
failure mode. Recorded as a deliberate accepted shortcut per iwc-shortcuts-anti-patterns
§2-3: the sibling tabular checkpoints carry the content, so a size band adds nothing a
min bound does not.
expected_value: 10000
tolerance:
kind: none
magnitude: null
rationale: "Min bound rather than a delta band; PDF size depends on the gene count that survives independent filtering and is not predicted here."
element_identifier: null
evidence: intent
confidence: medium
- workflow_label: "DESeq2 diagnostic plots: AR0387 vs AR0382"
label_status: resolved
description: Multi-page PDF for the second contrast. Same deliberate weak check.
output_kind: dataset
collection_shape: null
assertion_intent:
- family: has_size
intent: "Non-trivially sized PDF (min only, ~10 KB)."
expected_value: 10000
tolerance:
kind: none
magnitude: null
rationale: "Min bound rather than a delta band, as for the first contrast."
element_identifier: null
evidence: intent
confidence: medium
- id: topology-smoke-25k
doc: >-
Fast structural smoke test over the same six samples at 25,000 read pairs each (~7 MB
compressed total, small enough to commit in-repo). Makes NO biological claim: at that depth
SCF1 falls to roughly 50 fragments in AR0382 and near zero elsewhere, per-gene depth drops to
~4.5 reads/gene, and the significance tables may legitimately be empty. What it does buy is a
CI lane that is not hostage to a Zenodo fetch and that still exercises every structural thing
that can silently break: the sample_sheet:paired input shape, the flatten side-branch, the
six-element map-over identifier space, both condition-split joins, both DESeq2 reductions, and
BOTH cross-step invariants. Every assertion below is a subset of case 1's, with the depth
constant and the biological claims removed.
derived_from: intent
provenance: >-
test-data-refs.json `subsetting.shape_only_alternative`, which records this depth as a
committable option and states exactly what it loses. The fixtures at this depth were NOT
generated by phase 7 — only the 200k set was — so they must be produced by the same
deterministic recipe with `head -n 100000`.
job_inputs:
- workflow_label: RNA-seq reads (sample sheet)
label_status: resolved
description: >-
The same six samples and the same six element identifiers as case 1, at 25,000 read pairs
each. The identifier spine is unchanged, because it is the thing this case exists to
exercise.
collection_shape: sample_sheet:paired
datatype: fastqsanger.gz
fixture:
storage: in-repo
location: test-data/
checksum: null
provenance: >-
NOT YET GENERATED. Produce with the same deterministic head-subset recipe as the 200k
fixture but `head -n 100000` per mate, from the same six ENA runs; the source URLs and
source md5s are in test-data-refs.json `inputs["RNA-seq reads (sample sheet)"].elements`.
Record an uncompressed md5 per file, for the same gzip-instability reason. At ~7 MB total
this is under the ~1 MB-per-file IWC in-repo threshold only marginally, so a reviewer may
still ask for Zenodo hosting; see unresolved `smoke-fixture-not-generated`.
- workflow_label: Reference genome FASTA
label_status: resolved
description: Identical to case 1 — the reference is used whole at both depths.
collection_shape: null
datatype: fasta
fixture:
storage: remote-url
location: https://ftp.ncbi.nlm.nih.gov/genomes/all/GCA/002/759/435/GCA_002759435.2_Cand_auris_B8441_V2/GCA_002759435.2_Cand_auris_B8441_V2_genomic.fna.gz
checksum: MD5:a008b270d3aaa04736a8bb7daf3f6dd5
provenance: "Pinned NCBI FTP URL, staged with `decompress: true`. Same as case 1."
- workflow_label: Gene annotation GTF
label_status: resolved
description: Identical to case 1 — the annotation is what fixes the gene ID space, so it must not differ between cases.
collection_shape: null
datatype: gtf
fixture:
storage: remote-url
location: https://ftp.ncbi.nlm.nih.gov/genomes/all/GCA/002/759/435/GCA_002759435.2_Cand_auris_B8441_V2/GCA_002759435.2_Cand_auris_B8441_V2_genomic.gtf.gz
checksum: MD5:6e5b9528d48c0a8fc2c8588e6eeea929
provenance: "Pinned NCBI FTP URL, staged with `decompress: true`. Same as case 1."
- workflow_label: Reference condition level
label_status: resolved
description: Workflow default AR0382, as case 1.
collection_shape: null
datatype: null
fixture: {storage: null, location: null, checksum: null, provenance: "Workflow default."}
- workflow_label: First contrast condition level
label_status: resolved
description: Workflow default tnSWI1, as case 1.
collection_shape: null
datatype: null
fixture: {storage: null, location: null, checksum: null, provenance: "Workflow default."}
- workflow_label: Second contrast condition level
label_status: resolved
description: Workflow default AR0387, as case 1.
collection_shape: null
datatype: null
fixture: {storage: null, location: null, checksum: null, provenance: "Workflow default."}
- workflow_label: Strandedness
label_status: resolved
description: Workflow default `stranded - reverse`, as case 1. Strandedness is a property of the library, not of the subset depth.
collection_shape: null
datatype: null
fixture: {storage: null, location: null, checksum: null, provenance: "Workflow default, measured by phase 7."}
- workflow_label: Adjusted p-value threshold
label_status: resolved
description: Workflow default 0.05, as case 1 — left alone so the filter chain is exercised in its shipped configuration even though no significance claim is made.
collection_shape: null
datatype: null
fixture: {storage: null, location: null, checksum: null, provenance: "Workflow default."}
- workflow_label: log2 fold change threshold
label_status: resolved
description: Workflow default 1.0, as case 1.
collection_shape: null
datatype: null
fixture: {storage: null, location: null, checksum: null, provenance: "Workflow default."}
expected_outputs:
- workflow_label: "FastQC raw reads: text summary"
label_status: resolved
description: Twelve elements. The depth constant is the only thing that changes from case 1.
output_kind: collection
collection_shape: list
assertion_intent:
- family: has_text_matching
intent: "Each of the twelve elements reports exactly 25,000 sequences, proving the reduced fixture is the depth it claims and that the flatten side-branch fanned out to twelve."
expected_value: 'Total Sequences\s+25000'
tolerance: null
element_identifier: null
evidence: intent
confidence: high
- workflow_label: Gene counts per sample
label_status: resolved
description: >-
Six elements. The row count and gene ID space are depth-independent — featureCounts emits
one row per annotated gene regardless of coverage — so the annotation-space guard carries
over unchanged. The SCF1 count thresholds do NOT carry over.
output_kind: collection
collection_shape: list
assertion_intent:
- family: has_n_columns
intent: "Two-column gene-id/count table."
expected_value: 2
tolerance: null
element_identifier: null
evidence: intent
confidence: high
- family: has_n_lines
intent: "5586 annotated genes, depth-independent. Same ±1 header allowance as case 1."
expected_value: 5586
tolerance:
kind: delta
magnitude: 1
rationale: "Header-row presence on featureCounts output_short is inferred, not verified."
element_identifier: null
evidence: intent
confidence: medium
- family: has_line_matching
intent: "The annotation-space guard: at least 5000 rows are a bare B9J08_ locus tag plus an integer count."
expected_value: 'B9J08_[0-9]{6}\t[0-9]+'
tolerance: null
element_identifier: null
evidence: intent
confidence: high
- workflow_label: "DESeq2 results: tnSWI1 vs AR0382"
label_status: resolved
description: Raw result table, first contrast. Carries both invariant detectors, which are structural and therefore depth-independent.
output_kind: dataset
collection_shape: null
assertion_intent:
- family: has_n_columns
intent: "CROSS-STEP INVARIANT 1 DETECTOR: seven columns, i.e. `lfc_shrinkage_type` is still `none` and c7 is still padj."
expected_value: 7
tolerance: null
element_identifier: null
evidence: intent
confidence: high
- family: not_has_text
intent: "Headerless, the premise of `header_lines: '0'` on the significance filters."
expected_value: baseMean
tolerance: null
element_identifier: null
evidence: intent
confidence: high
- workflow_label: "DESeq2 results: AR0387 vs AR0382"
label_status: resolved
description: Raw result table, second contrast. Both invariant detectors again, because invariant 1 must hold on BOTH DESeq2 nodes.
output_kind: dataset
collection_shape: null
assertion_intent:
- family: has_n_columns
intent: "CROSS-STEP INVARIANT 1 DETECTOR on the second node."
expected_value: 7
tolerance: null
element_identifier: null
evidence: intent
confidence: high
- family: not_has_text
intent: "Headerless."
expected_value: baseMean
tolerance: null
element_identifier: null
evidence: intent
confidence: high
- workflow_label: "DESeq2 normalized counts: tnSWI1 vs AR0382"
label_status: resolved
description: Headered table. Carries the condition-split half of invariant 2 without needing any biological signal.
output_kind: dataset
collection_shape: null
assertion_intent:
- family: has_text
intent: >-
The header names exactly the four samples of this contrast — AR0382_A, AR0382_B,
AR0382_tnSWI1_A, AR0382_tnSWI1_B — each asserted separately.
expected_value: AR0382_tnSWI1_A
tolerance: null
element_identifier: null
evidence: intent
confidence: medium
- family: not_has_text
intent: "CROSS-STEP INVARIANT 2 DETECTOR (condition-split half): no AR0387 sample leaked into the first contrast."
expected_value: AR0387
tolerance: null
element_identifier: null
evidence: intent
confidence: high
- family: has_n_columns
intent: >-
Five fields on the first line — one padded row-name column plus four samples. A
`header_lines: '1'` regression on `Select first-contrast samples` would leak AR0382_A
into this contrast and make it six. See unresolved
`normalized-counts-header-width-unverified`: if the wrapper's header turns out NOT to
carry the leading blank field, the correct n is 4, and implement-galaxy-workflow-test
must settle this from the first real invocation.
expected_value: 5
tolerance: null
element_identifier: null
evidence: intent
confidence: medium
- workflow_label: "DESeq2 normalized counts: AR0387 vs AR0382"
label_status: resolved
description: Headered table, second contrast. Same split check against a tnSWI1 leak.
output_kind: dataset
collection_shape: null
assertion_intent:
- family: has_text
intent: "The header names exactly AR0382_A, AR0382_B, AR0387_A, AR0387_B, each asserted separately."
expected_value: AR0387_A
tolerance: null
element_identifier: null
evidence: intent
confidence: medium
- family: not_has_text
intent: "CROSS-STEP INVARIANT 2 DETECTOR on the second split: no tnSWI1 sample leaked in."
expected_value: tnSWI1
tolerance: null
element_identifier: null
evidence: intent
confidence: high
- family: has_n_columns
intent: "Five fields on the first line, same caveat as the first contrast."
expected_value: 5
tolerance: null
element_identifier: null
evidence: intent
confidence: medium
unresolved:
- kind: fixture
description: >-
The twelve 200,000-read-pair FASTQ fixtures for case 1 were generated and hashed by phase 7 but
never published; they existed only in that session's scratchpad. `location` is null and
`<FIXTURE_BASE>` in test-data-refs.json `planemo_test_job_block` has no value to substitute.
Case 1 cannot run until they are regenerated and hosted.
blocking: true
suggested_resolution: >-
Regenerate with test-data-refs.json `subsetting.recipe` (stream each ENA FASTQ, `head -n
800000`), verify each against its `md5_uncompressed`, publish to Zenodo, substitute the record
URL for `<FIXTURE_BASE>`. Mirrors open requirement `test-fixtures-not-hosted`.
- kind: fixture
description: >-
Case 2's 25,000-read-pair fixtures do not exist at all. Phase 7 costed and characterized this
depth (`subsetting.shape_only_alternative`) but generated only the 200k set.
blocking: true
suggested_resolution: >-
Same recipe with `head -n 100000` per mate from the same six ENA runs. Decide in-repo versus
Zenodo at that point: ~7 MB total is committable but a reviewer may still push back on twelve
binary files in `test-data/`. If the answer is Zenodo, case 2 loses its main advantage over
case 1 and should be reconsidered rather than kept for its own sake.
- kind: fixture
description: >-
Whether a `hashes:` block on an input that also sets `decompress: true` validates the
compressed bytes as fetched or the decompressed bytes was not established. The reference FASTA
and GTF carry md5s of the COMPRESSED NCBI files and are staged with `decompress: true`, so a
wrong assumption here fails the fetch with a hash mismatch rather than anything diagnostic.
blocking: false
suggested_resolution: >-
Confirm against Galaxy's fetch handler before adding `hashes:` to those two inputs; if it
cannot be confirmed cheaply, omit the hashes there (the URLs are stable NCBI FTP paths) and
keep them on the read fixtures, where no decompression happens.
- kind: fixture
description: >-
No `hashes:` block can be authored for the twelve read fixtures from what phase 7 supplies.
Phase 7 pinned UNCOMPRESSED md5s deliberately, because gzip output is not byte-stable across
gzip versions, but the staged file is the .gz and that is what a `hashes:` block would check.
IWC convention puts a hash on every remote input `location:`, so this will be noticed in review.
blocking: false
suggested_resolution: >-
Hash the published .gz files once, after hosting, and record those hashes in the test file —
the uncompressed md5s stay in test-data-refs.json as the regeneration check, and the published
.gz hashes become the fetch-integrity check. The two serve different purposes and both are
needed.
- kind: assertion
description: >-
Two of phase 7's biological claims are RANK claims and no tests-format assertion family can
express a rank or a row position: "SCF1 is within the 10 smallest padj rows" and "SCF1's
normalized count sits in the top 5% of the table". has_line_matching has no positional
anchoring; has_n_lines cannot bound where a match occurs. The padj-rank claim is TRUE as
measured — SCF1 is 5th of the 2393 genes carrying a padj in the first contrast and 4th of 1376
in the second — so it sits here because it is INEXPRESSIBLE, not because it is doubted. It is
also the weaker form of a claim this plan already asserts well: a rank of 5 has no margin
against the implementation difference recorded in warnings[]
`deseq2-statistics-measured-with-pydeseq2-not-r`, whereas the padj threshold it rests on clears
by 31 and 18 orders of magnitude. Promoting it is not worth a workflow change.
blocking: false
suggested_resolution: >-
Either accept membership in the significant-genes table as the proxy (what this plan does), or,
if the rank claim must be tested, add a `Select first` step after each DESeq2 node and promote
its output — which turns a rank into an ordinary membership assertion. That is a workflow
change, not a test change, and should be weighed against the output-list clutter.
- kind: assertion
description: >-
The `has_n_columns` value for the DESeq2 normalized-counts tables depends on whether
`col.names = NA` produces a header padded with a leading blank field (making the header the
same width as the data rows, n=5 for four samples) or an unpadded one (n=4). Galaxy's
has_n_columns reads the FIRST line only, so the two differ. Case 1 sidesteps this by asserting
on SCF1's data row instead; case 2 uses has_n_columns and carries the risk.
blocking: false
suggested_resolution: >-
Settle from the first successful invocation — `planemo workflow_test_init --from_invocation`
will show the real header — and correct case 2's n before committing the test file.
- kind: assertion
description: >-
Whether the DESeq2 wrapper names normalized-counts columns by the input dataset ELEMENT
IDENTIFIER (AR0382_A) or by some other dataset name is assumed, not verified. Every
sample-name has_text and not_has_text assertion on the two normalized-counts outputs, and
therefore the condition-split half of invariant 2, rests on it.
blocking: false
suggested_resolution: >-
Confirm from the first invocation. If the wrapper uses a different naming scheme, the
`not_has_text: AR0387` / `not_has_text: tnSWI1` negatives survive only if the condition token
still appears in the column name; otherwise fall back to the SCF1-row field-count assertion as
the sole split detector.
- kind: collection-shape
description: >-
The promoted per-sample outputs (FastQC text/HTML, Cutadapt report, Trimmed reads, STAR BAM and
summary, Gene counts, featureCounts summary) are recorded here with their BEHAVIOURAL shapes
`list` and `list:paired`, but a tool mapped over a `sample_sheet:paired` collection produces a
sample_sheet-shaped collection without `column_definitions`, whose declared collection_type
string is not `list`. This is open requirement `mapped-outputs-carry-sample-sheet-outer-axis`,
which was filed explicitly for this phase.
blocking: false
suggested_resolution: >-
Do NOT author `attributes: {collection_type: ...}` assertions on any of those eight outputs —
the plan asserts element identifiers instead, which is what actually matters and is
shape-agnostic. Record the real type strings from the first invocation and only then decide
whether a collection_type assertion is worth having.
- kind: assertion
description: >-
featureCounts `output_short` header-row presence was not verified. The DESeq2 steps bind
`header: true` on their counts inputs, which implies one, and the `has_n_lines` assertions
carry a ±1 allowance for it, but the featureCounts wrapper itself was not read in this run.
blocking: false
suggested_resolution: "Read the pinned iuc/featurecounts 2.1.1+galaxy1 wrapper, or take the line count from the first invocation, and tighten the ±1 to an exact value."
omissions:
- target: "FastQC raw reads: HTML report"
reason: >-
The sibling text summary is promoted precisely so it can carry the assertions, and it does —
exact sequence counts and the Basic Statistics module per element. The HTML embeds the FastQC
version banner and base64-encoded images, so any content assertion on it either restates the
text summary or pins a version string. Asserting both would be duplication, not coverage.
category: weak-output
- target: "STAR alignments (BAM)"
reason: >-
BAM is a gzipped block format whose header carries @PG lines with full command lines and Galaxy
job ids, so it is never byte-stable; and the workflow already promotes `STAR mapping summary`
as the assertable text behind it, which is asserted. Per planemo-asserts-idioms the fallback
would be has_size plus has_archive_member, which buys nothing the mapping summary does not
already prove.
category: weak-output
- target: "The paper's ~29-fold SCF1 expression difference between AR0382 and AR0387"
reason: >-
REFUSED DELIBERATELY. Direct measurement of the deposited data disagrees by roughly an order of
magnitude — ~270-fold on aligned fragments at fixture depth, ~158-fold by exact 31-mer matching
over 5.2 M reads. Reference bias is ruled out as the explanation, because AR0387 IS the B8441
reference strain. The 29-fold may be the RT-qPCR assay rather than the RNA-seq, a shrunk rather
than raw log2FC, or a different normalization; nothing in this run distinguishes them. Open
requirement `scf1-fold-change-magnitude-disagrees-with-paper`. Direction and significance are
asserted; magnitude is asserted only as a loose `<= -2` floor.
category: out-of-scope
- target: "SCF1 is the single top gene ranked by |log2 fold change|"
reason: >-
REFUSED DELIBERATELY, AND NOW MEASURED FALSE AS A CROSS-CONTRAST CLAIM. The mechanism was
predicted correctly: with `lfc_shrinkage_type: none` — which this workflow pins, for the
column-contract reason that is invariant 1 — low-count genes take extreme unshrunk log2FC
values and can outrank SCF1. Measurement confirms it, and also kills the fallback this rationale
previously proposed. Phase 8 ran pydeseq2 on a strand-aware count matrix built from phase 7's
six BAMs, as two separate two-level n=2 analyses with no shrinkage, mirroring the workflow's two
DESeq2 nodes. SCF1 (B9J08_001458) came out: tnSWI1 vs AR0382 — log2FC -6.93, padj 2.1e-31, 5th
of 2393 genes with a padj, 2nd of the 63 clearing padj<0.05 and |log2FC|>1 by |log2FC| (2nd of
the 52 down-regulated ones); AR0387 vs AR0382 — log2FC -7.95, padj 4.1e-18, 4th of 1376 by padj,
1st of 35 by |log2FC| (1st of 9 down-regulated). So NO rank-1 assertion holds in both contrasts:
not by |log2FC| (2nd, then 1st), and not by adjusted p-value either (5th, then 4th). The earlier
claim in this slot that "ranking by adjusted p-value is the safe form" was wrong and has been
removed. Membership plus a threshold is the honest form and is what the plan asserts: SCF1
appears in each significant-genes table — which by construction means padj < 0.05 and |log2FC| >
1 — with log2FoldChange <= -2. That form is also the robust one. It clears by 31 and 18 orders
of magnitude on padj and ~5 and ~6 log2 units on fold change, which is margin enough to survive
the fact that these numbers come from pydeseq2 over bwa-mem counts rather than the wrapper's R
DESeq2 over featureCounts -s 2 (warnings[] `deseq2-statistics-measured-with-pydeseq2-not-r`); a
rank of 1, 2, 4 or 5 has no such margin and would be a flaky assertion even if tests-format
could express it, which it cannot (see unresolved).
category: other
- target: "No significant dysregulation of the ALS or IFF/HYR adhesin families in the tnSWI1 contrast"
reason: >-
REFUSED DELIBERATELY. A negative claim over twelve named genes (B9J08_002582/_004498/_004112
ALS; _004100/_004109/_004098/_004110/_001531/_004892/_001155/_004451/_000675 IFF-HYR). At
200,000 read pairs most of those sit at counts where DESeq2 simply lacks power, so their
absence from the significant table is a depth artefact rather than evidence for the paper's
claim. Asserting it would be a fixture claiming an outcome it cannot produce — exactly the
defect the Foundry's fixture rule names. It would need full-depth runs.
category: out-of-scope
- target: "The tnBCY1 vs AR0382 contrast (Fig. S2)"
reason: >-
No tnBCY1 run was ever deposited in PRJNA904261; only three conditions exist. There is no test
input for it at any depth, and the workflow does not model it.
category: out-of-scope
- target: "Exact per-gene integer counts on `Gene counts per sample`"
reason: >-
The IWC comparison notes recommend being stricter than the corpus default here, and they are
right in principle — featureCounts on a fixed reference, annotation and strandedness is
deterministic. But phase 7 measured with bwa-mem plus a strand-agnostic exon counter, not with
RNA STAR plus featureCounts -s 2, so it cannot supply the exact integers and inventing them
would be a fixture asserting something nothing measured. The plan asserts thresholds with a
stated margin instead, and the exact-count upgrade is an explicit follow-up once a real
invocation exists.
category: other
- target: "Negative / failure-path test cases"
reason: >-
`expect_failure:` does not appear anywhere in the 115-file IWC test corpus. Error-path testing
belongs in tool wrappers, not workflow tests. No such case is authored here.
category: out-of-scope
warnings:
- code: fixture-measured-assertions-lack-an-evidence-class
message: >-
Every assertion carries `evidence: intent` because the schema's only alternative is
`test-evidence`, which means "translated from upstream test fixtures". A large share of these
assertions are neither: they were measured directly against the resolved fixtures by phase 7.
Confidence is raised to `high` where that is the case, but a reader filtering on `evidence`
will see a uniformly synthesized plan and undercount its grounding.
path: "test_cases[*].expected_outputs[*].assertion_intent[*].evidence"
- code: full-line-anchoring-convention
message: >-
Every `has_line_matching` expected_value in this plan is written to match a WHOLE line, because
Galaxy evaluates has_line_matching as `^(?:expression)$` under re.MULTILINE. The
`has_text_matching` values are written for `re.search` over the whole content instead, and `^`
would NOT be multiline there. Do not move an expression between the two families without
rewriting it.
path: "test_cases[*].expected_outputs[*].assertion_intent[*].expected_value"
- code: iwc-notes-and-test-data-refs-disagree-on-count-strictness
message: >-
iwc-comparison-notes.md §5 recommends exact per-gene integer counts on `Gene counts per
sample`; test-data-refs.json `unresolved.counts-measured-with-bwa-not-star` forbids pinning any
exact count. Resolved in favour of the later and better-grounded artifact: thresholds now,
exact counts as a recorded follow-up once a real invocation exists. Both positions are correct
about different moments in the run.
path: "test_cases[0].expected_outputs[4]"
- code: deseq2-at-reduced-depth-untested
message: >-
Case 2 assumes DESeq2 completes at 25,000 read pairs per sample (~4.5 reads/gene average). It
should — the gene count and sample count are unchanged and only the counts shrink — but no run
at that depth has happened, and a DESeq2 failure there would surface as a whole-case failure
rather than an assertion failure. Run case 2 once before relying on it as the fast CI lane.
path: "test_cases[1]"
- code: reference-level-filter-regression-undetectable
message: >-
Of the seven Filter1 steps that bind `header_lines: '0'`, this plan can detect a regression on
only two — `Select first-contrast samples` and `Select second-contrast samples`. On `Select
reference-level samples` the leaked row IS AR0382_A, which the `c2=='AR0382'` predicate keeps
anyway, so the output is identical and no assertion can see it. On the four significance
filters the leaked row is row 1 of a padj-sorted table, which passes both filters on its own
merits, so the regression is again invisible at workflow level. See the report for the full
accounting.
path: "test_cases[*].expected_outputs[*]"
- code: deseq2-statistics-measured-with-pydeseq2-not-r
message: >-
Every differential-expression number quoted in this plan — SCF1's log2 fold change and adjusted
p-value per contrast, and every rank figure in omissions[] and unresolved[] — was produced by
pydeseq2 over a strand-aware count matrix built from phase 7's six bwa-mem BAMs, as two separate
two-level n=2 analyses with no LFC shrinkage. The workflow runs the Galaxy DESeq2 wrapper, i.e.
R DESeq2, over RNA STAR plus featureCounts -s 2. pydeseq2 is a faithful reimplementation, but it
is not the same implementation and the counts are not the same counts, so exact values will
differ. NO number from that measurement may be turned into an equality assertion, and none is.
What survives the difference is margin: padj 2.1e-31 and 4.1e-18 against a 0.05 threshold,
log2FC -6.93 and -7.95 against a -2 floor. What does not survive it is rank — 5th and 4th by
padj, 2nd and 1st by |log2FC| — or any exact value, which is why neither is asserted.
path: "test_cases[*].expected_outputs[*].assertion_intent[*].intent"
- code: no-corpus-precedent-for-sample-sheet-test-fixture
message: >-
No IWC test fixture in the 115-file corpus declares a `sample_sheet` collection or any
`column_definitions`, so the job block for the reads input has no worked example to
pattern-match against. Phase 7 established from Galaxy's source that it IS expressible and
pinned the positional-list `rows:` shape; that reading is the plan's only authority here, and
the first `planemo test` run is what confirms it.
path: 'test_cases[*].job_inputs[0]'
Concrete gxformat2 workflow (`class: GalaxyWorkflow`) extracted from the fully-concretized draft at loop endstate via [[draft-extract]]: drafty steps dropped, `_plan_*` planning fields stripped, class promoted. The runnable, testable artifact that downstream Molds ([[implement-galaxy-workflow-test]], [[validate-galaxy-workflow]], [[run-workflow-test]]) consume.
GalaxyWorkflow — 27 step(s), 0 still drafty, 9 input(s), 16 output(s).
| step | label | tool | state | plan keys |
|---|---|---|---|---|
| Flatten reads for per-fastq QC | Flatten reads for per-fastq QC | __FLATTEN__ | resolved | — |
| Read quality report | Read quality report | toolshed.g2.bx.psu.edu/repos/devteam/fastqc/fastqc | resolved | — |
| Quality-trim reads | Quality-trim reads | toolshed.g2.bx.psu.edu/repos/lparsons/cutadapt/cutadapt | resolved | — |
| Splice-aware alignment | Splice-aware alignment | toolshed.g2.bx.psu.edu/repos/iuc/rgrnastar/rna_star | resolved | — |
| Get featureCounts strandedness parameter | Get featureCounts strandedness parameter | toolshed.g2.bx.psu.edu/repos/iuc/map_param_value/map_param_value | resolved | — |
| Count reads per gene | Count reads per gene | toolshed.g2.bx.psu.edu/repos/iuc/featurecounts/featurecounts | resolved | — |
| Project sample sheet to tabular | Project sample sheet to tabular | __SAMPLE_SHEET_TO_TABULAR__ | resolved | — |
| Build reference-level row predicate | Build reference-level row predicate | toolshed.g2.bx.psu.edu/repos/iuc/compose_text_param/compose_text_param | resolved | — |
| Select reference-level samples | Select reference-level samples | Filter1 | resolved | — |
| Reference-level identifier list | Reference-level identifier list | Cut1 | resolved | — |
| Counts for the reference level | Counts for the reference level | __FILTER_FROM_FILE__ | resolved | — |
| Build first-contrast row predicate | Build first-contrast row predicate | toolshed.g2.bx.psu.edu/repos/iuc/compose_text_param/compose_text_param | resolved | — |
| Select first-contrast samples | Select first-contrast samples | Filter1 | resolved | — |
| First-contrast identifier list | First-contrast identifier list | Cut1 | resolved | — |
| Counts for the first contrast level | Counts for the first contrast level | __FILTER_FROM_FILE__ | resolved | — |
| Build second-contrast row predicate | Build second-contrast row predicate | toolshed.g2.bx.psu.edu/repos/iuc/compose_text_param/compose_text_param | resolved | — |
| Select second-contrast samples | Select second-contrast samples | Filter1 | resolved | — |
| Second-contrast identifier list | Second-contrast identifier list | Cut1 | resolved | — |
| Counts for the second contrast level | Counts for the second contrast level | __FILTER_FROM_FILE__ | resolved | — |
| Differential expression: first contrast vs reference | Differential expression: first contrast vs reference | toolshed.g2.bx.psu.edu/repos/iuc/deseq2/deseq2 | resolved | — |
| Differential expression: second contrast vs reference | Differential expression: second contrast vs reference | toolshed.g2.bx.psu.edu/repos/iuc/deseq2/deseq2 | resolved | — |
| Build adjusted p-value predicate | Build adjusted p-value predicate | toolshed.g2.bx.psu.edu/repos/iuc/compose_text_param/compose_text_param | resolved | — |
| Build log2 fold-change predicate | Build log2 fold-change predicate | toolshed.g2.bx.psu.edu/repos/iuc/compose_text_param/compose_text_param | resolved | — |
| Filter with p-adj threshold: first contrast | Filter with p-adj threshold: first contrast | Filter1 | resolved | — |
| Filter with log2 FC threshold: first contrast | Filter with log2 FC threshold: first contrast | Filter1 | resolved | — |
| Filter with p-adj threshold: second contrast | Filter with p-adj threshold: second contrast | Filter1 | resolved | — |
| Filter with log2 FC threshold: second contrast | Filter with log2 FC threshold: second contrast | Filter1 | resolved | — |
class: GalaxyWorkflow
label: C. auris Scf1 RNA-seq differential expression (Santana et al. 2023)
doc: |-
Bulk RNA-seq differential expression for Candida auris adhesion mutants, reconstructed from the supplementary Materials and Methods of Santana et al. 2023 (Science 381:1461-1467). FastQC -> Cutadapt -> RNA STAR -> featureCounts -> DESeq2 -> significance filter, over 6 paired-end runs (BioProject PRJNA904261): 2 biological replicates each of the high-adhesion parent AR0382, the low-adhesion clinical isolate AR0387, and the AR0382 tnSWI1 insertional mutant. Two contrasts, both against AR0382. Reference genome is C. auris B8441, NCBI assembly GCA_002759435.2.
VERSION FIDELITY IS NOT CLAIMABLE. The paper names six tools and gives no version for any of them; only its non-Galaxy software is versioned. This workflow pins its own tool versions and reproduces the authors' method, never their software stack.
THE COUNTING ANNOTATION IS NOT THE PAPER'S. No GTF/GFF source is named anywhere in the supplement. Whoever supplies the annotation input determines the gene ID space, and therefore whether SCF1 appears as B9J08_001458 at all.
tags:
- transcriptomics
- RNAseq
inputs:
- id: RNA-seq reads (sample sheet)
type: collection
collection_type: sample_sheet:paired
format:
- fastqsanger.gz
optional: false
column_definitions:
- name: condition
type: string
optional: false
restrictions:
- AR0382
- AR0387
- tnSWI1
description: Experimental condition. AR0382 is the high-adhesion parent and the reference level for both contrasts.
- name: replicate
type: string
optional: false
restrictions:
- A
- B
description: Biological replicate label.
doc: "Six paired-end runs, 2 x 50 bp, Illumina NextSeq 2000 (PRJNA904261 / SRP409192). Element identifiers are the deposited sample names and are the spine of this workflow: AR0382_A, AR0382_B, AR0387_A, AR0387_B, AR0382_tnSWI1_A, AR0382_tnSWI1_B. They must survive map-over unchanged -- the condition split joins on them."
- id: Reference genome FASTA
type: data
format:
- fasta
optional: false
doc: C. auris B8441, NCBI assembly GCA_002759435.2 (Cand_auris_B8441_V2). Delivered as a history dataset rather than a built-in index because no public Galaxy server indexes this genome; RNA STAR builds its index inside each mapped job (~12.5 Mb, six builds).
- id: Gene annotation GTF
type: data
format:
- gtf
optional: false
doc: NOT NAMED BY THE PAPER. Consumed twice -- RNA STAR splice junctions and featureCounts gene assignment -- so the choice is load-bearing for both. NCBI RefSeq GFF and FungiDB GFF for this assembly differ in gene ID space and attribute keys. If the chosen source is GFF3 rather than GTF, this input's format changes or a conversion step is inserted ahead of both consumers.
- id: Reference condition level
type: text
default: AR0382
optional: false
doc: The condition value both contrasts are taken against. Must match a `condition` value in the sample sheet exactly.
- id: First contrast condition level
type: text
default: tnSWI1
optional: false
doc: "Condition value for the first contrast (paper: the AR0382 tnSWI1 insertional mutant). Changing it makes the `... tnSWI1 vs AR0382` output labels stale -- the labels are the public API and are not derived from this parameter."
- id: Second contrast condition level
type: text
default: AR0387
optional: false
doc: "Condition value for the second contrast (paper: the low-adhesion clinical isolate AR0387). Same label caveat as the first contrast level."
- id: Strandedness
type: text
default: stranded - reverse
optional: false
restrictions:
- stranded - forward
- stranded - reverse
- unstranded
doc: "INFERRED, NOT STATED. The paper names only the library kit (Illumina Stranded Total RNA Prep with Ribo-Zero Plus), which is reverse-stranded (dUTP) in Illumina's standard chemistry. The empirical check is the promoted featureCounts assignment summary: a wrong setting shows up as a large Unassigned_NoFeatures fraction."
- id: Adjusted p-value threshold
type: float
default: 0.05
optional: false
doc: "Stated by the paper: adjusted p-value < 0.05. Applied to DESeq2 column c7."
- id: log2 fold change threshold
type: float
default: 1
optional: false
doc: log2 fold change threshold. A log2 FC of 1 equals an absolute fold change of 2 (2^1), which is exactly the paper's |fold change| > 2 criterion; a log2 FC of 3 equals 8. Stated in log2 units because DESeq2 reports log2FoldChange and the filter compares the raw column with no conversion -- the corpus idiom (rnaseq-de at fe41a79). A linear 2.0 compared against abs(c3) would silently apply a 4-fold cut.
outputs:
- id: "FastQC raw reads: text summary"
outputSource: Read quality report/text_file
doc: "Twelve elements, not six: the reads are flattened per read direction before QC, so identifiers are <sample>_forward / <sample>_reverse."
- id: "FastQC raw reads: HTML report"
outputSource: Read quality report/html_file
- id: Cutadapt trimming report
outputSource: Quality-trim reads/report
- id: Trimmed reads
outputSource: Quality-trim reads/out_pairs
- id: STAR alignments (BAM)
outputSource: Splice-aware alignment/mapped_reads
- id: STAR mapping summary
outputSource: Splice-aware alignment/output_log
doc: Log.final.out -- the assertable text behind the binary BAM.
- id: Gene counts per sample
outputSource: Count reads per gene/output_short
doc: "The strongest deterministic checkpoint in this workflow: exact per-gene integer counts, addressable by gene id, keyed by the six sample element identifiers."
- id: featureCounts assignment summary
outputSource: Count reads per gene/output_summary
doc: Assigned / Unassigned_NoFeatures / Unassigned_Ambiguous totals -- also the direct read-out of whether Strandedness was set right.
- id: "DESeq2 normalized counts: tnSWI1 vs AR0382"
outputSource: "Differential expression: first contrast vs reference/counts_out"
- id: "DESeq2 normalized counts: AR0387 vs AR0382"
outputSource: "Differential expression: second contrast vs reference/counts_out"
- id: "DESeq2 results: tnSWI1 vs AR0382"
outputSource: "Differential expression: first contrast vs reference/deseq_out"
doc: Full result table. Expect SCF1 = B9J08_001458 present with a negative log2FC, and the strongest / most significant dysregulation in the table.
- id: "DESeq2 results: AR0387 vs AR0382"
outputSource: "Differential expression: second contrast vs reference/deseq_out"
doc: Full result table. Expect SCF1 the most down-regulated gene, ~29-fold per the paper.
- id: "Significant genes: tnSWI1 vs AR0382"
outputSource: "Filter with log2 FC threshold: first contrast/out_file1"
- id: "Significant genes: AR0387 vs AR0382"
outputSource: "Filter with log2 FC threshold: second contrast/out_file1"
- id: "DESeq2 diagnostic plots: tnSWI1 vs AR0382"
outputSource: "Differential expression: first contrast vs reference/plots"
- id: "DESeq2 diagnostic plots: AR0387 vs AR0382"
outputSource: "Differential expression: second contrast vs reference/plots"
steps:
- id: Flatten reads for per-fastq QC
label: Flatten reads for per-fastq QC
tool_id: __FLATTEN__
tool_version: 1.0.0
doc: "Tier: Resolved. sample_sheet:paired -> flat list of 12 fastq, identifiers <sample>_forward / <sample>_reverse. Side branch off the workflow input: it does not enter the map-over region, so the per-sample outputs keep the six-element sample identifier space untouched."
in:
input: RNA-seq reads (sample sheet)
out:
- id: output
hide: true
state:
join_identifier: _
- id: Read quality report
label: Read quality report
tool_id: toolshed.g2.bx.psu.edu/repos/devteam/fastqc/fastqc
tool_version: 0.74+galaxy1
tool_shed_repository:
changeset_revision: 2c64fded1286
name: fastqc
owner: devteam
tool_shed: toolshed.g2.bx.psu.edu
doc: "Tier: Resolved. Mapped over the FLATTENED 12-element list, not the six-element sample axis: FastQC takes one dataset at a time (`input_file` is a single `data` param), which is what the `__FLATTEN__` node upstream exists to produce. Twelve jobs, identifiers <sample>_forward / <sample>_reverse. Both outputs are promoted -- the text summary is the assertable checkpoint (per-module PASS/WARN/FAIL lines, total-sequence counts) behind the user-facing HTML binary."
in:
input_file: Flatten reads for per-fastq QC/output
out:
- id: text_file
- id: html_file
tool_state:
input_file:
__class__: ConnectedValue
adapters:
__class__: RuntimeValue
contaminants:
__class__: RuntimeValue
limits:
__class__: RuntimeValue
kmers: "7"
min_length: null
nogroup: false
- id: Quality-trim reads
label: Quality-trim reads
tool_id: toolshed.g2.bx.psu.edu/repos/lparsons/cutadapt/cutadapt
tool_version: 5.2+galaxy2
tool_shed_repository:
changeset_revision: f6168dd17f82
name: cutadapt
owner: lparsons
tool_shed: toolshed.g2.bx.psu.edu
doc: 'Tier: Resolved. Map-over, one job per sample row (6 jobs). Under library.type: paired_collection the wrapper emits ONE paired-inner collection (`out_pairs`, declared `type="paired"` with `format_source="library|input_1"`), so mapping it over the list:paired input yields list:paired again and no re-pair node is needed before RNA STAR.'
in:
library|input_1: RNA-seq reads (sample sheet)
out:
- id: out_pairs
- id: report
tool_state:
library:
type: paired_collection
__current_case__: 2
input_1:
__class__: ConnectedValue
r1:
adapters: []
front_adapters: []
anywhere_adapters: []
r2:
adapters2: []
front_adapters2: []
anywhere_adapters2: []
pair_adapters: false
other_trimming_options:
quality_cutoff: "20"
filter_options:
minimum_length: "1"
output_selector:
- report
- id: Splice-aware alignment
label: Splice-aware alignment
tool_id: toolshed.g2.bx.psu.edu/repos/iuc/rgrnastar/rna_star
tool_version: 2.7.11b+galaxy1
tool_shed_repository:
changeset_revision: d8d32326fbfb
name: rgrnastar
owner: iuc
tool_shed: toolshed.g2.bx.psu.edu
doc: "Tier: Resolved. Map-over, one job per sample row (6 jobs); the FASTA and GTF are data inputs wired into a mapped step, so they broadcast into every job. Each job builds its own temporary STAR index from the history FASTA (`--runMode genomeGenerate` into tempstargenomedir), which is why there is no separate index node to wire."
in:
singlePaired|input: Quality-trim reads/out_pairs
refGenomeSource|genomeFastaFiles: Reference genome FASTA
refGenomeSource|GTFconditional|sjdbGTFfile: Gene annotation GTF
out:
- id: mapped_reads
- id: output_log
tool_state:
singlePaired:
sPaired: paired_collection
__current_case__: 2
input:
__class__: ConnectedValue
refGenomeSource:
geneSource: history
__current_case__: 1
genomeFastaFiles:
__class__: ConnectedValue
genomeSAindexNbases: "10"
GTFconditional:
GTFselect: with-gtf
__current_case__: 0
sjdbGTFfile:
__class__: ConnectedValue
sjdbGTFfeatureExon: exon
sjdbOverhang: "49"
quantmode_output:
quantMode: "-"
__current_case__: 0
diploidconditional:
diploid: "No"
__current_case__: 1
twopass:
twopassMode: None
__current_case__: 0
twopass_read_subset: ""
sj_precalculated: ""
chimOutType: ""
oformat:
outSAMattributes:
- NH
- HI
- AS
- nM
- ch
HI_offset: "1"
outSAMprimaryFlag: OneBestScore
outSAMmapqUnique: "60"
wasp_conditional:
waspOutputMode: ""
__current_case__: 1
filter:
basic_filters: []
output_params2:
output_select2: "no"
__current_case__: 1
algo:
params:
settingsType: default
__current_case__: 0
perf:
outBAMsortingBinsN: "50"
winAnchorMultimapNmax: "50"
outWig:
outWigType: None
__current_case__: 0
outWigStrand: "false"
- id: Get featureCounts strandedness parameter
label: Get featureCounts strandedness parameter
tool_id: toolshed.g2.bx.psu.edu/repos/iuc/map_param_value/map_param_value
tool_version: 0.2.0
tool_shed_repository:
changeset_revision: 022a635f2283
name: map_param_value
owner: iuc
tool_shed: toolshed.g2.bx.psu.edu
doc: "Tier: Resolved. Single job; featureCounts is the only consumer. Maps the user-facing strandedness string onto featureCounts' numeric -s encoding (0 unstranded / 1 forward / 2 reverse). The VALUE it maps is still inferred from the library kit name rather than stated by the paper: open-requirements `rnaseq-strandedness-inferred-from-kit-name` stays OPEN. Concretizing how the parameter is expressed says nothing about confidence in the value flowing through it."
in:
input_param_type|input_param: Strandedness
out:
- id: output_param_text
hide: true
tool_state:
input_param_type:
type: text
__current_case__: 0
input_param:
__class__: ConnectedValue
mappings:
- __index__: 0
from: stranded - forward
to: "1"
- __index__: 1
from: stranded - reverse
to: "2"
- __index__: 2
from: unstranded
to: "0"
output_param_type: text
unmapped:
on_unmapped: fail
__current_case__: 1
- id: Count reads per gene
label: Count reads per gene
tool_id: toolshed.g2.bx.psu.edu/repos/iuc/featurecounts/featurecounts
tool_version: 2.1.1+galaxy1
tool_shed_repository:
changeset_revision: 37d067694d40
name: featurecounts
owner: iuc
tool_shed: toolshed.g2.bx.psu.edu
doc: "Tier: Resolved. Map-over, one job per sample row (6 jobs); the GTF is a data input wired into a mapped step, so it broadcasts into every job. Counts fragments against `exon` features grouped by `gene_id`, emitting the two-column gene-id/count table the DESeq2 per-level ports consume, plus the assignment summary that reads out whether the strandedness setting was right."
in:
alignment: Splice-aware alignment/mapped_reads
anno|reference_gene_sets: Gene annotation GTF
strand_specificity: Get featureCounts strandedness parameter/output_param_text
out:
- id: output_short
- id: output_summary
tool_state:
alignment:
__class__: ConnectedValue
anno:
anno_select: history
__current_case__: 2
reference_gene_sets:
__class__: ConnectedValue
gff_feature_type: exon
gff_feature_attribute: gene_id
summarization_level: false
format: tabdel_short
include_feature_length_file: false
pe_parameters:
paired_end_status: PE_fragments
__current_case__: 2
check_distance_enabled:
P: "false"
__current_case__: 1
only_both_ends: false
exclude_chimerics: true
strand_specificity:
__class__: ConnectedValue
read_filtering_parameters:
mapping_quality: "0"
splitonly: ""
primary: false
ignore_dup: false
extended_parameters:
multifeatures:
multifeat: ""
__current_case__: 0
exon_exon_junction_read_counting_enabled:
count_exon_exon_junction_reads: ""
__current_case__: 1
long_reads: false
by_read_group: false
largest_overlap: false
min_overlap: "1"
frac_overlap: "0"
frac_overlap_feature: "0"
read_extension_5p: "0"
read_extension_3p: "0"
read_reduction: ""
R: false
- id: Project sample sheet to tabular
label: Project sample sheet to tabular
tool_id: __SAMPLE_SHEET_TO_TABULAR__
tool_version: 1.0.0
doc: "Tier: Resolved. Single job, no map-over. Reads the workflow input directly -- the only edge in this workflow that reads column_definitions, and the only place the condition metadata still exists. Emits six headerless rows, three columns: element identifier, condition, replicate. Hidden because it is plumbing, but it is the single most useful thing to look at when the split misbehaves -- promote it to a workflow output temporarily when debugging rather than reasoning about it blind."
in:
input: RNA-seq reads (sample sheet)
out:
- id: output
hide: true
tool_state:
input:
__class__: ConnectedValue
include_headers: false
- id: Build reference-level row predicate
label: Build reference-level row predicate
tool_id: toolshed.g2.bx.psu.edu/repos/iuc/compose_text_param/compose_text_param
tool_version: 0.1.1
tool_shed_repository:
changeset_revision: e188c9826e0f
name: compose_text_param
owner: iuc
tool_shed: toolshed.g2.bx.psu.edu
doc: "Tier: Resolved. Single job. Composes the Filter1 predicate `c2=='<level>'` (e.g. `c2=='AR0382'`) and feeds the `cond` port of `Select reference-level samples`. Kept as a separate step rather than templated across the three levels: the three factor-level ports of the two DESeq2 jobs are structurally distinct and must not be collapsed. c2 is the condition column under the assumed (identifier, condition, replicate) ordering -- correct it together with the other five column bindings in this region once the sample-sheet-to-tabular output columns are known."
in:
components_1|param_type|component_value: Reference condition level
out:
- id: out1
hide: true
tool_state:
components:
- param_type:
select_param_type: text
component_value: c2=='
- param_type:
select_param_type: text
component_value:
__class__: ConnectedValue
- param_type:
select_param_type: text
component_value: "'"
- id: Select reference-level samples
label: Select reference-level samples
tool_id: Filter1
tool_version: 1.1.1
doc: "Tier: Resolved. Single job, no map-over. Row filter on the headerless sample metadata table: keeps the rows whose condition column (c2) matches the reference level, via the predicate built upstream. Output is the two-row slice for that level, feeding the element-identifier projection. Hidden because it is plumbing; promote it temporarily alongside `Project sample sheet to tabular` when the split misbehaves."
in:
cond: Build reference-level row predicate/out1
input: Project sample sheet to tabular/output
out:
- id: out_file1
hide: true
tool_state:
cond:
__class__: ConnectedValue
input:
__class__: ConnectedValue
header_lines: "0"
- id: Reference-level identifier list
label: Reference-level identifier list
tool_id: Cut1
tool_version: 1.0.2
doc: "Tier: Resolved. Single job, no map-over. Projects the element-identifier column (c1) out of the reference-level row slice, producing the one-column identifier list that `Counts for the reference level` matches the counts collection against. The projection is not cosmetic: __FILTER_FROM_FILE__ matches on file content, so handing it the full three-column table would match nothing and yield an empty collection rather than an error. Hidden because it is plumbing."
in:
input: Select reference-level samples/out_file1
out:
- id: out_file1
hide: true
tool_state:
columnList: c1
delimiter: T
input:
__class__: ConnectedValue
- id: Counts for the reference level
label: Counts for the reference level
tool_id: __FILTER_FROM_FILE__
tool_version: 1.1.0
doc: "Tier: Resolved. Single job, no map-over. Membership sync: filters the featureCounts collection down to the elements whose identifiers appear in the reference-level identifier list. Element identifier is the only key Galaxy preserves across map-over, so this join is what makes the condition split work. Expect a 2-element sub-collection. Consumed by BOTH DESeq2 nodes as their reference factor level. Hidden because it is plumbing for those nodes."
in:
input: Count reads per gene/output_short
how|filter_source: Reference-level identifier list/out_file1
out:
- id: output_filtered
hide: true
- id: output_discarded
hide: true
tool_state:
how:
how_filter: remove_if_absent
__current_case__: 0
filter_source:
__class__: ConnectedValue
input:
__class__: ConnectedValue
- id: Build first-contrast row predicate
label: Build first-contrast row predicate
tool_id: toolshed.g2.bx.psu.edu/repos/iuc/compose_text_param/compose_text_param
tool_version: 0.1.1
tool_shed_repository:
changeset_revision: e188c9826e0f
name: compose_text_param
owner: iuc
tool_shed: toolshed.g2.bx.psu.edu
doc: "Tier: Resolved. Single job. Composes the Filter1 predicate `c2=='<level>'` (e.g. `c2=='tnSWI1'`) and feeds the `cond` port of `Select first-contrast samples`. Kept as a separate step rather than templated across the three levels: the three factor-level ports of the two DESeq2 jobs are structurally distinct and must not be collapsed. c2 is the condition column under the assumed (identifier, condition, replicate) ordering -- correct it together with the other five column bindings in this region once the sample-sheet-to-tabular output columns are known."
in:
components_1|param_type|component_value: First contrast condition level
out:
- id: out1
hide: true
tool_state:
components:
- param_type:
select_param_type: text
component_value: c2=='
- param_type:
select_param_type: text
component_value:
__class__: ConnectedValue
- param_type:
select_param_type: text
component_value: "'"
- id: Select first-contrast samples
label: Select first-contrast samples
tool_id: Filter1
tool_version: 1.1.1
doc: "Tier: Resolved. Single job, no map-over. Row filter on the headerless sample metadata table: keeps the rows whose condition column (c2) matches the first-contrast level, via the predicate built upstream. Output is the two-row slice for that level, feeding the element-identifier projection. Hidden because it is plumbing; promote it temporarily alongside `Project sample sheet to tabular` when the split misbehaves."
in:
cond: Build first-contrast row predicate/out1
input: Project sample sheet to tabular/output
out:
- id: out_file1
hide: true
tool_state:
cond:
__class__: ConnectedValue
input:
__class__: ConnectedValue
header_lines: "0"
- id: First-contrast identifier list
label: First-contrast identifier list
tool_id: Cut1
tool_version: 1.0.2
doc: "Tier: Resolved. Single job, no map-over. Projects the element-identifier column (c1) out of the first-contrast row slice, producing the one-column identifier list that `Counts for the first contrast level` matches the counts collection against. The projection is not cosmetic: __FILTER_FROM_FILE__ matches on file content, so handing it the full three-column table would match nothing and yield an empty collection rather than an error. Hidden because it is plumbing."
in:
input: Select first-contrast samples/out_file1
out:
- id: out_file1
hide: true
tool_state:
columnList: c1
delimiter: T
input:
__class__: ConnectedValue
- id: Counts for the first contrast level
label: Counts for the first contrast level
tool_id: __FILTER_FROM_FILE__
tool_version: 1.1.0
doc: "Tier: Resolved. Single job, no map-over. Membership sync: filters the featureCounts collection down to the elements whose identifiers appear in the first-contrast identifier list. Element identifier is the only key Galaxy preserves across map-over, so this join is what makes the whole condition split work. Expect a 2-element sub-collection. Hidden because it is plumbing for the DESeq2 node."
in:
input: Count reads per gene/output_short
how|filter_source: First-contrast identifier list/out_file1
out:
- id: output_filtered
hide: true
- id: output_discarded
hide: true
tool_state:
how:
how_filter: remove_if_absent
__current_case__: 0
filter_source:
__class__: ConnectedValue
input:
__class__: ConnectedValue
- id: Build second-contrast row predicate
label: Build second-contrast row predicate
tool_id: toolshed.g2.bx.psu.edu/repos/iuc/compose_text_param/compose_text_param
tool_version: 0.1.1
tool_shed_repository:
changeset_revision: e188c9826e0f
name: compose_text_param
owner: iuc
tool_shed: toolshed.g2.bx.psu.edu
doc: "Tier: Resolved. Single job. Composes the Filter1 predicate `c2=='<level>'` (e.g. `c2=='AR0387'`) and feeds the `cond` port of `Select second-contrast samples`. Kept as a separate step rather than templated across the three levels: the three factor-level ports of the two DESeq2 jobs are structurally distinct and must not be collapsed. c2 is the condition column under the assumed (identifier, condition, replicate) ordering -- correct it together with the other five column bindings in this region once the sample-sheet-to-tabular output columns are known."
in:
components_1|param_type|component_value: Second contrast condition level
out:
- id: out1
hide: true
tool_state:
components:
- param_type:
select_param_type: text
component_value: c2=='
- param_type:
select_param_type: text
component_value:
__class__: ConnectedValue
- param_type:
select_param_type: text
component_value: "'"
- id: Select second-contrast samples
label: Select second-contrast samples
tool_id: Filter1
tool_version: 1.1.1
doc: "Tier: Resolved. Single job, no map-over. Row filter on the headerless sample metadata table: keeps the rows whose condition column (c2) matches the second-contrast level, via the predicate built upstream. Output is the two-row slice for that level, feeding the element-identifier projection. Hidden because it is plumbing; promote it temporarily alongside `Project sample sheet to tabular` when the split misbehaves."
in:
cond: Build second-contrast row predicate/out1
input: Project sample sheet to tabular/output
out:
- id: out_file1
hide: true
tool_state:
cond:
__class__: ConnectedValue
input:
__class__: ConnectedValue
header_lines: "0"
- id: Second-contrast identifier list
label: Second-contrast identifier list
tool_id: Cut1
tool_version: 1.0.2
doc: "Tier: Resolved. Single job, no map-over. Projects the element-identifier column (c1) out of the second-contrast row slice, producing the one-column identifier list that `Counts for the second contrast level` matches the counts collection against. The projection is not cosmetic: __FILTER_FROM_FILE__ matches on file content, so handing it the full three-column table would match nothing and yield an empty collection rather than an error. Hidden because it is plumbing."
in:
input: Select second-contrast samples/out_file1
out:
- id: out_file1
hide: true
tool_state:
columnList: c1
delimiter: T
input:
__class__: ConnectedValue
- id: Counts for the second contrast level
label: Counts for the second contrast level
tool_id: __FILTER_FROM_FILE__
tool_version: 1.1.0
doc: "Tier: Resolved. Single job, no map-over. Membership sync: filters the featureCounts collection down to the elements whose identifiers appear in the second-contrast identifier list. Element identifier is the only key Galaxy preserves across map-over, so this join is what makes the whole condition split work. Expect a 2-element sub-collection. Hidden because it is plumbing for the DESeq2 node."
in:
input: Count reads per gene/output_short
how|filter_source: Second-contrast identifier list/out_file1
out:
- id: output_filtered
hide: true
- id: output_discarded
hide: true
tool_state:
how:
how_filter: remove_if_absent
__current_case__: 0
filter_source:
__class__: ConnectedValue
input:
__class__: ConnectedValue
- id: "Differential expression: first contrast vs reference"
label: "Differential expression: first contrast vs reference"
tool_id: toolshed.g2.bx.psu.edu/repos/iuc/deseq2/deseq2
tool_version: 2.11.40.8+galaxy4
tool_shed_repository:
changeset_revision: 05f9e54d7e81
name: deseq2
owner: iuc
tool_shed: toolshed.g2.bx.psu.edu
doc: "Tier: Resolved. THE WORKFLOW'S ONLY REDUCTION: each factor-level port consumes a whole 2-element counts collection through the wrapper's multiple=true data input, so the per-sample axis collapses here. One job, no map-over. Paper contrast: tnSWI1 vs AR0382, taken as level-1-vs-level-2 so log2FC is signed tnSWI1-relative. Sees only its own four samples, which is why its normalized counts and plots are promoted under contrast-specific labels rather than as the workflow's."
in:
select_data|rep_factorName_0|rep_factorLevel_0|factorLevel: First contrast condition level
select_data|rep_factorName_0|rep_factorLevel_0|countsFile: Counts for the first contrast level/output_filtered
select_data|rep_factorName_0|rep_factorLevel_1|factorLevel: Reference condition level
select_data|rep_factorName_0|rep_factorLevel_1|countsFile: Counts for the reference level/output_filtered
output_options|alpha_ma: Adjusted p-value threshold
out:
- id: deseq_out
- id: counts_out
- id: plots
tool_state:
advanced_options:
esf_cond:
esf: ""
__current_case__: 0
fit_type: "1"
outlier_replace_off: false
outlier_filter_off: false
auto_mean_filter_off: false
prefilter_conditional:
prefilter: ""
__current_case__: 1
use_beta_priors: false
lfc_shrinkage_type: none
batch_factors:
__class__: RuntimeValue
header: true
output_options:
output_selector:
- pdf
- normCounts
alpha_ma:
__class__: ConnectedValue
select_data:
how: datasets_per_level
__current_case__: 1
rep_factorName:
- __index__: 0
factorName: condition
rep_factorLevel:
- __index__: 0
factorLevel:
__class__: ConnectedValue
countsFile:
__class__: ConnectedValue
- __index__: 1
factorLevel:
__class__: ConnectedValue
countsFile:
__class__: ConnectedValue
tximport:
tximport_selector: count
__current_case__: 1
- id: "Differential expression: second contrast vs reference"
label: "Differential expression: second contrast vs reference"
tool_id: toolshed.g2.bx.psu.edu/repos/iuc/deseq2/deseq2
tool_version: 2.11.40.8+galaxy4
tool_shed_repository:
changeset_revision: 05f9e54d7e81
name: deseq2
owner: iuc
tool_shed: toolshed.g2.bx.psu.edu
doc: "Tier: Resolved. REDUCTION, as for the first contrast: each factor-level port consumes a whole 2-element counts collection through the wrapper's multiple=true data input, so the per-sample axis collapses here. One job, no map-over. Paper contrast: AR0387 vs AR0382, taken as level-1-vs-level-2 so log2FC is signed AR0387-relative. Sees only its own four samples -- the two AR0387 replicates and the two AR0382 replicates -- which is why its normalized counts and plots are promoted under contrast-specific labels rather than as the workflow's. Its normalized counts are NOT interchangeable with the first contrast's: size factors are estimated over a different sample set."
in:
select_data|rep_factorName_0|rep_factorLevel_0|factorLevel: Second contrast condition level
select_data|rep_factorName_0|rep_factorLevel_0|countsFile: Counts for the second contrast level/output_filtered
select_data|rep_factorName_0|rep_factorLevel_1|factorLevel: Reference condition level
select_data|rep_factorName_0|rep_factorLevel_1|countsFile: Counts for the reference level/output_filtered
output_options|alpha_ma: Adjusted p-value threshold
out:
- id: deseq_out
- id: counts_out
- id: plots
tool_state:
advanced_options:
esf_cond:
esf: ""
__current_case__: 0
fit_type: "1"
outlier_replace_off: false
outlier_filter_off: false
auto_mean_filter_off: false
prefilter_conditional:
prefilter: ""
__current_case__: 1
use_beta_priors: false
lfc_shrinkage_type: none
batch_factors:
__class__: RuntimeValue
header: true
output_options:
output_selector:
- pdf
- normCounts
alpha_ma:
__class__: ConnectedValue
select_data:
how: datasets_per_level
__current_case__: 1
rep_factorName:
- __index__: 0
factorName: condition
rep_factorLevel:
- __index__: 0
factorLevel:
__class__: ConnectedValue
countsFile:
__class__: ConnectedValue
- __index__: 1
factorLevel:
__class__: ConnectedValue
countsFile:
__class__: ConnectedValue
tximport:
tximport_selector: count
__current_case__: 1
- id: Build adjusted p-value predicate
label: Build adjusted p-value predicate
tool_id: toolshed.g2.bx.psu.edu/repos/iuc/compose_text_param/compose_text_param
tool_version: 0.1.1
tool_shed_repository:
changeset_revision: e188c9826e0f
name: compose_text_param
owner: iuc
tool_shed: toolshed.g2.bx.psu.edu
doc: "Tier: Resolved. Single job. Composes the Filter1 predicate `c7<<threshold>` (e.g. `c7<0.05`) and feeds the `cond` port of BOTH contrasts' p-adj filter steps; the predicate text is identical for the two chains, so one job serves both. c7 is the adjusted p-value column of the raw DESeq2 result table."
in:
components_1|param_type|component_value: Adjusted p-value threshold
out:
- id: out1
hide: true
tool_state:
components:
- param_type:
select_param_type: text
component_value: c7<
- param_type:
select_param_type: float
component_value:
__class__: ConnectedValue
- id: Build log2 fold-change predicate
label: Build log2 fold-change predicate
tool_id: toolshed.g2.bx.psu.edu/repos/iuc/compose_text_param/compose_text_param
tool_version: 0.1.1
tool_shed_repository:
changeset_revision: e188c9826e0f
name: compose_text_param
owner: iuc
tool_shed: toolshed.g2.bx.psu.edu
doc: "Tier: Resolved. Single job. Composes the Filter1 predicate `abs(c3)><threshold>` (e.g. `abs(c3)>1.0`) and feeds the `cond` port of BOTH contrasts' log2FC filter steps; the predicate text is identical for the two chains, so one job serves both. c3 is the log2FoldChange column of the raw DESeq2 result table, and the comparison is against that RAW column -- the threshold parameter is already in log2 units, so there is no log2() call and no conversion step anywhere in this workflow."
in:
components_1|param_type|component_value: log2 fold change threshold
out:
- id: out1
hide: true
tool_state:
components:
- param_type:
select_param_type: text
component_value: abs(c3)>
- param_type:
select_param_type: float
component_value:
__class__: ConnectedValue
- id: "Filter with p-adj threshold: first contrast"
label: "Filter with p-adj threshold: first contrast"
tool_id: Filter1
tool_version: 1.1.1
doc: "Tier: Resolved. Single job, no map-over. First link of the first contrast's chain: keeps the rows of the raw DESeq2 result table whose adjusted p-value clears the threshold, via the `c7<<threshold>` predicate built upstream. Hidden; the chain's endpoint (the log2FC link) is what gets promoted."
in:
cond: Build adjusted p-value predicate/out1
input: "Differential expression: first contrast vs reference/deseq_out"
out:
- id: out_file1
hide: true
tool_state:
cond:
__class__: ConnectedValue
input:
__class__: ConnectedValue
header_lines: "0"
- id: "Filter with log2 FC threshold: first contrast"
label: "Filter with log2 FC threshold: first contrast"
tool_id: Filter1
tool_version: 1.1.1
doc: "Tier: Resolved. Single job, no map-over. Second link of the first contrast's chain and its endpoint: keeps the rows of the p-adj-filtered table whose |log2 fold change| clears the threshold, via the `abs(c3)><threshold>` predicate built upstream. Promoted as `Significant genes: tnSWI1 vs AR0382`."
in:
cond: Build log2 fold-change predicate/out1
input: "Filter with p-adj threshold: first contrast/out_file1"
out:
- id: out_file1
rename: Genes filtered with adj p-value and log2(FC) thresholds
tool_state:
cond:
__class__: ConnectedValue
input:
__class__: ConnectedValue
header_lines: "0"
- id: "Filter with p-adj threshold: second contrast"
label: "Filter with p-adj threshold: second contrast"
tool_id: Filter1
tool_version: 1.1.1
doc: "Tier: Resolved. Single job, no map-over. First link of the second contrast's chain: keeps the rows of the raw DESeq2 result table whose adjusted p-value clears the threshold, via the `c7<<threshold>` predicate built upstream. Hidden; the chain's endpoint (the log2FC link) is what gets promoted."
in:
cond: Build adjusted p-value predicate/out1
input: "Differential expression: second contrast vs reference/deseq_out"
out:
- id: out_file1
hide: true
tool_state:
cond:
__class__: ConnectedValue
input:
__class__: ConnectedValue
header_lines: "0"
- id: "Filter with log2 FC threshold: second contrast"
label: "Filter with log2 FC threshold: second contrast"
tool_id: Filter1
tool_version: 1.1.1
doc: "Tier: Resolved. Single job, no map-over. Second link of the second contrast's chain and its endpoint: keeps the rows of the p-adj-filtered table whose |log2 fold change| clears the threshold, via the `abs(c3)><threshold>` predicate built upstream. Promoted as `Significant genes: AR0387 vs AR0382`."
in:
cond: Build log2 fold-change predicate/out1
input: "Filter with p-adj threshold: second contrast/out_file1"
out:
- id: out_file1
rename: Genes filtered with adj p-value and log2(FC) thresholds
tool_state:
cond:
__class__: ConnectedValue
input:
__class__: ConnectedValue
header_lines: "0"
comments:
- type: frame
position:
- 0
- 0
size:
- 520
- 300
color: blue
title: Raw-read quality control
contains_steps:
- Flatten reads for per-fastq QC
- Read quality report
- type: frame
position:
- 560
- 0
size:
- 560
- 520
color: green
title: Trim, align and count (map-over, one job per sample)
contains_steps:
- Quality-trim reads
- Splice-aware alignment
- Count reads per gene
- type: frame
position:
- 560
- 560
size:
- 400
- 180
color: yellow
title: Map strandedness parameter
contains_steps:
- Get featureCounts strandedness parameter
- type: frame
position:
- 0
- 360
size:
- 520
- 900
color: red
title: Condition split from the sample sheet (no corpus precedent)
contains_steps:
- Project sample sheet to tabular
- Build reference-level row predicate
- Select reference-level samples
- Reference-level identifier list
- Counts for the reference level
- Build first-contrast row predicate
- Select first-contrast samples
- First-contrast identifier list
- Counts for the first contrast level
- Build second-contrast row predicate
- Select second-contrast samples
- Second-contrast identifier list
- Counts for the second contrast level
- type: markdown
position:
- 0
- 1280
size:
- 520
- 220
color: red
text: |
**Delimited region — the phase-2 alternative is one deletion away.**
Nothing in IWC at `fe41a79` uses a `sample_sheet` collection input or
`__SAMPLE_SHEET_TO_TABULAR__`, so this region has no worked precedent. To take the
recorded alternative: delete `Project sample sheet to tabular`, change the reads
input to `list:paired` (dropping `column_definitions`), add a `data` input
`Sample metadata table` (sample_id, condition, replicate), and repoint the three
`Select ...-level samples` steps at it. Every other step here, and the rest of the
workflow, is unchanged.
- type: frame
position:
- 1160
- 0
size:
- 460
- 420
color: turquoise
title: Differential expression (two contrasts, two jobs)
contains_steps:
- "Differential expression: first contrast vs reference"
- "Differential expression: second contrast vs reference"
- type: frame
position:
- 1160
- 460
size:
- 460
- 200
color: yellow
title: Build significance filter predicates
contains_steps:
- Build adjusted p-value predicate
- Build log2 fold-change predicate
- type: frame
position:
- 1160
- 700
size:
- 460
- 440
color: orange
title: Significance filtering (padj then log2FC, per contrast)
contains_steps:
- "Filter with p-adj threshold: first contrast"
- "Filter with log2 FC threshold: first contrast"
- "Filter with p-adj threshold: second contrast"
- "Filter with log2 FC threshold: second contrast"
gxformat2 draft (see [[galaxy-workflow-draft-format]]): topology fully resolved (workflow inputs, outputs, step set, edges); tool_id / state / tool_shed_repository and wrapper-determined port names may be TODO with free-text _plan_state / _plan_context / _plan_in / _plan_out per step for later implementation Molds.
GalaxyWorkflowDraft — 27 step(s), 0 still drafty, 9 input(s), 16 output(s).
| step | label | tool | state | plan keys |
|---|---|---|---|---|
| Flatten reads for per-fastq QC | Flatten reads for per-fastq QC | __FLATTEN__ | resolved | — |
| Read quality report | Read quality report | toolshed.g2.bx.psu.edu/repos/devteam/fastqc/fastqc | resolved | — |
| Quality-trim reads | Quality-trim reads | toolshed.g2.bx.psu.edu/repos/lparsons/cutadapt/cutadapt | resolved | — |
| Splice-aware alignment | Splice-aware alignment | toolshed.g2.bx.psu.edu/repos/iuc/rgrnastar/rna_star | resolved | — |
| Get featureCounts strandedness parameter | Get featureCounts strandedness parameter | toolshed.g2.bx.psu.edu/repos/iuc/map_param_value/map_param_value | resolved | — |
| Count reads per gene | Count reads per gene | toolshed.g2.bx.psu.edu/repos/iuc/featurecounts/featurecounts | resolved | — |
| Project sample sheet to tabular | Project sample sheet to tabular | __SAMPLE_SHEET_TO_TABULAR__ | resolved | — |
| Build reference-level row predicate | Build reference-level row predicate | toolshed.g2.bx.psu.edu/repos/iuc/compose_text_param/compose_text_param | resolved | — |
| Select reference-level samples | Select reference-level samples | Filter1 | resolved | — |
| Reference-level identifier list | Reference-level identifier list | Cut1 | resolved | — |
| Counts for the reference level | Counts for the reference level | __FILTER_FROM_FILE__ | resolved | — |
| Build first-contrast row predicate | Build first-contrast row predicate | toolshed.g2.bx.psu.edu/repos/iuc/compose_text_param/compose_text_param | resolved | — |
| Select first-contrast samples | Select first-contrast samples | Filter1 | resolved | — |
| First-contrast identifier list | First-contrast identifier list | Cut1 | resolved | — |
| Counts for the first contrast level | Counts for the first contrast level | __FILTER_FROM_FILE__ | resolved | — |
| Build second-contrast row predicate | Build second-contrast row predicate | toolshed.g2.bx.psu.edu/repos/iuc/compose_text_param/compose_text_param | resolved | — |
| Select second-contrast samples | Select second-contrast samples | Filter1 | resolved | — |
| Second-contrast identifier list | Second-contrast identifier list | Cut1 | resolved | — |
| Counts for the second contrast level | Counts for the second contrast level | __FILTER_FROM_FILE__ | resolved | — |
| Differential expression: first contrast vs reference | Differential expression: first contrast vs reference | toolshed.g2.bx.psu.edu/repos/iuc/deseq2/deseq2 | resolved | — |
| Differential expression: second contrast vs reference | Differential expression: second contrast vs reference | toolshed.g2.bx.psu.edu/repos/iuc/deseq2/deseq2 | resolved | — |
| Build adjusted p-value predicate | Build adjusted p-value predicate | toolshed.g2.bx.psu.edu/repos/iuc/compose_text_param/compose_text_param | resolved | — |
| Build log2 fold-change predicate | Build log2 fold-change predicate | toolshed.g2.bx.psu.edu/repos/iuc/compose_text_param/compose_text_param | resolved | — |
| Filter with p-adj threshold: first contrast | Filter with p-adj threshold: first contrast | Filter1 | resolved | — |
| Filter with log2 FC threshold: first contrast | Filter with log2 FC threshold: first contrast | Filter1 | resolved | — |
| Filter with p-adj threshold: second contrast | Filter with p-adj threshold: second contrast | Filter1 | resolved | — |
| Filter with log2 FC threshold: second contrast | Filter with log2 FC threshold: second contrast | Filter1 | resolved | — |
# Galaxy workflow draft — C. auris Scf1 RNA-seq differential expression
#
# Source chain: freeform-summary.md (Santana et al. 2023, Science 381:1461-1467,
# DOI 10.1126/science.adf8972) -> freeform-galaxy-interface.md (phase 2)
# -> freeform-galaxy-data-flow.md (phase 3) -> iwc-comparison-notes.md +
# iwc-exemplar.gxwf.yml (phase 4, corpus fe41a79).
#
# TOPOLOGY IS SETTLED HERE. Wrapper resolution is evidence-gated per step:
# Resolved - concrete tool_id + bound state, no _plan_*
# Identity-pinned - concrete tool_id, tool_version: TODO, _plan_* kept
# Deferred - tool_id: TODO, full _plan_*
# Per-step tier is stated in each step's doc.
class: GalaxyWorkflowDraft
label: "C. auris Scf1 RNA-seq differential expression (Santana et al. 2023)"
doc: >-
Bulk RNA-seq differential expression for Candida auris adhesion mutants, reconstructed
from the supplementary Materials and Methods of Santana et al. 2023 (Science
381:1461-1467). FastQC -> Cutadapt -> RNA STAR -> featureCounts -> DESeq2 -> significance
filter, over 6 paired-end runs (BioProject PRJNA904261): 2 biological replicates each of
the high-adhesion parent AR0382, the low-adhesion clinical isolate AR0387, and the AR0382
tnSWI1 insertional mutant. Two contrasts, both against AR0382. Reference genome is
C. auris B8441, NCBI assembly GCA_002759435.2.
VERSION FIDELITY IS NOT CLAIMABLE. The paper names six tools and gives no version for any
of them; only its non-Galaxy software is versioned. This workflow pins its own tool
versions and reproduces the authors' method, never their software stack.
THE COUNTING ANNOTATION IS NOT THE PAPER'S. No GTF/GFF source is named anywhere in the
supplement. Whoever supplies the annotation input determines the gene ID space, and
therefore whether SCF1 appears as B9J08_001458 at all.
tags:
- transcriptomics
- RNAseq
inputs:
# --- 1. Reads -------------------------------------------------------------
# sample_sheet:paired: per-sample typed metadata attached to paired fastq. The
# `condition` column is the only in-workflow source of the DESeq2 factor grouping, and
# it is readable ONLY at this input -- Galaxy does not propagate column metadata through
# map-over (data-flow brief section 4.1).
#
# NO CORPUS PRECEDENT: zero IWC workflows at fe41a79 declare a sample_sheet collection or
# a non-null column_definitions. See the delimited region frame in `comments:` and
# open-requirements entry `no-iwc-precedent-for-sample-sheet-workflow-input`.
- id: RNA-seq reads (sample sheet)
type: collection
collection_type: sample_sheet:paired
format:
- fastqsanger.gz
optional: false
column_definitions:
- name: condition
type: string
optional: false
restrictions:
- AR0382
- AR0387
- tnSWI1
description: >-
Experimental condition. AR0382 is the high-adhesion parent and the reference
level for both contrasts.
- name: replicate
type: string
optional: false
restrictions:
- A
- B
description: Biological replicate label.
doc: >-
Six paired-end runs, 2 x 50 bp, Illumina NextSeq 2000 (PRJNA904261 / SRP409192).
Element identifiers are the deposited sample names and are the spine of this
workflow: AR0382_A, AR0382_B, AR0387_A, AR0387_B, AR0382_tnSWI1_A, AR0382_tnSWI1_B.
They must survive map-over unchanged -- the condition split joins on them.
# --- 2-3. Reference data --------------------------------------------------
- id: Reference genome FASTA
type: data
format:
- fasta
optional: false
doc: >-
C. auris B8441, NCBI assembly GCA_002759435.2 (Cand_auris_B8441_V2). Delivered as a
history dataset rather than a built-in index because no public Galaxy server indexes
this genome; RNA STAR builds its index inside each mapped job (~12.5 Mb, six builds).
- id: Gene annotation GTF
type: data
format:
- gtf
optional: false
doc: >-
NOT NAMED BY THE PAPER. Consumed twice -- RNA STAR splice junctions and featureCounts
gene assignment -- so the choice is load-bearing for both. NCBI RefSeq GFF and
FungiDB GFF for this assembly differ in gene ID space and attribute keys. If the
chosen source is GFF3 rather than GTF, this input's format changes or a conversion
step is inserted ahead of both consumers.
# --- 4-6. Factor levels ---------------------------------------------------
# The three level literals are exposed rather than baked into the filter steps. Baking
# them would re-hard-code into the STEPS the three-level design that the sample-sheet
# input was chosen to keep out of the INTERFACE. Closes open-requirements entry
# `deseq2-factor-level-names-not-parameterized`; the interface brief's section 2.3
# parameter table needs the reflected edit.
- id: Reference condition level
type: text
default: AR0382
optional: false
doc: >-
The condition value both contrasts are taken against. Must match a `condition` value
in the sample sheet exactly.
- id: First contrast condition level
type: text
default: tnSWI1
optional: false
doc: >-
Condition value for the first contrast (paper: the AR0382 tnSWI1 insertional mutant).
Changing it makes the `... tnSWI1 vs AR0382` output labels stale -- the labels are the
public API and are not derived from this parameter.
- id: Second contrast condition level
type: text
default: AR0387
optional: false
doc: >-
Condition value for the second contrast (paper: the low-adhesion clinical isolate
AR0387). Same label caveat as the first contrast level.
# --- 7. Strandedness ------------------------------------------------------
# Corpus idiom (rnaseq-pe): one restricted human-readable string at the interface, mapped
# per consumer by map_param_value with on_unmapped: fail. Replaces the free-text input
# the interface brief declared.
- id: Strandedness
type: text
default: stranded - reverse
optional: false
restrictions:
- stranded - forward
- stranded - reverse
- unstranded
doc: >-
INFERRED, NOT STATED. The paper names only the library kit (Illumina Stranded Total
RNA Prep with Ribo-Zero Plus), which is reverse-stranded (dUTP) in Illumina's standard
chemistry. The empirical check is the promoted featureCounts assignment summary: a
wrong setting shows up as a large Unassigned_NoFeatures fraction.
# --- 8-9. Significance thresholds ----------------------------------------
- id: Adjusted p-value threshold
type: float
default: 0.05
optional: false
doc: "Stated by the paper: adjusted p-value < 0.05. Applied to DESeq2 column c7."
- id: log2 fold change threshold
type: float
default: 1.0
optional: false
doc: >-
log2 fold change threshold. A log2 FC of 1 equals an absolute fold change of 2 (2^1),
which is exactly the paper's |fold change| > 2 criterion; a log2 FC of 3 equals 8.
Stated in log2 units because DESeq2 reports log2FoldChange and the filter compares the
raw column with no conversion -- the corpus idiom (rnaseq-de at fe41a79). A linear
2.0 compared against abs(c3) would silently apply a 4-fold cut.
outputs:
- id: "FastQC raw reads: text summary"
outputSource: Read quality report/text_file
doc: >-
Twelve elements, not six: the reads are flattened per read direction before QC, so
identifiers are <sample>_forward / <sample>_reverse.
- id: "FastQC raw reads: HTML report"
outputSource: Read quality report/html_file
- id: Cutadapt trimming report
outputSource: Quality-trim reads/report
- id: Trimmed reads
outputSource: Quality-trim reads/out_pairs
- id: STAR alignments (BAM)
outputSource: Splice-aware alignment/mapped_reads
- id: STAR mapping summary
outputSource: Splice-aware alignment/output_log
doc: "Log.final.out -- the assertable text behind the binary BAM."
- id: Gene counts per sample
outputSource: Count reads per gene/output_short
doc: >-
The strongest deterministic checkpoint in this workflow: exact per-gene integer
counts, addressable by gene id, keyed by the six sample element identifiers.
- id: featureCounts assignment summary
outputSource: Count reads per gene/output_summary
doc: >-
Assigned / Unassigned_NoFeatures / Unassigned_Ambiguous totals -- also the direct
read-out of whether Strandedness was set right.
# Normalized counts and diagnostic plots are promoted PER CONTRAST, not once. With two
# DESeq2 jobs each is produced twice, and the two are not interchangeable: each job sees
# only its own four samples, so size factors, normalized values, and the PCA/dispersion
# plots are all contrast-scoped. Promoting one and calling it "the" normalized counts
# would be wrong rather than merely arbitrary. Either table carries both AR0382
# replicates, so the paper's "SCF1 in the top 2.5% of AR0382 expression" sanity check
# works against either.
- id: "DESeq2 normalized counts: tnSWI1 vs AR0382"
outputSource: "Differential expression: first contrast vs reference/counts_out"
- id: "DESeq2 normalized counts: AR0387 vs AR0382"
outputSource: "Differential expression: second contrast vs reference/counts_out"
- id: "DESeq2 results: tnSWI1 vs AR0382"
outputSource: "Differential expression: first contrast vs reference/deseq_out"
doc: >-
Full result table. Expect SCF1 = B9J08_001458 present with a negative log2FC, and the
strongest / most significant dysregulation in the table.
- id: "DESeq2 results: AR0387 vs AR0382"
outputSource: "Differential expression: second contrast vs reference/deseq_out"
doc: >-
Full result table. Expect SCF1 the most down-regulated gene, ~29-fold per the paper.
- id: "Significant genes: tnSWI1 vs AR0382"
outputSource: "Filter with log2 FC threshold: first contrast/out_file1"
- id: "Significant genes: AR0387 vs AR0382"
outputSource: "Filter with log2 FC threshold: second contrast/out_file1"
- id: "DESeq2 diagnostic plots: tnSWI1 vs AR0382"
outputSource: "Differential expression: first contrast vs reference/plots"
- id: "DESeq2 diagnostic plots: AR0387 vs AR0382"
outputSource: "Differential expression: second contrast vs reference/plots"
steps:
# =========================================================================
# REGION 1 -- RAW-READ QC (side branch off the reads input)
# =========================================================================
# RESOLVED. Built-in collection operation, corpus-confirmed state.
# The corpus flattens BEFORE per-fastq QC rather than promoting a nested collection
# (rnaseq-pe at fe41a79: __FLATTEN__ with join_identifier `_`, consuming subworkflow
# declares collection_type: list). This contradicts the data-flow brief's section 7.1
# recommendation; the corpus evidence is the later and stronger of the two.
- id: Flatten reads for per-fastq QC
label: Flatten reads for per-fastq QC
tool_id: __FLATTEN__
tool_version: 1.0.0
doc: >-
Tier: Resolved. sample_sheet:paired -> flat list of 12 fastq, identifiers
<sample>_forward / <sample>_reverse. Side branch off the workflow input: it does not
enter the map-over region, so the per-sample outputs keep the six-element sample
identifier space untouched.
in:
input: RNA-seq reads (sample sheet)
out:
- id: output
hide: true
state:
join_identifier: _
# RESOLVED, and the wrapper choice is the substantive decision here.
# Source: Santana et al. 2023, supplementary Materials and Methods, Pipeline A step 1
# ("FastQC"); no version given, as for every Galaxy step in this paper.
#
# FASTQC, NOT FALCO -- the corpus exemplar was overruled on two agreeing grounds.
# transcriptomics/rnaseq-pe/rnaseq-pe at IWC fe41a79 fills this slot with iuc/falco, a
# drop-in FastQC reimplementation exposing the same input_file port and the same
# html_file + text_file pair, which is why the template left this step Deferred. But
# (a) the paper names FastQC, and (b) the exemplar is not representative of the corpus:
# across IWC at fe41a79, devteam/fastqc/fastqc appears 40 times against falco's 4.
# Paper fidelity and corpus frequency point the same way, so falco is recorded as the
# documented alternative and not taken.
#
# Pin resolved fresh via Tool Shed discovery (the step was Deferred, `tool_id: TODO`):
# `tool-search fastqc` returns `devteam~fastqc~fastqc` at score 48.9, 1.7x the next hit
# and the only exact match on the XML id; `tool-versions` lists 0.74+galaxy1 as newest;
# `tool-revisions --latest` resolves that version to exactly one changeset, 2c64fded1286.
# Every IWC workflow at fe41a79 that pins 0.74+galaxy1 pins the same 2c64fded1286 --
# version and changeset agree, as they did for cutadapt and unlike the bridge tools.
#
# STATE IS MACHINE-VALIDATED, unlike the two steps before it. Both outputs are plain
# `data` outputs (html/txt, `from_work_dir`), so the shed's ParsedTool JSON decodes and
# `galaxy-tool-cache add` succeeds where it fails on every collection-output wrapper.
# The state below is nonetheless authored against the `gxwf convert --to format2`
# round-trip rather than the tool summary's published linked schema: two independent
# corpus workflows -- amplicon-mgnify/...-quality-control-paired-end.ga and
# VGP-assembly-v2/Scaffolding-HiC-VGP8.ga at fe41a79 -- round-trip to a byte-identical
# seven-key block, which is what is written here.
#
# WRAPPER DEFAULTS, exactly as the supplement implies. It says only "read quality was
# assessed" and names no contaminant, adapter, or limits file, so all three optional data
# params stay `{__class__: RuntimeValue}` -- the corpus round-trip's form for an unset
# optional data input, not a value this step chose. `kmers: '7'` and `nogroup: false` are
# the wrapper's own defaults written out explicitly so a later wrapper bump cannot change
# them silently; `min_length: null` is the unset optional integer. Adding a contaminant
# list would be adding method the paper does not describe.
- id: Read quality report
label: Read quality report
tool_id: toolshed.g2.bx.psu.edu/repos/devteam/fastqc/fastqc
tool_version: 0.74+galaxy1
doc: >-
Tier: Resolved. Mapped over the FLATTENED 12-element list, not the six-element sample
axis: FastQC takes one dataset at a time (`input_file` is a single `data` param), which
is what the `__FLATTEN__` node upstream exists to produce. Twelve jobs, identifiers
<sample>_forward / <sample>_reverse. Both outputs are promoted -- the text summary is
the assertable checkpoint (per-module PASS/WARN/FAIL lines, total-sequence counts)
behind the user-facing HTML binary.
in:
input_file: Flatten reads for per-fastq QC/output
out:
- id: text_file
- id: html_file
tool_state:
input_file:
__class__: ConnectedValue
adapters:
__class__: RuntimeValue
contaminants:
__class__: RuntimeValue
limits:
__class__: RuntimeValue
kmers: '7'
min_length: null
nogroup: false
# =========================================================================
# REGION 2 -- MAP-OVER SPINE (one job per sample row, six jobs per step)
# =========================================================================
# RESOLVED. First real bioinformatics tool of the workflow; everything before it is
# plumbing. The template's pinned identity is CONFIRMED, not adopted on trust: Tool Shed
# search returns `lparsons/cutadapt/cutadapt` as the canonical wrapper (a
# `jackcurragh/ribogalaxy_cutadapt` fork publishes the same XML id and actually outranks
# it -- the fork is not what the corpus or the paper mean), tool-versions lists 5.2+galaxy2
# as newest, and tool-revisions --latest pins that version to changeset f6168dd17f82.
# Every one of the eight lparsons/cutadapt steps in IWC at fe41a79 pins the same
# 5.2+galaxy2 at the same f6168dd17f82 -- version and changeset agree exactly, which is
# not the case for the compose_text_param and map_param_value bridges elsewhere here.
#
# THIS STEP'S STATE IS NOT MACHINE-VALIDATED. `galaxy-tool-cache add` fails on this
# wrapper at EVERY version tried (5.2+galaxy2, 5.2+galaxy0, 4.9+galaxy1, 3.7+galaxy0):
# the summary decoder rejects the shed's ParsedTool JSON because `outputs[0]` is a
# collection output whose `structure` key is absent. That is the same decode failure as
# `__FLATTEN__`, and this wrapper proves the cause is a collection-typed output, not
# built-in-ness -- `out_pairs` is declared `<collection name="out_pairs" type="paired">`.
# So draft-validate reports skip_tool_not_found for this step and checks nothing. The
# state below was bound by hand against two authoritative sources instead:
# - the wrapper XML at the pinned version (tools-iuc, @TOOL_VERSION@ 5.2 /
# @VERSION_SUFFIX@ 2), for parameter names, defaults and output filters;
# - `gxwf convert --to format2` over epigenetics/cutandrun/cutandrun.ga and
# VGP-assembly-v2/post-curation-processing/Post_Curation.ga at fe41a79, for the
# round-trip shape of the `library` conditional under paired_collection.
#
# QUALITY TRIMMING ONLY, and the empty adapter repeats are the substantive claim.
# The paper says "Cutadapt with a Phred cutoff score of 20" and names no adapter
# sequence anywhere in the 60-page supplement, so all six adapter repeats
# (adapters/front_adapters/anywhere_adapters on R1, adapters2/front_adapters2/
# anywhere_adapters2 on R2) are written explicitly empty. Empty repeats under
# paired_collection are corpus-observed, not invented: VGP-assembly-v2 step 24 runs
# exactly this shape. Adding an adapter would be adding method the paper does not
# describe. Open-requirements `cutadapt-adapter-and-length-filter-unstated` STAYS OPEN.
#
# `filter_options.minimum_length: '1'` IS THE WRAPPER DEFAULT, NOT THE PAPER'S PARAMETER.
# The XML comments its own choice: "the following param's default value is intentionally
# set to 1 and different from the commandline tool's 0 to avoid hard to debug issues with
# downstream tools". It is written out rather than left implicit so a later wrapper bump
# cannot change the length filter silently -- the corpus shows this value is genuinely
# chosen per workflow (cutandrun uses 15, the VGP workflows use 1). The paper states no
# minimum-length filter; that gap is the second half of the open-requirements entry.
#
# `output_selector: [report]` is load-bearing, not cosmetic: the `report` output carries
# `<filter>output_selector and 'report' in output_selector</filter>`, so dropping it
# deletes the promoted `Cutadapt trimming report` output. `out_pairs` carries
# `<filter>library['type'] == 'paired_collection'</filter>` plus a not-`multiple_output`
# filter, so the paired_collection branch and the absence of `multiple_output` are what
# keep the aligner's input wired.
- id: Quality-trim reads
label: Quality-trim reads
tool_id: toolshed.g2.bx.psu.edu/repos/lparsons/cutadapt/cutadapt
tool_version: 5.2+galaxy2
doc: >-
Tier: Resolved. Map-over, one job per sample row (6 jobs). Under
library.type: paired_collection the wrapper emits ONE paired-inner collection
(`out_pairs`, declared `type="paired"` with `format_source="library|input_1"`),
so mapping it over the list:paired input yields list:paired again and no re-pair
node is needed before RNA STAR.
in:
library|input_1: RNA-seq reads (sample sheet)
out:
- id: out_pairs
- id: report
tool_state:
library:
type: paired_collection
__current_case__: 2
input_1:
__class__: ConnectedValue
r1:
adapters: []
front_adapters: []
anywhere_adapters: []
r2:
adapters2: []
front_adapters2: []
anywhere_adapters2: []
pair_adapters: false
other_trimming_options:
quality_cutoff: '20'
filter_options:
minimum_length: '1'
output_selector:
- report
# RESOLVED. This is the step that closes
# `star-history-reference-wiring-has-no-corpus-precedent`.
#
# THE HISTORY BRANCH EXISTS AND THE WRAPPER SPECIFIES IT COMPLETELY. The corpus could not
# answer this — both IWC STAR steps at fe41a79 (rnaseq-pe, rnaseq-sr) use
# `geneSource: indexed` — so it was settled from the wrapper's own schema: a summarize
# pass on iuc/rgrnastar/rna_star @ 2.7.11b+galaxy1, cross-read against rg_rnaStar.xml and
# macros.xml at tools-iuc main, which carries @TOOL_VERSION@ 2.7.11b / @VERSION_SUFFIX@ 1,
# i.e. exactly this version. `refGenomeSource.geneSource` offers `indexed` and `history`;
# the `history` when carries `genomeFastaFiles` (data, fasta/fasta.gz, required),
# `genomeSAindexNbases`, its own TWO-case `GTFconditional`, and `diploidconditional`.
# So the two former TODO ports resolve to `refGenomeSource|genomeFastaFiles` and
# `refGenomeSource|GTFconditional|sjdbGTFfile` — NOT to the exemplar's
# `refGenomeSource|GTFconditional|genomeDir`, which exists only under `indexed`.
#
# CASE INDICES ARE CORROBORATED, NOT ASSUMED — and they are not readable off the dropdown.
# Under `history` the GTFconditional lists its OPTIONS as without-gtf, with-gtf but its
# `<when>` elements in the reverse order, so `with-gtf` is case 0. The rule (case index =
# `<when>` document order) is confirmed empirically: `gxwf convert --to format2` over
# transcriptomics/rnaseq-pe/rnaseq-pe.ga at fe41a79 emits
# `geneSource: indexed / __current_case__: 0` and
# `GTFselect: without-gtf-with-gtf / __current_case__: 1`, and the XML lists the indexed
# branch's whens as with-gtf, without-gtf-with-gtf, without-gtf. That same round-trip is
# the authority for the rest of the authoring form: integers serialize as STRINGS
# (`sjdbOverhang: "100"`, `outSAMmapqUnique: "255"` there), `__current_case__` sits on
# every conditional, and connected ports carry `{__class__: ConnectedValue}`.
#
# "DEFAULT PARAMETERS" IS TAKEN LITERALLY, WITH TWO NAMED EXCEPTIONS. Every section below
# is the wrapper default, written out rather than left implicit so a later wrapper bump
# cannot change the alignment silently: `algo.params.settingsType: default` (case 0), NOT
# the exemplar's `full` ENCODE long-RNA block, which is a deliberate non-default choice
# this paper does not make; `filter.output_params2.output_select2: no`;
# `outWig.outWigType: None`, which is why no signal_* outputs exist; `quantMode: '-'`,
# which is why `reads_per_gene` and `transcriptome_mapped_reads` do not exist — both carry
# `<filter>` expressions on it. The two exceptions:
# - `sjdbOverhang: '49'` against a wrapper default of 100. The wrapper's own help says
# "Ideal value is ReadLength-1" and the paper states 2 x 50 bp reads. Carried from the
# step plan.
# - `genomeSAindexNbases: '10'` against a wrapper default of 14. This parameter EXISTS
# ONLY IN THE HISTORY BRANCH: it configures the in-job `STAR --runMode genomeGenerate`
# that this workflow's own delivery choice introduces, so it is not a mapping
# parameter the paper could have defaulted. The wrapper's help carries STAR's formula
# verbatim — "For small genomes, the parameter --genomeSAindexNbases must be scaled
# down to min(14, log2(GenomeLength)/2 - 1)" — and for B8441 at ~12.5 Mb that is
# min(14, log2(12.5e6)/2 - 1) = min(14, 10.79) -> 10. Leaving 14 is the documented
# seg-fault-at-mapping hazard for a genome this small; the wrapper's own test data
# uses 5. The genome LENGTH behind the arithmetic is the assembly record's, not a
# measurement of whatever dataset is actually wired, which is why open-requirements
# `star-genome-length-drives-sa-index-parameter` is opened rather than nothing being
# recorded.
#
# BOTH PROMOTED OUTPUTS ARE STRUCTURALLY UNSUPPRESSIBLE. `output_log` (from_work_dir
# Log.final.out) and `mapped_reads` carry no `<filter>` at all in the wrapper's
# `<outputs>`, so no parameter choice here can delete them. The plan's constraint — that a
# configuration suppressing output_log is not acceptable — is met by the wrapper's shape,
# not by this step's luck.
#
# THE GTF PORT IS WIRED; WHICH GTF IS STILL UNKNOWN. `sjdbGTFfile` takes the declared
# workflow input `Gene annotation GTF`, and the wrapper's accepted formats (`gff3,gtf`)
# admit that input's `gtf`. That settles the PORT and nothing more.
# `featurecounts-annotation-source-unnamed` is about the SOURCE and STAYS OPEN — the paper
# names no annotation anywhere, and no amount of wrapper reading can say which one it was.
#
# `tool_state:`, not `state:`, per the draft-wide convention.
- id: Splice-aware alignment
label: Splice-aware alignment
tool_id: toolshed.g2.bx.psu.edu/repos/iuc/rgrnastar/rna_star
tool_version: 2.7.11b+galaxy1
doc: >-
Tier: Resolved. Map-over, one job per sample row (6 jobs); the FASTA and GTF are data
inputs wired into a mapped step, so they broadcast into every job. Each job builds its
own temporary STAR index from the history FASTA (`--runMode genomeGenerate` into
tempstargenomedir), which is why there is no separate index node to wire.
in:
singlePaired|input: Quality-trim reads/out_pairs
refGenomeSource|genomeFastaFiles: Reference genome FASTA
refGenomeSource|GTFconditional|sjdbGTFfile: Gene annotation GTF
out:
- id: mapped_reads
- id: output_log
tool_state:
singlePaired:
sPaired: paired_collection
__current_case__: 2
input:
__class__: ConnectedValue
refGenomeSource:
geneSource: history
__current_case__: 1
genomeFastaFiles:
__class__: ConnectedValue
genomeSAindexNbases: '10'
GTFconditional:
GTFselect: with-gtf
__current_case__: 0
sjdbGTFfile:
__class__: ConnectedValue
sjdbGTFfeatureExon: exon
sjdbOverhang: '49'
quantmode_output:
quantMode: '-'
__current_case__: 0
diploidconditional:
diploid: 'No'
__current_case__: 1
twopass:
twopassMode: 'None'
__current_case__: 0
twopass_read_subset: ''
sj_precalculated: ''
chimOutType: ''
oformat:
outSAMattributes:
- NH
- HI
- AS
- nM
- ch
HI_offset: '1'
outSAMprimaryFlag: OneBestScore
outSAMmapqUnique: '60'
wasp_conditional:
waspOutputMode: ''
__current_case__: 1
filter:
basic_filters: []
output_params2:
output_select2: 'no'
__current_case__: 1
algo:
params:
settingsType: default
__current_case__: 0
perf:
outBAMsortingBinsN: '50'
winAnchorMultimapNmax: '50'
outWig:
outWigType: 'None'
__current_case__: 0
outWigStrand: 'false'
# RESOLVED. Parameter-derivation step. Corpus idiom from rnaseq-pe: one map_param_value
# per consumer, translating one user-facing vocabulary into each wrapper's own encoding,
# with on_unmapped: fail so a bad value stops the run instead of silently defaulting to
# unstranded -- which would produce plausible counts that are quietly wrong.
#
# WRAPPER CONFIRMED, NOT COPIED. The id lived only in this step's plan text, so it was
# re-derived against the Tool Shed rather than adopted: search resolves one dominant
# iuc-owned candidate (`Map parameter value`), tool-versions lists 0.1.0 / 0.1.1 / 0.2.0,
# and tool-revisions pins newest-0.2.0 to changeset 022a635f2283. Corpus
# transcriptomics/rnaseq-pe/rnaseq-pe at fe41a79 pins the same 0.2.0 at its own earlier
# changeset 5ac8a4bf7a8d; same tool version, different bytes, so the newest changeset is
# taken here.
#
# THE MAPPINGS ARE THE CORPUS'S, VERBATIM, and must stay in lockstep with the
# `Strandedness` input's `restrictions:` list. on_unmapped: fail converts any drift
# between the two into a hard run failure -- that is the design, not a hazard.
#
# STATE SHAPE IS THE ROUND-TRIP SHAPE. `__current_case__` on both conditionals and
# `__index__` on each mappings entry are kept deliberately: `gxwf convert --to format2`
# of the corpus .ga emits exactly this block, so this is what Galaxy round-trips. Note
# the tool summary disagrees -- `input_schemas.workflow_step_linked` is
# additionalProperties: false on these objects and would reject both keys. The validator
# sides with the converter (all four shapes probed here pass), so the published schema is
# the outlier and is not the authority on authoring form.
#
# `tool_state:`, not `state:`, per the draft-wide convention. Here the connected
# `input_param` is optional: true in the text branch, so unlike the compose_text_param
# bridges both keys validate and neither the key choice nor the presence of the
# ConnectedValue marker is enforced by any gate -- the convention is held by hand.
- id: Get featureCounts strandedness parameter
label: Get featureCounts strandedness parameter
tool_id: toolshed.g2.bx.psu.edu/repos/iuc/map_param_value/map_param_value
tool_version: 0.2.0
doc: >-
Tier: Resolved. Single job; featureCounts is the only consumer. Maps the user-facing
strandedness string onto featureCounts' numeric -s encoding (0 unstranded /
1 forward / 2 reverse). The VALUE it maps is still inferred from the library kit name
rather than stated by the paper: open-requirements
`rnaseq-strandedness-inferred-from-kit-name` stays OPEN. Concretizing how the
parameter is expressed says nothing about confidence in the value flowing through it.
in:
input_param_type|input_param: Strandedness
out:
- id: output_param_text
hide: true
tool_state:
input_param_type:
type: text
__current_case__: 0
input_param:
__class__: ConnectedValue
mappings:
- __index__: 0
from: stranded - forward
to: "1"
- __index__: 1
from: stranded - reverse
to: "2"
- __index__: 2
from: unstranded
to: "0"
output_param_type: text
unmapped:
on_unmapped: fail
__current_case__: 1
# RESOLVED. Identity was pinned by the template and is CONFIRMED rather than re-derived:
# `iuc/featurecounts/featurecounts` @ 2.1.1+galaxy1, changeset 37d067694d40. tool-versions
# lists 28 published versions with 2.1.1+galaxy1 newest; tool-revisions pins that version
# to exactly one changeset; and corpus transcriptomics/rnaseq-pe/rnaseq-pe at fe41a79 pins
# the same version at the same changeset (rnaseq-sr does too). Shed-newest and corpus agree,
# so no choice had to be made between them. It caches and summarizes cleanly -- all seven
# featureCounts outputs are plain `data` outputs, so the collection-output decode failure
# that blocks `__FLATTEN__` and lparsons/cutadapt does not apply here, and this step's state
# is validated for real rather than skipped.
#
# STATE IS THE CORPUS ROUND-TRIP, VERBATIM apart from the exemplar's `when` port (it gates
# its featureCounts on an optional-step boolean this workflow does not have). Every key
# below is what `gxwf convert --to format2` emits from rnaseq-pe.ga at fe41a79 for its own
# featureCounts step at this exact version, cross-read against the cached wrapper schema.
# That includes the integer-declared parameters serialized as STRINGS (`min_overlap: '1'`,
# `mapping_quality: '0'`, the frac_* and read_extension_* keys) -- the same
# wrapper-declares-integer / corpus-serializes-string pattern already settled for Filter1's
# `header_lines`.
#
# CASE INDICES ARE READ OFF `<when>` ORDER, NOT THE DROPDOWNS. `anno` (builtin, cached,
# history) puts `history` at 2, and `pe_parameters` (single_end, PE_individual,
# PE_fragments) puts `PE_fragments` at 2 -- there the option order and the `<when>` order
# happen to agree. `extended_parameters.exon_exon_junction_read_counting_enabled` is where
# they do not: its `<when>`s are ('-J', ''), so the OFF value '' is case 1, not case 0. All
# of them, plus `multifeatures` '' at case 0 and `check_distance_enabled` 'false' at case 1,
# match the round-trip exactly.
#
# FOUR NON-DEFAULT BINDINGS, ALL DELIBERATE. `anno_select: history` (wrapper default is
# `cached`) because the annotation is a declared workflow input -- the same dataset the
# aligner uses -- and the cached branch would need an admin-installed B8441 annotation that
# nothing in this run checked for (`reference-genome-delivery-shape-unverified`).
# `paired_end_status: PE_fragments` (default single_end) because the library is paired-end
# and a fragment is one observation, not two. `format: tabdel_short` because the DESeq2
# per-level ports take a two-column gene-id/count table. `strand_specificity` is CONNECTED
# rather than bound, so the wrapper default '0' (unstranded) never applies -- it arrives
# from `Get featureCounts strandedness parameter`, whose on_unmapped: fail turns any drift
# from the `Strandedness` input's restrictions into a hard failure. Everything else below is
# the wrapper default, written out explicitly so a later wrapper bump cannot change the
# counts silently.
#
# `gff_feature_attribute: gene_id` IS AN ANNOTATION-SHAPED GUESS, NOT A PAPER FACT. It is
# the wrapper default and the corpus value and it is right for a GTF, but a FungiDB B8441
# GFF3 keys on `ID` instead, and a wrong -g produces a full table of the WRONG identifiers
# rather than an error. Open-requirements `featurecounts-annotation-source-unnamed` STAYS
# OPEN and now carries this binding as well as the source question.
# `rnaseq-strandedness-inferred-from-kit-name` also STAYS OPEN: wiring the parameter port
# says nothing about the value flowing through it, which is still inferred from the library
# kit name. The promoted `output_summary` is what settles it empirically.
#
# BOTH OUTPUTS ARE PROMOTED, so neither carries `hide: true` -- the corpus hides both
# because its featureCounts feeds a MultiQC node this workflow does not have. `output_short`
# is ALSO consumed by the three condition-split filters, which join the metadata table to
# this collection on element identifier: the six sample names must survive this step
# unchanged, and nothing here renames them.
#
# `tool_state:`, not `state:`, per the draft-wide convention.
- id: Count reads per gene
label: Count reads per gene
tool_id: toolshed.g2.bx.psu.edu/repos/iuc/featurecounts/featurecounts
tool_version: 2.1.1+galaxy1
doc: >-
Tier: Resolved. Map-over, one job per sample row (6 jobs); the GTF is a data input wired
into a mapped step, so it broadcasts into every job. Counts fragments against `exon`
features grouped by `gene_id`, emitting the two-column gene-id/count table the DESeq2
per-level ports consume, plus the assignment summary that reads out whether the
strandedness setting was right.
in:
alignment: Splice-aware alignment/mapped_reads
anno|reference_gene_sets: Gene annotation GTF
strand_specificity: Get featureCounts strandedness parameter/output_param_text
out:
- id: output_short
- id: output_summary
tool_state:
alignment:
__class__: ConnectedValue
anno:
anno_select: history
__current_case__: 2
reference_gene_sets:
__class__: ConnectedValue
gff_feature_type: exon
gff_feature_attribute: gene_id
summarization_level: false
format: tabdel_short
include_feature_length_file: false
pe_parameters:
paired_end_status: PE_fragments
__current_case__: 2
check_distance_enabled:
P: 'false'
__current_case__: 1
only_both_ends: false
exclude_chimerics: true
strand_specificity:
__class__: ConnectedValue
read_filtering_parameters:
mapping_quality: '0'
splitonly: ''
primary: false
ignore_dup: false
extended_parameters:
multifeatures:
multifeat: ''
__current_case__: 0
exon_exon_junction_read_counting_enabled:
count_exon_exon_junction_reads: ''
__current_case__: 1
long_reads: false
by_read_group: false
largest_overlap: false
min_overlap: '1'
frac_overlap: '0'
frac_overlap_feature: '0'
read_extension_5p: '0'
read_extension_3p: '0'
read_reduction: ''
R: false
# =========================================================================
# REGION 3 -- CONDITION SPLIT FROM THE SAMPLE SHEET
#
# ############ DELIMITED REGION -- NO CORPUS PRECEDENT ####################
# Searched across all of workflows/ at IWC fe41a79: `sample_sheet` as a collection type
# appears in ZERO workflows, `__SAMPLE_SHEET_TO_TABULAR__` in ZERO, and
# `column_definitions` only ever as the literal null. `__FILTER_FROM_FILE__` is real
# (6 workflows) but none in transcriptomics and none for a condition split. Nothing in
# the corpus refutes this route -- it is unprecedented, not wrong -- but it is the one
# region of this workflow with no worked example to pattern-match against.
#
# THE PHASE-2 ALTERNATIVE IS ONE DELETION AWAY. If
# `sample-sheet-to-tabular-identifier-column-unverified` or
# `sample-sheet-input-test-fixture-expressibility` resolves against the sample sheet:
# 1. delete step `Project sample sheet to tabular`;
# 2. change the reads input to `collection_type: list:paired` (drop
# column_definitions) and rename it `RNA-seq reads`;
# 3. add a `data` input `Sample metadata table` (tabular: sample_id, condition,
# replicate);
# 4. repoint the three `Select ...-level samples` steps' `input` port at that input.
# Every other step in this region, and the whole of regions 1, 2, 4 and 5, is unchanged.
# That is why the metadata is routed through a tabular intermediate at all.
# #########################################################################
# =========================================================================
# RESOLVED. Built-in collection operation, but NOT a collection op in the plumbing sense:
# its own help says "This tool creates a new tabular dataset (not a collection
# operation)", and unlike `__FLATTEN__` / `__FILTER_FROM_FILE__` it caches cleanly --
# its single output is a `data` output, so gxwf's tool-summary decoder does not trip on
# the missing collection `structure`. Resolved by bare id against the Tool Shed API at
# v1.0.0, the only version it serves; summary cached for draft-validate.
#
# THE COLUMN QUESTION IS SETTLED, from the wrapper's own help rather than any exemplar:
# "The first column is always the element identifier (sample name). The remaining
# columns match the metadata fields defined in the sample sheet." With headers enabled
# the first column is literally named `element_identifier`. So the ordering this region
# assumed throughout -- (element identifier, condition, replicate) -- is CONFIRMED:
# c1 = element identifier, c2 = condition, c3 = replicate, in column_definitions order.
# All six provisional column bindings downstream (three Filter1 predicates on c2, three
# Cut1 projections of c1) stand exactly as written; none needed correcting. The
# Apply Rules substitute node named in the plan is not needed and is not built.
#
# `include_headers` is written explicitly at its wrapper default `false` rather than
# left implicit, because it is the binding the three `Select ...-level samples` filters
# read off: headerless output means `header_lines: 0` there. It is also the safer
# setting downstream -- Filter1 passes a retained header row through, Cut1 projects it,
# and `__FILTER_FROM_FILE__` would then try to match a literal `element_identifier`
# against the counts collection.
#
# The four value-replacement parameters (none_replace, empty_replace, bool_true_replace,
# bool_false_replace) are left at their defaults: both column_definitions on the reads
# input are `optional: false` strings, so no null and no boolean can reach the table.
#
# Still no corpus precedent -- nothing in IWC at fe41a79 uses this tool. The evidence
# here is the wrapper's own documented contract, not a worked example.
- id: Project sample sheet to tabular
label: Project sample sheet to tabular
tool_id: __SAMPLE_SHEET_TO_TABULAR__
tool_version: 1.0.0
doc: >-
Tier: Resolved. Single job, no map-over. Reads the workflow input directly -- the
only edge in this workflow that reads column_definitions, and the only place the
condition metadata still exists. Emits six headerless rows, three columns:
element identifier, condition, replicate. Hidden because it is plumbing, but it is
the single most useful thing to look at when the split misbehaves -- promote it to a
workflow output temporarily when debugging rather than reasoning about it blind.
in:
input: RNA-seq reads (sample sheet)
out:
- id: output
hide: true
tool_state:
input:
__class__: ConnectedValue
include_headers: false
# --- Reference level (AR0382) -------------------------------------------
# RESOLVED. Filter1's predicate is a TEXT parameter, so a workflow string parameter
# cannot reach it directly -- the same constraint the corpus documents for the
# significance thresholds. One compose_text_param per level builds the predicate.
# Same wrapper and version as the other four bridges: 0.1.1, changeset e188c9826e0f,
# the newest published version. Corpus:
# transcriptomics/rnaseq-de/rnaseq-de-filtering-plotting at fe41a79 pins
# iuc/compose_text_param/compose_text_param/0.1.1 and uses this bridge twice, for its two
# numeric thresholds; this is the same idiom applied to a string parameter. All THREE
# components take the TEXT case here -- the connected value is a condition-level string,
# not the float the two threshold bridges connect. Written under `tool_state:`, not
# `state:`, for one convention across the five bridges; see the state-key note on `Build
# adjusted p-value predicate` for why that distinction is not cosmetic.
- id: Build reference-level row predicate
label: Build reference-level row predicate
tool_id: toolshed.g2.bx.psu.edu/repos/iuc/compose_text_param/compose_text_param
tool_version: 0.1.1
doc: >-
Tier: Resolved. Single job. Composes the Filter1 predicate `c2=='<level>'` (e.g.
`c2=='AR0382'`) and feeds the `cond` port of `Select reference-level samples`. Kept
as a separate step rather than templated across the three levels: the three
factor-level ports of the two DESeq2 jobs are structurally distinct and must not be
collapsed. c2 is the condition column under the assumed (identifier, condition,
replicate) ordering -- correct it together with the other five column bindings in
this region once the sample-sheet-to-tabular output columns are known.
in:
components_1|param_type|component_value: Reference condition level
out:
- id: out1
hide: true
tool_state:
components:
- param_type:
select_param_type: text
component_value: "c2=='"
- param_type:
select_param_type: text
component_value:
__class__: ConnectedValue
- param_type:
select_param_type: text
component_value: "'"
# RESOLVED -- direct sibling of `Select first-contrast samples`. It adopts the four-step
# `Filter1` convention settled in the comment block above THAT step (bare stock id, no
# `tool_shed_repository` and no version suffix, tool_version 1.1.1, `cond`/`input` as
# ConnectedValue under `tool_state:`, string-encoded `header_lines`); that block is
# written once to be referenced, so it is not repeated here.
#
# `header_lines: '0'` is load-bearing at THIS step in particular. The reference level is
# the group both DESeq2 contrasts share, so binding '1' against the headerless table
# `Project sample sheet to tabular` emits would drop one of its two replicates and
# degrade both contrasts at once -- silently, with no error anywhere.
- id: Select reference-level samples
label: Select reference-level samples
tool_id: Filter1
tool_version: 1.1.1
doc: >-
Tier: Resolved. Single job, no map-over. Row filter on the headerless sample
metadata table: keeps the rows whose condition column (c2) matches the reference
level, via the predicate built upstream. Output is the two-row slice for that
level, feeding the element-identifier projection. Hidden because it is plumbing;
promote it temporarily alongside `Project sample sheet to tabular` when the split
misbehaves.
in:
cond: Build reference-level row predicate/out1
input: Project sample sheet to tabular/output
out:
- id: out_file1
hide: true
tool_state:
cond:
__class__: ConnectedValue
input:
__class__: ConnectedValue
header_lines: '0'
# RESOLVED -- the second of the three identifier-list projections. It adopts the `Cut1`
# convention settled in the CANONICAL comment block above `First-contrast identifier
# list` (bare stock id, NO `tool_shed_repository` and no version suffix, tool_version
# 1.0.2, ports `input` -> `out_file1`, `columnList` a STRING of `cN` tokens, `delimiter`
# the select's option VALUE `T` -- which has no default and must be written -- `input`
# as ConnectedValue under `tool_state:`, and no header parameter at all); that block is
# written once to be referenced, so it is not repeated here. `Cut1` @ 1.0.2 was already
# in the shared tool cache from that iteration, so no re-resolution was needed and none
# was done.
#
# Two plan assumptions are discharged by that block rather than carried: the port names
# are confirmed from the wrapper summary and 112/112 corpus occurrences, not merely
# assumed from Filter1; and the plan's informal "delimiter: tab" is written here as the
# wrapper's option value `T`. c1 = element identifier is settled from
# `__SAMPLE_SHEET_TO_TABULAR__`'s own packaged help, so the plan's provisional note on
# column ordering is closed.
- id: Reference-level identifier list
label: Reference-level identifier list
tool_id: Cut1
tool_version: 1.0.2
doc: >-
Tier: Resolved. Single job, no map-over. Projects the element-identifier column (c1)
out of the reference-level row slice, producing the one-column identifier list that
`Counts for the reference level` matches the counts collection against. The
projection is not cosmetic: __FILTER_FROM_FILE__ matches on file content, so handing
it the full three-column table would match nothing and yield an empty collection
rather than an error. Hidden because it is plumbing.
in:
input: Select reference-level samples/out_file1
out:
- id: out_file1
hide: true
tool_state:
columnList: c1
delimiter: T
input:
__class__: ConnectedValue
# RESOLVED. The CANONICAL `__FILTER_FROM_FILE__` note for all three counts filters sits
# above `Counts for the first contrast level` further down this file. Read it rather than
# this block for: why the step is NOT CHECKED BY ANY GATE (both outputs are
# `type: collection`, the shed serializes them flat, gxwf's decoder wants them nested
# under `structure`, so the tool cannot be cached and the step reports
# `skip_tool_not_found` under a green `Concrete: OK`); the SPLIT corpus behind the `1.1.0`
# pin (7 x 1.0.0 vs 6 x 1.1.0, the two raw TRS payloads diffed state-identical); the
# QUALIFIED `how|filter_source` port that the `TODO_filter_source` sentinel misnames; and
# the `__current_case__: 0` reading off `<when>` order. All of it applies here verbatim --
# only the identifier-list source below differs. Re-confirmed for this step: the cache add
# at 1.1.0 fails with the same `["outputs"][0]["structure"] is missing` decode mismatch.
#
# THIS filter carries more risk than its two siblings. The reference level is the group
# BOTH DESeq2 contrasts consume -- it feeds `rep_factorLevel_1|countsFile` on each of the
# two DESeq2 nodes -- so a wrong `how|filter_source` here corrupts both contrasts at once,
# and no gate in this run would say so. Checked by eye against the canonical block: the
# key is qualified `how|filter_source`, and its source is `Reference-level identifier
# list/out_file1` -- the reference-level projection immediately above, not either contrast
# list. On the eyeball-review list with its two siblings.
- id: Counts for the reference level
label: Counts for the reference level
tool_id: __FILTER_FROM_FILE__
tool_version: 1.1.0
doc: >-
Tier: Resolved. Single job, no map-over. Membership sync: filters the featureCounts
collection down to the elements whose identifiers appear in the reference-level
identifier list. Element identifier is the only key Galaxy preserves across map-over,
so this join is what makes the condition split work. Expect a 2-element
sub-collection. Consumed by BOTH DESeq2 nodes as their reference factor level.
Hidden because it is plumbing for those nodes.
in:
input: Count reads per gene/output_short
how|filter_source: Reference-level identifier list/out_file1
out:
- id: output_filtered
hide: true
- id: output_discarded
hide: true
tool_state:
how:
how_filter: remove_if_absent
__current_case__: 0
filter_source:
__class__: ConnectedValue
input:
__class__: ConnectedValue
# --- First contrast level (tnSWI1) --------------------------------------
# RESOLVED. Same wrapper and version as `Build adjusted p-value predicate`: 0.1.1,
# changeset e188c9826e0f, the newest published version. All THREE components take the
# TEXT case here -- the connected value is a condition-level string, not the float the
# two threshold bridges connect. Written under `tool_state:`, not `state:`, for one
# convention across the five bridges; see the state-key note on `Build adjusted p-value
# predicate` for why that distinction is not cosmetic.
- id: Build first-contrast row predicate
label: Build first-contrast row predicate
tool_id: toolshed.g2.bx.psu.edu/repos/iuc/compose_text_param/compose_text_param
tool_version: 0.1.1
doc: >-
Tier: Resolved. Single job. Composes the Filter1 predicate `c2=='<level>'` (e.g.
`c2=='tnSWI1'`) and feeds the `cond` port of `Select first-contrast samples`. Kept as
a separate step rather than templated across the three levels: the three factor-level
ports of the two DESeq2 jobs are structurally distinct and must not be collapsed. c2
is the condition column under the assumed (identifier, condition, replicate)
ordering -- correct it together with the other five column bindings in this region
once the sample-sheet-to-tabular output columns are known.
in:
components_1|param_type|component_value: First contrast condition level
out:
- id: out1
hide: true
tool_state:
components:
- param_type:
select_param_type: text
component_value: "c2=='"
- param_type:
select_param_type: text
component_value:
__class__: ConnectedValue
- param_type:
select_param_type: text
component_value: "'"
# RESOLVED -- and this settles the convention for ALL FOUR `Filter1` steps in this
# workflow (the three condition-split row filters and the two significance filters,
# which share the same binding shape). Stock built-in: the bare id IS the wrapper
# identity, so no Tool Shed path, no `tool_shed_repository` block, and no version
# suffix on `tool_id` -- exactly how the corpus round-trip writes it.
#
# Version 1.1.1, unanimous across the corpus: 35 `Filter1` occurrences in 16 IWC
# workflows at fe41a79, every one at 1.1.1, and it is the only version the shed serves.
# Resolution needed an explicit `--tool-version`; a bare `galaxy-tool-cache add Filter1`
# fails misleadingly (TRS 500, then `/versions/_default_` 404). Its single output is a
# plain `data` output, so the summary caches and draft-validate checks this state for
# real -- unlike the collection-output steps in this workflow.
#
# `header_lines: '0'` is SETTLED, not defensive. `Project sample sheet to tabular` is
# pinned with `include_headers: false`, so the table reaching this filter is headerless.
# Binding '1' here would not error -- it would unconditionally drop the first data row,
# costing exactly one sample from the group and silently degrading a 2-replicate level
# to 1 replicate. The wrapper's own default is 0, and it is written explicitly anyway
# because it is load-bearing and reads off a sibling step's binding.
#
# Encoding is the corpus's, not the summary's: the wrapper declares `header_lines` as
# `gx_integer`, but all 21 corpus Filter1 states serialize it as a STRING ('0' x14,
# '1' x7). Written as '0' to match what Galaxy actually round-trips.
#
# Both connected ports carry `{__class__: ConnectedValue}` under `tool_state:`. That is
# not cosmetic here: `cond` is `optional: false` and carries an `empty_field` validator,
# so a `state:` block -- from which gxwf 1.10.1 drops the ConnectedValue marker -- would
# fail tool-state validation, and "fixing" it with a literal predicate would leave a
# shadow default behind a connected parameter.
#
# Corpus shape confirmed by `gxwf convert --to format2` round-trip of
# transcriptomics/rnaseq-de/rnaseq-de-filtering-plotting at fe41a79, steps
# `Filter with p-adj threshold` and `Filter with log2 FC threshold`: bare `Filter1`,
# tool_version 1.1.1, `tool_state` with `cond`/`input` as ConnectedValue and a string
# `header_lines`. That workflow filters a DESeq2 table rather than a sample sheet, so it
# fixes the BINDING SHAPE only -- the condition-split use of that shape remains
# unprecedented in IWC, as this region declares.
- id: Select first-contrast samples
label: Select first-contrast samples
tool_id: Filter1
tool_version: 1.1.1
doc: >-
Tier: Resolved. Single job, no map-over. Row filter on the headerless sample
metadata table: keeps the rows whose condition column (c2) matches the
first-contrast level, via the predicate built upstream. Output is the two-row
slice for that level, feeding the element-identifier projection. Hidden because it
is plumbing; promote it temporarily alongside `Project sample sheet to tabular`
when the split misbehaves.
in:
cond: Build first-contrast row predicate/out1
input: Project sample sheet to tabular/output
out:
- id: out_file1
hide: true
tool_state:
cond:
__class__: ConnectedValue
input:
__class__: ConnectedValue
header_lines: '0'
# RESOLVED -- and this block is the CANONICAL `Cut1` note for all THREE identifier-list
# projections in this region. The other two reference it rather than repeat it.
#
# Stock built-in, resolved like `Filter1`: bare `Cut1` as `tool_id`, NO
# `tool_shed_repository` block, NO version suffix on the id. `tool_version: 1.0.2` --
# unanimous in the corpus at fe41a79 (112 occurrences across 35 workflows, not one other
# version), then confirmed by a successful `galaxy-tool-cache add Cut1 --tool-version
# 1.0.2`. The explicit `--tool-version` is required: a bare add fails misleadingly for a
# stock tool. Its single output is a plain `data` output, so it caches and its state is
# checked for real by draft-validate --concrete -- unlike the collection operations here.
#
# PORTS: `input` -> `out_file1`. Wrapper summary and 112/112 corpus occurrences agree;
# the assumption recorded in the plan (same built-in convention as Filter1) is confirmed,
# not just adopted.
#
# STATE is exactly three keys, and two of them are easy to get wrong:
# - `columnList` is a STRING of comma-separated `cN` tokens -- `c1` here, one column.
# It is `gx_text` (optional, no default), never a list and never an integer.
# - `delimiter` is the SELECT's option VALUE, `T`, not the word `tab` the plan used
# informally. The wrapper offers Tab/Whitespace/Dot/Comma/Dash/Underscore/Pipe with
# values T/Sp/Dt/C/D/U/P; 110 of the 112 corpus states write `T`. No option carries
# `selected`, so `delimiter` has NO default and MUST be written explicitly.
# - `input` carries `{__class__: ConnectedValue}` under `tool_state:`, per the one
# state-key convention used throughout this draft.
# Corroborated by round-trip, not just by frequency: VGP-assembly-v2/kmer-profiling-hifi-VGP1
# at fe41a79 through `gxwf convert --to format2` emits exactly this four-line `tool_state`.
#
# HEADERS: `Cut1` has NO header parameter -- it projects every row it is handed. That is
# consistent with, and does not reintroduce an assumption against, the headerless
# contract this region is built on (`include_headers: false` on `Project sample sheet to
# tabular`, `header_lines: '0'` on all three selectors): every row reaching this step is a
# sample row, so the projection is exactly the level's element identifiers and nothing
# else. c1 = element identifier is settled from the projection tool's own packaged help.
- id: First-contrast identifier list
label: First-contrast identifier list
tool_id: Cut1
tool_version: 1.0.2
doc: >-
Tier: Resolved. Single job, no map-over. Projects the element-identifier column (c1)
out of the first-contrast row slice, producing the one-column identifier list that
`Counts for the first contrast level` matches the counts collection against. The
projection is not cosmetic: __FILTER_FROM_FILE__ matches on file content, so handing
it the full three-column table would match nothing and yield an empty collection
rather than an error. Hidden because it is plumbing.
in:
input: Select first-contrast samples/out_file1
out:
- id: out_file1
hide: true
tool_state:
columnList: c1
delimiter: T
input:
__class__: ConnectedValue
# RESOLVED -- and this block is the CANONICAL `__FILTER_FROM_FILE__` note for all THREE
# counts filters in this region. The other two reference it rather than repeat it.
#
# Stock built-in, resolved like `Filter1` and `Cut1`: bare `__FILTER_FROM_FILE__` as
# `tool_id`, NO `tool_shed_repository` block, NO version suffix on the id.
#
# NOT CHECKED BY ANY GATE -- read this before trusting a green verdict over this step.
# Both outputs are `type: collection`, and the shed serializes a collection output FLAT
# while gxwf's decoder wants `collection_type` / `collection_type_source` /
# `structured_like` nested inside a `structure` object. So
# `galaxy-tool-cache add __FILTER_FROM_FILE__ --tool-version 1.1.0` fails with
# `["outputs"][0]["structure"] is missing` -- a decode mismatch, not a transport error,
# and nothing is missing upstream. This step therefore reports `skip_tool_not_found`, and
# `draft-validate --concrete` checks NOTHING about its state: the green `Concrete: OK`
# line prints over an unvalidated step. Everything below was bound by hand against the
# raw TRS payload `GET /api/tools/__FILTER_FROM_FILE__/versions/1.1.0` (200) and a corpus
# round-trip, and the step is on the eyeball-review list.
#
# VERSION: `1.1.0`, and the corpus is SPLIT -- do not read this as unanimous the way the
# `Filter1` and `Cut1` pins are. 13 occurrences across 6 workflows at fe41a79: 7 at 1.0.0
# (Scaffolding-HiC-VGP8, influenza-consensus-and-subtyping, multiplex-tma) and 6 at 1.1.0
# (the three mgnify-amplicon-pipeline-v5 workflows). The two raw TRS payloads were
# fetched and diffed: inputs, the `how` conditional, both `whens` and both outputs are
# IDENTICAL, so 1.1.0 changes nothing that touches tool state. Taking the newer of two
# state-equivalent versions.
#
# PORTS -- and one of them is a trap the template's own sentinel sets. `input` is a
# top-level `data_collection` port, but `filter_source` lives INSIDE the `how`
# conditional, so its `in:` key must be QUALIFIED as `how|filter_source`. The
# `TODO_filter_source` sentinel implies a flat top-level port and is misleading; a bare
# `filter_source:` key would not resolve. Outputs are `output_filtered` /
# `output_discarded`.
#
# STATE, and the case index is the load-bearing part:
# - `how_filter: remove_if_absent` is `__current_case__: 0`. Read off `<when>` ORDER,
# per the standing convention -- it is `whens[0]` and carries `is_default_when: true`.
# Option order happens to agree here, so there is no trap, but the index was taken
# from `<when>` order regardless and not from the dropdown. Semantically
# `remove_if_absent` = drop the elements ABSENT from the identifier list = keep this
# level's samples, which is what the plan asks for; `remove_if_present` (case 1) would
# keep exactly the four samples we want dropped.
# - both connected ports carry `{__class__: ConnectedValue}` under `tool_state:`, per
# the one state-key convention used throughout this draft. `input` and `filter_source`
# are both `optional: false`, so this is not cosmetic.
#
# OUTPUTS: `output_discarded` is DECLARED and hidden, not omitted. The plan's "do not
# promote the discarded branch" is satisfied by hiding it -- promotion is the top-level
# `outputs:` list, which neither branch appears in. `output_filtered` is hidden too: it is
# not a promoted workflow output, and every non-promoted step output in this draft is
# hidden. All six corpus 1.1.0 instances hide both.
#
# SHAPE: both outputs declare `collection_type_source: input`, so the filtered collection
# inherits the counts collection's sample_sheet-shaped outer axis rather than flattening
# to a literal `list` -- which is what the DESeq2 node downstream reduces over.
# Round-trip: mgnify-amplicon-pipeline-v5-rrna-prediction at fe41a79 through
# `gxwf convert --to format2` (`_unlabeled_step_17`) emits exactly the `in:` and
# `tool_state:` written below, port for port and key for key.
- id: Counts for the first contrast level
label: Counts for the first contrast level
tool_id: __FILTER_FROM_FILE__
tool_version: 1.1.0
doc: >-
Tier: Resolved. Single job, no map-over. Membership sync: filters the featureCounts
collection down to the elements whose identifiers appear in the first-contrast
identifier list. Element identifier is the only key Galaxy preserves across map-over,
so this join is what makes the whole condition split work. Expect a 2-element
sub-collection. Hidden because it is plumbing for the DESeq2 node.
in:
input: Count reads per gene/output_short
how|filter_source: First-contrast identifier list/out_file1
out:
- id: output_filtered
hide: true
- id: output_discarded
hide: true
tool_state:
how:
how_filter: remove_if_absent
__current_case__: 0
filter_source:
__class__: ConnectedValue
input:
__class__: ConnectedValue
# --- Second contrast level (AR0387) -------------------------------------
# RESOLVED. Same wrapper and version as the other four bridges: 0.1.1, changeset
# e188c9826e0f, the newest published version. Corpus:
# transcriptomics/rnaseq-de/rnaseq-de-filtering-plotting at fe41a79 pins
# iuc/compose_text_param/compose_text_param/0.1.1 and uses this bridge twice, for its two
# numeric thresholds; this is the same idiom applied to a string parameter. All THREE
# components take the TEXT case here -- the connected value is a condition-level string,
# not the float the two threshold bridges connect. Written under `tool_state:`, not
# `state:`, for one convention across the five bridges; see the state-key note on `Build
# adjusted p-value predicate` for why that distinction is not cosmetic.
- id: Build second-contrast row predicate
label: Build second-contrast row predicate
tool_id: toolshed.g2.bx.psu.edu/repos/iuc/compose_text_param/compose_text_param
tool_version: 0.1.1
doc: >-
Tier: Resolved. Single job. Composes the Filter1 predicate `c2=='<level>'` (e.g.
`c2=='AR0387'`) and feeds the `cond` port of `Select second-contrast samples`. Kept
as a separate step rather than templated across the three levels: the three
factor-level ports of the two DESeq2 jobs are structurally distinct and must not be
collapsed. c2 is the condition column under the assumed (identifier, condition,
replicate) ordering -- correct it together with the other five column bindings in
this region once the sample-sheet-to-tabular output columns are known.
in:
components_1|param_type|component_value: Second contrast condition level
out:
- id: out1
hide: true
tool_state:
components:
- param_type:
select_param_type: text
component_value: "c2=='"
- param_type:
select_param_type: text
component_value:
__class__: ConnectedValue
- param_type:
select_param_type: text
component_value: "'"
# RESOLVED -- the last of the three condition-split row filters, and a direct sibling of
# `Select first-contrast samples`. It adopts the `Filter1` convention settled in the
# comment block above THAT step (bare stock id, no `tool_shed_repository` and no version
# suffix, tool_version 1.1.1, `cond`/`input` as ConnectedValue under `tool_state:`,
# string-encoded `header_lines`); that block is written once to be referenced, so it is
# not repeated here.
#
# `header_lines: '0'` for the same reason it is '0' at the other two: `Project sample
# sheet to tabular` pins `include_headers: false`, so there is no header row to skip and
# '1' would drop this level's first replicate -- leaving the second contrast a 1-vs-2
# comparison with no error raised anywhere.
- id: Select second-contrast samples
label: Select second-contrast samples
tool_id: Filter1
tool_version: 1.1.1
doc: >-
Tier: Resolved. Single job, no map-over. Row filter on the headerless sample
metadata table: keeps the rows whose condition column (c2) matches the
second-contrast level, via the predicate built upstream. Output is the two-row
slice for that level, feeding the element-identifier projection. Hidden because it
is plumbing; promote it temporarily alongside `Project sample sheet to tabular`
when the split misbehaves.
in:
cond: Build second-contrast row predicate/out1
input: Project sample sheet to tabular/output
out:
- id: out_file1
hide: true
tool_state:
cond:
__class__: ConnectedValue
input:
__class__: ConnectedValue
header_lines: '0'
# RESOLVED -- the third and last of the three identifier-list projections. It adopts the
# `Cut1` convention settled in the CANONICAL comment block above `First-contrast
# identifier list` (bare stock id, NO `tool_shed_repository` and no version suffix,
# tool_version 1.0.2, ports `input` -> `out_file1`, `columnList` a STRING of `cN` tokens,
# `delimiter` the select's option VALUE `T` -- which has no default and must be written
# -- `input` as ConnectedValue under `tool_state:`, and no header parameter at all); that
# block is written once to be referenced, so it is not repeated here. `Cut1` @ 1.0.2 was
# already in the shared tool cache, so no re-resolution was needed and none was done.
#
# With this step all three projections are concrete and byte-identical in state, which is
# the point: they differ only in which row slice they are wired to.
#
# Two plan assumptions are discharged by that block rather than carried: the port names
# are confirmed from the wrapper summary and 112/112 corpus occurrences, not merely
# assumed from Filter1; and the plan's informal "delimiter: tab" is written here as the
# wrapper's option value `T`. c1 = element identifier is settled from
# `__SAMPLE_SHEET_TO_TABULAR__`'s own packaged help, so the plan's provisional note on
# column ordering is closed.
- id: Second-contrast identifier list
label: Second-contrast identifier list
tool_id: Cut1
tool_version: 1.0.2
doc: >-
Tier: Resolved. Single job, no map-over. Projects the element-identifier column (c1)
out of the second-contrast row slice, producing the one-column identifier list that
`Counts for the second contrast level` matches the counts collection against. The
projection is not cosmetic: __FILTER_FROM_FILE__ matches on file content, so handing
it the full three-column table would match nothing and yield an empty collection
rather than an error. Hidden because it is plumbing.
in:
input: Select second-contrast samples/out_file1
out:
- id: out_file1
hide: true
tool_state:
columnList: c1
delimiter: T
input:
__class__: ConnectedValue
# RESOLVED -- the third and last of the three counts filters, and the one that completes
# the condition-split region. The CANONICAL `__FILTER_FROM_FILE__` note sits above
# `Counts for the first contrast level` above. Read it rather than this block for: why
# the step is NOT CHECKED BY ANY GATE (both outputs are `type: collection`, the shed
# serializes them flat, gxwf's decoder wants them nested under `structure`, so the tool
# cannot be cached and the step reports `skip_tool_not_found` under a green
# `Concrete: OK`); the SPLIT corpus behind the `1.1.0` pin; the QUALIFIED
# `how|filter_source` port that the `TODO_filter_source` sentinel misnames; and the
# `__current_case__: 0` read off `<when>` order. All of it applies here verbatim -- only
# the identifier-list source below differs. Re-confirmed for this step: the cache add at
# 1.1.0 fails with the same `["outputs"][0]["structure"] is missing` decode mismatch.
#
# Checked by eye against the canonical block, key for key: `input` is the top-level port,
# `how|filter_source` is qualified, and its source is `Second-contrast identifier
# list/out_file1` -- the second-contrast projection immediately above, not the reference
# or first-contrast list. A structural diff of the three siblings' parsed `in` / `out` /
# `tool_state` shows them identical key for key, differing only in that one source.
# On the eyeball-review list with its two siblings.
- id: Counts for the second contrast level
label: Counts for the second contrast level
tool_id: __FILTER_FROM_FILE__
tool_version: 1.1.0
doc: >-
Tier: Resolved. Single job, no map-over. Membership sync: filters the featureCounts
collection down to the elements whose identifiers appear in the second-contrast
identifier list. Element identifier is the only key Galaxy preserves across map-over,
so this join is what makes the whole condition split work. Expect a 2-element
sub-collection. Hidden because it is plumbing for the DESeq2 node.
in:
input: Count reads per gene/output_short
how|filter_source: Second-contrast identifier list/out_file1
out:
- id: output_filtered
hide: true
- id: output_discarded
hide: true
tool_state:
how:
how_filter: remove_if_absent
__current_case__: 0
filter_source:
__class__: ConnectedValue
input:
__class__: ConnectedValue
# =========================================================================
# REGION 4 -- DIFFERENTIAL EXPRESSION (the workflow's only reduction)
#
# TWO DESeq2 jobs, each with a two-level factor, the reference-level collection feeding
# both. Settled on corpus evidence: the exemplar's `deseq_out` is a single dataset (it
# feeds single-dataset tools and its sibling test asserts on it as a dataset), so one job
# yields one results table, and the interface's two distinctly labelled result tables
# therefore require two jobs. A three-level factor is expressible via the rep_factorLevel
# repeat but would still emit one deseq_out.
# =========================================================================
# RESOLVED -- the first of the two DESeq2 nodes, and the workflow's ONLY reduction:
# everything upstream of here is map-over, and each `countsFile` port swallows a whole
# 2-element counts collection through the wrapper's `multiple="true"` data input, so the
# per-sample axis collapses at this step.
#
# NOT CHECKED BY ANY GATE -- read this before trusting a green verdict over this step.
# `galaxy-tool-cache add toolshed.g2.bx.psu.edu/repos/iuc/deseq2/deseq2 --tool-version
# 2.11.40.8+galaxy4` fails with `["outputs"][0]["structure"] is missing`, the same
# flat-vs-nested collection-output decode mismatch that blocks the collection operations
# in region 3 -- and here it is NOT a collection operation: the offending output is
# `split_output` (`type: collection`, `collection_type: list`, emitted only under
# `output_selector: many_contrasts`, which this step does not select). One unselectable
# collection output in the wrapper's declaration is enough to make the whole tool
# uncacheable, so this step reports `skip_tool_not_found` under a green `Concrete: OK`
# and its state is bound BY HAND. It is on the eyeball-review list.
#
# Hand-binding sources, all at the pinned version:
# - raw Tool Shed payload `GET /api/tools/iuc~deseq2~deseq2/versions/2.11.40.8+galaxy4`
# -> 200 (the full input tree, the output list, and `repository_revision`);
# - `gxwf convert --to format2` of the corpus workflow
# transcriptomics/rnaseq-de/rnaseq-de-filtering-plotting at fe41a79, step
# `Differential Analysis`, whose tool_state this block follows key for key;
# - the wrapper's own `deseq2.R` and `deseq2.xml` at @TOOL_VERSION@ 2.11.40.8 /
# @VERSION_SUFFIX@ 4 -- i.e. exactly this version -- for the output-header question.
#
# VERSION. Identity was template-pinned; the version is resolved against the Tool Shed:
# `2.11.40.8+galaxy4` is numeric revision 35, changeset `05f9e54d7e81`, the NEWEST
# published revision of iuc/deseq2 (revisions 28-35 enumerated via
# `/api/repositories/1f158f7565dc70f9/metadata?downloadable_only=true`), and it is what
# the one corpus workflow that uses DESeq2 pins. The corpus is unanimous but n = 1
# workflow, so the shed check is what carries the pin, not the corpus count. Per this
# draft's convention: no `tool_shed_repository` block and no version suffix on `tool_id`.
#
# `deseq_out` IS A SINGLE DATASET -- the plan's arity caveat is now discharged from the
# wrapper itself, not inferred from what the corpus feeds it: the payload lists
# `deseq_out` as `type: data, format: tabular`. The two-node decision stands.
#
# `__current_case__: 1` FOR `select_data.how: datasets_per_level` -- and this is exactly
# the trap convention 8 warns about. The `how` select lists its options
# `datasets_per_level, group_tags, sample_sheet_contrasts`, which would suggest case 0.
# The `<when>` blocks are ordered `group_tags, datasets_per_level,
# sample_sheet_contrasts`, so the correct index is 1 -- read off `<when>` order, then
# corroborated by the corpus round-trip, which also emits 1.
#
# LEVEL ORDER IS THE SIGN OF log2FC. The wrapper help states it outright: "Output log2
# fold changes are based on primary factor level 1 vs. factor level 2". Index 0 is the
# CHANGED level (the first contrast level, tnSWI1) and index 1 the BASE level (the
# reference, AR0382) -- the corpus exemplar's MainFactor / BaseFactor ordering. Swapped,
# SCF1 comes out up-regulated: a silent inversion of the paper's finding, not an error.
#
# FACTOR LEVEL NAMES ARE CONNECTED, NOT BAKED. The corpus writes the literals
# `MainFactor` / `BaseFactor`; this step instead wires `factorLevel` at both indices to
# the workflow's own level parameters, which keeps the closure of resolved entry
# `deseq2-factor-level-names-not-parameterized` ("no level literal is baked into any
# step") true of this step too, and makes the diagnostic plots' contrast titles track the
# parameters. The key shape -- a text param connected inside two nested repeats -- is
# corpus-attested: 75 doubly-nested `in:` keys at fe41a79, the closest being
# `conditions_0|filters_0|bam_property|bam_property_value` on devteam/bamtools_filter in
# Scaffolding-HiC-VGP8, which serializes exactly as here: `__index__` on each repeat
# entry and `{__class__: ConnectedValue}` at the leaf. `factorName` is NOT a level
# literal, so it is baked as `condition`, matching the sample sheet's column name.
#
# `header: true` is baked, not exposed. featureCounts `format: tabdel_short` writes a
# header line; the corpus exposes this as a boolean workflow input precisely because STAR
# count files do not have one, but featureCounts is the only count producer this workflow
# has. If that ever changes, this becomes a real input.
#
# `batch_factors: {__class__: RuntimeValue}` -- no batch factor: n = 2 per level, one
# factor, no blocking variable. RuntimeValue rather than `null` is both the corpus form
# for this exact parameter and this draft's settled convention for an unset optional data
# param (see `Assess raw read quality`, which leaves three of them that way).
#
# `output_selector: [pdf, normCounts]` -- both promoted, and both CONTRAST-SCOPED: this
# job sees only its own four samples, so its size factors, normalized values and
# PCA/dispersion plots are computed over that subset. That is why resolved entry
# `deseq2-normalized-counts-and-plots-promoted-per-contrast` promotes four outputs where
# the interface declared two. `alpha_ma` is connected to the adjusted-p threshold but
# only shades the MA plot; it does NOT filter the result table. The real cut is the
# Filter1 chain downstream, which is why the same parameter feeds both.
#
# COLUMN CONTRACT, and what it depends on. `get_result_output_columns()` in deseq2.R
# returns `geneID, baseMean, log2FoldChange, lfcSE, stat, pvalue, padj` -- c1..c7, so the
# downstream filters' c3 = log2FC and c7 = padj hold -- but ONLY while
# `lfc_shrinkage_type` is `none`. Under any shrinkage the `stat` column is dropped and
# padj becomes c6. `lfc_shrinkage_type: none` is therefore load-bearing for the four
# significance filters, not a default written out for completeness.
#
# HEADER QUESTION SETTLED: `deseq_out` HAS NO HEADER ROW, so the four significance
# filters bind `header_lines: '0'`. Closes `deseq2-result-table-header-presence-unverified`
# on three independent lines of evidence at the pinned version:
# 1. deseq2.R writes the result table with
# `write.table(out_df, file = opt$outfile, sep = "\t", quote = FALSE,
# row.names = FALSE, col.names = FALSE)` -- col.names FALSE. By contrast the
# normalized-counts writer one screen up uses `col.names = NA`, so `counts_out` DOES
# carry a header. Same tool, opposite answers for the two outputs.
# 2. deseq2.xml's tests assert `deseq_out` content as
# `FBgn0003360\t1933.9504...\t-2.8399...` with `has_n_lines n="3999"` -- a data row
# first and no header assertion, while the `vst_out` and `counts_out` assertions in
# the same test DO assert a sample-name header line.
# 3. The corpus chain explains itself: rnaseq-de MANUFACTURES the header it later skips
# -- `tp_text_file_with_recurring_lines` emits one line
# `GeneID__tc__Base mean__tc__log2(FC)...`, `tp_sed_tool` turns `__tc__` into tabs,
# and `tp_cat` (labelled `Annotate DESeq2 table`) concatenates it on top of the
# deg_annotate output. Only THEN do its two Filter1 steps bind `header_lines: "1"`.
# Its "1" is about the manufactured header, not about deseq_out. This workflow omits
# the annotate/cat pair and filters `deseq_out` directly, so it takes '0'.
- id: "Differential expression: first contrast vs reference"
label: "Differential expression: first contrast vs reference"
tool_id: toolshed.g2.bx.psu.edu/repos/iuc/deseq2/deseq2
tool_version: 2.11.40.8+galaxy4
doc: >-
Tier: Resolved. THE WORKFLOW'S ONLY REDUCTION: each factor-level port consumes a whole
2-element counts collection through the wrapper's multiple=true data input, so the
per-sample axis collapses here. One job, no map-over. Paper contrast: tnSWI1 vs
AR0382, taken as level-1-vs-level-2 so log2FC is signed tnSWI1-relative. Sees only its
own four samples, which is why its normalized counts and plots are promoted under
contrast-specific labels rather than as the workflow's.
in:
select_data|rep_factorName_0|rep_factorLevel_0|factorLevel: First contrast condition level
select_data|rep_factorName_0|rep_factorLevel_0|countsFile: Counts for the first contrast level/output_filtered
select_data|rep_factorName_0|rep_factorLevel_1|factorLevel: Reference condition level
select_data|rep_factorName_0|rep_factorLevel_1|countsFile: Counts for the reference level/output_filtered
output_options|alpha_ma: Adjusted p-value threshold
out:
- id: deseq_out
- id: counts_out
- id: plots
tool_state:
advanced_options:
esf_cond:
esf: ""
__current_case__: 0
fit_type: "1"
outlier_replace_off: false
outlier_filter_off: false
auto_mean_filter_off: false
prefilter_conditional:
prefilter: ""
__current_case__: 1
use_beta_priors: false
lfc_shrinkage_type: none
batch_factors:
__class__: RuntimeValue
header: true
output_options:
output_selector:
- pdf
- normCounts
alpha_ma:
__class__: ConnectedValue
select_data:
how: datasets_per_level
__current_case__: 1
rep_factorName:
- __index__: 0
factorName: condition
rep_factorLevel:
- __index__: 0
factorLevel:
__class__: ConnectedValue
countsFile:
__class__: ConnectedValue
- __index__: 1
factorLevel:
__class__: ConnectedValue
countsFile:
__class__: ConnectedValue
tximport:
tximport_selector: count
__current_case__: 1
# RESOLVED. Same wrapper, same pin, same state as the first contrast: the pin was taken
# from the sibling step rather than re-discovered
# (toolshed.g2.bx.psu.edu/repos/iuc/deseq2/deseq2 @ 2.11.40.8+galaxy4, changeset
# 05f9e54d7e81, confirmed in the block above `Differential expression: first contrast vs
# reference`). Read that block for the full derivation: the `__current_case__` traps, the
# level-order/sign argument, why the factor levels are connected rather than baked, and
# the column contract. Only what is SPECIFIC to this node is recorded here.
#
# THIS NODE IS KEY-FOR-KEY IDENTICAL TO THE FIRST CONTRAST except two bindings:
# `rep_factorLevel_0|countsFile` takes `Counts for the second contrast level` and
# `rep_factorLevel_0|factorLevel` takes `Second contrast condition level`. Index 1 --
# both the reference counts collection and `Reference condition level` -- is shared with
# the first contrast verbatim. Every other key, value and case index is byte-identical.
#
# LEVEL ORDER, CHECKED BY EYE. The wrapper help is explicit: "Output log2 fold changes
# are based on primary factor level 1 vs. factor level2 ... DESeq2 computes fold changes
# of 'Treated' samples against 'Untreated'". Level 1 is repeat index 0. This contrast is
# AR0387 vs AR0382, so index 0 MUST be `Second contrast condition level` (AR0387) and
# index 1 `Reference condition level` (AR0382). Swapping them inverts the sign of every
# log2FC in the table with no error and no warning -- SCF1 would come out up-regulated,
# the exact opposite of the paper's ~29-fold down. A path-resolution check cannot catch
# this: both indices resolve to real keys either way. It was verified by reading.
#
# `lfc_shrinkage_type: none` IS LOAD-BEARING HERE TOO, and nothing in the draft enforces
# it. Under `none` the result table has 7 columns, so `c3` = log2FoldChange and `c7` =
# padj -- which is what `Build adjusted p-value predicate` ("c7<") and `Build log2
# fold-change predicate` ("abs(c3)>") already bake as literals, in DIFFERENT steps. Any
# other shrinkage drops the `stat` column, padj moves to c6, and "c7<" then names a
# column that does not exist. The coupling spans three steps and no gate checks it.
#
# NOT VALIDATED BY ANY GATE. Like the first contrast, this step reports
# `skip_tool_not_found`: `galaxy-tool-cache add ... --tool-version 2.11.40.8+galaxy4`
# fails with `["outputs"][0]["structure"] is missing`, because output 0 is `split_output`
# -- a `type: collection` output serialized FLAT by the shed, under the `many_contrasts`
# branch this workflow does not select. The unselected branch still makes the whole
# wrapper undecodable. Every `in:` key, `out:` id and `tool_state` leaf path below was
# resolved against the wrapper's own parsed input tree from the raw TRS payload
# (`GET /api/tools/.../versions/2.11.40.8+galaxy4`), and every `__current_case__`
# re-derived from `<when>` order. This step is on the eyeball-review list.
- id: "Differential expression: second contrast vs reference"
label: "Differential expression: second contrast vs reference"
tool_id: toolshed.g2.bx.psu.edu/repos/iuc/deseq2/deseq2
tool_version: 2.11.40.8+galaxy4
doc: >-
Tier: Resolved. REDUCTION, as for the first contrast: each factor-level port consumes
a whole 2-element counts collection through the wrapper's multiple=true data input,
so the per-sample axis collapses here. One job, no map-over. Paper contrast: AR0387
vs AR0382, taken as level-1-vs-level-2 so log2FC is signed AR0387-relative. Sees only
its own four samples -- the two AR0387 replicates and the two AR0382 replicates --
which is why its normalized counts and plots are promoted under contrast-specific
labels rather than as the workflow's. Its normalized counts are NOT interchangeable
with the first contrast's: size factors are estimated over a different sample set.
in:
select_data|rep_factorName_0|rep_factorLevel_0|factorLevel: Second contrast condition level
select_data|rep_factorName_0|rep_factorLevel_0|countsFile: Counts for the second contrast level/output_filtered
select_data|rep_factorName_0|rep_factorLevel_1|factorLevel: Reference condition level
select_data|rep_factorName_0|rep_factorLevel_1|countsFile: Counts for the reference level/output_filtered
output_options|alpha_ma: Adjusted p-value threshold
out:
- id: deseq_out
- id: counts_out
- id: plots
tool_state:
advanced_options:
esf_cond:
esf: ""
__current_case__: 0
fit_type: "1"
outlier_replace_off: false
outlier_filter_off: false
auto_mean_filter_off: false
prefilter_conditional:
prefilter: ""
__current_case__: 1
use_beta_priors: false
lfc_shrinkage_type: none
batch_factors:
__class__: RuntimeValue
header: true
output_options:
output_selector:
- pdf
- normCounts
alpha_ma:
__class__: ConnectedValue
select_data:
how: datasets_per_level
__current_case__: 1
rep_factorName:
- __index__: 0
factorName: condition
rep_factorLevel:
- __index__: 0
factorLevel:
__class__: ConnectedValue
countsFile:
__class__: ConnectedValue
- __index__: 1
factorLevel:
__class__: ConnectedValue
countsFile:
__class__: ConnectedValue
tximport:
tximport_selector: count
__current_case__: 1
# =========================================================================
# REGION 5 -- SIGNIFICANCE FILTERING
#
# Two thresholds, two chained Filter1 steps per contrast, and one compose_text_param
# bridge per threshold SHARED across both contrasts (the predicate text is identical, so
# the built dataset feeds both chains). Not one compound predicate: the corpus chains
# them, and each intermediate is separately inspectable.
# =========================================================================
# RESOLVED. Identity was corpus-pinned; the version is resolved against the Tool Shed:
# 0.1.1 (changeset e188c9826e0f) is the newest published version and is exactly what
# rnaseq-de at fe41a79 pins for this state. The second component is the FLOAT case, not
# the text case -- the threshold parameter is a float and the wrapper's conditional
# branches on it; `component_value` there is the ConnectedValue placeholder the `in:`
# connection fills.
#
# NOTE ON THE STATE KEY: this step writes `tool_state:`, not the `state:` used by
# `Flatten reads for per-fastq QC`. The two are semantically identical, but gxwf 1.10.1
# drops `{__class__: ConnectedValue}` from a `state:` block before tool-state validation,
# which then fails the wrapper's float branch on a missing required `component_value`.
# Under `tool_state:` the same bytes validate, and `tool_state:` is what `gxwf convert`
# itself emits for format2. This bites any step that connects into a non-text parameter
# case -- here, this bridge and `Build log2 fold-change predicate`. The three row
# predicates connect a string into the text case, whose schema branch has no required
# fields, so they validate either way; write them as `tool_state:` too, for one
# convention rather than two.
- id: Build adjusted p-value predicate
label: Build adjusted p-value predicate
tool_id: toolshed.g2.bx.psu.edu/repos/iuc/compose_text_param/compose_text_param
tool_version: 0.1.1
doc: >-
Tier: Resolved. Single job. Composes the Filter1 predicate `c7<<threshold>` (e.g.
`c7<0.05`) and feeds the `cond` port of BOTH contrasts' p-adj filter steps; the
predicate text is identical for the two chains, so one job serves both. c7 is the
adjusted p-value column of the raw DESeq2 result table.
in:
components_1|param_type|component_value: Adjusted p-value threshold
out:
- id: out1
hide: true
tool_state:
components:
- param_type:
select_param_type: text
component_value: "c7<"
- param_type:
select_param_type: float
component_value:
__class__: ConnectedValue
# RESOLVED. Same wrapper and version as `Build adjusted p-value predicate`: 0.1.1
# (changeset e188c9826e0f), the newest published version and what rnaseq-de at fe41a79
# pins for this exact state. Second component is the FLOAT case -- so `tool_state:`, not
# `state:`, for the reason given on the p-adj bridge above: gxwf 1.10.1 drops
# `{__class__: ConnectedValue}` from a `state:` block and then fails the float branch on
# a missing required `component_value`.
- id: Build log2 fold-change predicate
label: Build log2 fold-change predicate
tool_id: toolshed.g2.bx.psu.edu/repos/iuc/compose_text_param/compose_text_param
tool_version: 0.1.1
doc: >-
Tier: Resolved. Single job. Composes the Filter1 predicate `abs(c3)><threshold>`
(e.g. `abs(c3)>1.0`) and feeds the `cond` port of BOTH contrasts' log2FC filter
steps; the predicate text is identical for the two chains, so one job serves both.
c3 is the log2FoldChange column of the raw DESeq2 result table, and the comparison is
against that RAW column -- the threshold parameter is already in log2 units, so there
is no log2() call and no conversion step anywhere in this workflow.
in:
components_1|param_type|component_value: log2 fold change threshold
out:
- id: out1
hide: true
tool_state:
components:
- param_type:
select_param_type: text
component_value: "abs(c3)>"
- param_type:
select_param_type: float
component_value:
__class__: ConnectedValue
# RESOLVED -- and this block is the CANONICAL note for ALL FOUR significance filters
# (`Filter with p-adj threshold` / `Filter with log2 FC threshold`, each contrast). The
# other three reference it rather than repeat it.
#
# WRAPPER: bare stock `Filter1`, NO `tool_shed_repository` block, NO version suffix on
# the id, `tool_version: 1.1.1` -- the same pin the three sample selectors in the
# condition-split region carry (35 corpus occurrences across 16 IWC workflows at fe41a79,
# every one 1.1.1). Already in the shared cache, so no re-resolution was needed here.
# PORTS `cond` / `input` -> `out_file1`, confirmed from the wrapper summary. Its single
# output is a plain `data` output, so this step's state IS checked for real by
# draft-validate --concrete.
#
# Both connected ports carry `{__class__: ConnectedValue}` under `tool_state:` -- `cond`
# is `optional: false` with an `empty_field` validator, and gxwf 1.10.1 drops the marker
# from a `state:` block. The predicate arrives CONNECTED from `Build adjusted p-value
# predicate` ("c7<" + the threshold float); never inline a literal predicate here.
#
# `header_lines: '0'` -- A STRING, and the ZERO IS THE POINT.
# - String, not integer: the wrapper declares `gx_integer` (default 0), but all 21
# corpus `Filter1` states at fe41a79 serialize it as a string and none as an integer.
# - Zero, because `deseq_out` CARRIES NO HEADER at the pinned DESeq2 version
# 2.11.40.8+galaxy4. `deseq2.R` writes the result table with `col.names = FALSE` at
# both call sites, and `deseq2.xml`'s own tests assert `deseq_out` content starting
# from a DATA row while asserting a sample-name header for `vst_out` / `counts_out`.
# One tool, opposite answers for its two tabular outputs -- which is why this could
# not be settled by analogy. (The DESeq2 `header: true` binding is the INPUT side --
# featureCounts `tabdel_short` writes a header DESeq2 must skip -- and says nothing
# about the output side.) Closes open-requirements
# `deseq2-result-table-header-presence-unverified`.
# - The corpus `'1'` IS A TRAP, not an answer: rnaseq-de-filtering-plotting
# MANUFACTURES the header it then skips (tp_text_file_with_recurring_lines -> tp_sed
# -> tp_cat, "Annotate DESeq2 table") and only then filters. This workflow omits that
# pair and filters `deseq_out` directly. Binding '1' would silently discard row 1 of
# a table sorted by padj -- the most significant gene, here *SCF1* itself, the
# paper's entire finding. No error, no warning, just a missing top hit.
#
# COLUMN INDICES: `c7<` (padj) is correct only because BOTH DESeq2 nodes pin
# `advanced_options.lfc_shrinkage_type: none` -> 7-column output (c3 = log2FoldChange,
# c7 = padj). Under any other shrinkage the `stat` column is dropped and padj moves to
# c6. Nothing in the draft or any gate enforces that coupling across the steps; both
# nodes were re-checked by hand here and carry `none`.
- id: "Filter with p-adj threshold: first contrast"
label: "Filter with p-adj threshold: first contrast"
tool_id: Filter1
tool_version: 1.1.1
doc: >-
Tier: Resolved. Single job, no map-over. First link of the first contrast's chain:
keeps the rows of the raw DESeq2 result table whose adjusted p-value clears the
threshold, via the `c7<<threshold>` predicate built upstream. Hidden; the chain's
endpoint (the log2FC link) is what gets promoted.
in:
cond: Build adjusted p-value predicate/out1
input: "Differential expression: first contrast vs reference/deseq_out"
out:
- id: out_file1
hide: true
tool_state:
cond:
__class__: ConnectedValue
input:
__class__: ConnectedValue
header_lines: '0'
# RESOLVED. Third of the four significance filters, and the FIRST CHAIN ENDPOINT. The
# shared wrapper pin, port names, `tool_state:` shape and the `header_lines: '0'`
# reasoning are ALL in the canonical block above `Filter with p-adj threshold: first
# contrast` -- read it there, it is not repeated.
#
# What differs from the two p-adj links: `input` chains off the p-adj filter's
# `out_file1` rather than off `deseq_out` (two chained Filter1 steps, not one compound
# predicate -- `Filter1` takes a single text predicate), `cond` comes from the log2FC
# bridge ("abs(c3)>" + threshold), and `out_file1` is PROMOTED rather than hidden: no
# `hide:`, and a `rename:`. The rename string is deliberately identical on both chain
# endpoints -- the contrast-specific distinction lives in the workflow `outputs:` label
# (`Significant genes: tnSWI1 vs AR0382`), not in the rename.
#
# This step's output IS the paper's headline result: the significant-gene table in which
# *SCF1* (B9J08_001458) should be the top down-regulated row. The table is sorted by padj,
# so `header_lines: '0'` is what keeps row 1 -- the most significant gene -- in it.
- id: "Filter with log2 FC threshold: first contrast"
label: "Filter with log2 FC threshold: first contrast"
tool_id: Filter1
tool_version: 1.1.1
doc: >-
Tier: Resolved. Single job, no map-over. Second link of the first contrast's chain and
its endpoint: keeps the rows of the p-adj-filtered table whose |log2 fold change|
clears the threshold, via the `abs(c3)><threshold>` predicate built upstream. Promoted
as `Significant genes: tnSWI1 vs AR0382`.
in:
cond: Build log2 fold-change predicate/out1
input: "Filter with p-adj threshold: first contrast/out_file1"
out:
- id: out_file1
rename: Genes filtered with adj p-value and log2(FC) thresholds
tool_state:
cond:
__class__: ConnectedValue
input:
__class__: ConnectedValue
header_lines: '0'
# RESOLVED. Second of the four significance filters; the shared wrapper pin, port names,
# `tool_state:` shape and the `header_lines: '0'` reasoning are ALL in the canonical block
# above `Filter with p-adj threshold: first contrast` -- read it there, it is not repeated.
# Only `in:` and `out:` differ: this link reads the SECOND contrast's `deseq_out`, and like
# its first-contrast twin it is link 1 of 2, so `out_file1` is hidden rather than promoted.
# The chain endpoint (the log2FC link below) carries the contrast-specific rename.
- id: "Filter with p-adj threshold: second contrast"
label: "Filter with p-adj threshold: second contrast"
tool_id: Filter1
tool_version: 1.1.1
doc: >-
Tier: Resolved. Single job, no map-over. First link of the second contrast's chain:
keeps the rows of the raw DESeq2 result table whose adjusted p-value clears the
threshold, via the `c7<<threshold>` predicate built upstream. Hidden; the chain's
endpoint (the log2FC link) is what gets promoted.
in:
cond: Build adjusted p-value predicate/out1
input: "Differential expression: second contrast vs reference/deseq_out"
out:
- id: out_file1
hide: true
tool_state:
cond:
__class__: ConnectedValue
input:
__class__: ConnectedValue
header_lines: '0'
# RESOLVED. Fourth of the four significance filters, and the SECOND CHAIN ENDPOINT. The
# shared wrapper pin, port names, `tool_state:` shape and the `header_lines: '0'`
# reasoning are ALL in the canonical block above `Filter with p-adj threshold: first
# contrast` -- read it there, it is not repeated.
#
# Structurally identical to `Filter with log2 FC threshold: first contrast`: `cond` comes
# from the log2FC bridge ("abs(c3)>" + threshold), `input` chains off THIS contrast's
# p-adj filter rather than off `deseq_out`, and `out_file1` is PROMOTED -- no `hide:` and a
# `rename:` whose string is deliberately IDENTICAL to the other endpoint's. The contrast
# distinction lives in the workflow `outputs:` label (`Significant genes: AR0387 vs
# AR0382`), not in the rename.
#
# From the plan (was `_plan_out`): the paper expects *SCF1* (B9J08_001458) as the most
# down-regulated gene in this contrast, ~29-fold. The table is sorted by padj, so
# `header_lines: '0'` is what keeps row 1 -- that top hit -- in the result.
- id: "Filter with log2 FC threshold: second contrast"
label: "Filter with log2 FC threshold: second contrast"
tool_id: Filter1
tool_version: 1.1.1
doc: >-
Tier: Resolved. Single job, no map-over. Second link of the second contrast's chain and
its endpoint: keeps the rows of the p-adj-filtered table whose |log2 fold change|
clears the threshold, via the `abs(c3)><threshold>` predicate built upstream. Promoted
as `Significant genes: AR0387 vs AR0382`.
in:
cond: Build log2 fold-change predicate/out1
input: "Filter with p-adj threshold: second contrast/out_file1"
out:
- id: out_file1
rename: Genes filtered with adj p-value and log2(FC) thresholds
tool_state:
cond:
__class__: ConnectedValue
input:
__class__: ConnectedValue
header_lines: '0'
comments:
- type: frame
position: [0, 0]
size: [520, 300]
color: blue
title: Raw-read quality control
contains_steps:
- Flatten reads for per-fastq QC
- Read quality report
- type: frame
position: [560, 0]
size: [560, 520]
color: green
title: Trim, align and count (map-over, one job per sample)
contains_steps:
- Quality-trim reads
- Splice-aware alignment
- Count reads per gene
- type: frame
position: [560, 560]
size: [400, 180]
color: yellow
title: Map strandedness parameter
contains_steps:
- Get featureCounts strandedness parameter
- type: frame
position: [0, 360]
size: [520, 900]
color: red
title: Condition split from the sample sheet (no corpus precedent)
contains_steps:
- Project sample sheet to tabular
- Build reference-level row predicate
- Select reference-level samples
- Reference-level identifier list
- Counts for the reference level
- Build first-contrast row predicate
- Select first-contrast samples
- First-contrast identifier list
- Counts for the first contrast level
- Build second-contrast row predicate
- Select second-contrast samples
- Second-contrast identifier list
- Counts for the second contrast level
- type: markdown
position: [0, 1280]
size: [520, 220]
color: red
text: |
**Delimited region — the phase-2 alternative is one deletion away.**
Nothing in IWC at `fe41a79` uses a `sample_sheet` collection input or
`__SAMPLE_SHEET_TO_TABULAR__`, so this region has no worked precedent. To take the
recorded alternative: delete `Project sample sheet to tabular`, change the reads
input to `list:paired` (dropping `column_definitions`), add a `data` input
`Sample metadata table` (sample_id, condition, replicate), and repoint the three
`Select ...-level samples` steps at it. Every other step here, and the rest of the
workflow, is unchanged.
- type: frame
position: [1160, 0]
size: [460, 420]
color: turquoise
title: Differential expression (two contrasts, two jobs)
contains_steps:
- "Differential expression: first contrast vs reference"
- "Differential expression: second contrast vs reference"
- type: frame
position: [1160, 460]
size: [460, 200]
color: yellow
title: Build significance filter predicates
contains_steps:
- Build adjusted p-value predicate
- Build log2 fold-change predicate
- type: frame
position: [1160, 700]
size: [460, 440]
color: orange
title: Significance filtering (padj then log2FC, per contrast)
contains_steps:
- "Filter with p-adj threshold: first contrast"
- "Filter with log2 FC threshold: first contrast"
- "Filter with p-adj threshold: second contrast"
- "Filter with log2 FC threshold: second contrast"
Galaxy workflow test file (tests-format) with job inputs, expected outputs, assertions; passes static schema + label cross-check. Named as the workflow basename + `-tests.yml` so Planemo discovers it as the companion of `galaxy-workflow.gxwf.yml`.
# Galaxy workflow tests for galaxy-workflow.gxwf.yml
#
# Two cases:
# 1. scf1-both-contrasts-200k -- the biological case. Six ENA runs at 200,000 read pairs.
# 2. topology-smoke-25k -- structural smoke test at 25,000 read pairs, no biology.
#
# READ FIXTURES. test-data/*.fastq.gz (200k) and test-data/smoke-25k/*.fastq.gz (25k) are
# deterministic head subsets of the six PRJNA904261 runs, produced by
# curl -L <ENA url> | gzip -dc | head -n 800000 (200k pairs; -n 100000 for 25k pairs)
# and gzipped with `gzip -n`. Each 200k .fastq was verified byte-for-byte against the
# UNCOMPRESSED md5 pinned in test-data-refs.json before compression -- all twelve matched --
# and that md5 is the regeneration check, because gzip output is not byte-stable across gzip
# versions. The 25k set was then derived locally as the first 100,000 lines of each VERIFIED
# 200k .fastq, which is identical to re-streaming the ENA source with `head -n 100000`. The
# SHA-1 values in the `hashes:` blocks below are of the .gz artifacts as produced here and
# are the fetch-integrity check. Both are needed and they check different things. If these
# fixtures are published (Zenodo), replace each `path:` with the published `location:` and
# keep the SHA-1 unchanged -- it pins exactly these bytes.
#
# REFERENCE. Genome FASTA and GTF are pinned NCBI FTP URLs staged with `decompress: true`.
# They carry NO `hashes:` block: test-data-refs.json pins md5s of the COMPRESSED files as
# served, and whether a `hashes:` block on a `decompress: true` input checks the fetched or
# the decompressed bytes was not established. Omitting is the resolution the test plan
# names for that case (unresolved `hash-vs-decompress-ordering`).
#
# SETTLED HERE, FROM THE PINNED WRAPPER SOURCES (both are the versions this workflow pins).
# * featurecounts 2.1.1+galaxy1 builds output_short as `grep -v "^#" output | cut -f 1,7`,
# which strips only the "# Program:" comment and keeps the header row, renamed to the
# element identifier. output_short is therefore 1 header + one row per annotated gene.
# The NCBI GTF for GCA_002759435.2 carries exactly 5586 distinct gene_id values, all of
# them matching B9J08_[0-9]{6} -- hence has_n_lines n: 5587 and the annotation guard.
# * deseq2 2.11.40.8+galaxy4's deseq2.R writes counts_out with `col.names = NA`, so its
# header is padded with a leading blank field: 4 samples -> 5 tab-separated fields on
# line 1. has_n_columns n: 5 on both normalized-counts tables is therefore verified,
# not assumed, and the test plan's `normalized-counts-header-width-unverified` is closed.
# * Both wrappers name count columns by $file.element_identifier, so the normalized-counts
# header really does carry AR0382_A / AR0382_tnSWI1_A and the sample-name assertions
# below are well founded (test plan's `normalized-counts-column-naming-assumed`, closed).
#
# HEADER ASYMMETRY. deseq2.R writes `deseq_out` with col.names = FALSE (NO header, which is
# why the significance filters bind header_lines: '0') and `counts_out` with col.names = NA
# (header present, padded with a leading blank field). The assertions below respect that:
# `not_has_text: baseMean` on the raw result tables, sample-name `has_text` on the
# normalized-counts tables.
#
# NOT ASSERTED, DELIBERATELY. No rank claim (SCF1 is 5th/4th by padj and 2nd/1st by
# |log2FC| in the two contrasts -- no rank-1 assertion is true in both, and tests-format
# cannot express a rank anyway). No exact DESeq2 value, no exact per-gene count, not the
# paper's ~29-fold magnitude, and not the negative ALS / IFF-HYR adhesin claim. See the
# test plan's omissions[] for each refusal.
- doc: >-
End-to-end run over all six deposited runs of PRJNA904261 (SRP409192), each subset to
the first 200,000 read pairs, against the NCBI GCA_002759435.2 (Cand_auris_B8441_V2)
reference and its own GTF. Exercises the whole DAG - flatten side-branch, FastQC,
Cutadapt, RNA STAR with a history reference, featureCounts, the sample-sheet condition
split, both DESeq2 reductions and both two-link significance chains - and asserts the
paper's central result: SCF1 (B9J08_001458) is significantly down-regulated in BOTH
contrasts against the AR0382 reference level. Membership in each significant-genes
table means padj < 0.05 AND |log2FC| > 1 by construction; the extra floor asserted
here is log2FC <= -2. No rank and no measured value is asserted anywhere: the
grounding measurements come from pydeseq2 over bwa-mem counts, not from this
workflow's R DESeq2 over featureCounts -s 2, so only margins survive the difference.
job:
RNA-seq reads (sample sheet):
class: Collection
collection_type: sample_sheet:paired
# rows is a mapping of element identifier -> POSITIONAL LIST of column values,
# ordered to match the input's column_definitions (condition, replicate).
# Not a dict of column-name -> value: validate_row() zips row against
# column_definitions and rejects on length mismatch.
rows:
AR0382_A: [AR0382, A]
AR0382_B: [AR0382, B]
AR0387_A: [AR0387, A]
AR0387_B: [AR0387, B]
AR0382_tnSWI1_A: [tnSWI1, A]
AR0382_tnSWI1_B: [tnSWI1, B]
elements:
- class: Collection
collection_type: paired
identifier: AR0382_A
elements:
- class: File
identifier: forward
path: test-data/AR0382_A_1.fastq.gz
filetype: fastqsanger.gz
hashes:
- hash_function: SHA-1
hash_value: 81847033e7a1bd1430efc68499b08ec2a3de619e
- class: File
identifier: reverse
path: test-data/AR0382_A_2.fastq.gz
filetype: fastqsanger.gz
hashes:
- hash_function: SHA-1
hash_value: bc59eb5c253f42454631792d59405a019c56dcfe
- class: Collection
collection_type: paired
identifier: AR0382_B
elements:
- class: File
identifier: forward
path: test-data/AR0382_B_1.fastq.gz
filetype: fastqsanger.gz
hashes:
- hash_function: SHA-1
hash_value: a11438bf4c886b081b887ca650642bf925014177
- class: File
identifier: reverse
path: test-data/AR0382_B_2.fastq.gz
filetype: fastqsanger.gz
hashes:
- hash_function: SHA-1
hash_value: ab93048b0e16a0729fb0e38ed206cf398ed798df
- class: Collection
collection_type: paired
identifier: AR0387_A
elements:
- class: File
identifier: forward
path: test-data/AR0387_A_1.fastq.gz
filetype: fastqsanger.gz
hashes:
- hash_function: SHA-1
hash_value: 0bcb624b2347148ab740c81ebcb19894449c30b4
- class: File
identifier: reverse
path: test-data/AR0387_A_2.fastq.gz
filetype: fastqsanger.gz
hashes:
- hash_function: SHA-1
hash_value: 4590862aba8082c42557f31cbe62c12d694de4b6
- class: Collection
collection_type: paired
identifier: AR0387_B
elements:
- class: File
identifier: forward
path: test-data/AR0387_B_1.fastq.gz
filetype: fastqsanger.gz
hashes:
- hash_function: SHA-1
hash_value: f0e1a1e97ab993d91baf38ae4ffdd3edf5028905
- class: File
identifier: reverse
path: test-data/AR0387_B_2.fastq.gz
filetype: fastqsanger.gz
hashes:
- hash_function: SHA-1
hash_value: 7ccf13c15a7a0ac4571bd2b06796243050d4a17c
- class: Collection
collection_type: paired
identifier: AR0382_tnSWI1_A
elements:
- class: File
identifier: forward
path: test-data/AR0382_tnSWI1_A_1.fastq.gz
filetype: fastqsanger.gz
hashes:
- hash_function: SHA-1
hash_value: 8c76f5caecf41f7a9e62689f5cbdac2e877c31c7
- class: File
identifier: reverse
path: test-data/AR0382_tnSWI1_A_2.fastq.gz
filetype: fastqsanger.gz
hashes:
- hash_function: SHA-1
hash_value: 8d8d57b28c41d50a6f5dd13074440deeb801d1da
- class: Collection
collection_type: paired
identifier: AR0382_tnSWI1_B
elements:
- class: File
identifier: forward
path: test-data/AR0382_tnSWI1_B_1.fastq.gz
filetype: fastqsanger.gz
hashes:
- hash_function: SHA-1
hash_value: d666ddad269a39d17b736079f15d5fb98d980851
- class: File
identifier: reverse
path: test-data/AR0382_tnSWI1_B_2.fastq.gz
filetype: fastqsanger.gz
hashes:
- hash_function: SHA-1
hash_value: 7a7d6932aed4f46604cd030b04eab9355106e5f9
Reference genome FASTA:
class: File
location: https://ftp.ncbi.nlm.nih.gov/genomes/all/GCA/002/759/435/GCA_002759435.2_Cand_auris_B8441_V2/GCA_002759435.2_Cand_auris_B8441_V2_genomic.fna.gz
filetype: fasta
decompress: true
Gene annotation GTF:
class: File
location: https://ftp.ncbi.nlm.nih.gov/genomes/all/GCA/002/759/435/GCA_002759435.2_Cand_auris_B8441_V2/GCA_002759435.2_Cand_auris_B8441_V2_genomic.gtf.gz
filetype: gtf
decompress: true
Reference condition level: AR0382
First contrast condition level: tnSWI1
Second contrast condition level: AR0387
Strandedness: 'stranded - reverse'
Adjusted p-value threshold: 0.05
log2 fold change threshold: 1.0
outputs:
'FastQC raw reads: text summary':
class: Collection
element_count: 12
element_tests:
AR0382_A_forward:
asserts:
- that: has_text_matching
expression: 'Total Sequences\s+200000\b'
- that: has_text
text: '>>Basic Statistics'
AR0382_A_reverse:
asserts:
- that: has_text_matching
expression: 'Total Sequences\s+200000\b'
- that: has_text
text: '>>Basic Statistics'
AR0382_B_forward:
asserts:
- that: has_text_matching
expression: 'Total Sequences\s+200000\b'
- that: has_text
text: '>>Basic Statistics'
AR0382_B_reverse:
asserts:
- that: has_text_matching
expression: 'Total Sequences\s+200000\b'
- that: has_text
text: '>>Basic Statistics'
AR0387_A_forward:
asserts:
- that: has_text_matching
expression: 'Total Sequences\s+200000\b'
- that: has_text
text: '>>Basic Statistics'
AR0387_A_reverse:
asserts:
- that: has_text_matching
expression: 'Total Sequences\s+200000\b'
- that: has_text
text: '>>Basic Statistics'
AR0387_B_forward:
asserts:
- that: has_text_matching
expression: 'Total Sequences\s+200000\b'
- that: has_text
text: '>>Basic Statistics'
AR0387_B_reverse:
asserts:
- that: has_text_matching
expression: 'Total Sequences\s+200000\b'
- that: has_text
text: '>>Basic Statistics'
AR0382_tnSWI1_A_forward:
asserts:
- that: has_text_matching
expression: 'Total Sequences\s+200000\b'
- that: has_text
text: '>>Basic Statistics'
AR0382_tnSWI1_A_reverse:
asserts:
- that: has_text_matching
expression: 'Total Sequences\s+200000\b'
- that: has_text
text: '>>Basic Statistics'
AR0382_tnSWI1_B_forward:
asserts:
- that: has_text_matching
expression: 'Total Sequences\s+200000\b'
- that: has_text
text: '>>Basic Statistics'
AR0382_tnSWI1_B_reverse:
asserts:
- that: has_text_matching
expression: 'Total Sequences\s+200000\b'
- that: has_text
text: '>>Basic Statistics'
Cutadapt trimming report:
class: Collection
element_count: 6
element_tests:
AR0382_A:
asserts:
- that: has_text_matching
expression: 'Total read pairs processed:\s+200,000'
AR0382_B:
asserts:
- that: has_text_matching
expression: 'Total read pairs processed:\s+200,000'
AR0387_A:
asserts:
- that: has_text_matching
expression: 'Total read pairs processed:\s+200,000'
AR0387_B:
asserts:
- that: has_text_matching
expression: 'Total read pairs processed:\s+200,000'
AR0382_tnSWI1_A:
asserts:
- that: has_text_matching
expression: 'Total read pairs processed:\s+200,000'
AR0382_tnSWI1_B:
asserts:
- that: has_text_matching
expression: 'Total read pairs processed:\s+200,000'
Trimmed reads:
class: Collection
element_count: 6
element_tests:
AR0382_A:
class: Collection
elements:
forward:
asserts:
- that: has_size
min: 1000
reverse:
asserts:
- that: has_size
min: 1000
AR0382_B:
class: Collection
elements:
forward:
asserts:
- that: has_size
min: 1000
reverse:
asserts:
- that: has_size
min: 1000
AR0387_A:
class: Collection
elements:
forward:
asserts:
- that: has_size
min: 1000
reverse:
asserts:
- that: has_size
min: 1000
AR0387_B:
class: Collection
elements:
forward:
asserts:
- that: has_size
min: 1000
reverse:
asserts:
- that: has_size
min: 1000
AR0382_tnSWI1_A:
class: Collection
elements:
forward:
asserts:
- that: has_size
min: 1000
reverse:
asserts:
- that: has_size
min: 1000
AR0382_tnSWI1_B:
class: Collection
elements:
forward:
asserts:
- that: has_size
min: 1000
reverse:
asserts:
- that: has_size
min: 1000
STAR mapping summary:
class: Collection
element_count: 6
element_tests:
AR0382_A:
asserts:
- that: has_text
text: 'Uniquely mapped reads %'
- that: has_line_matching
expression: '\s*Uniquely mapped reads % \|\s+(?:8[0-9]|9[0-9]|100)\.[0-9]+%'
AR0382_B:
asserts:
- that: has_text
text: 'Uniquely mapped reads %'
- that: has_line_matching
expression: '\s*Uniquely mapped reads % \|\s+(?:8[0-9]|9[0-9]|100)\.[0-9]+%'
AR0387_A:
asserts:
- that: has_text
text: 'Uniquely mapped reads %'
- that: has_line_matching
expression: '\s*Uniquely mapped reads % \|\s+(?:8[0-9]|9[0-9]|100)\.[0-9]+%'
AR0387_B:
asserts:
- that: has_text
text: 'Uniquely mapped reads %'
- that: has_line_matching
expression: '\s*Uniquely mapped reads % \|\s+(?:8[0-9]|9[0-9]|100)\.[0-9]+%'
AR0382_tnSWI1_A:
asserts:
- that: has_text
text: 'Uniquely mapped reads %'
- that: has_line_matching
expression: '\s*Uniquely mapped reads % \|\s+(?:8[0-9]|9[0-9]|100)\.[0-9]+%'
AR0382_tnSWI1_B:
asserts:
- that: has_text
text: 'Uniquely mapped reads %'
- that: has_line_matching
expression: '\s*Uniquely mapped reads % \|\s+(?:8[0-9]|9[0-9]|100)\.[0-9]+%'
Gene counts per sample:
class: Collection
element_count: 6
element_tests:
AR0382_A:
asserts:
- that: has_n_columns
n: 2
- that: has_n_lines
n: 5587
delta: 1
- that: has_line_matching
expression: 'B9J08_[0-9]{6}\t[0-9]+'
min: 5000
- that: has_line_matching
expression: 'B9J08_001458\t[0-9]+'
- that: has_line_matching
expression: 'B9J08_001458\t[0-9]{3,}'
AR0382_B:
asserts:
- that: has_n_columns
n: 2
- that: has_n_lines
n: 5587
delta: 1
- that: has_line_matching
expression: 'B9J08_[0-9]{6}\t[0-9]+'
min: 5000
- that: has_line_matching
expression: 'B9J08_001458\t[0-9]+'
- that: has_line_matching
expression: 'B9J08_001458\t[0-9]{3,}'
AR0387_A:
asserts:
- that: has_n_columns
n: 2
- that: has_n_lines
n: 5587
delta: 1
- that: has_line_matching
expression: 'B9J08_[0-9]{6}\t[0-9]+'
min: 5000
- that: has_line_matching
expression: 'B9J08_001458\t[0-9]+'
- that: has_line_matching
expression: 'B9J08_001458\t[0-9]{1,2}'
AR0387_B:
asserts:
- that: has_n_columns
n: 2
- that: has_n_lines
n: 5587
delta: 1
- that: has_line_matching
expression: 'B9J08_[0-9]{6}\t[0-9]+'
min: 5000
- that: has_line_matching
expression: 'B9J08_001458\t[0-9]+'
- that: has_line_matching
expression: 'B9J08_001458\t[0-9]{1,2}'
AR0382_tnSWI1_A:
asserts:
- that: has_n_columns
n: 2
- that: has_n_lines
n: 5587
delta: 1
- that: has_line_matching
expression: 'B9J08_[0-9]{6}\t[0-9]+'
min: 5000
- that: has_line_matching
expression: 'B9J08_001458\t[0-9]+'
- that: has_line_matching
expression: 'B9J08_001458\t[0-9]{1,2}'
AR0382_tnSWI1_B:
asserts:
- that: has_n_columns
n: 2
- that: has_n_lines
n: 5587
delta: 1
- that: has_line_matching
expression: 'B9J08_[0-9]{6}\t[0-9]+'
min: 5000
- that: has_line_matching
expression: 'B9J08_001458\t[0-9]+'
- that: has_line_matching
expression: 'B9J08_001458\t[0-9]{1,2}'
featureCounts assignment summary:
class: Collection
element_count: 6
element_tests:
AR0382_A:
asserts:
- that: has_line_matching
expression: 'Assigned\t1[5-9][0-9]{4}'
- that: has_line_matching
expression: 'Unassigned_NoFeatures\t[0-9]{1,5}'
AR0382_B:
asserts:
- that: has_line_matching
expression: 'Assigned\t1[5-9][0-9]{4}'
- that: has_line_matching
expression: 'Unassigned_NoFeatures\t[0-9]{1,5}'
AR0387_A:
asserts:
- that: has_line_matching
expression: 'Assigned\t1[5-9][0-9]{4}'
- that: has_line_matching
expression: 'Unassigned_NoFeatures\t[0-9]{1,5}'
AR0387_B:
asserts:
- that: has_line_matching
expression: 'Assigned\t1[5-9][0-9]{4}'
- that: has_line_matching
expression: 'Unassigned_NoFeatures\t[0-9]{1,5}'
AR0382_tnSWI1_A:
asserts:
- that: has_line_matching
expression: 'Assigned\t1[5-9][0-9]{4}'
- that: has_line_matching
expression: 'Unassigned_NoFeatures\t[0-9]{1,5}'
AR0382_tnSWI1_B:
asserts:
- that: has_line_matching
expression: 'Assigned\t1[5-9][0-9]{4}'
- that: has_line_matching
expression: 'Unassigned_NoFeatures\t[0-9]{1,5}'
'DESeq2 results: tnSWI1 vs AR0382':
asserts:
- that: has_n_columns
n: 7
- that: not_has_text
text: baseMean
- that: has_line_matching
expression: 'B9J08_001458(\t[^\t]*){6}'
- that: has_line_matching
expression: 'B9J08_001458\t[^\t]+\t-[0-9.]+(\t[^\t]*){4}'
'DESeq2 results: AR0387 vs AR0382':
asserts:
- that: has_n_columns
n: 7
- that: not_has_text
text: baseMean
- that: has_line_matching
expression: 'B9J08_001458(\t[^\t]*){6}'
- that: has_line_matching
expression: 'B9J08_001458\t[^\t]+\t-[0-9.]+(\t[^\t]*){4}'
'DESeq2 normalized counts: tnSWI1 vs AR0382':
asserts:
- that: has_n_columns
n: 5
- that: has_line_matching
expression: 'B9J08_001458(\t[0-9.eE+-]+){4}'
- that: has_text
text: AR0382_A
- that: has_text
text: AR0382_B
- that: has_text
text: AR0382_tnSWI1_A
- that: has_text
text: AR0382_tnSWI1_B
- that: not_has_text
text: AR0387
'DESeq2 normalized counts: AR0387 vs AR0382':
asserts:
- that: has_n_columns
n: 5
- that: has_line_matching
expression: 'B9J08_001458(\t[0-9.eE+-]+){4}'
- that: has_text
text: AR0382_A
- that: has_text
text: AR0382_B
- that: has_text
text: AR0387_A
- that: has_text
text: AR0387_B
- that: not_has_text
text: tnSWI1
'Significant genes: tnSWI1 vs AR0382':
asserts:
- that: has_n_columns
n: 7
- that: has_line_matching
expression: 'B9J08_001458(\t[^\t]*){6}'
- that: has_line_matching
expression: 'B9J08_001458\t[^\t]+\t-(?:[2-9]|[1-9][0-9]+)(?:\.[0-9]+)?(\t[^\t]*){4}'
'Significant genes: AR0387 vs AR0382':
asserts:
- that: has_n_columns
n: 7
- that: has_line_matching
expression: 'B9J08_001458(\t[^\t]*){6}'
- that: has_line_matching
expression: 'B9J08_001458\t[^\t]+\t-(?:[2-9]|[1-9][0-9]+)(?:\.[0-9]+)?(\t[^\t]*){4}'
'DESeq2 diagnostic plots: tnSWI1 vs AR0382':
asserts:
- that: has_size
min: 10000
'DESeq2 diagnostic plots: AR0387 vs AR0382':
asserts:
- that: has_size
min: 10000
- doc: >-
Fast structural smoke test over the same six samples at 25,000 read pairs each
(~7 MB compressed total). Makes NO biological claim: at that depth SCF1 falls to
roughly 50 fragments in AR0382 and near zero elsewhere, and the significance tables
may legitimately be empty. What it does exercise is everything that can silently
break without erroring - the sample_sheet:paired input shape, the flatten
side-branch, the six-element map-over identifier space, both condition-split joins,
both DESeq2 reductions, and both cross-step invariants. Every assertion here is a
subset of case 1's, with the depth constant changed and the biological claims removed.
job:
RNA-seq reads (sample sheet):
class: Collection
collection_type: sample_sheet:paired
# rows is a mapping of element identifier -> POSITIONAL LIST of column values,
# ordered to match the input's column_definitions (condition, replicate).
# Not a dict of column-name -> value: validate_row() zips row against
# column_definitions and rejects on length mismatch.
rows:
AR0382_A: [AR0382, A]
AR0382_B: [AR0382, B]
AR0387_A: [AR0387, A]
AR0387_B: [AR0387, B]
AR0382_tnSWI1_A: [tnSWI1, A]
AR0382_tnSWI1_B: [tnSWI1, B]
elements:
- class: Collection
collection_type: paired
identifier: AR0382_A
elements:
- class: File
identifier: forward
path: test-data/smoke-25k/AR0382_A_1.fastq.gz
filetype: fastqsanger.gz
hashes:
- hash_function: SHA-1
hash_value: 7686fa073a0089b0376bbe1d3b819b4ea14e3a67
- class: File
identifier: reverse
path: test-data/smoke-25k/AR0382_A_2.fastq.gz
filetype: fastqsanger.gz
hashes:
- hash_function: SHA-1
hash_value: f1749c97d907abffd4f5ee36139c22b15376d2af
- class: Collection
collection_type: paired
identifier: AR0382_B
elements:
- class: File
identifier: forward
path: test-data/smoke-25k/AR0382_B_1.fastq.gz
filetype: fastqsanger.gz
hashes:
- hash_function: SHA-1
hash_value: a120941043541873bd674fc8e5cab4bf7d8dcc37
- class: File
identifier: reverse
path: test-data/smoke-25k/AR0382_B_2.fastq.gz
filetype: fastqsanger.gz
hashes:
- hash_function: SHA-1
hash_value: 8f315372a23e305fc5dc5ee04d31d8a771ccb8f3
- class: Collection
collection_type: paired
identifier: AR0387_A
elements:
- class: File
identifier: forward
path: test-data/smoke-25k/AR0387_A_1.fastq.gz
filetype: fastqsanger.gz
hashes:
- hash_function: SHA-1
hash_value: d8e170332a2ae20242b8f79c66f287b79fa2c56a
- class: File
identifier: reverse
path: test-data/smoke-25k/AR0387_A_2.fastq.gz
filetype: fastqsanger.gz
hashes:
- hash_function: SHA-1
hash_value: 89a32f456f670a00719362b8d72a1a90110c7e94
- class: Collection
collection_type: paired
identifier: AR0387_B
elements:
- class: File
identifier: forward
path: test-data/smoke-25k/AR0387_B_1.fastq.gz
filetype: fastqsanger.gz
hashes:
- hash_function: SHA-1
hash_value: 022d0851c4b94b919c62d6c1af64e3713bb8e138
- class: File
identifier: reverse
path: test-data/smoke-25k/AR0387_B_2.fastq.gz
filetype: fastqsanger.gz
hashes:
- hash_function: SHA-1
hash_value: 28d347eddac1455d3df000846cffb4dc0a472c4d
- class: Collection
collection_type: paired
identifier: AR0382_tnSWI1_A
elements:
- class: File
identifier: forward
path: test-data/smoke-25k/AR0382_tnSWI1_A_1.fastq.gz
filetype: fastqsanger.gz
hashes:
- hash_function: SHA-1
hash_value: 3b6109b5392bba5cb75f312fd7c078b67346013c
- class: File
identifier: reverse
path: test-data/smoke-25k/AR0382_tnSWI1_A_2.fastq.gz
filetype: fastqsanger.gz
hashes:
- hash_function: SHA-1
hash_value: e5599b61d6ec223f9cd88e7eb7e7877c6d7cf29c
- class: Collection
collection_type: paired
identifier: AR0382_tnSWI1_B
elements:
- class: File
identifier: forward
path: test-data/smoke-25k/AR0382_tnSWI1_B_1.fastq.gz
filetype: fastqsanger.gz
hashes:
- hash_function: SHA-1
hash_value: f803a1ee765bd35a575ab66f80133ad09e50a4ea
- class: File
identifier: reverse
path: test-data/smoke-25k/AR0382_tnSWI1_B_2.fastq.gz
filetype: fastqsanger.gz
hashes:
- hash_function: SHA-1
hash_value: 97a623f6a006e90740a44d6391685ede0157acf4
Reference genome FASTA:
class: File
location: https://ftp.ncbi.nlm.nih.gov/genomes/all/GCA/002/759/435/GCA_002759435.2_Cand_auris_B8441_V2/GCA_002759435.2_Cand_auris_B8441_V2_genomic.fna.gz
filetype: fasta
decompress: true
Gene annotation GTF:
class: File
location: https://ftp.ncbi.nlm.nih.gov/genomes/all/GCA/002/759/435/GCA_002759435.2_Cand_auris_B8441_V2/GCA_002759435.2_Cand_auris_B8441_V2_genomic.gtf.gz
filetype: gtf
decompress: true
Reference condition level: AR0382
First contrast condition level: tnSWI1
Second contrast condition level: AR0387
Strandedness: 'stranded - reverse'
Adjusted p-value threshold: 0.05
log2 fold change threshold: 1.0
outputs:
'FastQC raw reads: text summary':
class: Collection
element_count: 12
element_tests:
AR0382_A_forward:
asserts:
- that: has_text_matching
expression: 'Total Sequences\s+25000\b'
AR0382_A_reverse:
asserts:
- that: has_text_matching
expression: 'Total Sequences\s+25000\b'
AR0382_B_forward:
asserts:
- that: has_text_matching
expression: 'Total Sequences\s+25000\b'
AR0382_B_reverse:
asserts:
- that: has_text_matching
expression: 'Total Sequences\s+25000\b'
AR0387_A_forward:
asserts:
- that: has_text_matching
expression: 'Total Sequences\s+25000\b'
AR0387_A_reverse:
asserts:
- that: has_text_matching
expression: 'Total Sequences\s+25000\b'
AR0387_B_forward:
asserts:
- that: has_text_matching
expression: 'Total Sequences\s+25000\b'
AR0387_B_reverse:
asserts:
- that: has_text_matching
expression: 'Total Sequences\s+25000\b'
AR0382_tnSWI1_A_forward:
asserts:
- that: has_text_matching
expression: 'Total Sequences\s+25000\b'
AR0382_tnSWI1_A_reverse:
asserts:
- that: has_text_matching
expression: 'Total Sequences\s+25000\b'
AR0382_tnSWI1_B_forward:
asserts:
- that: has_text_matching
expression: 'Total Sequences\s+25000\b'
AR0382_tnSWI1_B_reverse:
asserts:
- that: has_text_matching
expression: 'Total Sequences\s+25000\b'
Gene counts per sample:
class: Collection
element_count: 6
element_tests:
AR0382_A:
asserts:
- that: has_n_columns
n: 2
- that: has_n_lines
n: 5587
delta: 1
- that: has_line_matching
expression: 'B9J08_[0-9]{6}\t[0-9]+'
min: 5000
AR0382_B:
asserts:
- that: has_n_columns
n: 2
- that: has_n_lines
n: 5587
delta: 1
- that: has_line_matching
expression: 'B9J08_[0-9]{6}\t[0-9]+'
min: 5000
AR0387_A:
asserts:
- that: has_n_columns
n: 2
- that: has_n_lines
n: 5587
delta: 1
- that: has_line_matching
expression: 'B9J08_[0-9]{6}\t[0-9]+'
min: 5000
AR0387_B:
asserts:
- that: has_n_columns
n: 2
- that: has_n_lines
n: 5587
delta: 1
- that: has_line_matching
expression: 'B9J08_[0-9]{6}\t[0-9]+'
min: 5000
AR0382_tnSWI1_A:
asserts:
- that: has_n_columns
n: 2
- that: has_n_lines
n: 5587
delta: 1
- that: has_line_matching
expression: 'B9J08_[0-9]{6}\t[0-9]+'
min: 5000
AR0382_tnSWI1_B:
asserts:
- that: has_n_columns
n: 2
- that: has_n_lines
n: 5587
delta: 1
- that: has_line_matching
expression: 'B9J08_[0-9]{6}\t[0-9]+'
min: 5000
'DESeq2 results: tnSWI1 vs AR0382':
asserts:
- that: has_n_columns
n: 7
- that: not_has_text
text: baseMean
'DESeq2 results: AR0387 vs AR0382':
asserts:
- that: has_n_columns
n: 7
- that: not_has_text
text: baseMean
'DESeq2 normalized counts: tnSWI1 vs AR0382':
asserts:
- that: has_n_columns
n: 5
- that: has_text
text: AR0382_A
- that: has_text
text: AR0382_B
- that: has_text
text: AR0382_tnSWI1_A
- that: has_text
text: AR0382_tnSWI1_B
- that: not_has_text
text: AR0387
'DESeq2 normalized counts: AR0387 vs AR0382':
asserts:
- that: has_n_columns
n: 5
- that: has_text
text: AR0382_A
- that: has_text
text: AR0382_B
- that: has_text
text: AR0387_A
- that: has_text
text: AR0387_B
- that: not_has_text
text: tnSWI1
Terminal gxwf validation handoff: the exact command run, a pass/fail/not-run status, the classified workflow-level diagnostics, and the residual runtime risks static validation cannot settle.
{
"artifact": "galaxy-workflow-validation-result",
"phase": 10,
"skill": "validate-galaxy-workflow",
"workflow_path": "galaxy-workflow.gxwf.yml",
"validated_at": "2026-09-18",
"tool": {
"name": "gxwf",
"package": "@galaxy-tool-util/cli",
"package_version": "1.12.0",
"self_reported_version": "1.0.0",
"self_reported_version_note": "`gxwf --version` reports a stale 1.0.0; the installed package is 1.12.0. Still open as feedback `gxwf-version-flag-reports-stale-version` — re-confirmed on 1.12.0."
},
"command": "gxwf validate galaxy-workflow.gxwf.yml --json --cache-dir <run-cache>/gxwf-cache",
"command_notes": [
"`--cache-dir` is still load-bearing, but the failure mode moved. With a network, an EMPTY cache dir is now self-healing: gxwf fetched 13 unique tools and validated all 27 steps in 2.4s. The vacuous green is still reachable offline — `--offline` against an empty cache returns 27 of 27 `skip_tool_not_found` and exit 0. Read the split, never the exit code.",
"`--connections` still cannot be used: gxwf crashes before producing any report (diagnostic `gxwf-connections-crash`). The collection-algebra / map-over check did NOT run in this phase either.",
"`--strict-structure --strict-encoding` was run as a separate pass (diagnostic `format2-state-key-encoding`); `--strict-state` was also run and exits 0 now that nothing is skipped."
],
"status": "pass",
"status_rationale": "gxwf 1.12.0 exits 0 with 27 of 27 steps tool-state validated, 0 failures, 0 structure errors and 0 encoding errors under default strictness. The 7 steps the 1.10.1 run could not check now decode and validate, so the `pass-with-unvalidated-steps` qualifier no longer applies. Two holes remain and are recorded below: connection validation still does not run at all, and a stray `in:` key on a step that carries `tool_state` is still not caught.",
"not_run_reason": null,
"coverage": {
"steps_total": 27,
"tool_state_validated": 27,
"tool_state_skipped": 0,
"tool_state_failed": 0,
"structure_errors": 0,
"encoding_errors_default_mode": 0,
"connection_check_ran": false,
"skipped_steps": [],
"skip_cause": null,
"cold_cache_behaviour": "Cache dir created empty for this run; gxwf populated it with 13 unique tool entries during validation. The 7 previously undecodable tools (__FLATTEN__, __FILTER_FROM_FILE__, cutadapt, deseq2) now decode."
},
"non_vacuity_evidence": [
"Negative control, re-measured on gxwf 1.12.0: injecting `header_lines: 'notanint'` into step 8 (Filter1) turns the summary into 26 ok / 1 fail with specific diagnostics (`Expected number, actual \"notanint\"`). The 27 validated steps are genuinely being checked.",
"Tolerance measured on 1.10.1 and not re-measured here: an unknown extra tool_state key (`bogus_param_probe`) and a dropped required parameter (`cond`) both still validated green under every strictness flag gxwf offers. The tool-state check verifies the values of the parameters present, not the completeness or legality of the parameter set.",
"Connection-key tolerance, measured on 1.12.0: a stray `in:` key validates green on any step that carries `tool_state` (diagnostic `in-key-check-masked-by-tool-state`)."
],
"out_of_band_verification": {
"why": "On 1.10.1 gxwf validated nothing on 7 steps and checked no `in:` key on any step; both holes were closed by hand for phase 10. On 1.12.0 the first hole is gone (27 of 27 validated) and the second is half gone — see diagnostic `in-key-check-masked-by-tool-state`. The hand check below therefore still carries the `in:` key names for the 26 steps that declare `tool_state`.",
"in_key_names_vs_tool_schema": {
"method": "Every `in:` key and every tool_state parameter path was resolved against the real tool input tree (conditionals via test_param/cases, repeats via `<name>_<n>`, sections). The 20 cached tools came from the gxwf tool cache; the 7 uncached tools were fetched from `https://usegalaxy.org/api/tools/<id>?io_details=true`.",
"result": "27 of 27 steps clean. 0 mismatches on `in:` keys, 0 mismatches on tool_state parameter names, 0 illegal select-option or boolean values on the 7 previously unchecked steps.",
"significance": "This is the check no gate in this run performs. A wrong unqualified `filter_source:` in place of `how|filter_source` was measured to validate byte-identically green; confirmed independently here, with a bogus `in:` key (`not_a_real_tool_input`) on a fully-cached Filter1 step passing under `--strict-structure --strict-state`."
}
},
"cross_step_invariants": [
{
"id": "deseq2-lfc-shrinkage-none",
"statement": "`advanced_options.lfc_shrinkage_type: 'none'` on BOTH DESeq2 nodes. Any other value makes `get_result_output_columns()` drop the `stat` column, moving padj from c7 to c6 and making the `\"c7<\"` predicate filter on nothing, silently.",
"verdict": "HOLDS",
"evidence": "step 19 and step 20 both bind `advanced_options.lfc_shrinkage_type: 'none'`. Neither node selects `many_contrasts` / `split_output`; both are `select_data.how: datasets_per_level`.",
"enforced_by_a_gate": false,
"consumers": "One compose_text_param bridge hard-codes the padj predicate: step 21 `\"c7<\"`, consumed by Filter1 steps 23 and 25. The log2FC bridge (step 22, `\"abs(c3)>\"`, consumed by steps 24 and 26) is on c3, which sits ahead of `stat` and does not move under shrinkage — so this invariant protects the padj predicate specifically."
},
{
"id": "filter1-header-lines-zero",
"statement": "`header_lines: '0'` on ALL SEVEN Filter1 steps. DESeq2's `deseq_out` is written with `col.names = FALSE` and has no header; binding `'1'` silently discards the top row, which on a padj-sorted table is the most significant gene (SCF1, the paper's finding).",
"verdict": "HOLDS",
"evidence": "All 7 Filter1 steps (8, 12, 16, 23, 24, 25, 26) bind `header_lines: '0'` as the string `'0'`. Filter1 step count is exactly 7.",
"enforced_by_a_gate": false
},
{
"id": "filter-from-file-qualified-key",
"statement": "The three `__FILTER_FROM_FILE__` steps must carry the QUALIFIED `how|filter_source` key, not a bare `filter_source`.",
"verdict": "HOLDS",
"evidence": "Steps 10, 14 and 18 each carry `in: {input, how|filter_source}`. Confirmed against the live `__FILTER_FROM_FILE__` 1.1.0 schema from the Galaxy tool API: inputs are `input` (data_collection) and the conditional `how` (test param `how_filter`) with `filter_source` inside each case. The tool_state matches: `how.how_filter: remove_if_absent`, `__current_case__: 0`.",
"enforced_by_a_gate": false
}
],
"structural_confirmation": {
"class": "GalaxyWorkflow",
"steps": 27,
"inputs": 9,
"outputs": 16,
"all_steps_have_in": true,
"unresolved_in_sources": 0,
"unresolved_output_sources": 0,
"all_outputs_labelled": true
},
"diagnostics": [
{
"id": "gxwf-connections-crash",
"severity": "warning",
"target": "the validation run itself, not the workflow",
"message": "`gxwf validate --connections` aborts with `TypeError: step.in is not iterable` at @galaxy-tool-util/schema/dist/workflow/normalized/toNative.js, inside `_extractConnections` reached from `_buildToolStep`. Unchanged on 1.12.0: exit 1, 0 bytes on stdout. The connection-type / collection-algebra / map-over check did not run.",
"workflow_is_at_fault": false,
"isolation": "Reproduced on 20-line minimal format2 workflows with a single cached Filter1 step, and unchanged on gxwf 1.12.0. Root cause is `_isNormalizedFormat2` in toNative: it infers \"already normalized\" from `class` plus array-valued `inputs`/`steps`, which says nothing about per-step shape, so a workflow written with list-form `inputs:`/`steps:` and mapping-form `in:` - this workflow's dialect - skips `normalizedFormat2` and reaches `_extractConnections` unnormalized. The same input crashes `gxwf convert --to native` with no connection flag involved; map-form `inputs:` or `steps:`, or list-form `in:`, convert cleanly.",
"routes_to": "gxwf (upstream). Filed as feedback `tonative-shape-sniff-skips-normalization-on-list-form-format2`; fix submitted as jmchilton/galaxy-tool-util-ts#179 (green, unmerged at the time of this run). Not in any release yet.",
"consequence_for_this_run": "Collection-shape compatibility is unproven by any static gate. See residual risk `collection-algebra-unchecked`."
},
{
"id": "format2-state-key-encoding",
"severity": "advisory",
"target": "all 26 tool steps (step 0 has no tool_state)",
"message": "Under `--strict-encoding`, gxwf reports `step N: uses \"tool_state\" instead of \"state\" (format2 should use \"state\")` for every one of the 26 steps that carry tool state, and exits 2. Re-measured unchanged on 1.12.0.",
"workflow_is_at_fault": false,
"why_not_a_defect_here": "`tool_state:` is forced. gxwf's own tool-state validator drops a `{__class__: ConnectedValue}` placeholder when the state is written under `state:` and honours it under `tool_state:` — filed in phase 6 as `gxwf-drops-connectedvalue-under-format2-state-key`. Switching to the key `--strict-encoding` demands would break the compose_text_param steps. `gxwf convert --to format2` also emits `tool_state:`, as does the IWC exemplar this run compared against.",
"routes_to": "gxwf (upstream). Filed as feedback `gxwf-strict-encoding-demands-the-state-key-its-validator-mishandles`.",
"consequence_for_this_run": "None at runtime. Galaxy and gxwf both accept `tool_state:`. Recorded so a later maturation pass does not read the strict-encoding failure as a workflow defect and 'fix' it into the broken key."
},
{
"id": "json-interface-unreliable",
"severity": "advisory",
"target": "the validation run itself",
"message": "PARTIALLY FIXED on 1.12.0. The ~55 lines of Effect schema text that used to precede the JSON document on stdout are gone: `--json` now yields a parseable document with a cold cache and a warm one alike. What survives is the strict-mode breach — `--json --strict-structure --strict-encoding` exits 2 with 0 bytes on stdout and plain text on stderr, so no harness can read a strict verdict as JSON.",
"workflow_is_at_fault": false,
"routes_to": "gxwf (upstream). Filed as feedback `gxwf-validate-json-output-is-not-machine-parseable`.",
"consequence_for_this_run": "None for the verdict. The default-mode report parsed directly this time; the strict pass still had to be read from stderr."
},
{
"id": "in-key-check-masked-by-tool-state",
"severity": "warning",
"target": "the gate's coverage of all 26 steps that carry tool_state",
"message": "gxwf 1.12.0 rejects an `in:` key that names no parameter of the pinned tool ONLY when the step carries no explicit `tool_state` block. When `tool_state` is present and names the real port, a stray connection key in `in:` validates green.",
"workflow_is_at_fault": false,
"isolation": "Two 15-line Filter1 workflows differing only by a `tool_state` block: without it the bogus key fails with `No parameter definition matching connection key`, with it the same bogus key returns `ok`. Reproduced on this workflow: renaming `Read quality report`'s `input_file:` connection to `not_a_real_tool_input` leaves all 27 steps `ok`, because its `tool_state` still carries `input_file: {__class__: ConnectedValue}`.",
"routes_to": "gxwf (upstream). Sharpens the existing feedback entry `gxwf-validate-never-checks-in-key-names`, which this run's evidence shows is only half fixed.",
"consequence_for_this_run": "The hand check recorded under `out_of_band_verification` is still the only thing covering `in:` key names on 26 of 27 steps."
}
],
"residual_runtime_risks": [
{
"id": "collection-algebra-unchecked",
"risk": "No static check has confirmed collection-shape compatibility anywhere in the workflow, because `--connections` crashes. The map-over structure is the load-bearing part of this design: `sample_sheet:paired` in → `__FLATTEN__` → FastQC (per-fastq fan-out), the paired collection into Cutadapt `library|input_1` and on to RNA STAR `singlePaired|input`, three `__FILTER_FROM_FILE__` reductions on `Count reads per gene/output_short`, and the workflow's only reduction where DESeq2's multiple=true `countsFile` ports consume whole collections.",
"proved_or_disproved_by": "The first successful invocation. Per-step HDCA population and `step_jobs_summary` from `GET /api/invocations/{id}/step_jobs_summary`; a shape mismatch surfaces as invocation message `collection_failed` or as an unexpected job count on a mapped step.",
"severity": "major"
},
{
"id": "sample-sheet-collection-support",
"risk": "The workflow's primary input is `collection_type: sample_sheet:paired` with `column_definitions`, and step 6 is `__SAMPLE_SHEET_TO_TABULAR__`. Both are recent Galaxy features. gxwf accepted them structurally, which says nothing about whether the Galaxy instance planemo targets supports them. Related open requirement: `no-iwc-precedent-for-sample-sheet-workflow-input`; related filed feedback: `galaxy-test-staging-drops-sample-sheet-column-definitions`.",
"proved_or_disproved_by": "Phase 11. Failure would appear at request-time validation or input materialization, before useful invocation state — an API error or `dataset_failed`, not a tool stderr.",
"severity": "major"
},
{
"id": "deseq2-column-contract-at-runtime",
"risk": "The c3 = log2FoldChange / c7 = padj contract, and the headerless premise under it, are verified here only as authored bindings. That `deseq_out` really arrives 7-column and headerless from this wrapper at this version is a runtime fact.",
"proved_or_disproved_by": "Phase 11. The test plan already asserts `has_n_columns: 7` on both raw result tables and `not_has_text: baseMean`, so a regression fails the tests rather than filtering silently. Tracked by open requirement `deseq2-statistics-unverified-against-the-galaxy-wrapper`.",
"severity": "major"
},
{
"id": "star-index-parameter-for-a-substituted-reference",
"risk": "`genomeSAindexNbases: '10'` is correct for the 12,365,959 bp B8441 reference this run's tests supply, and silently wrong for any other genome a user wires in. Open requirement `star-genome-length-drives-sa-index-parameter`.",
"proved_or_disproved_by": "The promoted `STAR mapping summary` / job stderr on the first real run — STAR prints its own recommended value when the supplied one is too large.",
"severity": "minor"
},
{
"id": "strandedness-inferred-not-stated",
"risk": "`Strandedness` defaults to `stranded - reverse`, inferred from the library kit rather than stated by the paper. A wrong setting produces a plausible but wrong count matrix, not an error. Open requirement `cutadapt-adapter-and-length-filter-unstated` is adjacent.",
"proved_or_disproved_by": "The promoted `featureCounts assignment summary` on the first real run: a wrong setting shows a large `Unassigned_NoFeatures` fraction.",
"severity": "minor"
},
{
"id": "flatten-join-identifier-defaulted",
"risk": "Step 0 (`__FLATTEN__`) carries no tool_state at all, so `join_identifier` takes the wrapper default. That default determines the element identifiers FastQC's per-read outputs carry. The sample-sheet element identifiers are declared the spine of this workflow.",
"proved_or_disproved_by": "Phase 11: the element identifiers on the promoted `FastQC raw reads` outputs.",
"severity": "minor"
}
],
"handoff_to_phase_11_12": [
"Collection algebra was NOT statically checked, on 1.12.0 either. `--connections` still crashes. Treat the first invocation as the first check of map-over shape, and read `step_jobs_summary` per step rather than only the terminal state.",
"The 7 formerly-skipped steps are now covered by the gate itself (27 of 27 validated on gxwf 1.12.0). The phase-10 hand check of their `in:` keys and state parameter names no longer needs re-doing after an edit for tool-state purposes.",
"`in:` key names on the 26 steps that carry `tool_state` are still NOT covered by any gate — gxwf's new connection-key check is masked by the presence of `tool_state`. The phase-10 hand check remains the only evidence there.",
"All three cross-step invariants hold in the assembled artifact. None is enforced by a gate, and the `doc:` half of `cross-step-invariants-survive-only-in-the-draft` is still not done: neither DESeq2 node's `doc:` mentions shrinkage, and none of the seven Filter1 `doc:` strings says why `header_lines` is '0'. The four significance filters do name the `c7<` / `abs(c3)>` predicates in their docs; the two DESeq2 nodes name nothing.",
"`galaxy-workflow-draft.gxwf.yml` still carries the invariant rationale in YAML comments that `draft-extract` strips. Do not discard it before the `doc:` strings are written.",
"A workflow-level green from this gate now covers 27 of 27 steps' tool state and no `in:` key name on any step that declares `tool_state`. Any later phase quoting 'validation passed' should quote that scope with it."
],
"previous_run": {
"file": "galaxy-workflow-validation-result.json.gxwf-1.10.1.bak",
"gxwf": "1.10.1",
"validated_at": "2026-09-16",
"status": "pass-with-unvalidated-steps",
"note": "Superseded by this run. Kept because its 20/7 split is cited by open-requirements and feedback entries."
},
"resolved_since_previous_run": [
{
"feedback_entry": "collection-output-decode-is-a-flat-vs-nested-shape-mismatch-not-a-missing-field",
"resolution": "Fixed upstream between 1.10.1 and 1.12.0.",
"evidence": "Same workflow, same flags, cold cache: 20 validated / 7 skipped on 1.10.1 → 27 validated / 0 skipped on 1.12.0. Step 0 `__FLATTEN__` and both DESeq2 nodes report `ok`."
},
{
"feedback_entry": "gxwf-cannot-decode-collection-outputs-of-builtin-collection-operations",
"resolution": "Resolved with the entry above, which already superseded it.",
"evidence": "`__FLATTEN__` and `__FILTER_FROM_FILE__` both decode and validate on 1.12.0."
},
{
"feedback_entry": "gxwf-validate-json-output-is-not-machine-parseable",
"resolution": "PARTIAL — breach (1) only.",
"evidence": "`--json` now writes parseable JSON and nothing else to stdout, with both a warm and a cold cache; `json.load(stdout)` succeeds. Breach (2) survives: adding any strict flag still yields exit 2, 0 bytes on stdout and plain text on stderr."
},
{
"feedback_entry": "gxwf-validate-never-checks-in-key-names",
"resolution": "PARTIAL — and the remaining half is what this workflow needs.",
"evidence": "A bogus `in:` key on a step with NO `tool_state` now fails: `No parameter definition matching connection key \"not_a_real_port\"`. The same bogus key on a step that CARRIES a `tool_state` block naming the real port still validates green — isolated on two 15-line Filter1 workflows differing only by the presence of `tool_state`, and confirmed on this workflow by renaming step `Read quality report`'s `input_file:` connection to `not_a_real_tool_input` (27 of 27 still `ok`). 26 of this workflow's 27 steps carry `tool_state`, so the check does not cover them."
}
],
"resolved_risks": [
{
"id": "seven-steps-never-schema-checked-by-a-gate",
"risk": "Steps 0, 2, 10, 14, 18, 19 and 20 will be unvalidated by gxwf on every future run of this gate, permanently — the decoder mismatch is not clearable by priming the cache. This phase checked them by hand against the live tool API and found them clean, but that check is not repeatable by a gate and will not be redone automatically after any later edit.",
"proved_or_disproved_by": "Nothing static, until the gxwf decoder is fixed. Phase 11's invocation is the first machine check these 7 steps get. A later maturation pass that edits any of them re-opens the hole.",
"severity": "major",
"resolved_on": "gxwf 1.12.0",
"resolution": "The decoder mismatch is fixed. All 7 steps now validate under the standard gate on every future run, and the hand check no longer has to be re-done after an edit."
}
]
}Structural diff against the nearest IWC exemplar(s); guidance for the downstream *-summary-to-galaxy-template Mold before per-step authoring. Carries an inline, bounded gxformat2 excerpt of the nearest exemplar's relevant subgraph under a labeled section, cross-referencing the iwc-exemplar-gxformat2 sibling file.
# IWC exemplar comparison — *C. auris* Scf1 RNA-seq differential expression
Source handoffs: `freeform-galaxy-interface.md` (phase 2) and `freeform-galaxy-data-flow.md` (phase 3). Where the two conflict, the data-flow brief governs (its §7 corrects six declared shapes); this note takes that as given and diffs against the corrected reading.
Consumers: `freeform-summary-to-galaxy-template` (phase 5), `freeform-summary-to-galaxy-test-plan` (phase 8).
## Corpus provenance
| | |
|---|---|
| Corpus | `https://github.com/galaxyproject/iwc` |
| HEAD | `fe41a79` (`main`; merge of PR #1337) |
| Clone | shallow, taken 2026-09-16, read-only |
| Normalization | `gxwf convert <file>.ga --to format2 --compact` (`@galaxy-tool-util/cli`, `gxwf --version` reports `1.0.0`) |
**Environment deviation.** The Mold's procedure says to clone or pull the corpus to `~/.foundry/iwc`. That path is not creatable on this machine (`mkdir ~/.foundry` → `Operation not permitted`, with and without the Bash sandbox), so the harness supplied a fresh clone elsewhere and this phase read it in place. No `git pull` was run and nothing was written inside the clone, so the comparison is pinned to `fe41a79` rather than to corpus `main` at read time. Raised as feedback against the Mold: the path is hard-coded with no environment override.
---
## 1. Ranking
Candidates were drawn by tool family across the whole corpus, not by directory name: `deseq2` matches 2 workflows, `featurecounts` 3, `rgrnastar` 2, `cutadapt` 10.
| Rank | IWC workflow ID | Covers | Confidence |
|---|---|---|---|
| 1 | `transcriptomics/rnaseq-de/rnaseq-de-filtering-plotting` | the DE tail — data-flow nodes H, I | **High** |
| 2 | `transcriptomics/rnaseq-pe/rnaseq-pe` | the map-over head — nodes A–D | **Medium** |
| 3 | `epigenetics/cutandrun/cutandrun` | the Cutadapt step only | **Low / tool-level evidence, not a domain exemplar** |
| — | `transcriptomics/rnaseq-sr/rnaseq-sr` | single-read sibling of #2 | not ranked — wrong read topology, no independent signal |
| — | `scRNAseq/pseudobulk-worflow-decoupler-edger` | pseudobulk DE | no match — edgeR/decoupler, single-cell domain; the `deseq2` grep hit is a doc mention, not a step |
### 1.1 Why rank 1 is High
Same domain and subdomain (bulk RNA-seq differential expression). Same input topology at the boundary it covers: `list` collections of per-sample count tables, reduced into DESeq2's `multiple=true` factor-level ports — which is exactly the reduction the data-flow brief specifies at node H. Same primary tool family (`iuc/deseq2`). Same DAG motif: DESeq2 → annotate → threshold filter. Same output surface as the subject's outputs 9–13 (normalized counts, results table, filtered gene list). Matching test-fixture shape: remote Zenodo count tables with explicit per-element `identifier`, and `has_text_matching` regexes on the tabular results.
### 1.2 Why rank 2 is Medium, not High
Same domain and same input topology (`list:paired` reads). Same tool families for the two steps that matter most structurally (`iuc/rgrnastar`, `iuc/featurecounts`), including the same `output_short` / `output_summary` output pair the subject promotes as outputs 7 and 8. But: the trimmer is fastp, not Cutadapt; the reference is a built-in index, not a history FASTA; and roughly two-thirds of the workflow (Cufflinks, StringTie, bigwig coverage, MultiQC, Picard/RSeQC QC) has no counterpart in the subject's deliberately closed tool set. Partial tool-family and output match is the Medium band by definition.
### 1.3 The headline structural finding
**IWC has no single workflow spanning FastQC → DESeq2.** The corpus publishes this journey as two workflows joined at the count-table boundary: `rnaseq-pe` ends at per-sample count tables, `rnaseq-de` begins there. The subject is one workflow spanning both halves.
This is not a cosmetic packaging difference — it is *where the condition grouping lives*. Because `rnaseq-de` starts from count tables, it can demand pre-grouped collections as workflow inputs and never needs a metadata-driven split at all. The subject, being one workflow, cannot: it receives ungrouped reads and must reconstruct the grouping internally. Everything in §3 below follows from that one divergence. Recorded as open-requirements entry `iwc-splits-rnaseq-de-at-the-count-table-boundary`.
---
## 2. The log2FC question — settled by the corpus
This is the entry phase 3 flagged as the sharpest conflict, and the corpus answers it without ambiguity.
**What IWC does.** `rnaseq-de` exposes the effect-size threshold in **log2 units at the interface**:
```yaml
- id: log2 fold change threshold
type: float
optional: false
default: 1
doc: >-
log2 fold change threshold to filter for highly regulated genes.
A log2 FC of 3 equals to an absolute fold change of 8 (2^3).
```
and filters with the raw column, no conversion anywhere:
```yaml
component_value: abs(c3)> # c3 == log2(FC)
```
There is no linear fold-change parameter in the workflow, no `log2()` call, and no conversion node. The user is asked for the number the tool actually reports, and the doc string teaches the conversion in prose.
**What this settles.** `fold-change-threshold-linear-vs-deseq2-log2fc` closes on the third of its three options: **restate interface input 7 as a log2 threshold with a changed label**, not a conversion node and not an inline conversion in the filter expression. Default `1.0`, which is precisely the paper's `|fold change| > 2`. The corpus default is the same number for the same reason.
**Three details the corpus supplies alongside the answer:**
1. **Column indices.** `c3` is log2FC, `c7` is adjusted p-value. These are the raw DESeq2 output columns — `deg_annotate` appends columns 8–13 (chromosome, start, end, strand, feature, gene name) and leaves 1–7 untouched, so the indices hold whether or not the subject adds an annotation step. The header the exemplar generates names them: `GeneID, Base mean, log2(FC), StdErr, Wald-Stats, P-value, P-adj, …`.
2. **`Filter1` cannot take a numeric parameter directly.** Its predicate is a text parameter. The corpus idiom is one `iuc/compose_text_param` step per threshold, concatenating a literal prefix (`c7<`, `abs(c3)>`) with the connected float. So the subject's node I is **two Galaxy steps per filter**, not one.
3. **Two chained `Filter1` steps, not one compound predicate.** `Filter with p-adj threshold` → `Filter with log2 FC threshold`, each with `header_lines: "1"`. Each intermediate is separately promotable, which is also why the exemplar can rename them distinctly.
**The consequence the ledger entry warned about is real and the corpus confirms the trap.** Had the subject kept a linear `2.0` and compared `abs(c3) > 2.0`, it would have applied a 4-fold cut — a plausible-looking table that silently omits most of the paper's gene list. Nothing in the pipeline would have caught it.
### 2.1 Inline excerpt — the parameter-to-filter bridge
From `transcriptomics/rnaseq-de/rnaseq-de-filtering-plotting`, steps `_unlabeled_step_10` and `Filter with log2 FC threshold`. Fuller subgraph in the sibling file `iwc-exemplar.gxwf.yml`, document 1.
```yaml
- id: _unlabeled_step_10
tool_id: toolshed.g2.bx.psu.edu/repos/iuc/compose_text_param/compose_text_param/0.1.1
in:
- id: components_1|param_type|component_value
source: log2 fold change threshold
out:
- id: out1
hide: true
tool_state:
components:
- __index__: 0
param_type: {select_param_type: text, __current_case__: 0, component_value: "abs(c3)>"}
- __index__: 1
param_type: {select_param_type: float, __current_case__: 2, component_value: {__class__: ConnectedValue}}
- id: Filter with log2 FC threshold
label: Filter with log2 FC threshold
tool_id: Filter1
in:
- id: cond
source: _unlabeled_step_10/out1
- id: input
source: Filter with p-adj threshold/out_file1
out:
- id: out_file1
rename: Genes filtered with adj p-value and log2(FC) thresholds
tool_state:
header_lines: "1"
```
---
## 3. Structural divergences that matter for template authoring
Ordered by how much authoring effort they redirect.
### 3.1 The condition split has no corpus precedent at all
The data-flow brief's §4 route — `__SAMPLE_SHEET_TO_TABULAR__` → row filter → `__FILTER_FROM_FILE__` → per-level counts collections — was the run's most carefully reasoned decision. The corpus does not support it, and does not contradict it either. It simply has nothing.
Searched across all of `workflows/` at `fe41a79`:
| Token | Hits |
|---|---|
| `sample_sheet` (any collection type) | **0** |
| `__SAMPLE_SHEET_TO_TABULAR__` | **0** |
| `column_definitions` with a non-null value | **0** (the token appears only as `"column_definitions": null` on ordinary collection inputs, a serialization artifact of newer Galaxy) |
| `__FILTER_FROM_FILE__` | 6 workflows — none in transcriptomics, none for a condition split |
So: the `__FILTER_FROM_FILE__` half of the route is a real, used Galaxy idiom; the sample-sheet half is unprecedented in published IWC practice. The subject would be the first.
**What IWC does instead** is to push the grouping into the interface — two pre-grouped `list` collections, `Counts from changed condition` and `Counts from reference condition`. That is *the same shape* as the alternative phase 2 explicitly rejected ("three condition-scoped `list:paired` inputs … hard-codes the three-level design into the interface"). The rejection reasoning was sound on its own terms; it is worth knowing that the corpus made the opposite trade, and bought a simpler workflow with it.
**Guidance for the template.** Build the §4 split as designed — nothing in the corpus refutes it, and the phase-3 reasoning stands. But build it as a clearly delimited template region with the phase-2 fallback (a `data` input `Sample metadata table`) reachable by deleting one node, because it is the one region of this workflow with no worked precedent to pattern-match against, and two open entries (`sample-sheet-to-tabular-identifier-column-unverified`, `sample-sheet-input-test-fixture-expressibility`) still ride on it. Recorded as `no-iwc-precedent-for-sample-sheet-workflow-input`.
### 3.2 DESeq2 node arity — two nodes, settled
`deseq2-contrast-realization-unsettled` closes: **two DESeq2 nodes, each with a two-level factor, the reference-level collection feeding both.**
Evidence, all from the exemplar:
- The factor-level ports are `select_data|rep_factorName_0|rep_factorLevel_0|countsFile` and `…|rep_factorLevel_1|countsFile`, under `how: datasets_per_level`. Each port consumes a whole collection — confirming the data-flow brief's node-H reduction exactly.
- `deseq_out` is **a single dataset, not a collection**. Downstream of it sit `deg_annotate`, `tp_cat` and two `Filter1` steps — all single-dataset tools that would map over a collection rather than consume it whole — and the sibling `-tests.yml` asserts `has_text_matching` on the derived output as a dataset.
- The workflow's own README bounds it: *"works only with an experimental setup containing exactly 2 conditions with at least 2 replicates per condition."*
One DESeq2 job therefore yields one results table. The interface fixes two distinctly labelled results tables (outputs 10 and 11), so it needs two jobs. The `rep_factorLevel` repeat means a three-level factor is *expressible*, but a three-level run still emits one `deseq_out`, which cannot satisfy a two-output interface — so the arity question is answered by the output surface regardless of what the wrapper does with three levels.
This costs the template nothing upstream: as §4.4 of the data-flow brief predicted, both realizations consume the same per-level counts collections, so nodes E, F₁–F₃, G₁–G₃ are unchanged. Node H becomes H₁ (`G_AR0382`, `G_tnSWI1`) and H₂ (`G_AR0382`, `G_AR0387`).
One consequence worth flagging forward: with two runs, `DESeq2 normalized counts` (output 9) and `DESeq2 diagnostic plots` (output 14) are now produced twice. The interface declares one of each. Either promote from one designated run and say which, or relabel per contrast. The template should not leave this implicit.
### 3.3 FastQC fan-out — flatten, contra the data-flow brief
`fastqc-per-read-fanout-not-a-flat-list` closes **in favour of the option the data-flow brief rated "no analytical gain"**.
`rnaseq-pe` puts an explicit `__FLATTEN__` between the `list:paired` reads input and the per-fastq QC tool:
```yaml
- id: _unlabeled_step_11
tool_id: __FLATTEN__
in:
- id: input
source: Collection paired FASTQ files
out:
- id: output
hide: true
tool_state:
join_identifier: _
```
and the consuming subworkflow declares its input as `collection_type: list` — flat. Identifiers become `<sample>_forward` / `<sample>_reverse`.
So the corpus idiom is: flatten first, run QC over a flat list. This satisfies the interface's declared `list` shape for outputs 1–2 as originally written, and it is the published convention rather than a workaround. The data-flow brief's preference for promoting the nested collection was a reasonable call made with no corpus evidence available; the evidence now exists and points the other way.
**For the test plan:** the identifier vocabulary on outputs 1 and 2 doubles to twelve — `AR0382_A_forward`, `AR0382_A_reverse`, and so on. Outputs 3–8 keep the six-element sample identifier space untouched, because the flatten is a side branch off the workflow input and does not touch the map-over region.
### 3.4 Cutadapt emits one paired collection — no re-pair node
`trimmed-reads-paired-reassembly-conditional` closes. Neither transcriptomics exemplar uses Cutadapt (both use fastp), so this comes from `epigenetics/cutandrun` — a different domain, cited for the wrapper's IO shape only and for nothing else.
`lparsons/cutadapt/cutadapt/5.2+galaxy2` driven from a `list:paired` input with `library.type: paired_collection` declares outputs `out_pairs` (type `input`, i.e. the input collection's own shape) and `report`. One paired-inner collection out, one report per element.
So placeholder transformation 5.6 (re-pair node) is **not needed** and node C takes a single collection input, as the data-flow brief's primary reading assumed. `rnaseq-pe`'s fastp behaves identically (`output_paired_coll`), so the finding is consistent across both wrappers that could fill the trimmer slot.
### 3.5 RNA STAR from a history FASTA has no corpus precedent
Every STAR step in IWC — `rnaseq-pe` and `rnaseq-sr`, the only two — uses `refGenomeSource.geneSource: indexed` with a built-in `genomeDir` selected through a `restrictOnConnections: true` string parameter, plus `sjdbGTFfile` from the history. The test jobs pass a plain genome string (`Reference genome: sacCer3`).
The subject settled input 2 as a history FASTA on portability grounds (`reference-genome-delivery-shape-unverified`), which is the right call for *C. auris* B8441 — a genome no public server indexes — but it means the template has **no worked example of the history-reference conditional branch** to pattern-match against. That branch is a different `__current_case__` in the wrapper with a different set of required sub-parameters. Recorded as `star-history-reference-wiring-has-no-corpus-precedent`; it is a step-implementation obligation, best discharged by a `summarize-galaxy-tool` pass on `iuc/rgrnastar` rather than guessed at in the template.
This does not disturb the topology. The data-flow brief's §8 note stands: six in-job index builds of a ~12.5 Mb genome, no separate index node, and the graph is insensitive to how the entry eventually resolves.
### 3.6 Strandedness is a mapped parameter, not a raw one
`rnaseq-pe` exposes a restricted human-readable string:
```yaml
- id: Strandedness
type: string
restrictions: ["stranded - forward", "stranded - reverse", "unstranded"]
```
and translates it per consumer with one `iuc/map_param_value` step each (`Get featureCounts strandedness parameter`, and siblings for Cufflinks and StringTie), with `unmapped.on_unmapped: fail` so an unrecognized value stops the run rather than silently defaulting.
The subject's interface input 5 is a free `text` parameter with allowed values named only in prose. The corpus idiom is strictly better here: it type-restricts at the interface, it fails loudly on a bad value, and — because the subject's featureCounts step is the only consumer — it costs exactly one extra step. Worth adopting. This does not close `rnaseq-strandedness-inferred-from-kit-name` (the *value* is still an inference from the kit name; only its expression improves), and the empirical check via output 8 remains as the data-flow brief wired it.
---
## 4. Where the corpus confirms the design
Reported so the template does not mistake silence for doubt.
- **The map-over spine A→B→C→D is conventional and matches.** `rnaseq-pe` runs trim → STAR → featureCounts over a `list:paired` collection with the GTF broadcast into every mapped job and `anno.anno_select: history`, precisely as the data-flow brief's §2.2 edge table specifies. Input 3 is consumed twice there too, by STAR's `sjdbGTFfile` and featureCounts' `reference_gene_sets` — the same double consumption §8 flagged.
- **featureCounts' output pair is the right promotion.** `output_short` and `output_summary` are exactly interface outputs 7 and 8, and the exemplar promotes the counts through to its own `Counts Table`.
- **STAR's `output_log` is the assertable text behind the BAM.** The exemplar carries it, matching the interface's deliberate binary/text pairing for outputs 5 and 6.
- **No merged count matrix.** The exemplar's DESeq2 consumes per-sample count files through collection ports; the interface's refusal to invent a concatenate node is correct.
- **No collection-cleanup node.** Neither transcriptomics exemplar filters failed or empty elements out of a dense homogeneous collection. §3 of the data-flow brief called this right.
- **Absent MultiQC is a defensible divergence, not an omission.** `rnaseq-pe` does run MultiQC, but behind a `Generate additional QC reports` boolean and over QC tools the subject does not have. Adding it would be inventing method the paper does not name, and the interface's closed tool set is the better call.
---
## 5. Test-fixture guidance (for phase 8)
Corpus-observed, from the two sibling `-tests.yml` files. This is guidance for the test-plan Mold, not work this phase owns.
- **Collections are declared inline with explicit per-element identifiers.** `rnaseq-pe-tests.yml` nests `class: Collection` / `collection_type: paired` inside `collection_type: list:paired`, with `identifier: forward` / `reverse` on the inner files. That is the fixture form for the phase-2 fallback shape.
- **No corpus fixture declares a `sample_sheet` collection or any `column_definitions`.** `sample-sheet-input-test-fixture-expressibility` therefore remains genuinely open — the corpus offers no worked example either way, and its absence is weak evidence at best.
- **Remote Zenodo URLs with SHA-1 `hashes:` on inputs.** Hashes appear on inputs only; the corpus uses no checksum assertions on outputs.
- **Assertions are tolerant by default** — `has_size` + `delta`, `has_text_matching` regexes with digit wildcards on floating-point columns. The subject's declared assertion intents fit this vocabulary. The one place to be *stricter* than the corpus default is output 7: featureCounts on a fixed reference, annotation and strandedness is deterministic, and an existence-only probe there would be the smell the packaged anti-patterns note names. Exact per-gene integer counts, as the interface intends, is the right call.
- **Labels are the API.** Both exemplars key every job entry and every output assertion by label. If §3.2's duplicated normalized-counts/plots outputs get relabelled per contrast, that is a breaking change to be made once, in the interface, before any test is written.
---
## 6. Findings routed by authoring surface
| # | Finding | Owner |
|---|---|---|
| 1 | Restate input 7 as a log2 threshold, default 1.0, relabel (§2) | interface brief, then template |
| 2 | Node I is two steps per filter: `compose_text_param` → `Filter1`, chained padj → log2FC (§2) | template |
| 3 | Node H splits into H₁ and H₂ (§3.2) | template |
| 4 | Outputs 9 and 14 are produced twice under two DESeq2 nodes — designate or relabel (§3.2) | interface brief, then template |
| 5 | Insert `__FLATTEN__` before node A; outputs 1–2 are flat `list` (§3.3) | template |
| 6 | Drop placeholder transformation 5.6; node C takes one paired collection (§3.4) | template |
| 7 | STAR history-reference branch needs wrapper evidence (§3.5) | per-step loop (`summarize-galaxy-tool` on `iuc/rgrnastar`) |
| 8 | Restrict input 5 and add a `map_param_value` bridge (§3.6) | interface brief, then template |
| 9 | Build the §4 split as a delimited region with the fallback one deletion away (§3.1) | template |
| 10 | `sample_sheet` fixture expressibility; tolerant-assertion vocabulary; label stability (§5) | test plan |
| 11 | The corpus idioms in §2 (parameter→text-filter bridge) and §3.6 (`map_param_value` fan-out) recur across IWC and are candidates for pattern pages | Foundry pattern tier |
None of these blocks downstream authoring. Findings 1–6 are applicable as written; 7 defers to the per-step loop by design; 9 is a structuring instruction, not a correction.
---
## 7. Open-requirements ledger changes
Closed by this phase (4): `fold-change-threshold-linear-vs-deseq2-log2fc`, `deseq2-contrast-realization-unsettled`, `fastqc-per-read-fanout-not-a-flat-list`, `trimmed-reads-paired-reassembly-conditional`.
Raised by this phase (3): `iwc-splits-rnaseq-de-at-the-count-table-boundary`, `no-iwc-precedent-for-sample-sheet-workflow-input`, `star-history-reference-wiring-has-no-corpus-precedent`.
Left open (12, unchanged and passed through with provenance intact). Corpus evidence bearing on entries this phase did not close, recorded here rather than by editing another Mold's entries:
- **`deseq2-factor-level-names-not-parameterized`** — the corpus offers no level-name parameterization because it has no in-workflow split to parameterize; it puts the grouping in the interface instead (§3.1). The entry's two named options both stand; the corpus adds a third, which is the phase-2 fallback shape. Still open.
- **`sample-sheet-to-tabular-identifier-column-unverified`** — zero corpus uses of `__SAMPLE_SHEET_TO_TABULAR__`. The entry's note nominated "the IWC exemplar comparison" as one of three possible closers; that route is now exhausted, and the remaining two (a `summarize-galaxy-tool` pass, or the first real run) are the live ones.
- **`sample-sheet-input-test-fixture-expressibility`** — see §5. No corpus fixture either way.
- **`reference-genome-delivery-shape-unverified`** — the corpus idiom is a built-in index, which B8441 cannot use; the history-FASTA decision stands on its portability reasoning, and §3.5 records the wiring gap it creates. The entry is about *whether usegalaxy.org carries the index*, which the corpus cannot answer. Still open.
- **`rnaseq-strandedness-inferred-from-kit-name`** — §3.6 improves how the parameter is expressed, not what its value should be. Unchanged.
- **`featurecounts-annotation-source-unnamed`** — both exemplars take the GTF as a history `data` input with `gff_feature_attribute: gene_id` and `gff_feature_type: exon`, confirming the shape but naming no source for *C. auris*. Unchanged.
- **`cutadapt-adapter-and-length-filter-unstated`** — `cutandrun` wires adapter sequences from text parameters, and `rnaseq-pe` exposes optional forward/reverse adapter strings. That is an interface convention, not evidence about this paper's library. Unchanged; adding an adapter would still be adding method.
- **`mapped-outputs-carry-sample-sheet-outer-axis`**, **`galaxy-tool-versions-unpinnable-from-source`**, **`tnbcy1-contrast-not-carried`**, **`pipeline-b-tdna-mapping-not-carried`** — no corpus bearing. Unchanged.
- **`sample-sheet-condition-to-deseq2-factor-wiring`** — already `resolved` by phase 3. The corpus does not refute its `because`, so no `supersedes` is warranted; §3.1 records that the route is unprecedented, which is a different claim from wrong.
Cleaned gxformat2 conversion (via [[convert]] --to format2 --compact) of the nearest IWC exemplar's relevant subgraph — the concrete idiom the downstream template draft pattern-matches against. Bounded to the relevant subgraph, not the whole workflow. Absent when no nearest exemplar is found.
# Nearest IWC exemplar(s) — bounded subgraphs
#
# Corpus: https://github.com/galaxyproject/iwc
# Corpus HEAD: fe41a79 (galaxyproject/iwc main, shallow clone taken 2026-09-16)
# Produced by: gxwf convert <workflow>.ga --to format2 --compact, then bounded to the
# relevant subgraph and stripped of tool_state keys that carry no structural
# signal. Load-bearing tool_state is kept verbatim.
#
# THIS FILE IS A READING AID, NOT A RUNNABLE WORKFLOW. Steps have been removed and
# tool_state elided; every elision is marked with a `# [elided]` comment. For the full
# exemplars, convert the corpus files named in each document header.
#
# The subject workflow (C. auris Scf1 RNA-seq DE: FastQC -> Cutadapt -> RNA STAR ->
# featureCounts -> DESeq2 -> significance filter) spans a journey that IWC publishes as
# TWO workflows joined at the count-table boundary. Both halves are given below.
# See iwc-comparison-notes.md for the structural diff and the ranking.
---
# ============================================================================
# DOCUMENT 1 — PRIMARY EXEMPLAR (High confidence) for the DE tail (nodes H, I)
#
# IWC workflow ID: transcriptomics/rnaseq-de/rnaseq-de-filtering-plotting
# Release: 0.12
# Steps covered: Differential Analysis, the two compose_text_param parameter
# bridges, Annotate DESeq2 table, Filter with p-adj threshold,
# Filter with log2 FC threshold
# Steps dropped: volcano plot, both heatmaps, the normalized-counts join/cut
# chain, and the recurring-header generator (visualization tail;
# the subject brief exposes no counterpart)
#
# This is the document that settles `fold-change-threshold-linear-vs-deseq2-log2fc`
# and `deseq2-contrast-realization-unsettled`.
# ============================================================================
class: GalaxyWorkflow
label: RNA-Seq Differential Expression Analysis with Visualization
doc: >-
Identifies differentially expressed genes between exactly two experimental conditions
from count tables. [elided: full doc string]
inputs:
# NOTE: the condition grouping lives in the INTERFACE, as two pre-grouped `list`
# collections of per-sample count tables. There is no metadata-driven split anywhere
# in this workflow. Contrast with the subject's data-flow brief section 4.
- id: Counts from changed condition
type: collection
collection_type: list
optional: false
doc: Counts from experimental condition or changed condition.
- id: Counts from reference condition
type: collection
collection_type: list
optional: false
doc: Counts from reference condition or base condition.
- id: Count files have header
type: boolean
optional: false
doc: >-
featureCounts count files have a header line; RNA-STAR count files do not.
- id: Gene Annotaton
type: data
optional: false
doc: The same annotation GTF used for mapping and counting
- id: Adjusted p-value threshold
type: float
optional: false
default: 0.05
# *** THE log2FC ANSWER ***
# IWC states the effect-size threshold in log2 units at the interface. It does NOT
# expose a linear fold change and convert internally. Default 1.0 == linear 2-fold,
# which is exactly the subject paper's |FC| > 2 criterion.
- id: log2 fold change threshold
type: float
optional: false
default: 1
doc: >-
log2 fold change threshold to filter for highly regulated genes.
A log2 FC of 3 equals to an absolute fold change of 8 (2^3).
outputs:
- id: DESeq2 Plots
outputSource: Differential Analysis/plots
- id: DESeq2 Normalized Counts
outputSource: Differential Analysis/counts_out
- id: Annotated DESeq2 results table
outputSource: Annotate DESeq2 table/out_file1
- id: Significantly differentially expressed genes
outputSource: Filter with log2 FC threshold/out_file1
steps:
# --- Node H equivalent: ONE DESeq2 job == ONE contrast ---------------------
# `how: datasets_per_level` with a `rep_factorLevel` repeat: each level port takes a
# COLLECTION of per-sample count files, reduced into the tool's multiple=true input.
# Exactly two levels here, and `deseq_out` is a single dataset (proved downstream: it
# feeds deg_annotate / tp_cat / Filter1, all single-dataset tools, and the sibling
# -tests.yml asserts has_text_matching on it as a dataset, not as a collection).
- id: Differential Analysis
label: Differential Analysis
tool_id: toolshed.g2.bx.psu.edu/repos/iuc/deseq2/deseq2/2.11.40.8+galaxy4
tool_version: 2.11.40.8+galaxy4
tool_shed_repository:
changeset_revision: 05f9e54d7e81
name: deseq2
owner: iuc
tool_shed: toolshed.g2.bx.psu.edu
in:
- id: header
source: Count files have header
- id: output_options|alpha_ma
source: Adjusted p-value threshold
- id: select_data|rep_factorName_0|rep_factorLevel_0|countsFile
source: Counts from changed condition
- id: select_data|rep_factorName_0|rep_factorLevel_1|countsFile
source: Counts from reference condition
out:
- id: deseq_out
hide: true
tool_state:
# [elided] advanced_options, batch_factors, tximport
output_options:
output_selector:
- pdf
- normCounts
alpha_ma:
__class__: ConnectedValue
select_data:
how: datasets_per_level
__current_case__: 1
rep_factorName:
- __index__: 0
factorName: DEFactor
rep_factorLevel:
- __index__: 0
factorLevel: MainFactor
countsFile:
__class__: ConnectedValue
- __index__: 1
factorLevel: BaseFactor
countsFile:
__class__: ConnectedValue
# --- Annotation: gene positions / biotype / symbol onto the results table --
# Appends columns 8-13; columns 1-7 (GeneID, BaseMean, log2FC, StdErr, Wald, pval,
# padj) are unchanged, so the c3 / c7 filter indices below hold with or without it.
- id: _unlabeled_step_11
tool_id: toolshed.g2.bx.psu.edu/repos/iuc/deg_annotate/deg_annotate/1.1.0+galaxy1
tool_version: 1.1.0+galaxy1
in:
- id: annotation
source: Gene Annotaton
- id: input_table
source: Differential Analysis/deseq_out
out:
- id: output
hide: true
tool_state:
advanced_parameters:
gff_feature_type: exon
gff_feature_attribute: gene_id
gff_transcript_attribute: transcript_id
gff_attributes: gene_biotype, gene_name
mode: degseq
# [elided] chromInfo, ConnectedValue stubs
- id: Annotate DESeq2 table
label: Annotate DESeq2 table
tool_id: toolshed.g2.bx.psu.edu/repos/bgruening/text_processing/tp_cat/9.11+galaxy0
tool_version: 9.11+galaxy0
in:
# [elided] `inputs` comes from a generated single-line header dataset
- id: queries_0|inputs2
source: _unlabeled_step_11
out:
- id: out_file1
change_datatype: tabular
rename: Annotated DESeq2 results
# --- PARAMETER -> FILTER EXPRESSION BRIDGE --------------------------------
# `Filter1` takes its predicate as a TEXT parameter, so a numeric workflow input
# cannot reach it directly. The corpus idiom is one compose_text_param step per
# filter: a literal prefix plus the connected float. Two steps per threshold.
- id: _unlabeled_step_8
tool_id: toolshed.g2.bx.psu.edu/repos/iuc/compose_text_param/compose_text_param/0.1.1
tool_version: 0.1.1
in:
- id: components_1|param_type|component_value
source: Adjusted p-value threshold
out:
- id: out1
hide: true
tool_state:
components:
- __index__: 0
param_type:
select_param_type: text
__current_case__: 0
component_value: c7< # c7 == P-adj
- __index__: 1
param_type:
select_param_type: float
__current_case__: 2
component_value:
__class__: ConnectedValue
- id: _unlabeled_step_10
tool_id: toolshed.g2.bx.psu.edu/repos/iuc/compose_text_param/compose_text_param/0.1.1
tool_version: 0.1.1
in:
- id: components_1|param_type|component_value
source: log2 fold change threshold
out:
- id: out1
hide: true
tool_state:
components:
- __index__: 0
param_type:
select_param_type: text
__current_case__: 0
component_value: abs(c3)> # c3 == log2(FC); NO conversion applied
- __index__: 1
param_type:
select_param_type: float
__current_case__: 2
component_value:
__class__: ConnectedValue
# --- Node I equivalent: TWO CHAINED Filter1 STEPS, not one ----------------
- id: Filter with p-adj threshold
label: Filter with p-adj threshold
tool_id: Filter1
tool_version: 1.1.1
in:
- id: cond
source: _unlabeled_step_8/out1
- id: input
source: Annotate DESeq2 table/out_file1
out:
- id: out_file1
hide: true
rename: Genes filtered with adj p-value threshold
tool_state:
header_lines: "1"
- id: Filter with log2 FC threshold
label: Filter with log2 FC threshold
tool_id: Filter1
tool_version: 1.1.1
in:
- id: cond
source: _unlabeled_step_10/out1
- id: input
source: Filter with p-adj threshold/out_file1
out:
- id: out_file1
rename: Genes filtered with adj p-value and log2(FC) thresholds
tool_state:
header_lines: "1"
tags:
- transcriptomics
- RNAseq
license: MIT
release: "0.12"
---
# ============================================================================
# DOCUMENT 2 — SECONDARY EXEMPLAR (Medium confidence) for the map-over head
# (nodes A-D)
#
# IWC workflow ID: transcriptomics/rnaseq-pe/rnaseq-pe
# Steps covered: __FLATTEN__, fastp (the trimmer slot), RNA STAR, the
# map_param_value strandedness bridge, featureCounts, and the
# `More QC` subworkflow reduced to its Falco (FastQC-equivalent) step
# Steps dropped: Cufflinks, StringTie, all coverage/bigwig generation, MultiQC,
# Picard / RSeQC / idxstats QC, the reference-genome text bridge
# (the subject brief has no counterpart for any of these)
#
# This is the document that settles `fastqc-per-read-fanout-not-a-flat-list` and
# supplies the strandedness-parameter idiom.
# ============================================================================
class: GalaxyWorkflow
label: "RNA-Seq Analysis: Paired-End Read Processing and Quantification"
inputs:
- id: Collection paired FASTQ files
type: collection
collection_type: list:paired
optional: false
doc: Should be a list of paired-end RNA-seq fastqs
# Built-in STAR index selected by name. `restrictOnConnections: true` narrows the
# option list to the genomes STAR actually has indexed on the server.
# The subject's C. auris B8441 has no such entry — see iwc-comparison-notes.md.
- id: Reference genome
type: string
optional: false
restrictOnConnections: true
- id: GTF file of annotation
type: data
optional: false
- id: Strandedness
type: string
optional: false
restrictions:
- stranded - forward
- stranded - reverse
- unstranded
outputs:
- id: Mapped Reads
outputSource: "STAR: map and count and coverage splitted/mapped_reads"
- id: Counts Table
outputSource: _unlabeled_step_25/Counts Table # [elided] relabel step
steps:
# --- *** THE FastQC FAN-OUT ANSWER *** ------------------------------------
# IWC does NOT promote a nested collection from per-fastq QC. It flattens the
# list:paired collection FIRST, with an explicit join_identifier, and runs the
# single-dataset QC tool over the resulting flat `list`. Element identifiers
# become <sample>_forward / <sample>_reverse.
- id: _unlabeled_step_11
tool_id: __FLATTEN__
tool_version: 1.0.0
in:
- id: input
source: Collection paired FASTQ files
out:
- id: output
hide: true
tool_state:
input:
__class__: ConnectedValue
join_identifier: _
# --- Trimmer slot. rnaseq-pe uses fastp, the subject's paper names Cutadapt. ---
# Both emit ONE paired-inner collection when driven by a list:paired input:
# fastp -> output_paired_coll; Cutadapt (lparsons/cutadapt, library.type
# `paired_collection`) -> out_pairs + report. See the Cutadapt excerpt below.
- id: remove adapters + bad quality bases
label: remove adapters + bad quality bases
tool_id: toolshed.g2.bx.psu.edu/repos/iuc/fastp/fastp/1.3.6+galaxy0
tool_version: 1.3.6+galaxy0
in:
- id: single_paired|paired_input
source: Collection paired FASTQ files
# [elided] adapter_sequence1 / adapter_sequence2 from optional string inputs
out:
- id: output_paired_coll
hide: true
- id: report_json
hide: true
tool_state:
filter_options:
quality_filtering_options:
disable_quality_filtering: false
qualified_quality_phred: "30"
# [elided] duplicated_reads, read_mod_options, output_options
# --- Aligner. ENCODE long-RNA parameter set, INDEXED reference. -----------
- id: "STAR: map and count and coverage splitted"
label: "STAR: map and count and coverage splitted"
tool_id: toolshed.g2.bx.psu.edu/repos/iuc/rgrnastar/rna_star/2.7.11b+galaxy1
tool_version: 2.7.11b+galaxy1
tool_shed_repository:
changeset_revision: 55c9ac3aa8f4
name: rgrnastar
owner: iuc
tool_shed: toolshed.g2.bx.psu.edu
in:
- id: refGenomeSource|GTFconditional|genomeDir
source: Reference genome
- id: refGenomeSource|GTFconditional|sjdbGTFfile
source: GTF file of annotation
- id: singlePaired|input
source: remove adapters + bad quality bases/output_paired_coll
out:
- id: output_log # Log.final.out — the assertable text behind the BAM
hide: true
- id: reads_per_gene
hide: true
rename: Reads per gene from STAR
- id: mapped_reads
rename: Mapped Reads
# [elided] signal_unique_str1/2, signal_uniquemultiple_str1/2, splice_junctions
tool_state:
refGenomeSource:
geneSource: indexed # <-- every STAR step in IWC uses `indexed`
__current_case__: 0
GTFconditional:
GTFselect: without-gtf-with-gtf
__current_case__: 1
genomeDir:
__class__: ConnectedValue
sjdbGTFfile:
__class__: ConnectedValue
sjdbGTFfeatureExon: exon
sjdbOverhang: "100"
# [elided] full ENCODE algo.params block (seed / align / junction settings)
# --- User-facing strandedness string -> per-tool parameter value ----------
# One map_param_value step per consumer (featureCounts, Cufflinks, StringTie).
# Keeps ONE user-facing vocabulary while each wrapper gets its own encoding.
- id: Get featureCounts strandedness parameter
label: Get featureCounts strandedness parameter
tool_id: toolshed.g2.bx.psu.edu/repos/iuc/map_param_value/map_param_value/0.2.0
tool_version: 0.2.0
in:
- id: input_param_type|input_param
source: Strandedness
out:
- id: output_param_text
tool_state:
# [elided] the value_map repeat: 'unstranded'->0, 'stranded - forward'->1,
# 'stranded - reverse'->2
output_param_type: text
unmapped:
on_unmapped: fail
__current_case__: 1
- id: _unlabeled_step_21
tool_id: toolshed.g2.bx.psu.edu/repos/iuc/featurecounts/featurecounts/2.1.1+galaxy1
tool_version: 2.1.1+galaxy1
tool_shed_repository:
changeset_revision: 37d067694d40
name: featurecounts
owner: iuc
tool_shed: toolshed.g2.bx.psu.edu
in:
- id: alignment
source: "STAR: map and count and coverage splitted/mapped_reads"
- id: anno|reference_gene_sets
source: GTF file of annotation
- id: strand_specificity
source: Get featureCounts strandedness parameter/output_param_text
- id: when
source: Use featureCounts for generating count tables
out:
- id: output_short # subject interface output 7
hide: true
- id: output_summary # subject interface output 8
hide: true
tool_state:
anno:
anno_select: history # GTF from the history, same dataset as STAR's
__current_case__: 2
reference_gene_sets:
__class__: ConnectedValue
gff_feature_type: exon
gff_feature_attribute: gene_id
summarization_level: false
format: tabdel_short
pe_parameters:
paired_end_status: PE_fragments
__current_case__: 2
exclude_chimerics: true
# [elided] extended_parameters, read_filtering_parameters
when: $(inputs.when)
# --- QC subworkflow, reduced to the per-fastq QC step --------------------
- id: More QC
label: More QC
run:
class: GalaxyWorkflow
label: RNA-seq-QC
inputs:
- id: FASTQ collection
type: collection
collection_type: list # <-- FLAT, because of __FLATTEN__ upstream
optional: false
outputs:
- id: Falco text output
outputSource: _unlabeled_step_3/text_file
steps:
# Falco is a drop-in FastQC reimplementation; same single-dataset input,
# same html_file + text_file output pair the subject promotes as outputs 1-2.
- id: _unlabeled_step_3
tool_id: toolshed.g2.bx.psu.edu/repos/iuc/falco/falco/1.3.2+galaxy0
tool_version: 1.3.2+galaxy0
in:
- id: input_file
source: FASTQ collection
out:
- id: html_file
hide: true
- id: text_file
hide: true
# [elided] gtftobed12, samtools view/idxstats, Picard MarkDuplicates,
# RSeQC read_distribution and geneBody_coverage, and their three other inputs
in:
- id: FASTQ collection
source: _unlabeled_step_11 # the FLATTEN output
- id: STAR BAM
source: "STAR: map and count and coverage splitted/mapped_reads"
- id: reference_annotation_gtf
source: GTF file of annotation
- id: when
source: Generate additional QC reports
when: $(inputs.when)
---
# ============================================================================
# DOCUMENT 3 — TOOL-LEVEL EVIDENCE ONLY (cross-domain, NOT a domain exemplar)
#
# IWC workflow ID: epigenetics/cutandrun/cutandrun
# Steps covered: the Cutadapt step only
#
# Cited for one fact the subject needs and the transcriptomics exemplars cannot
# supply, because neither uses Cutadapt: the output shape of the IUC Cutadapt
# wrapper when mapped over a list:paired collection. This settles
# `trimmed-reads-paired-reassembly-conditional`. CUT&RUN is a different domain
# and this document must not be read as a structural exemplar for anything else.
# ============================================================================
class: GalaxyWorkflow
label: CUT&RUN / CUT&TAG analysis
inputs:
- id: PE fastq input
type: collection
collection_type: list:paired
optional: false
steps:
- id: Cutadapt (remove adapter + bad quality bases)
label: Cutadapt (remove adapter + bad quality bases)
tool_id: toolshed.g2.bx.psu.edu/repos/lparsons/cutadapt/cutadapt/5.2+galaxy2
tool_version: 5.2+galaxy2
in:
- id: library|input_1
source: PE fastq input
# [elided] library|r1|adapters_0|... and library|r2|adapters2_0|... adapter
# sequences, wired from two text parameter inputs
out:
# ONE paired collection out, shape `input` (i.e. same as the input collection),
# plus one report per element. No re-pair node is needed downstream.
- id: out_pairs
- id: report
rename: cutadapt report
tool_state:
library:
type: paired_collection
__current_case__: 2
input_1:
__class__: ConnectedValue
pair_adapters: false
# [elided] r1 / r2 adapter repeats
Carried obligations ledger re-emitted by this step: entries it appended or closed updated, every other entry passed through with its provenance intact.
# open-requirements-ledger — run auris-scf1 (paper-to-galaxy)
# Started empty by freeform-summary-to-galaxy-interface, the first Mold of this run to carry it.
entries:
- id: featurecounts-annotation-source-unnamed
status: resolved
raised_by: freeform-summary-to-galaxy-interface
unmet: "gene annotation for workflow input `Gene annotation GTF`"
missing: >-
The paper names the genome assembly (GCA_002759435.2, C. auris B8441) but never names a
GTF/GFF for it. featureCounts requires one and RNA STAR uses one for splice junctions.
NCBI RefSeq GFF and FungiDB B8441 GFF differ in gene ID space and attribute keys
(`gene_id` vs `ID`), which changes the featureCounts `-g` attribute, every downstream gene
identifier, and whether SCF1 appears as `B9J08_001458` at all. FungiDB and CGOB are cited in
the paper only for synteny inspection, not as the counting annotation. The interface fixes
the datatype as `gtf`; a GFF3 source would require a different datatype or a conversion step.
resolved_by: paper-to-test-data
supersedes: null
note: >-
Largest single gap for reproducing this analysis. Whoever picks a source must record which
one, because count values and gene IDs are not comparable across the two.
CARRIED FORWARD by advance-galaxy-draft-step at `Count reads per gene`, which is now
concrete and STILL DOES NOT CLOSE THIS. Both consumers are wired to the declared
`Gene annotation GTF` input (RNA STAR `sjdbGTFfile`, featureCounts
`anno|reference_gene_sets` under `anno_select: history`, case 2), so the PORTS are
settled. What this step adds to the entry is a second dependent binding:
`gff_feature_attribute: gene_id` (featureCounts `-g`), taken from the wrapper default and
from corpus transcriptomics/rnaseq-pe/rnaseq-pe at fe41a79. That is correct for a GTF and
wrong for a FungiDB B8441 GFF3, which keys on `ID` — and the failure mode is a complete,
plausible counts table of the WRONG identifiers, not an error. `gff_feature_type: exon`
has the same shape of exposure. So this entry now gates three things, not one: the input's
datatype, the `-g` attribute, and the gene ID space every downstream result is addressed
in. Naming the source settles all three at once; nothing else will.
CLOSED by paper-to-test-data. The annotation is NCBI's own GTF for the exact accession the
paper names: `GCA_002759435.2_Cand_auris_B8441_V2_genomic.gtf.gz` under
`https://ftp.ncbi.nlm.nih.gov/genomes/all/GCA/002/759/435/GCA_002759435.2_Cand_auris_B8441_V2/`
(md5 6e5b9528d48c0a8fc2c8588e6eeea929). It was downloaded and inspected, not merely cited.
It settles all three things this entry gated, in the direction the workflow already assumes:
it is a true GTF (`#gtf-version 2.2`), so the input's `gtf` datatype needs no conversion
step; its `gene_id` values ARE the paper's locus tags, so `gff_feature_attribute: gene_id`
is correct as bound and SCF1 resolves as `B9J08_001458` with no identifier translation
(PEKT02000003.1:864995-867292, + strand, single exon, 2298 bp); and it carries 6057 `exon`
features across 5586 distinct genes, so `gff_feature_type: exon` is correct too. The
FungiDB-GFF3 hazard this entry described is avoided by not using FungiDB — nothing about
that hazard was wrong, it simply does not arise for this source.
- id: rnaseq-strandedness-inferred-from-kit-name
status: resolved
raised_by: freeform-summary-to-galaxy-interface
unmet: "value for workflow parameter `featureCounts strandedness`"
missing: >-
The paper states only the library kit (Illumina Stranded Total RNA Prep with Ribo-Zero Plus).
Reverse-stranded (dUTP) is inferred from that kit name and is not stated anywhere in the
supplement. The interface exposes the parameter with default `reverse` so the inference is
visible and changeable rather than buried in a step default.
resolved_by: paper-to-test-data
supersedes: null
note: >-
Empirically checkable without new information: the promoted output
`featureCounts assignment summary` shows a large Unassigned_NoFeatures fraction when the
setting is wrong. A test-plan or run phase can close this from evidence.
CLOSED by paper-to-test-data, empirically rather than by argument. The entry itself named
the check; it was performed a step earlier than expected, on alignments rather than on a
featureCounts summary. 200,000 read pairs from each of AR0382_A, AR0387_A and tnSWI1_A were
aligned to GCA_002759435.2 with bwa-mem and each R1 that unambiguously overlapped a single
annotated gene was compared against that gene's strand. 98.4% / 98.4% / 98.2% map ANTISENSE.
That is the dUTP reverse-stranded signature and it is not a close call. `stranded - reverse`
→ featureCounts `-s 2` is correct; the value is now measured, not inferred from the kit
name. The `featureCounts assignment summary` check this entry proposed remains valid as a
regression guard and is carried into the test plan as an assertion.
- id: sample-sheet-condition-to-deseq2-factor-wiring
status: resolved
raised_by: freeform-summary-to-galaxy-interface
unmet: "path from per-sample `condition` metadata to DESeq2's factor-level inputs"
missing: >-
The interface carries condition and replicate as `column_definitions` on a
`sample_sheet:paired` reads input. Galaxy does not propagate `column_definitions` or per-row
`columns` through map-over, so by the time featureCounts has produced counts the condition
metadata is gone and an explicit step must reattach it
(`__SAMPLE_SHEET_TO_TABULAR__` plus a filter/split, or the rules DSL) before DESeq2 can
receive one multi-data input per factor level.
resolved_by: freeform-summary-to-galaxy-data-flow
supersedes: null
note: >-
Named fallback if the wiring proves unbuildable: replace the sample-sheet input with a
`list:paired` reads collection plus a `data` input `Sample metadata table`
(tabular: sample_id, condition, replicate) and split on that table. Taking the fallback
changes workflow input 1 and must be reflected back into the interface brief, not applied
silently in the template.
Closed by the data-flow brief, section 4. The condition reaches DESeq2 by an
identifier-keyed split taken off the workflow input, where the column metadata still lives:
`__SAMPLE_SHEET_TO_TABULAR__` projects the sample sheet to a tabular (element identifier,
condition, replicate); a row filter plus column projection yields one identifier list per
factor level; `__FILTER_FROM_FILE__` filters the featureCounts collection to each level's
identifiers; each per-level counts sub-collection reduces into one DESeq2 multi-data factor
port. The map-over region and every promoted per-sample output are untouched, and the join
key is the element identifier, which Galaxy preserves across map-over. The route is
independent of how the DESeq2 contrasts are realized
(`deseq2-contrast-realization-unsettled`) and survives the named fallback intact: under the
fallback the tabular node simply disappears and the user-supplied metadata table lands in
its place, with the filter and collection-split nodes unchanged. Two narrower successors
carry what remains: `sample-sheet-to-tabular-identifier-column-unverified` (is the element
identifier emitted as a column) and `deseq2-factor-level-names-not-parameterized` (where the
per-level literal comes from). Rejected alternatives and why are recorded in the brief's
section 4.6 — notably filtering by element-identifier regex, which is unsafe here because
`AR0382_tnSWI1_A` contains the reference level's own identifier as a substring.
- id: sample-sheet-input-test-fixture-expressibility
status: resolved
raised_by: freeform-summary-to-galaxy-interface
unmet: "test-fixture form for the `sample_sheet:paired` workflow input"
missing: >-
Whether a `sample_sheet`-family workflow input — element identifiers plus per-row typed
`columns` and collection-level `column_definitions` — can be expressed in a Planemo/IWC
`-tests.yml` job block. If it cannot, the workflow's primary input is untestable as designed
and the fallback in `sample-sheet-condition-to-deseq2-factor-wiring` becomes mandatory.
resolved_by: paper-to-test-data
supersedes: null
note: >-
Not verified in this phase; no fixture syntax for sample-sheet inputs appears in the
references packaged with this Mold. Naturally closed by the test-plan phase.
CLOSED by paper-to-test-data: YES, it is expressible, and the `list:paired` fallback is not
required. Galaxy's job-block loader has an explicit branch for it —
`lib/galaxy/tool_util/cwl/util.py`, `replacement_collection()`:
`if collection_type.startswith("sample_sheet"): kwds["rows"] = value.get("rows")`, carried
to the collections API by `lib/galaxy/tool_util/client/staging.py`. The nested
`sample_sheet:paired` shape specifically is covered by
`test/unit/tool_util/test_cwl_util.py::test_galactic_job_json_sample_sheet_paired_collection`,
and an end-to-end worked example ships as
`lib/galaxy_test/workflow/collection_semantics_cat_sample_sheet.gxwf-tests.yml`.
Two things a test author must get right. (1) `rows` is a mapping of element identifier to a
POSITIONAL LIST of column values ordered to match `column_definitions` — authority is
`validate_row()` in
`lib/galaxy/model/dataset_collections/types/sample_sheet_util.py`, which rejects on
`len(row) != len(column_definitions)` then `zip(row, column_definitions)`. The dict form
appearing in Galaxy's own non-paired unit tests never reaches that validator and will not
work. (2) Neither `galactic_job_json` nor `staging.py` passes `column_definitions` when
creating the collection, although the API payload supports it, so a test-staged sample sheet
has collection-level `column_definitions: None`. This is NOT fatal: per-element `columns`
are still populated from `rows`, and the two `column_definitions_compatible()` call sites
are both in `DataCollectionToolParameter` option-building (UI dropdown filtering), which a
test bypasses by supplying the HDCA by id. The one real consequence is that
`__SAMPLE_SHEET_TO_TABULAR__` emits its header line only
`#if $include_headers and $input.collection.column_definitions` — harmless here because
`Project sample sheet to tabular` sets `include_headers: false`, but a later phase that
flips it to true will see the header silently vanish under test while it appears in the UI.
Concrete job block is in `test-data-refs.json` under `planemo_test_job_block`.
- id: deseq2-contrast-realization-unsettled
status: resolved
raised_by: freeform-summary-to-galaxy-interface
unmet: "step realization behind the two contrast outputs"
missing: >-
The paper reports two contrasts (tnSWI1 vs AR0382, AR0387 vs AR0382) and states the
significance thresholds, but writes no design formula. One factor, three levels, n = 2 is
inferred from the six deposited runs. Whether the two contrasts come from one DESeq2 run over
a three-level factor or two runs over two-level factors depends on the chosen wrapper's
contrast handling, which is not known at interface time.
resolved_by: compare-against-iwc-exemplar
supersedes: null
note: >-
The interface fixes only the output surface — two result tables and two filtered tables,
labelled per contrast. Either realization satisfies it.
Closed by the IWC exemplar comparison, section 3.2, against
transcriptomics/rnaseq-de/rnaseq-de-filtering-plotting at corpus fe41a79. Two DESeq2
nodes, each with a two-level factor, the reference-level counts collection feeding both:
H1 (AR0382, tnSWI1) and H2 (AR0382, AR0387). Evidence: the exemplar's DESeq2 step uses
`select_data.how: datasets_per_level` with a `rep_factorName_0.rep_factorLevel` repeat of
exactly two levels, each port consuming a whole collection through the wrapper's
multiple=true input; its `deseq_out` is a single dataset, not a collection, proved by the
single-dataset tools it feeds (deg_annotate, tp_cat, two Filter1 steps) and by the sibling
-tests.yml asserting has_text_matching on the derived output as a dataset; and the
workflow README bounds itself to "exactly 2 conditions with at least 2 replicates per
condition". One DESeq2 job therefore yields one results table, so the interface's two
distinctly labelled result tables require two jobs. A three-level factor is expressible
via the repeat but would still emit one `deseq_out`, which cannot satisfy a two-output
interface — so the output surface settles the arity regardless of the wrapper's
three-level contrast behaviour. Upstream wiring is unaffected, exactly as the data-flow
brief's section 4.4 predicted: nodes E, F1-F3 and G1-G3 are identical either way. One
consequence for the interface: under two DESeq2 nodes, `DESeq2 normalized counts`
(output 9) and `DESeq2 diagnostic plots` (output 14) are each produced twice while the
interface declares one of each. Promote from one designated run and say which, or relabel
per contrast; do not leave it implicit in the template.
- id: reference-genome-delivery-shape-unverified
status: open
raised_by: freeform-summary-to-galaxy-interface
unmet: "verified delivery shape for the C. auris B8441 reference genome"
missing: >-
The interface settles the genome as a history `fasta` dataset on portability grounds (a
remote-URL fixture resolves on any server; a CVMFS built-in index does not), with RNA STAR
building its index at run time. Whether usegalaxy.org carries a built-in index for
GCA_002759435.2 was not checked, and this run's phase roster contains no reference-data Mold
that owns the question.
resolved_by: null
supersedes: null
note: >-
Provisionally settled, not verified. A built-in index would be cheaper at run time but less
portable for tests; revisiting it changes workflow input 2.
- id: cutadapt-adapter-and-length-filter-unstated
status: open
raised_by: freeform-summary-to-galaxy-interface
unmet: "Cutadapt adapter sequence and minimum-length filter"
missing: >-
The paper gives only "Cutadapt with a Phred cutoff score of 20". No adapter sequence appears
anywhere in the supplement, and no minimum-length filter is stated. The interface reads the
step as quality trimming only (`-q 20`, baked in) and exposes no adapter input.
resolved_by: null
supersedes: null
note: >-
If a later phase adds adapter trimming it is adding method the paper does not describe, and
that addition has to be labelled as such rather than presented as a faithful port.
STILL OPEN after phase 6 iteration 8 concretized `Quality-trim reads`, and deliberately so.
The step pins lparsons/cutadapt 5.2+galaxy2 with all six adapter repeats written explicitly
empty (adapters / front_adapters / anywhere_adapters on R1, the adapters2 / front_adapters2 /
anywhere_adapters2 trio on R2) and `other_trimming_options.quality_cutoff: '20'`, which is
the one Cutadapt parameter the paper actually states. The unstated minimum-length filter is
now recorded concretely: the pinned wrapper's default is `filter_options.minimum_length: 1`,
which the XML flags as a deliberate wrapper-side departure from cutadapt's own default of 0
("intentionally set to 1 ... to avoid hard to debug issues with downstream tools"). The step
writes that 1 out explicitly so a later wrapper bump cannot move it silently -- but it is the
WRAPPER's default, not the paper's parameter, and the corpus shows the value is genuinely
chosen per workflow (cutandrun at fe41a79 uses 15, the VGP workflows use 1). Closing this
entry requires a stated adapter and a stated length cut, neither of which exists in the
source.
- id: galaxy-tool-versions-unpinnable-from-source
status: open
raised_by: freeform-summary-to-galaxy-interface
unmet: "tool versions matching the published analysis"
missing: >-
The paper gives no version for any Galaxy step (FastQC, Cutadapt, RNA STAR, featureCounts,
DESeq2). Only non-Galaxy software is versioned (R 4.0.3, DescTools 0.99.49, survminer 0.4.9,
Fiji 1.52, CellProfiler 3.1.9). The constructed workflow will pin its own versions and can
reproduce the authors' method but never their software stack.
resolved_by: null
supersedes: null
note: >-
Unclosable from the source. Expected to be surrendered at the terminal and stated on the
workflow itself, so a reader does not mistake the run for a version-faithful reproduction.
- id: tnbcy1-contrast-not-carried
status: open
kind: dropped
raised_by: freeform-summary-to-galaxy-interface
units: "the tnBCY1 (B9J08_002818) vs AR0382 transcriptome comparison reported in Fig. S2"
because: >-
No tnBCY1 runs exist in BioProject PRJNA904261. Only three conditions were deposited
(AR0382, AR0387, AR0382 tnSWI1), across six runs SRR22376027–SRR22376032.
unmet: "one of the three transcriptome comparisons the paper reports"
missing: >-
The workflow can build only the two contrasts whose input data is public. A test that tries
to reproduce Fig. S2 has no input.
resolved_by: null
supersedes: null
note: >-
Cut by data availability, not by a design decision. Adding a fourth condition later needs
no interface change: the reads input is a sample sheet whose `condition` restrictions widen.
- id: pipeline-b-tdna-mapping-not-carried
status: open
kind: dropped
raised_by: freeform-summary-to-galaxy-interface
units: >-
the whole of Pipeline B — AtMT T-DNA insertion-site mapping: FastQC, Trimmomatic,
BWA-MEM against linearized pTO128 (seed 50, band width 2), extractSoftClipped,
BWA-MEM of the soft-clipped flanks against C. auris B8441 (5 computational steps,
2 deposited runs SRR22376033–SRR22376034)
because: >-
Two blockers, both recorded in the source summary's open questions (4 and 5) and neither
independently verified in this phase. (a) `extractSoftClipped` from SE-MEI
(github.com/dpryan79/SE-MEI) has no known Galaxy Tool Shed wrapper, so the pipeline cannot be
assembled from stock tools as written; substituting a samtools/awk soft-clip extraction would
change the method. (b) The pTO128 (pPZP-NAT) plasmid reference has no public accession in the
paper — it is cited to the prior AtMT method paper (ref. 52) — so step 3's reference is
unresolvable from the publication alone.
unmet: "the second of the paper's two sequencing analyses"
missing: >-
No T-DNA integration sites are produced by this run. The scope decision was taken by the
harness before this phase.
resolved_by: null
supersedes: null
note: >-
Uncited in the strict sense: no Tool Shed search for extractSoftClipped and no Addgene or
ref. 52 lookup for pTO128 was run in this phase; both reasons are carried from the source
summary. Treat as a debt, not a finding. If a later phase's discovery step contradicts either
reason, this entry's `because` is refuted and the entry must be reopened rather than left
standing.
- id: sample-sheet-to-tabular-identifier-column-unverified
status: resolved
raised_by: freeform-summary-to-galaxy-data-flow
step: sample_metadata_table
unmet: "confirmation that the sample-sheet-to-tabular bridge emits the element identifier"
missing: >-
The settled condition wiring joins the sample metadata table to the featureCounts collection
on element identifier, so node E's output must carry that identifier as a column. The
packaged note galaxy-sample-sheet-collections documents `__SAMPLE_SHEET_TO_TABULAR__` only as
iterating elements and tab-joining "for downstream tabular consumers"; it does not state the
output column set or ordering, and no other packaged reference covers it. Galaxy source was
not readable from inside this run.
resolved_by: advance-galaxy-draft-step
supersedes: null
note: >-
RESOLVED, affirmatively, by the tool summary itself. `__SAMPLE_SHEET_TO_TABULAR__` v1.0.0
caches cleanly from the Tool Shed API by bare id (unlike `__FLATTEN__`, whose collection
output defeats gxwf's summary decoder), and its packaged help states the contract
directly: "The first column is always the element identifier (sample name). The remaining
columns match the metadata fields defined in the sample sheet." With the optional
`include_headers` enabled the first header cell is literally `element_identifier`. So the
identifier IS emitted, as column 1, and the ordering this region assumed throughout --
(element identifier, condition, replicate), metadata in column_definitions order -- is
confirmed. All six provisional column bindings downstream (three Filter1 predicates on c2,
three Cut1 projections of c1) stand unchanged; the Apply Rules substitute node named in
the data-flow brief section 4.2 is not needed and was not built. The step is pinned with
`include_headers: false`, which also settles the three row filters' `header_lines` at 0 --
that binding is no longer blocked on this entry. Evidence is the wrapper's own documented
contract, not a corpus exemplar; `no-iwc-precedent-for-sample-sheet-workflow-input` is
unaffected and stays open.
- id: deseq2-factor-level-names-not-parameterized
status: resolved
raised_by: freeform-summary-to-galaxy-data-flow
step: select_level_L
unmet: "a source for the two non-reference condition level literals"
missing: >-
The condition split needs one literal condition value per factor level to filter the sample
metadata table (AR0382, AR0387, tnSWI1). The interface exposes only `Reference condition
level` (input 4, default AR0382). The two contrast levels have no parameter, so as the
interface stands they would be baked into two filter steps — re-hard-coding into the steps
the three-level design that the sample-sheet input was chosen to keep out of the public
interface.
resolved_by: freeform-summary-to-galaxy-template
supersedes: null
note: >-
Two ways to settle it, both changing the interface's parameter surface rather than the
topology: expose two more text parameters (one per contrast level) alongside the existing
reference-level parameter, or accept the bake and state plainly in the interface that the
three level names are fixed in the steps. Whichever is chosen must be reflected back into
freeform-galaxy-interface.md section 2.3, not applied silently in the template.
Closed by the template on the first of those two options. The draft exposes two further
`text` workflow inputs, `First contrast condition level` (default tnSWI1) and
`Second contrast condition level` (default AR0387), alongside the existing
`Reference condition level` (default AR0382). Each feeds a `compose_text_param` step that
builds the row predicate for its level, so no level literal is baked into any step.
The deciding argument is symmetry: the reference level was already a parameter for exactly
this reason, and baking the other two would have left the interface half-generalized while
re-hard-coding into the steps the design the sample-sheet input was chosen to keep out of
the interface. The cost is two inputs the phase-2 brief does not declare, which is one facet
of the interface drift recorded in
`interface-brief-output-and-parameter-surface-drifted-from-draft`; that entry carries the
owed edit to freeform-galaxy-interface.md section 2.3, since this Mold does not own that
artifact. One caveat the draft states on both new inputs: the contrast output labels
(`... tnSWI1 vs AR0382`, `... AR0387 vs AR0382`) are the public API and are not derived
from these parameters, so changing a level value makes the labels stale.
- id: fold-change-threshold-linear-vs-deseq2-log2fc
status: resolved
raised_by: freeform-summary-to-galaxy-data-flow
step: filter_significant
unmet: "unit agreement between workflow input 7 and the DESeq2 result column it thresholds"
missing: >-
Workflow input 7 is `Minimum absolute fold change`, default 2.0, in linear units — the form
the paper states (|fold change| > 2). DESeq2 reports log2FoldChange. The significance filter
must therefore compare abs(log2FoldChange) > log2(threshold), which needs either a conversion
the filter expression may not support or a restatement of the parameter. Nothing in the
interface brief notes the mismatch.
resolved_by: compare-against-iwc-exemplar
supersedes: null
note: >-
Consequential if missed rather than merely untidy: comparing abs(log2FoldChange) > 2.0
applies a 4-fold cut and silently fails to reproduce the paper's gene lists, while still
producing a plausible-looking filtered table. Options are a log2 conversion node before the
filter, a filter expression that computes the conversion inline, or restating input 7 as a
log2 threshold (default 1.0) with its label changed — the last changes the interface.
Closed by the IWC exemplar comparison, section 2, on the third of those three options.
transcriptomics/rnaseq-de/rnaseq-de-filtering-plotting at corpus fe41a79 exposes
`log2 fold change threshold` as a float workflow input, default 1.0, documented as
"A log2 FC of 3 equals to an absolute fold change of 8 (2^3)", and filters with the raw
column and no conversion anywhere in the workflow: `abs(c3)>` concatenated with the
connected float. There is no linear fold-change parameter, no log2() call and no
conversion node in the corpus exemplar. So: restate interface input 7 as a log2 threshold
with default 1.0 — which is exactly the paper's |fold change| > 2 — and change its label;
teach the conversion in the doc string as the corpus does. Three implementation details
come with the answer. (a) Column indices are c3 for log2FC and c7 for adjusted p-value;
deg_annotate appends columns 8-13 and leaves 1-7 untouched, so the indices hold whether or
not an annotation step is added. (b) Filter1's predicate is a text parameter, so a numeric
workflow input cannot reach it directly: the corpus idiom is one iuc/compose_text_param
step per threshold concatenating a literal prefix (`c7<`, `abs(c3)>`) with the connected
float, making node I two Galaxy steps per filter rather than one. (c) The two thresholds
are two chained Filter1 steps, p-adj then log2FC, each with header_lines "1", not one
compound predicate. Restating input 7 changes the interface brief's section 2.3 and its
output-3 table entry; that edit must be made there, not silently in the template.
- id: fastqc-per-read-fanout-not-a-flat-list
status: resolved
raised_by: freeform-summary-to-galaxy-data-flow
step: qc_raw_reads
unmet: "agreement between the declared shape of outputs 1-2 and what map-over actually produces"
missing: >-
The interface declares `FastQC raw reads: text summary` and `: HTML report` as `list`
collections. FastQC consumes a single dataset, so mapping it over a `sample_sheet:paired`
input fans out over the inner paired axis as well — 12 jobs, and outputs nested one report
per read direction per sample, not six flat elements. As declared, outputs 1 and 2 are not
what the workflow computes.
resolved_by: compare-against-iwc-exemplar
supersedes: null
note: >-
The data-flow brief (section 7.1) recommends promoting the nested collection and correcting
the interface: per-read-direction QC is what a reader wants from raw-read QC, and it
preserves the element identifier space that every checkpoint assertion keys on. The
alternative is an explicit flatten node, which satisfies the declared `list` but rewrites
identifiers to a doubled vocabulary (`AR0382_A_forward`) for no analytical gain. Either way
the test plan must know which, because it changes every assertion on outputs 1 and 2.
Closed by the IWC exemplar comparison, section 3.3, in favour of the explicit flatten —
the option the data-flow brief rated as having no analytical gain. The brief's preference
was a reasonable call made with no corpus evidence available; the evidence now exists and
points the other way. transcriptomics/rnaseq-pe/rnaseq-pe at corpus fe41a79 puts an
explicit `__FLATTEN__` step with `join_identifier: _` between the list:paired reads input
and the per-fastq QC tool, and the consuming QC subworkflow declares its input as
`collection_type: list` — flat. This is published IWC convention rather than a workaround,
and it satisfies the interface's declared `list` shape for outputs 1 and 2 as originally
written, so no interface correction is needed. Consequences the test plan must key on: the
identifier vocabulary on outputs 1 and 2 doubles to twelve (AR0382_A_forward,
AR0382_A_reverse, and so on for each of the six samples), while outputs 3-8 keep the
six-element sample identifier space untouched, because the flatten is a side branch off the
workflow input and does not enter the map-over region. The data-flow brief's placeholder
transformation 5.5 is therefore required, not conditional.
- id: trimmed-reads-paired-reassembly-conditional
status: resolved
raised_by: freeform-summary-to-galaxy-data-flow
step: trim_reads
unmet: "the output collection shape of the paired-aware trimming step"
missing: >-
The interface declares `Trimmed reads` as `list:paired`. Whether the trimmer emits one
paired-inner collection per sample or two parallel single-ended collections (R1, R2) is
wrapper-dependent and was not resolvable in this phase, which pins no Tool Shed tools. If it
is the latter, the design needs a re-pair node between trimming and alignment, or the
alignment step must take two parallel collections in dot-product.
resolved_by: compare-against-iwc-exemplar
supersedes: null
note: >-
Conditional shape repair, not method: recorded in the data-flow brief as placeholder
transformation 5.6 so the template does not assume one reading. Naturally closed by the IWC
exemplar comparison or by tool discovery on the trimmer.
Closed by the IWC exemplar comparison, section 3.4: one paired-inner collection per sample,
no re-pair node. Neither transcriptomics exemplar uses Cutadapt — both use fastp — so the
evidence comes from epigenetics/cutandrun/cutandrun at corpus fe41a79, cited for the
wrapper's IO shape only and for nothing else, since CUT&RUN is a different domain.
toolshed.g2.bx.psu.edu/repos/lparsons/cutadapt/cutadapt/5.2+galaxy2 driven from a
list:paired collection with `library.type: paired_collection` declares outputs `out_pairs`
(type `input`, i.e. the input collection's own shape) and `report` — one paired collection
plus one report per element. Placeholder transformation 5.6 is therefore not needed and
node C takes a single collection input, as the data-flow brief's primary reading assumed.
The finding is consistent across both wrappers that could fill the trimmer slot:
transcriptomics/rnaseq-pe/rnaseq-pe's fastp emits `output_paired_coll` the same way. The
residual is version-scoped rather than structural — a future Cutadapt wrapper could change
its output set, so the per-step loop should confirm the output name against whatever
version it pins.
- id: mapped-outputs-carry-sample-sheet-outer-axis
status: open
raised_by: freeform-summary-to-galaxy-data-flow
unmet: "accurate declared collection types for the promoted per-sample outputs"
missing: >-
A tool mapped over a `sample_sheet`-family collection produces a `sample_sheet`-shaped output
without `column_definitions` (packaged note galaxy-sample-sheet-collections, "Mapping
rules"). The interface's output table calls outputs 1-8 `list` / `list:paired`. Behaviourally
that is accurate — such a collection maps, reduces and filters exactly like a list — but the
declared collection type string is not `list`, which matters to anything that type-checks,
including workflow test assertions on collection type.
resolved_by: null
supersedes: null
note: >-
No topology consequence; the fix is either a corrected type column in the interface brief or
an explicit statement that the promoted outputs are sample_sheet-shaped lists without column
metadata. Flagged primarily for the test-plan phase, which is where a wrong collection_type
assertion would surface as a confusing failure.
ADVANCED, NOT CLOSED, by implement-galaxy-workflow-test. The test file authors no `attributes:
{collection_type: ...}` assertion on any of the eight mapped outputs, as the test plan's `promoted-
collection-type-string-unverified` instructs. What it asserts instead is `element_count` plus a named
`element_tests` entry per identifier - twelve for the flattened FastQC outputs, six for every output on
the sample axis - which is shape-agnostic and pins the thing that actually matters, the identifier space
the condition split joins on. So no assertion in this run depends on the declared type string and nothing
here forces the question. It stays open as an interface-brief accuracy item, and the real type strings
should be read off the first successful invocation before anyone decides whether a collection_type
assertion is worth having.
- id: iwc-splits-rnaseq-de-at-the-count-table-boundary
status: open
raised_by: compare-against-iwc-exemplar
unmet: "agreement between this run's workflow scope and published IWC practice"
missing: >-
IWC has no single workflow spanning FastQC to DESeq2. At corpus fe41a79 the journey is
published as two workflows joined at the count-table boundary:
transcriptomics/rnaseq-pe/rnaseq-pe ends at per-sample count tables, and
transcriptomics/rnaseq-de/rnaseq-de-filtering-plotting begins there, taking pre-grouped
`list` collections of count tables as its workflow inputs. This run's design is one
workflow spanning both halves, which is why it needs an in-workflow condition split at all
— a workflow that starts from count tables can demand pre-grouped collections at its
interface and never reconstruct grouping internally.
resolved_by: null
supersedes: null
note: >-
Not a defect in either design, and not a reason to re-scope: the paper describes one
analysis and the run was asked to build it. Recorded because it is the root of the single
largest divergence from corpus practice (see
`no-iwc-precedent-for-sample-sheet-workflow-input`) and because any later
mature-galaxy-workflow-for-iwc pass will meet it as a packaging question — a reviewer may
ask why this is not two workflows. Whoever revisits it should weigh that the split also
costs reusability of the DE half, which is presumably why IWC made it.
- id: no-iwc-precedent-for-sample-sheet-workflow-input
status: open
raised_by: compare-against-iwc-exemplar
step: sample_metadata_table
unmet: "a worked corpus precedent for the sample-sheet condition split"
missing: >-
Searched across all of `workflows/` at corpus fe41a79: `sample_sheet` as a collection type
appears in zero workflows; `__SAMPLE_SHEET_TO_TABULAR__` appears in zero workflows;
`column_definitions` appears only as the literal `null` that newer Galaxy serializes onto
ordinary collection inputs, never with a value. `__FILTER_FROM_FILE__` does appear, in six
workflows, but none in transcriptomics and none for a condition split. So the second half
of the data-flow brief's section 4.2 route is a real used Galaxy idiom while the
sample-sheet half is unprecedented in published IWC practice, and no corpus fixture
declares a sample-sheet collection either.
resolved_by: null
supersedes: null
note: >-
Absence of precedent is not refutation, and nothing in the corpus contradicts the route —
the phase-3 reasoning stands on its own. What changes is the risk profile: this is the one
region of the workflow with no worked example to pattern-match against, and two open
entries ride on it (`sample-sheet-to-tabular-identifier-column-unverified`,
`sample-sheet-input-test-fixture-expressibility`). Guidance for the template, from the
comparison notes section 3.1: build the split as a clearly delimited region with the
phase-2 fallback (a `data` input `Sample metadata table`) reachable by deleting one node,
so that if either open entry resolves against the sample sheet the cost is a deletion
rather than a redesign. Worth knowing for the record that IWC made the opposite trade
deliberately — it pushes grouping into the interface as two pre-grouped count collections,
which is the same shape the interface brief's section 2.1 rejected for hard-coding the
design into the public API.
- id: star-history-reference-wiring-has-no-corpus-precedent
status: resolved
raised_by: compare-against-iwc-exemplar
step: align_reads
unmet: "worked wiring for the RNA STAR history-reference conditional branch"
missing: >-
Every RNA STAR step in the corpus at fe41a79 — there are exactly two, in
transcriptomics/rnaseq-pe/rnaseq-pe and transcriptomics/rnaseq-sr/rnaseq-sr — uses
`refGenomeSource.geneSource: indexed`, selecting a built-in `genomeDir` through a
`restrictOnConnections: true` string parameter with `sjdbGTFfile` supplied from the
history. Their test jobs pass a plain genome string (`Reference genome: sacCer3`). The
interface settles input 2 as a history FASTA, which is the right call for C. auris B8441
since no public server indexes it, but that is a different `__current_case__` in the
iuc/rgrnastar wrapper with a different set of required sub-parameters, and the corpus has
no example of it.
resolved_by: advance-galaxy-draft-step
supersedes: null
note: >-
RESOLVED, affirmatively, from the wrapper rather than from the corpus — which is what the
original note asked for. iuc/rgrnastar/rna_star @ 2.7.11b+galaxy1 caches and summarizes
cleanly (all ten of its outputs are plain `data`, so the collection-output decode failure
that blocks `__FLATTEN__` and lparsons/cutadapt does not apply here), and its schema
answers every part of the question. `refGenomeSource.geneSource` publishes exactly two
options, `indexed` and `history`; the HISTORY branch is real, is `__current_case__: 1`, and
carries `genomeFastaFiles` (gx_data, formats fasta/fasta.gz, optional false),
`genomeSAindexNbases` (integer, min 2 max 16, default 14), its own TWO-case
`GTFconditional`, and `diploidconditional` (Yes=0 / No=1, default No). So the two TODO
ports resolve to `refGenomeSource|genomeFastaFiles` and
`refGenomeSource|GTFconditional|sjdbGTFfile`; the exemplar's
`refGenomeSource|GTFconditional|genomeDir` exists only under `indexed` and is correctly
absent. Under history, GTFconditional's option order (without-gtf, with-gtf) is the
REVERSE of its `<when>` order (with-gtf, without-gtf), so `with-gtf` is case 0 — a case
index that cannot be read off the dropdown. That the case index follows `<when>` document
order is not assumed: `gxwf convert --to format2` over
transcriptomics/rnaseq-pe/rnaseq-pe.ga at fe41a79 emits `geneSource: indexed` with
`__current_case__: 0` and `GTFselect: without-gtf-with-gtf` with `__current_case__: 1`,
matching the indexed branch's `<when>` order (with-gtf, without-gtf-with-gtf, without-gtf)
and not its option order. Cross-read against rg_rnaStar.xml and macros.xml at tools-iuc
main, which carries @TOOL_VERSION@ 2.7.11b / @VERSION_SUFFIX@ 1 — this exact version.
Two consequences worth carrying: the history branch builds its index in-job via
`STAR --runMode genomeGenerate` into tempstargenomedir, confirming the data-flow brief's
no-separate-index-node call; and `output_log` and `mapped_reads` carry no `<filter>`
whatsoever in `<outputs>`, so the promoted `STAR mapping summary` cannot be suppressed by
any parameter choice here. The step now validates for real against the cache
(`12 ok, 0 fail, 2 skip`), not by skip. `reference-genome-delivery-shape-unverified` is
untouched and STAYS OPEN — this entry establishes that the history route WORKS, never
that it is preferable to a built-in index nobody checked for.
`featurecounts-annotation-source-unnamed` also stays open: the GTF PORT is now wired, the
GTF SOURCE is still unnamed by the paper.
- id: star-genome-length-drives-sa-index-parameter
status: open
raised_by: advance-galaxy-draft-step
step: "Splice-aware alignment"
unmet: "a measured length for the reference actually wired into `Reference genome FASTA`"
missing: >-
`genomeSAindexNbases` exists ONLY in the RNA STAR history branch — it configures the
in-job `--runMode genomeGenerate` that this workflow's history-FASTA delivery choice
introduces, so it is not a mapping parameter the paper's "default parameters" could have
covered. The wrapper's own help carries STAR's formula verbatim: "For small genomes, the
parameter --genomeSAindexNbases must be scaled down to min(14, log2(GenomeLength)/2 - 1)".
The step binds `'10'`, from min(14, log2(12.5e6)/2 - 1) = min(14, 10.79). The 12.5 Mb is
the GCA_002759435.2 assembly record's length as carried through this run's briefs; nothing
in this run measured the dataset that will actually be wired.
resolved_by: null
supersedes: null
note: >-
Silent-degradation class, not a hard gate: the wrapper default of 14 is the documented
seg-fault-at-mapping hazard for a genome this small (the wrapper's own test data uses 5),
and a value too small merely costs search speed. Two ways this becomes wrong: a different
reference is supplied at run time (a mammalian genome would want 14 back), or the B8441
FASTA in hand differs materially in length from the assembly record. Settle it by reading
the length of the actual dataset, or by reading STAR's own recommendation out of the
promoted `STAR mapping summary` / job stderr on the first real run — STAR prints the
recommended value when the supplied one is too large. Tied to
`reference-genome-delivery-shape-unverified`: if that resolves to a built-in index, this
parameter disappears with the branch.
ADVANCED, NOT CLOSED, by paper-to-test-data. The entry asked for a measured length of the
reference actually wired in. The reference this run will wire is now pinned — NCBI
`GCA_002759435.2_Cand_auris_B8441_V2_genomic.fna.gz`, md5 a008b270d3aaa04736a8bb7daf3f6dd5 —
and it was downloaded and measured: 12,365,959 bp across 15 contigs. min(14,
log2(12365959)/2 - 1) = min(14, 10.78), so the bound `'10'` is correct for this dataset.
What keeps the entry open is exactly the exposure it already named: the parameter is
correct for the reference the TEST supplies, and a user who supplies a different genome at
run time still gets a silently wrong value. The first real STAR run should confirm from the
promoted `STAR mapping summary`.
- id: deseq2-normalized-counts-and-plots-promoted-per-contrast
status: resolved
raised_by: freeform-summary-to-galaxy-template
step: "Differential expression: first contrast vs reference"
unmet: >-
a decision on the two DESeq2 outputs the interface declares once and two DESeq2 jobs
produce twice
missing: >-
`deseq2-contrast-realization-unsettled` closed on two DESeq2 nodes, which makes
`DESeq2 normalized counts` (interface output 9) and `DESeq2 diagnostic plots` (interface
output 14) each produced twice while the interface declares one of each. The IWC comparison
flagged the consequence and instructed the template not to leave it implicit: promote from
one designated run and say which, or relabel per contrast.
resolved_by: freeform-summary-to-galaxy-template
supersedes: null
note: >-
Settled by relabelling per contrast. The draft promotes four outputs where the interface
declared two: `DESeq2 normalized counts: tnSWI1 vs AR0382`,
`DESeq2 normalized counts: AR0387 vs AR0382`, `DESeq2 diagnostic plots: tnSWI1 vs AR0382`
and `DESeq2 diagnostic plots: AR0387 vs AR0382`, taking the workflow to sixteen outputs.
This is not a tie broken on taste. Each DESeq2 job sees only its own four samples, so its
size factors, normalized values and PCA/dispersion plots are computed over that subset: the
two normalized-count tables are different tables, not two copies of one. Promoting either
and calling it "the" normalized counts would be wrong rather than merely arbitrary, and
dropping one would discard a real artifact while leaving the surviving one silently
contrast-scoped. Relabelling also makes the four outputs symmetric with the result and
filtered-gene tables, which are already labelled per contrast.
One consequence for the test plan: the paper's sanity check that SCF1 sits in the top 2.5%
of AR0382 expression can be asserted against either normalized-counts table, because both
carry the two AR0382 replicates. Assert it on one and say which.
The owed edit to freeform-galaxy-interface.md section 3 rides on
`interface-brief-output-and-parameter-surface-drifted-from-draft`; labels are the public API
and this is a breaking change to be made once, before any test is written.
- id: interface-brief-output-and-parameter-surface-drifted-from-draft
status: open
raised_by: freeform-summary-to-galaxy-template
unmet: >-
agreement between freeform-galaxy-interface.md and the settled draft's public input and
output surface
missing: >-
Phases 4 and 5 changed the interface in four places that the phase-2 brief still states in
its original form. The brief is the artifact the test plan reads for labels, and labels are
the API, so the drift has to be visible rather than inferred by diffing two artifacts.
(a) Interface input 7 `Minimum absolute fold change` (float, linear, default 2.0) is in the
draft `log2 fold change threshold` (float, log2 units, default 1.0) — closed on corpus
evidence by `fold-change-threshold-linear-vs-deseq2-log2fc`.
(b) Interface input 5 `featureCounts strandedness` (free text, allowed values named only in
prose) is in the draft `Strandedness` (text restricted to `stranded - forward` /
`stranded - reverse` / `unstranded`, default `stranded - reverse`) feeding a
`map_param_value` bridge with `on_unmapped: fail` — the corpus idiom adopted per the IWC
comparison section 3.6.
(c) Two inputs the brief does not declare at all: `First contrast condition level` and
`Second contrast condition level` — see `deseq2-factor-level-names-not-parameterized`.
(d) Fourteen declared outputs become sixteen — see
`deseq2-normalized-counts-and-plots-promoted-per-contrast`.
resolved_by: null
supersedes: null
note: >-
Bookkeeping, not a design gap: every one of the four changes is itself settled and recorded,
and the draft is the current statement of the interface. What is unmet is that no Mold in
this run's roster owns freeform-galaxy-interface.md after phase 2, so the edits have no
writer. Recorded here so the test-plan phase reads labels off the draft rather than off the
stale brief, and so a later maturation pass knows which artifact was authoritative. Closable
by editing the brief's sections 2.3 and 3, or by declaring the draft authoritative for the
interface and saying so in the brief.
- id: deseq2-result-table-header-presence-unverified
status: resolved
raised_by: freeform-summary-to-galaxy-template
step: "Filter with p-adj threshold: first contrast"
unmet: "the `header_lines` binding for the four Filter1 steps of the significance chains"
missing: >-
The corpus exemplar binds `header_lines: "1"` on both of its Filter1 steps, but it filters a
table that has been through `deg_annotate` and then `tp_cat`, and the tp_cat step exists
precisely to concatenate a separately generated single-line header onto the DESeq2 output.
That strongly implies the raw `deseq_out` is headerless. This workflow omits the annotation
pair — the paper names no annotation step and the interface's tool set is closed — so its
Filter1 steps consume `deseq_out` directly and the corpus binding cannot be copied across.
Whether the pinned iuc/deseq2 wrapper writes a header row on `deseq_out` is not established
by anything this run read.
resolved_by: advance-galaxy-draft-step
supersedes: null
note: >-
Consequential if missed rather than untidy, which is why it is a ledger entry and not only a
`_plan_state` line. Binding "1" against a headerless table silently drops the first gene
row of every filtered result; binding "0" against a headed table passes the header line into
a numeric comparison, where Filter1 discards it with a warning rather than an error. Either
way the workflow produces a plausible table. The same binding applies to the three row
filters in the condition-split region, where the unknown is the header behaviour of
`__SAMPLE_SHEET_TO_TABULAR__` rather than of DESeq2 — a different producer, the same class
of silent loss, tracked there by
`sample-sheet-to-tabular-identifier-column-unverified`. Closable by a summarize-galaxy-tool
pass on iuc/deseq2 during the per-step loop, or by inspecting the first real run's output.
SETTLED in the per-step loop, at the step that pins the producer
(`Differential expression: first contrast vs reference`, iuc/deseq2/deseq2 @
2.11.40.8+galaxy4, changeset 05f9e54d7e81): `deseq_out` carries NO header row, so all four
significance filters bind `header_lines: '0'` — the same value the three condition-split row
filters already use, for an unrelated reason. Three independent lines of evidence, all at
the pinned version. (1) The wrapper's own `deseq2.R` writes the result table with
`write.table(out_df, file = opt$outfile, sep = "\t", quote = FALSE, row.names = FALSE,
col.names = FALSE)` — `col.names = FALSE`, at both of the two call sites that write a result
table (the single-contrast path and the `many_contrasts` loop). The same script writes
`counts_out` with `col.names = NA`, so the normalized-counts table DOES carry a header:
one tool, opposite answers for its two tabular outputs, which is exactly why this could not
be settled by analogy. (2) `deseq2.xml`'s test assertions for `deseq_out` match a data row
first (`FBgn0003360\t1933.9504…\t-2.8399…`) with `has_n_lines n="3999"` and assert no
header, while `vst_out` and `counts_out` assertions in the same test block DO assert a
sample-name header line. (3) The corpus chain explains its own "1": rnaseq-de MANUFACTURES
the header it later skips — `tp_text_file_with_recurring_lines` emits the single line
`GeneID__tc__Base mean__tc__log2(FC)…`, `tp_sed_tool` turns `__tc__` into tabs, and `tp_cat`
(labelled `Annotate DESeq2 table`) concatenates it on top of the deg_annotate output; only
then do its two Filter1 steps bind `header_lines: "1"`. That "1" is about the manufactured
header, not about `deseq_out`, which confirms rather than contradicts the reading above.
One rider the four filters inherit: the c3 = log2FC / c7 = padj column contract holds only
while `lfc_shrinkage_type` is `none` — `get_result_output_columns()` drops the `stat` column
under any shrinkage, moving padj to c6. The DESeq2 steps pin `none` explicitly for that
reason.
- id: cross-step-invariants-survive-only-in-the-draft
status: open
raised_by: advance-galaxy-draft-step
step: "Differential expression: first contrast vs reference / second contrast vs reference; the four significance filters"
unmet: "a durable record, in the runnable artifact, of the two cross-step couplings that keep the filter predicates pointing at the right columns"
missing: >-
Two couplings hold this workflow together and neither is expressible in gxformat2 nor
checked by any gate. (1) `advanced_options.lfc_shrinkage_type: none` on BOTH DESeq2 nodes
is what makes `deseq_out` a 7-column table, so `c3` = log2FoldChange and `c7` = padj — the
columns the `compose_text_param` bridges hard-code as `"abs(c3)>"` and `"c7<"` in four
OTHER steps. Under any other shrinkage the `stat` column is dropped, padj moves to c6, and
`"c7<"` silently filters on nothing. (2) `header_lines: '0'` on all four significance
filters, against a corpus exemplar that binds `'1'` — the exemplar manufactures the header
it then skips, this workflow does not. `'1'` would discard row 1 of a padj-sorted table,
i.e. *SCF1*, the paper's finding. Both were established by hand and recorded only in YAML
comments above the steps in `galaxy-workflow-draft.gxwf.yml`. `gxwf draft-extract`
re-serializes from a parsed model and destroys every comment, so
`galaxy-workflow.gxwf.yml` — the artifact the test, validate, run and any later maturation
phase actually consume — carries no trace of either. The values themselves are correct in
the extract (verified at iteration 26); what is missing is any reason a later editor would
know not to change them.
resolved_by: null
supersedes: null
note: >-
Not closable inside the per-step loop: the loop's only writable surface is the draft, and
the draft's comments are exactly what the extract drops. Closable by (a) folding both
couplings into the step `doc:` strings of the two DESeq2 nodes and the four filters — `doc:`
survives extraction and is visible in the Galaxy editor — or (b) making the test plan assert
the column contract directly (a test on `deseq_out` that the padj column is c7, or on a
filtered table that its top row survived), which converts a silent invariant into a failing
test. (a) is the cheap one and should be done before the draft is discarded; (b) is the one
that would actually hold. Filed upstream as `draft-extract-destroys-yaml-comments` in
`foundry-feedback.ledger.yml`; that is the tool-side gap, this entry is the run-side
obligation it leaves behind.
ADVANCED, NOT CLOSED, by freeform-summary-to-galaxy-test-plan. Route (b) was taken and it
lands unevenly across the two invariants. INVARIANT 1 is now fully detectable: both test cases
assert `has_n_columns: 7` on BOTH `DESeq2 results:` tables, and case 1 asserts it again on both
`Significant genes:` tables, so any shrinkage setting other than `none` on either node drops
the `stat` column and fails the assertion before the `c7<` predicate can silently filter on
nothing. A `not_has_text: baseMean` assertion on both raw tables covers the headerless premise
the same predicates rest on. INVARIANT 2 is only partly detectable, and the plan says so in
warnings[] `reference-level-filter-regression-undetectable`: of the seven Filter1 steps binding
`header_lines: '0'`, a regression is visible on two — `Select first-contrast samples` and
`Select second-contrast samples`, where the leaked metadata row would add a fifth sample column
to that contrast's normalized-counts table, which both cases assert against. On `Select
reference-level samples` the leaked row IS `AR0382_A`, which the `c2=='AR0382'` predicate keeps
anyway, so the output is byte-identical and no assertion can see it. On the four significance
filters the leaked row is row 1 of a padj-sorted table, which passes both filters on its own
merits, so the regression is invisible at workflow level. Route (a) — folding both couplings
into the `doc:` strings of the two DESeq2 nodes and the seven filters, which survives
`draft-extract` — was NOT done and remains the cheap half of this entry. It is now the only
protection the three undetectable reference-level and significance filters have. Do it before
`galaxy-workflow-draft.gxwf.yml` is discarded.
FURTHER ADVANCED by implement-galaxy-workflow-test, on the invariant-2 half only. The normalized-counts
header width is no longer an assumption: deseq2.R at the pinned 2.11.40.8+galaxy4 writes `counts_out` with
`write.table(..., col.names = NA)`, which emits a header padded with a leading blank field, so a four-
sample contrast has exactly five tab-separated fields on line 1. Both test cases now assert
`has_n_columns: 5` on both normalized-counts tables, and case 1 keeps the SCF1 data-row field-count
assertion alongside it. A `header_lines: '1'` regression on either contrast-selecting Filter1 leaks a
fifth sample column and fails both. The three undetectable filters - `Select reference-level samples` and
the four significance filters - are unchanged, and route (a), folding both couplings into the `doc:`
strings that survive `draft-extract`, is still not done and is still the only protection they have.
RE-VERIFIED AT THE TERMINAL GATE by validate-galaxy-workflow, still not closed. Both couplings
hold in `galaxy-workflow.gxwf.yml` as assembled: `advanced_options.lfc_shrinkage_type: 'none'`
on both DESeq2 nodes (neither selects `many_contrasts`/`split_output`; both are
`select_data.how: datasets_per_level`), and `header_lines: '0'` as the string `'0'` on all
seven Filter1 steps (8, 12, 16, 23, 24, 25, 26 — the Filter1 count is exactly seven). Two
refinements to the entry's own statement of invariant 1, from reading the assembled artifact.
The `"c7<"` predicate is hard-coded in ONE `compose_text_param` bridge, step 21, consumed by
two Filter1 steps (23 and 25); the second bridge, step 22, is `"abs(c3)>"`, consumed by 24 and
26. Since `stat` sits at c5, dropping it moves padj from c7 to c6 but leaves log2FoldChange at
c3 — so the shrinkage coupling protects the padj predicate specifically, and the `abs(c3)>`
bridge is not exposed to it. Route (a) remains not done and the terminal artifact confirms
where the hole is: the four significance filters' `doc:` strings do name their `c7<` /
`abs(c3)>` predicates and the three condition-split filters' docs do say "headerless", but
NEITHER DESeq2 node's `doc:` mentions shrinkage at all, and no filter doc says why
`header_lines` is `'0'`. An editor changing `lfc_shrinkage_type` has nothing in the runnable
artifact to warn them. `galaxy-workflow-draft.gxwf.yml` still holds the rationale in the YAML
comments `draft-extract` strips; do not discard it before the `doc:` strings are written.
- id: scf1-fold-change-magnitude-disagrees-with-paper
status: open
raised_by: paper-to-test-data
unmet: "reconciliation of the paper's ~29-fold SCF1 figure with the deposited read data"
missing: >-
The paper states a ~29-fold SCF1 expression difference between AR0382 and AR0387. Direct
measurement of the deposited runs disagrees on magnitude by roughly an order of magnitude:
aligned fragments over the SCF1 exon at 200,000 read pairs per sample give 414 and 391 in
the two AR0382 replicates against 2 and 1 in AR0387 (~270-fold), and exact 31-mer matching
against the SCF1 CDS over 5.2 M reads gives 1942.7 vs 12.3 per million (~158-fold).
Direction and significance are not in doubt; only the ratio is. Candidates not
distinguished here: the 29-fold may be the RT-qPCR assay rather than the RNA-seq, a shrunk
rather than raw log2FC, or a different normalization. Reference bias is ruled out as the
explanation — AR0387 IS the B8441 reference strain, so its SCF1 reads cannot be undercounted;
if anything AR0382's divergent allele is undercounted, which makes the true ratio larger,
not smaller.
resolved_by: null
supersedes: null
note: >-
Consequence for the test plan, and the reason this is filed rather than glossed: no
assertion may pin the paper's 29-fold figure. `test-data-refs.json` asserts direction and
significance only (`log2FoldChange < -2`, `padj < 0.05`), which both the paper and the data
support. Settling this needs the paper's source data (Data S1) or the authors, not more
compute.
REINFORCED by freeform-summary-to-galaxy-test-plan, and one candidate explanation is now the
leading one. The disagreement was previously measured only on raw counts and k-mer matches;
phase 8 ran an actual differential-expression model — pydeseq2, two separate two-level n=2
analyses with NO LFC shrinkage, mirroring the workflow's two DESeq2 nodes, on a strand-aware
count matrix built from phase 7's six BAMs. AR0387 vs AR0382 gives SCF1 log2FC -7.95 at padj
4.1e-18, i.e. ~247-fold, consistent with the ~270-fold raw fragment ratio and not with the
paper's ~29-fold. tnSWI1 vs AR0382 gives log2FC -6.93 at padj 2.1e-31. Because that run applied
no shrinkage, it does not rule out the "shrunk rather than raw log2FC" candidate — log2(29) =
4.86 against an unshrunk 7.95 is exactly the direction and roughly the magnitude a shrinkage
estimator would move a gene whose counts are near zero in one group, so that candidate is now
the most plausible of the three and the RT-qPCR candidate is not excluded either. This matters
for the workflow, not just the paper: the workflow pins `lfc_shrinkage_type: none`, so ITS
log2FoldChange for SCF1 will be the unshrunk ~-8 and will never reproduce the paper's figure by
construction. Do not treat that as a defect when the first real invocation lands. Caveat on all
of the above: pydeseq2 is not the R DESeq2 the Galaxy wrapper runs, and the counts came from
bwa-mem rather than RNA STAR plus featureCounts -s 2; see
`deseq2-statistics-unverified-against-the-galaxy-wrapper`.
- id: test-fixtures-not-hosted
status: open
raised_by: paper-to-test-data
unmet: "a resolvable URL for each of the 12 read fixtures"
missing: >-
The 200,000-read-pair subsets were generated and hashed but not published. They exist only
in the producing session's scratchpad and do not survive it. At ~61 MB compressed for 12
files they are too large to commit alongside the workflow, so the IWC idiom applies: host
them (Zenodo or equivalent) and reference them by URL. `test-data-refs.json` carries
`<FIXTURE_BASE>` as the placeholder in `planemo_test_job_block`.
resolved_by: null
supersedes: null
note: >-
Not a research gap — the recipe is deterministic and the artifact pins it. Regenerate by
streaming each ENA FASTQ and taking `head -n 800000`, then verify against the
`md5_uncompressed` value recorded per file (hash the UNCOMPRESSED `.fastq`; gzip output is
not byte-stable across gzip versions). The reference genome and annotation need no hosting:
both are pinned to stable NCBI FTP URLs with md5s and are staged with `decompress: true`.
ADVANCED, NOT CLOSED, by implement-galaxy-workflow-test. All twelve 200,000-read-pair fixtures were
regenerated in this phase with the recorded recipe (`curl -L <ENA url> | gzip -dc | head -n 800000`) and
every one verified byte-for-byte against its `md5_uncompressed` in test-data-refs.json before compression,
so the recipe and the hashes are now proven reproducible rather than merely recorded. They live at
`<run>/test-data/*.fastq.gz` (59 MB) and `galaxy-workflow.gxwf-tests.yml` addresses them as local `path:`
entries, each with the SHA-1 of the `.gz` artifact as produced here. A twelve-file 25,000-read-pair set
for the smoke case was derived from the verified 200k files by taking their first 100,000 lines, at
`<run>/test-data/smoke-25k/` (6.7 MB). The test therefore runs against real data in this run directory and
nowhere else: the fixtures are still unhosted and do not survive the run, which is the whole of what
remains unmet. Publishing them is now a mechanical step - upload the exact `.gz` bytes, replace each
`path:` with the published `location:`, and leave the SHA-1 values unchanged, because they pin those
bytes. Keep the uncompressed md5s in test-data-refs.json as the regeneration check; the SHA-1s are the
fetch-integrity check and the two answer different questions.
- id: deseq2-statistics-unverified-against-the-galaxy-wrapper
status: open
raised_by: freeform-summary-to-galaxy-test-plan
step: "Differential expression: first contrast vs reference / second contrast vs reference"
unmet: "confirmation that the workflow's own DESeq2 node reproduces the differential-expression result the test plan asserts"
missing: >-
Every differential-expression figure this run has is from a different implementation than the
workflow runs. Phase 8 measured SCF1's log2 fold change and adjusted p-value with pydeseq2 over
a strand-aware count matrix built from phase 7's six bwa-mem BAMs, as two separate two-level
n=2 analyses with no LFC shrinkage — tnSWI1 vs AR0382 log2FC -6.93 / padj 2.1e-31, AR0387 vs
AR0382 log2FC -7.95 / padj 4.1e-18. The workflow runs the Galaxy DESeq2 wrapper, i.e. R DESeq2,
over RNA STAR plus featureCounts -s 2. pydeseq2 is a faithful reimplementation, but it is not
the same implementation and the counts are not the same counts, so the exact values will
differ. What is unverified is not the biology — the separation is ~200-fold at the counts level
and the padj margins are 18 and 31 orders of magnitude — but that the workflow's own node
produces a row for B9J08_001458 clearing `padj < 0.05` and `|log2FC| > 1` into each significant
table with `log2FoldChange <= -2`, which is what the test plan asserts, and that the raw table
really is the 7-column shape both cases assert `has_n_columns: 7` on.
resolved_by: null
supersedes: null
note: >-
Closable by the first successful invocation and nothing cheaper; phase 11 (run-workflow-test)
is where it lands, and phase 9 should not try to settle it by adding assertions. The plan is
already written so that this can only bite as a margin question rather than a value question:
no assertion anywhere pins an exact statistic, and the recorded measurements are stated as
headroom over thresholds (~5 and ~6 log2 units over the -2 floor; 31 and 18 orders of magnitude
over the 0.05 cut). Three things to check against the first invocation while it is open — the
sign and magnitude of SCF1's c3 in both contrasts, that c7 is padj and not something else, and
whether R DESeq2's independent filtering keeps SCF1 in the padj-carrying set at this depth as
pydeseq2 did (2393 and 1376 genes retained). Recorded in the plan as warnings[]
`deseq2-statistics-measured-with-pydeseq2-not-r`.
- id: collection-algebra-never-statically-checked
status: open
raised_by: validate-galaxy-workflow
step: whole workflow; acutely steps 0-3, 5, 10/14/18, 19-20
unmet: any static confirmation that the workflow's collection shapes and map-over structure are compatible end to end
missing: >-
Terminal validation could not run the one check that covers this. `gxwf validate --connections`
— the flag whose entire purpose is connection-type compatibility, collection algebra and
map-over — aborts with `TypeError: step.in is not iterable` before emitting any report, on a
format2 workflow written the way this one is: list-form `inputs:`/`steps:` with mapping-form
`in:`, which gxwf's `toNative` mistakes for an already-normalized workflow and therefore never
normalizes. Reproduced on a 20-line minimal case and unchanged on gxwf 1.12.0, so it is a gxwf
defect and not a property of this workflow (filed as
`tonative-shape-sniff-skips-normalization-on-list-form-format2`; fix submitted as
jmchilton/galaxy-tool-util-ts#179). Two routes could still close this without an upstream
release: rewriting the workflow's `in:` blocks in list form, or running `--connections` against
a patched gxwf. What the default validation does cover is per-step tool state on the 20
decodable steps, and nothing about how shapes compose.
The map-over structure is the load-bearing part of this design and none of it is proven: the
`sample_sheet:paired` input fanning through `__FLATTEN__` into FastQC's per-fastq axis; the
paired collection entering Cutadapt at `library|input_1` and leaving as `out_pairs` into RNA
STAR's `singlePaired|input`; three `__FILTER_FROM_FILE__` reductions over `Count reads per
gene/output_short`; and the workflow's only true reduction, where each DESeq2 node's
multiple=true `countsFile` ports consume whole 2-element collections and the per-sample axis
collapses.
resolved_by: null
supersedes: null
note: >-
Not a defect claim against the workflow — every `in:` source resolves, every `in:` key name and
tool_state parameter name was checked by hand against the real tool input trees for all 27
steps (the 20 cached tools from the gxwf cache, the 7 undecodable ones fetched from
`GET /api/tools/<id>?io_details=true`) with zero mismatches, and every select and boolean value
on the previously unchecked steps is legal. What is unmet is that shape COMPOSITION has no
static witness, and cannot get one until gxwf is fixed. Closable only by the first successful
invocation. Phase 11 should read `GET /api/invocations/{id}/step_jobs_summary` per step rather
than only the terminal invocation state: a shape mismatch surfaces as invocation message
`collection_failed`, or as an unexpected job count on a mapped step, and a workflow can report
`completed` while a mapped step ran the wrong number of jobs. Two adjacent exposures ride along
and are settled by the same run: whether the target Galaxy supports the `sample_sheet:paired`
input with `column_definitions` and `__SAMPLE_SHEET_TO_TABULAR__` at all (see
`no-iwc-precedent-for-sample-sheet-workflow-input` and the filed
`galaxy-test-staging-drops-sample-sheet-column-definitions`), which would fail at request-time
validation or input materialization rather than in any tool's stderr; and the element
identifiers `__FLATTEN__` produces, since step 0 carries no tool_state and `join_identifier`
therefore takes the wrapper default, while the sample-sheet element identifiers are declared
the spine of this workflow.
Resolved workflow test inputs and expected outputs derived from paper evidence (URLs, file shapes, expected hashes).
{
"schema": "test-data-refs",
"schema_version": 1,
"produced_by": "paper-to-test-data",
"run_slug": "auris-scf1",
"branch_path_selected": "paper-to-test-data",
"target_workflow": "galaxy-workflow.gxwf.yml",
"generated": "2026-09-16",
"source": {
"paper": "Santana DJ et al. 2023, Science 381(6665):1461-1467",
"doi": "10.1126/science.adf8972",
"pipeline": "Pipeline A - bulk RNA-seq differential expression",
"bioproject": "PRJNA904261",
"sra_study": "SRP409192",
"taxon": 498019
},
"verdict": {
"tier": "biology",
"summary": "The fixture reproduces the paper's central SCF1 finding in BOTH contrasts, not shape only. This is measured, not projected: a 200,000-read-pair head subset of each of the six deposited runs was aligned to GCA_002759435.2 with bwa-mem and counted against the NCBI GTF's exon features. SCF1 (B9J08_001458) carries 414/391 fragments in the two AR0382 replicates against 2/1 in AR0387 and 4/3 in tnSWI1 - a separation DESeq2 will call significant at n=2 with wide margin.",
"not_reproduced": [
"The paper's ~29-fold AR0387-vs-AR0382 magnitude. Measured raw fragment ratio at fixture depth is ~270-fold; see unresolved `scf1-fold-change-magnitude-disagrees-with-paper`.",
"The claim that SCF1 is THE single top gene ranked by |log2 fold change|. At this depth, and with the workflow's `lfc_shrinkage_type: none`, low-count genes take extreme unshrunk log2FC values and can outrank it. Rank by adjusted p-value is the safe form and is what the assertions use.",
"Fig. S2 / tnBCY1. No tnBCY1 run was ever deposited; no test input exists for it at any depth."
],
"caveat": "Counts above come from bwa-mem (unspliced) plus a strand-agnostic exon-overlap counter, not from RNA STAR plus featureCounts with -s 2. They establish that the SIGNAL survives subsetting; they are not predictions of the workflow's exact integer counts. No assertion below pins an exact count."
},
"inputs": {
"RNA-seq reads (sample sheet)": {
"workflow_input_type": "collection",
"collection_type": "sample_sheet:paired",
"format": "fastqsanger.gz",
"column_definitions_order": [
"condition",
"replicate"
],
"elements": [
{
"element_identifier": "AR0382_A",
"sra_run": "SRR22376032",
"condition": "AR0382",
"replicate": "A",
"source_reads_full": 26239060,
"source": {
"forward": {
"url": "https://ftp.sra.ebi.ac.uk/vol1/fastq/SRR223/032/SRR22376032/SRR22376032_1.fastq.gz",
"md5": "8bbc89136c3e8f9692172ab245e1bb3c"
},
"reverse": {
"url": "https://ftp.sra.ebi.ac.uk/vol1/fastq/SRR223/032/SRR22376032/SRR22376032_2.fastq.gz",
"md5": "195a5f65d3af18006a7a47fae8b43245"
}
},
"fixture": {
"forward": {
"basename": "AR0382_A_1.fastq.gz",
"md5_uncompressed": "638d1c0b75d9e688fc5ef9340f4cc3eb",
"approx_bytes_gz": 4620000
},
"reverse": {
"basename": "AR0382_A_2.fastq.gz",
"md5_uncompressed": "74178e6e71d82fd12edd7a1b578fa80c",
"approx_bytes_gz": 4625000
},
"read_pairs": 200000,
"hosting": "NOT HOSTED. Regenerate with `subsetting.recipe`, then publish (Zenodo or equivalent) and substitute the URL into `location`."
},
"measured_at_fixture_depth": {
"r1_primary_mapped": 198486,
"assigned_to_genes": 181378,
"genes_nonzero": 5101,
"genes_ge_10": 3112,
"scf1": 414,
"scf1_rank": 71
}
},
{
"element_identifier": "AR0382_B",
"sra_run": "SRR22376031",
"condition": "AR0382",
"replicate": "B",
"source_reads_full": 23724430,
"source": {
"forward": {
"url": "https://ftp.sra.ebi.ac.uk/vol1/fastq/SRR223/031/SRR22376031/SRR22376031_1.fastq.gz",
"md5": "cc9d1404ad4a4d1932bb43333fdf310e"
},
"reverse": {
"url": "https://ftp.sra.ebi.ac.uk/vol1/fastq/SRR223/031/SRR22376031/SRR22376031_2.fastq.gz",
"md5": "d34f7aee48c7e2bf0d43998e1edecd61"
}
},
"fixture": {
"forward": {
"basename": "AR0382_B_1.fastq.gz",
"md5_uncompressed": "1027b9ff9f53914d870ed99cec61e618",
"approx_bytes_gz": 4620000
},
"reverse": {
"basename": "AR0382_B_2.fastq.gz",
"md5_uncompressed": "2f49cc2d14949c0f415a500505a02a59",
"approx_bytes_gz": 4625000
},
"read_pairs": 200000,
"hosting": "NOT HOSTED. Regenerate with `subsetting.recipe`, then publish (Zenodo or equivalent) and substitute the URL into `location`."
},
"measured_at_fixture_depth": {
"r1_primary_mapped": 198505,
"assigned_to_genes": 181360,
"genes_nonzero": 5076,
"genes_ge_10": 3142,
"scf1": 391,
"scf1_rank": 73
}
},
{
"element_identifier": "AR0387_A",
"sra_run": "SRR22376030",
"condition": "AR0387",
"replicate": "A",
"source_reads_full": 25524969,
"source": {
"forward": {
"url": "https://ftp.sra.ebi.ac.uk/vol1/fastq/SRR223/030/SRR22376030/SRR22376030_1.fastq.gz",
"md5": "eb20211e3ac46e4c06b7f304d8d69c22"
},
"reverse": {
"url": "https://ftp.sra.ebi.ac.uk/vol1/fastq/SRR223/030/SRR22376030/SRR22376030_2.fastq.gz",
"md5": "a8be593535c43e2f618efaac05bbbeba"
}
},
"fixture": {
"forward": {
"basename": "AR0387_A_1.fastq.gz",
"md5_uncompressed": "75fea1effe836256ee8f1205a6281037",
"approx_bytes_gz": 4620000
},
"reverse": {
"basename": "AR0387_A_2.fastq.gz",
"md5_uncompressed": "1b5e3f732f9ebb2722a10110198cfef3",
"approx_bytes_gz": 4625000
},
"read_pairs": 200000,
"hosting": "NOT HOSTED. Regenerate with `subsetting.recipe`, then publish (Zenodo or equivalent) and substitute the URL into `location`."
},
"measured_at_fixture_depth": {
"r1_primary_mapped": 198586,
"assigned_to_genes": 179176,
"genes_nonzero": 5053,
"genes_ge_10": 2934,
"scf1": 2,
"scf1_rank": 4541
}
},
{
"element_identifier": "AR0387_B",
"sra_run": "SRR22376029",
"condition": "AR0387",
"replicate": "B",
"source_reads_full": 22010492,
"source": {
"forward": {
"url": "https://ftp.sra.ebi.ac.uk/vol1/fastq/SRR223/029/SRR22376029/SRR22376029_1.fastq.gz",
"md5": "67c93fa57ef2d7bac8c309e1a86b1311"
},
"reverse": {
"url": "https://ftp.sra.ebi.ac.uk/vol1/fastq/SRR223/029/SRR22376029/SRR22376029_2.fastq.gz",
"md5": "2a701a987851bbeb4870766f1a263e56"
}
},
"fixture": {
"forward": {
"basename": "AR0387_B_1.fastq.gz",
"md5_uncompressed": "46b3b7ee5daeeadd3e3aab3d5c89c3fd",
"approx_bytes_gz": 4620000
},
"reverse": {
"basename": "AR0387_B_2.fastq.gz",
"md5_uncompressed": "1d72c57ae5be084075022be86de3b3c3",
"approx_bytes_gz": 4625000
},
"read_pairs": 200000,
"hosting": "NOT HOSTED. Regenerate with `subsetting.recipe`, then publish (Zenodo or equivalent) and substitute the URL into `location`."
},
"measured_at_fixture_depth": {
"r1_primary_mapped": 198443,
"assigned_to_genes": 178899,
"genes_nonzero": 5042,
"genes_ge_10": 2890,
"scf1": 1,
"scf1_rank": 4775
}
},
{
"element_identifier": "AR0382_tnSWI1_A",
"sra_run": "SRR22376028",
"condition": "tnSWI1",
"replicate": "A",
"source_reads_full": 21590744,
"source": {
"forward": {
"url": "https://ftp.sra.ebi.ac.uk/vol1/fastq/SRR223/028/SRR22376028/SRR22376028_1.fastq.gz",
"md5": "5cc6fb05f47fa6e9c519439afae0f487"
},
"reverse": {
"url": "https://ftp.sra.ebi.ac.uk/vol1/fastq/SRR223/028/SRR22376028/SRR22376028_2.fastq.gz",
"md5": "e110cbb1ef1ae24810f57e226851a07b"
}
},
"fixture": {
"forward": {
"basename": "AR0382_tnSWI1_A_1.fastq.gz",
"md5_uncompressed": "15fc0849808eafcc468d4e25a87a8566",
"approx_bytes_gz": 4620000
},
"reverse": {
"basename": "AR0382_tnSWI1_A_2.fastq.gz",
"md5_uncompressed": "b1decf99a029e855e46837452e019e85",
"approx_bytes_gz": 4625000
},
"read_pairs": 200000,
"hosting": "NOT HOSTED. Regenerate with `subsetting.recipe`, then publish (Zenodo or equivalent) and substitute the URL into `location`."
},
"measured_at_fixture_depth": {
"r1_primary_mapped": 198245,
"assigned_to_genes": 180354,
"genes_nonzero": 5139,
"genes_ge_10": 3262,
"scf1": 4,
"scf1_rank": 4242
}
},
{
"element_identifier": "AR0382_tnSWI1_B",
"sra_run": "SRR22376027",
"condition": "tnSWI1",
"replicate": "B",
"source_reads_full": 29894351,
"source": {
"forward": {
"url": "https://ftp.sra.ebi.ac.uk/vol1/fastq/SRR223/027/SRR22376027/SRR22376027_1.fastq.gz",
"md5": "7e0650d15a982323294839fdeda80e56"
},
"reverse": {
"url": "https://ftp.sra.ebi.ac.uk/vol1/fastq/SRR223/027/SRR22376027/SRR22376027_2.fastq.gz",
"md5": "43b800cdb2648d22e01710ea06ae24a4"
}
},
"fixture": {
"forward": {
"basename": "AR0382_tnSWI1_B_1.fastq.gz",
"md5_uncompressed": "064e50e58ffabb652aa97ba14c976b57",
"approx_bytes_gz": 4620000
},
"reverse": {
"basename": "AR0382_tnSWI1_B_2.fastq.gz",
"md5_uncompressed": "1b2b6be866f978a441acd292253bba94",
"approx_bytes_gz": 4625000
},
"read_pairs": 200000,
"hosting": "NOT HOSTED. Regenerate with `subsetting.recipe`, then publish (Zenodo or equivalent) and substitute the URL into `location`."
},
"measured_at_fixture_depth": {
"r1_primary_mapped": 198468,
"assigned_to_genes": 180955,
"genes_nonzero": 5108,
"genes_ge_10": 3283,
"scf1": 3,
"scf1_rank": 4475
}
}
]
},
"Reference genome FASTA": {
"workflow_input_type": "data",
"format": "fasta",
"assembly": "GCA_002759435.2 (Cand_auris_B8441_V2), C. auris B8441",
"url": "https://ftp.ncbi.nlm.nih.gov/genomes/all/GCA/002/759/435/GCA_002759435.2_Cand_auris_B8441_V2/GCA_002759435.2_Cand_auris_B8441_V2_genomic.fna.gz",
"md5_gz": "a008b270d3aaa04736a8bb7daf3f6dd5",
"decompress": true,
"measured": {
"total_bp": 12365959,
"contigs": 15,
"largest_contig_bp": 3195935
},
"note": "Used whole. At 12.4 Mb it is small enough to be a test fixture unmodified, so no reference subsetting is needed and the gene ID space stays exactly the real one."
},
"Gene annotation GTF": {
"workflow_input_type": "data",
"format": "gtf",
"url": "https://ftp.ncbi.nlm.nih.gov/genomes/all/GCA/002/759/435/GCA_002759435.2_Cand_auris_B8441_V2/GCA_002759435.2_Cand_auris_B8441_V2_genomic.gtf.gz",
"md5_gz": "6e5b9528d48c0a8fc2c8588e6eeea929",
"decompress": true,
"resolves_open_requirement": "featurecounts-annotation-source-unnamed",
"measured": {
"gtf_version": "2.2",
"lines": 33953,
"genes": 5586,
"exon_features": 6057,
"gene_id_example": "B9J08_001458",
"scf1_locus": "PEKT02000003.1:864995-867292(+), single exon, 2298 bp"
},
"why_defensible": "It is NCBI's own annotation of the exact accession the paper names - not a different assembly, not a different resource. It is a true GTF (#gtf-version 2.2), so the workflow's `format: gtf` needs no conversion step. Its gene_id values ARE the paper's B9J08_* locus tags, so SCF1 appears as B9J08_001458 with no identifier translation. It carries `exon` features, matching the step's `gff_feature_type: exon`. This makes the step's existing `gff_feature_attribute: gene_id` correct as bound - the FungiDB-GFF3 hazard the ledger recorded (keyed on `ID`) is avoided by not using FungiDB."
},
"Reference condition level": {
"value": "AR0382"
},
"First contrast condition level": {
"value": "tnSWI1"
},
"Second contrast condition level": {
"value": "AR0387"
},
"Strandedness": {
"value": "stranded - reverse",
"resolves_open_requirement": "rnaseq-strandedness-inferred-from-kit-name",
"evidence": "Measured, no longer inferred from the kit name. Of R1 reads unambiguously overlapping a single annotated gene, 98.4% / 98.4% / 98.2% (AR0382_A / AR0387_A / tnSWI1_A) map ANTISENSE to that gene. That is the dUTP reverse-stranded signature. featureCounts -s 2 is correct."
},
"Adjusted p-value threshold": {
"value": 0.05,
"source": "stated by the paper"
},
"log2 fold change threshold": {
"value": 1,
"source": "paper's |fold change| > 2, in log2 units"
}
},
"subsetting": {
"why": "Six runs at 22-30 M pairs each is ~6.5 GB of source FASTQ. Unusable as a fixture.",
"strategy": "Deterministic head subset - first 200,000 read pairs of each mate, taken in file order.",
"why_a_head_subset_is_valid_here": "Tested, not assumed. SCF1-matching reads were counted per 1,000,000-read block across the first 5 M reads of AR0382_A (1938, 1896, 2012, 1899, 1950) and AR0387_A (10, 12, 13, 14, 14). The rate is flat, so position in the file carries no bias for this transcript and no alignment-based region targeting is needed. The simpler, fully reproducible recipe is therefore also the correct one.",
"why_200k": "SCF1 is extremely highly expressed in AR0382 - rank ~71 of ~5,100 detected genes, the top 1.4%, which independently corroborates the paper's 'top 2.5%' background fact. At 200,000 pairs it still draws ~400 fragments there while AR0387 and tnSWI1 draw 1-4, and ~3,000 genes still clear 10 reads, which is ample for DESeq2 size factors and dispersion fitting. Below ~50,000 pairs the transcriptome-wide depth thins enough that DESeq2's independent filtering starts removing genes wholesale and the significant-gene tables can come back empty.",
"recipe": [
"for each run R in {SRR22376032,SRR22376031,SRR22376030,SRR22376029,SRR22376028,SRR22376027}:",
" for mate m in {1,2}:",
" curl -L <ENA url for R,m> | gzip -dc | head -n 800000 > <element_identifier>_<m>.fastq",
" gzip -n <element_identifier>_<m>.fastq",
"Verify against `fixture.*.md5_uncompressed` on the UNCOMPRESSED .fastq (gzip output is not byte-stable across gzip versions).",
"Total: 12 files, ~61 MB compressed. Too large to commit; host and reference by URL."
],
"total_fixture_bytes_gz": 55500000,
"shape_only_alternative": {
"read_pairs": 25000,
"total_bytes_gz": 7000000,
"committable": true,
"loses": "SCF1 falls to ~50 fragments in AR0382 and ~0 elsewhere; per-gene depth drops to ~4.5 reads/gene average. Workflow shape still exercises end to end, but the significant-gene tables may be empty and no SCF1 assertion is safe. Use only if hosting is impossible."
}
},
"sample_sheet_expressibility": {
"resolves_open_requirement": "sample-sheet-input-test-fixture-expressibility",
"answer": "YES - a sample_sheet:paired input is expressible in a Planemo/gxwf test job block. The phase-2 list:paired fallback is NOT required.",
"evidence": [
"galaxy/lib/galaxy/tool_util/cwl/util.py, replacement_collection(): `if collection_type.startswith(\"sample_sheet\"): kwds[\"rows\"] = value.get(\"rows\")` - the job-block loader has an explicit sample_sheet branch.",
"galaxy/lib/galaxy/tool_util/client/staging.py, create_collection_func(): carries `rows` through to the dataset_collections API.",
"galaxy/lib/galaxy_test/workflow/collection_semantics_cat_sample_sheet.gxwf-tests.yml - a real end-to-end gxwf test that drives a sample_sheet input from a job block.",
"galaxy/test/unit/tool_util/test_cwl_util.py::test_galactic_job_json_sample_sheet_paired_collection - exercises the NESTED sample_sheet:paired shape specifically, producing `src: new_collection` sub-collections of type `paired`."
],
"rows_shape": {
"form": "mapping of element identifier -> POSITIONAL LIST of column values, ordered to match column_definitions",
"authority": "galaxy/lib/galaxy/model/dataset_collections/types/sample_sheet_util.py, validate_row(): rejects on `len(row) != len(column_definitions)` and then `zip(row, column_definitions)`.",
"warning": "test_cwl_util.py's non-paired unit tests write rows as a DICT ({'el1': {'condition': 'treatment'}}). That shape never reaches validate_row because those tests mock collection_create_func, and it will not validate against the real API. Use the list form."
},
"column_definitions_caveat": {
"fact": "Neither galactic_job_json nor staging.py passes `column_definitions` when creating the collection, though the API payload (CreateNewCollectionPayload) supports it. A test-staged sample sheet therefore has collection-level column_definitions = None.",
"is_it_fatal": "No. Per-element `columns` are still populated from `rows` (SampleSheetDatasetCollectionType.generate_elements sets them regardless), and validate_row short-circuits when column_definitions is absent. The two column_definitions_compatible() call sites are both in DataCollectionToolParameter option-building (match_collections / _classify_hdca) - UI dropdown filtering, not invocation-time validation. Supplying the HDCA by id, as a test does, bypasses them.",
"one_real_consequence": "__SAMPLE_SHEET_TO_TABULAR__ emits its header line only `#if $include_headers and $input.collection.column_definitions`. This workflow's `Project sample sheet to tabular` step sets `include_headers: false`, so it is unaffected. If a later phase flips that to true, the header will silently vanish under test while appearing in the UI."
}
},
"planemo_test_job_block": "# job: block for the workflow's -tests.yml. Substitute hosted URLs for <FIXTURE_BASE>.\nRNA-seq reads (sample sheet):\n class: Collection\n collection_type: sample_sheet:paired\n rows:\n AR0382_A: [AR0382, A]\n AR0382_B: [AR0382, B]\n AR0387_A: [AR0387, A]\n AR0387_B: [AR0387, B]\n AR0382_tnSWI1_A: [tnSWI1, A]\n AR0382_tnSWI1_B: [tnSWI1, B]\n elements:\n - identifier: AR0382_A\n class: Collection\n type: paired\n elements:\n - identifier: forward\n class: File\n location: <FIXTURE_BASE>/AR0382_A_1.fastq.gz\n filetype: fastqsanger.gz\n - identifier: reverse\n class: File\n location: <FIXTURE_BASE>/AR0382_A_2.fastq.gz\n filetype: fastqsanger.gz\n - identifier: AR0382_B\n class: Collection\n type: paired\n elements:\n - identifier: forward\n class: File\n location: <FIXTURE_BASE>/AR0382_B_1.fastq.gz\n filetype: fastqsanger.gz\n - identifier: reverse\n class: File\n location: <FIXTURE_BASE>/AR0382_B_2.fastq.gz\n filetype: fastqsanger.gz\n - identifier: AR0387_A\n class: Collection\n type: paired\n elements:\n - identifier: forward\n class: File\n location: <FIXTURE_BASE>/AR0387_A_1.fastq.gz\n filetype: fastqsanger.gz\n - identifier: reverse\n class: File\n location: <FIXTURE_BASE>/AR0387_A_2.fastq.gz\n filetype: fastqsanger.gz\n - identifier: AR0387_B\n class: Collection\n type: paired\n elements:\n - identifier: forward\n class: File\n location: <FIXTURE_BASE>/AR0387_B_1.fastq.gz\n filetype: fastqsanger.gz\n - identifier: reverse\n class: File\n location: <FIXTURE_BASE>/AR0387_B_2.fastq.gz\n filetype: fastqsanger.gz\n - identifier: AR0382_tnSWI1_A\n class: Collection\n type: paired\n elements:\n - identifier: forward\n class: File\n location: <FIXTURE_BASE>/AR0382_tnSWI1_A_1.fastq.gz\n filetype: fastqsanger.gz\n - identifier: reverse\n class: File\n location: <FIXTURE_BASE>/AR0382_tnSWI1_A_2.fastq.gz\n filetype: fastqsanger.gz\n - identifier: AR0382_tnSWI1_B\n class: Collection\n type: paired\n elements:\n - identifier: forward\n class: File\n location: <FIXTURE_BASE>/AR0382_tnSWI1_B_1.fastq.gz\n filetype: fastqsanger.gz\n - identifier: reverse\n class: File\n location: <FIXTURE_BASE>/AR0382_tnSWI1_B_2.fastq.gz\n filetype: fastqsanger.gz\nReference genome FASTA:\n class: File\n location: https://ftp.ncbi.nlm.nih.gov/genomes/all/GCA/002/759/435/GCA_002759435.2_Cand_auris_B8441_V2/GCA_002759435.2_Cand_auris_B8441_V2_genomic.fna.gz\n filetype: fasta\n decompress: true\nGene annotation GTF:\n class: File\n location: https://ftp.ncbi.nlm.nih.gov/genomes/all/GCA/002/759/435/GCA_002759435.2_Cand_auris_B8441_V2/GCA_002759435.2_Cand_auris_B8441_V2_genomic.gtf.gz\n filetype: gtf\n decompress: true\nReference condition level: AR0382\nFirst contrast condition level: tnSWI1\nSecond contrast condition level: AR0387\nStrandedness: stranded - reverse\nAdjusted p-value threshold: 0.05\nlog2 fold change threshold: 1.0\n",
"planemo_test_job_notes": [
"`decompress: true` is honoured only on the fetch-API staging path (cwl_util.py reads it into FileUploadTarget properties; staging.py passes it to the fetch payload). The reads must NOT set it - the workflow input declares fastqsanger.gz and the .gz must survive.",
"Element identifiers are the ENA library_name values and match the workflow input's documented spine exactly. They are load-bearing: the condition split joins the featureCounts collection back to the sample sheet on them."
],
"expected_outputs": {
"safe": [
{
"output": "FastQC raw reads: text summary",
"assert": "collection of 12 elements, identifiers <sample>_forward / <sample>_reverse",
"basis": "structural"
},
{
"output": "Trimmed reads",
"assert": "list:paired of 6 elements",
"basis": "structural"
},
{
"output": "STAR mapping summary",
"assert": "has_text 'Uniquely mapped reads %'; uniquely-mapped fraction > 80%",
"basis": "measured - bwa-mem primary-mapped 198.2-198.6k of 200k R1 (>99%); STAR with the same reference will be lower but comfortably above 80%"
},
{
"output": "Gene counts per sample",
"assert": "6 elements; each has 5586 data rows; column 1 matches ^B9J08_[0-9]{6}$; has_text 'B9J08_001458'",
"basis": "measured - the NCBI GTF carries exactly 5586 distinct gene_id values"
},
{
"output": "featureCounts assignment summary",
"assert": "Assigned fraction > 70%; Unassigned_NoFeatures fraction < 20%",
"basis": "measured - 90-91% of primary-mapped R1 fell inside an annotated exon; this also functions as the strandedness read-out, since -s 1 would invert it"
},
{
"output": "DESeq2 results: tnSWI1 vs AR0382",
"assert": "has_text 'B9J08_001458'; 7 columns",
"basis": "structural"
},
{
"output": "DESeq2 results: AR0387 vs AR0382",
"assert": "has_text 'B9J08_001458'; 7 columns",
"basis": "structural"
},
{
"output": "Significant genes: tnSWI1 vs AR0382",
"assert": "non-empty; has_text 'B9J08_001458'",
"basis": "measured count separation 414/391 vs 4/3"
},
{
"output": "Significant genes: AR0387 vs AR0382",
"assert": "non-empty; has_text 'B9J08_001458'",
"basis": "measured count separation 414/391 vs 2/1"
}
],
"biological": [
{
"claim": "SCF1 is significantly down-regulated in tnSWI1 vs AR0382",
"assert": "row B9J08_001458 in the significant-genes table has log2FoldChange < -2 and padj < 0.05",
"paper": "Fig. 1D / Fig. S5A - SCF1 the strongest, most significant dysregulation",
"basis": "measured fragment counts 414/391 (AR0382) vs 4/3 (tnSWI1) at fixture depth"
},
{
"claim": "SCF1 is significantly down-regulated in AR0387 vs AR0382",
"assert": "row B9J08_001458 has log2FoldChange < -2 and padj < 0.05",
"paper": "SCF1 the most down-regulated gene between the two wild-type isolates",
"basis": "measured fragment counts 414/391 vs 2/1"
},
{
"claim": "SCF1 is among the most significantly dysregulated genes in the AR0387 contrast",
"assert": "B9J08_001458 within the 10 smallest padj rows",
"note": "Rank by padj, never by |log2FC| - see verdict.not_reproduced."
},
{
"claim": "SCF1 is highly expressed in AR0382",
"assert": "SCF1's normalized count in the AR0382 columns sits in the top 5% of the table",
"paper": "top 2.5% of all genes",
"basis": "measured rank 71 and 73 of ~5,100 detected genes = top 1.4%; asserted at top 5% to leave headroom for the STAR/featureCounts difference"
}
],
"not_assertable": [
{
"claim": "No significant dysregulation of the ALS or IFF/HYR adhesin families in the tnSWI1 contrast",
"why": "A negative claim over 12 named genes (B9J08_002582/_004498/_004112 ALS; _004100/_004109/_004098/_004110/_001531/_004892/_001155/_004451/_000675 IFF-HYR). At 200k pairs most of these sit at counts where DESeq2 simply lacks power, so their absence from the significant table is a depth artefact and not evidence for the paper's claim. Asserting it would be a fixture claiming an outcome it cannot produce.",
"would_need": "full-depth runs"
},
{
"claim": "~29-fold AR0382 vs AR0387",
"why": "see unresolved `scf1-fold-change-magnitude-disagrees-with-paper`"
}
]
},
"unresolved": [
{
"id": "scf1-fold-change-magnitude-disagrees-with-paper",
"what": "The paper states a ~29-fold SCF1 expression difference between AR0382 and AR0387. Direct measurement at fixture depth gives ~270-fold on aligned fragments (414/391 vs 2/1) and ~158-fold on exact 31-mer read matching (1943 vs 12.3 per million). Direction and significance are not in doubt; the magnitude is off by roughly an order of magnitude. Candidate explanations not distinguished here: the paper's 29-fold may come from RT-qPCR rather than RNA-seq, from shrunk rather than raw log2FC, or from a different normalization. Note AR0387 IS the B8441 reference strain, so its SCF1 reads cannot be undercounted by reference bias - if anything AR0382's divergent allele is undercounted, which would make the true ratio larger still, not smaller.",
"consequence": "Do not assert the paper's 29-fold figure. Assert direction and significance only.",
"new": true
},
{
"id": "fixture-hosting-not-done",
"what": "The 12 fixture files were generated and hashed but not published. They live only in this session's scratchpad and will not survive it.",
"consequence": "Phase 9/11 must regenerate via `subsetting.recipe` and host before any test can run. The recipe is deterministic and the uncompressed md5s pin it.",
"new": true
},
{
"id": "deseq2-not-executed",
"what": "No DESeq2 run was performed. The significance claims are inferred from measured count separation, which at 414/391 vs 1-4 with n=2 is about as unambiguous as DESeq2 input gets, but they are inference.",
"consequence": "The first real workflow run settles them. Nothing here should be reported as a passed test.",
"new": true
},
{
"id": "counts-measured-with-bwa-not-star",
"what": "Counts came from bwa-mem plus a strand-agnostic exon-overlap counter, because STAR and featureCounts are not available in this environment.",
"consequence": "No assertion pins an exact integer count. All count-derived assertions are expressed as fractions, thresholds, or ranks.",
"new": true
}
],
"open_requirements_effect": {
"closed": [
"featurecounts-annotation-source-unnamed",
"rnaseq-strandedness-inferred-from-kit-name",
"sample-sheet-input-test-fixture-expressibility"
],
"advanced_not_closed": [
"star-genome-length-drives-sa-index-parameter"
],
"opened": [
"scf1-fold-change-magnitude-disagrees-with-paper",
"test-fixtures-not-hosted"
]
}
}Failure-surface classification with captured job/invocation/collection/assertion evidence and a recommended next step or reference-gap follow-up.
not on disk.
Structured status plus captured evidence — Planemo result, invocation/history/workflow ids, artifact paths, Galaxy mode, and (on failure) the observed modality and next reference surface — for debug-galaxy-workflow-output. Also the faithful handoff when no test exists or none could be run.
not on disk.
15 open, 13 resolved, 0 surrendered, 2 deliberately dropped. 0 open entries are blocking.
Terminal validation could not run the one check that covers this. `gxwf validate --connections` — the flag whose entire purpose is connection-type compatibility, collection algebra and map-over — aborts with `TypeError: step.in is not iterable` before emitting any report, on a format2 workflow written the way this one is: list-form `inputs:`/`steps:` with mapping-form `in:`, which gxwf's `toNative` mistakes for an already-normalized workflow and therefore never normalizes. Reproduced on a 20-line minimal case and unchanged on gxwf 1.12.0, so it is a gxwf defect and not a property of this workflow (filed as `tonative-shape-sniff-skips-normalization-on-list-form-format2`; fix submitted as jmchilton/galaxy-tool-util-ts#179). Two routes could still close this without an upstream release: rewriting the workflow's `in:` blocks in list form, or running `--connections` against a patched gxwf. What the default validation does cover is per-step tool state on the 20 decodable steps, and nothing about how shapes compose. The map-over structure is the load-bearing part of this design and none of it is proven: the `sample_sheet:paired` input fanning through `__FLATTEN__` into FastQC's per-fastq axis; the paired collection entering Cutadapt at `library|input_1` and leaving as `out_pairs` into RNA STAR's `singlePaired|input`; three `__FILTER_FROM_FILE__` reductions over `Count reads per gene/output_short`; and the workflow's only true reduction, where each DESeq2 node's multiple=true `countsFile` ports consume whole 2-element collections and the per-sample axis collapses.
Not a defect claim against the workflow — every `in:` source resolves, every `in:` key name and tool_state parameter name was checked by hand against the real tool input trees for all 27 steps (the 20 cached tools from the gxwf cache, the 7 undecodable ones fetched from `GET /api/tools/<id>?io_details=true`) with zero mismatches, and every select and boolean value on the previously unchecked steps is legal. What is unmet is that shape COMPOSITION has no static witness, and cannot get one until gxwf is fixed. Closable only by the first successful invocation. Phase 11 should read `GET /api/invocations/{id}/step_jobs_summary` per step rather than only the terminal invocation state: a shape mismatch surfaces as invocation message `collection_failed`, or as an unexpected job count on a mapped step, and a workflow can report `completed` while a mapped step ran the wrong number of jobs. Two adjacent exposures ride along and are settled by the same run: whether the target Galaxy supports the `sample_sheet:paired` input with `column_definitions` and `__SAMPLE_SHEET_TO_TABULAR__` at all (see `no-iwc-precedent-for-sample-sheet-workflow-input` and the filed `galaxy-test-staging-drops-sample-sheet-column-definitions`), which would fail at request-time validation or input materialization rather than in any tool's stderr; and the element identifiers `__FLATTEN__` produces, since step 0 carries no tool_state and `join_identifier` therefore takes the wrapper default, while the sample-sheet element identifiers are declared the spine of this workflow.
Two couplings hold this workflow together and neither is expressible in gxformat2 nor checked by any gate. (1) `advanced_options.lfc_shrinkage_type: none` on BOTH DESeq2 nodes is what makes `deseq_out` a 7-column table, so `c3` = log2FoldChange and `c7` = padj — the columns the `compose_text_param` bridges hard-code as `"abs(c3)>"` and `"c7<"` in four OTHER steps. Under any other shrinkage the `stat` column is dropped, padj moves to c6, and `"c7<"` silently filters on nothing. (2) `header_lines: '0'` on all four significance filters, against a corpus exemplar that binds `'1'` — the exemplar manufactures the header it then skips, this workflow does not. `'1'` would discard row 1 of a padj-sorted table, i.e. *SCF1*, the paper's finding. Both were established by hand and recorded only in YAML comments above the steps in `galaxy-workflow-draft.gxwf.yml`. `gxwf draft-extract` re-serializes from a parsed model and destroys every comment, so `galaxy-workflow.gxwf.yml` — the artifact the test, validate, run and any later maturation phase actually consume — carries no trace of either. The values themselves are correct in the extract (verified at iteration 26); what is missing is any reason a later editor would know not to change them.
Not closable inside the per-step loop: the loop's only writable surface is the draft, and the draft's comments are exactly what the extract drops. Closable by (a) folding both couplings into the step `doc:` strings of the two DESeq2 nodes and the four filters — `doc:` survives extraction and is visible in the Galaxy editor — or (b) making the test plan assert the column contract directly (a test on `deseq_out` that the padj column is c7, or on a filtered table that its top row survived), which converts a silent invariant into a failing test. (a) is the cheap one and should be done before the draft is discarded; (b) is the one that would actually hold. Filed upstream as `draft-extract-destroys-yaml-comments` in `foundry-feedback.ledger.yml`; that is the tool-side gap, this entry is the run-side obligation it leaves behind. ADVANCED, NOT CLOSED, by freeform-summary-to-galaxy-test-plan. Route (b) was taken and it lands unevenly across the two invariants. INVARIANT 1 is now fully detectable: both test cases assert `has_n_columns: 7` on BOTH `DESeq2 results:` tables, and case 1 asserts it again on both `Significant genes:` tables, so any shrinkage setting other than `none` on either node drops the `stat` column and fails the assertion before the `c7<` predicate can silently filter on nothing. A `not_has_text: baseMean` assertion on both raw tables covers the headerless premise the same predicates rest on. INVARIANT 2 is only partly detectable, and the plan says so in warnings[] `reference-level-filter-regression-undetectable`: of the seven Filter1 steps binding `header_lines: '0'`, a regression is visible on two — `Select first-contrast samples` and `Select second-contrast samples`, where the leaked metadata row would add a fifth sample column to that contrast's normalized-counts table, which both cases assert against. On `Select reference-level samples` the leaked row IS `AR0382_A`, which the `c2=='AR0382'` predicate keeps anyway, so the output is byte-identical and no assertion can see it. On the four significance filters the leaked row is row 1 of a padj-sorted table, which passes both filters on its own merits, so the regression is invisible at workflow level. Route (a) — folding both couplings into the `doc:` strings of the two DESeq2 nodes and the seven filters, which survives `draft-extract` — was NOT done and remains the cheap half of this entry. It is now the only protection the three undetectable reference-level and significance filters have. Do it before `galaxy-workflow-draft.gxwf.yml` is discarded. FURTHER ADVANCED by implement-galaxy-workflow-test, on the invariant-2 half only. The normalized-counts header width is no longer an assumption: deseq2.R at the pinned 2.11.40.8+galaxy4 writes `counts_out` with `write.table(..., col.names = NA)`, which emits a header padded with a leading blank field, so a four- sample contrast has exactly five tab-separated fields on line 1. Both test cases now assert `has_n_columns: 5` on both normalized-counts tables, and case 1 keeps the SCF1 data-row field-count assertion alongside it. A `header_lines: '1'` regression on either contrast-selecting Filter1 leaks a fifth sample column and fails both. The three undetectable filters - `Select reference-level samples` and the four significance filters - are unchanged, and route (a), folding both couplings into the `doc:` strings that survive `draft-extract`, is still not done and is still the only protection they have. RE-VERIFIED AT THE TERMINAL GATE by validate-galaxy-workflow, still not closed. Both couplings hold in `galaxy-workflow.gxwf.yml` as assembled: `advanced_options.lfc_shrinkage_type: 'none'` on both DESeq2 nodes (neither selects `many_contrasts`/`split_output`; both are `select_data.how: datasets_per_level`), and `header_lines: '0'` as the string `'0'` on all seven Filter1 steps (8, 12, 16, 23, 24, 25, 26 — the Filter1 count is exactly seven). Two refinements to the entry's own statement of invariant 1, from reading the assembled artifact. The `"c7<"` predicate is hard-coded in ONE `compose_text_param` bridge, step 21, consumed by two Filter1 steps (23 and 25); the second bridge, step 22, is `"abs(c3)>"`, consumed by 24 and 26. Since `stat` sits at c5, dropping it moves padj from c7 to c6 but leaves log2FoldChange at c3 — so the shrinkage coupling protects the padj predicate specifically, and the `abs(c3)>` bridge is not exposed to it. Route (a) remains not done and the terminal artifact confirms where the hole is: the four significance filters' `doc:` strings do name their `c7<` / `abs(c3)>` predicates and the three condition-split filters' docs do say "headerless", but NEITHER DESeq2 node's `doc:` mentions shrinkage at all, and no filter doc says why `header_lines` is `'0'`. An editor changing `lfc_shrinkage_type` has nothing in the runnable artifact to warn them. `galaxy-workflow-draft.gxwf.yml` still holds the rationale in the YAML comments `draft-extract` strips; do not discard it before the `doc:` strings are written.
The paper gives only "Cutadapt with a Phred cutoff score of 20". No adapter sequence appears anywhere in the supplement, and no minimum-length filter is stated. The interface reads the step as quality trimming only (`-q 20`, baked in) and exposes no adapter input.
If a later phase adds adapter trimming it is adding method the paper does not describe, and that addition has to be labelled as such rather than presented as a faithful port. STILL OPEN after phase 6 iteration 8 concretized `Quality-trim reads`, and deliberately so. The step pins lparsons/cutadapt 5.2+galaxy2 with all six adapter repeats written explicitly empty (adapters / front_adapters / anywhere_adapters on R1, the adapters2 / front_adapters2 / anywhere_adapters2 trio on R2) and `other_trimming_options.quality_cutoff: '20'`, which is the one Cutadapt parameter the paper actually states. The unstated minimum-length filter is now recorded concretely: the pinned wrapper's default is `filter_options.minimum_length: 1`, which the XML flags as a deliberate wrapper-side departure from cutadapt's own default of 0 ("intentionally set to 1 ... to avoid hard to debug issues with downstream tools"). The step writes that 1 out explicitly so a later wrapper bump cannot move it silently -- but it is the WRAPPER's default, not the paper's parameter, and the corpus shows the value is genuinely chosen per workflow (cutandrun at fe41a79 uses 15, the VGP workflows use 1). Closing this entry requires a stated adapter and a stated length cut, neither of which exists in the source.
Every differential-expression figure this run has is from a different implementation than the workflow runs. Phase 8 measured SCF1's log2 fold change and adjusted p-value with pydeseq2 over a strand-aware count matrix built from phase 7's six bwa-mem BAMs, as two separate two-level n=2 analyses with no LFC shrinkage — tnSWI1 vs AR0382 log2FC -6.93 / padj 2.1e-31, AR0387 vs AR0382 log2FC -7.95 / padj 4.1e-18. The workflow runs the Galaxy DESeq2 wrapper, i.e. R DESeq2, over RNA STAR plus featureCounts -s 2. pydeseq2 is a faithful reimplementation, but it is not the same implementation and the counts are not the same counts, so the exact values will differ. What is unverified is not the biology — the separation is ~200-fold at the counts level and the padj margins are 18 and 31 orders of magnitude — but that the workflow's own node produces a row for B9J08_001458 clearing `padj < 0.05` and `|log2FC| > 1` into each significant table with `log2FoldChange <= -2`, which is what the test plan asserts, and that the raw table really is the 7-column shape both cases assert `has_n_columns: 7` on.
Closable by the first successful invocation and nothing cheaper; phase 11 (run-workflow-test) is where it lands, and phase 9 should not try to settle it by adding assertions. The plan is already written so that this can only bite as a margin question rather than a value question: no assertion anywhere pins an exact statistic, and the recorded measurements are stated as headroom over thresholds (~5 and ~6 log2 units over the -2 floor; 31 and 18 orders of magnitude over the 0.05 cut). Three things to check against the first invocation while it is open — the sign and magnitude of SCF1's c3 in both contrasts, that c7 is padj and not something else, and whether R DESeq2's independent filtering keeps SCF1 in the padj-carrying set at this depth as pydeseq2 did (2393 and 1376 genes retained). Recorded in the plan as warnings[] `deseq2-statistics-measured-with-pydeseq2-not-r`.
The paper gives no version for any Galaxy step (FastQC, Cutadapt, RNA STAR, featureCounts, DESeq2). Only non-Galaxy software is versioned (R 4.0.3, DescTools 0.99.49, survminer 0.4.9, Fiji 1.52, CellProfiler 3.1.9). The constructed workflow will pin its own versions and can reproduce the authors' method but never their software stack.
Unclosable from the source. Expected to be surrendered at the terminal and stated on the workflow itself, so a reader does not mistake the run for a version-faithful reproduction.
Phases 4 and 5 changed the interface in four places that the phase-2 brief still states in its original form. The brief is the artifact the test plan reads for labels, and labels are the API, so the drift has to be visible rather than inferred by diffing two artifacts. (a) Interface input 7 `Minimum absolute fold change` (float, linear, default 2.0) is in the draft `log2 fold change threshold` (float, log2 units, default 1.0) — closed on corpus evidence by `fold-change-threshold-linear-vs-deseq2-log2fc`. (b) Interface input 5 `featureCounts strandedness` (free text, allowed values named only in prose) is in the draft `Strandedness` (text restricted to `stranded - forward` / `stranded - reverse` / `unstranded`, default `stranded - reverse`) feeding a `map_param_value` bridge with `on_unmapped: fail` — the corpus idiom adopted per the IWC comparison section 3.6. (c) Two inputs the brief does not declare at all: `First contrast condition level` and `Second contrast condition level` — see `deseq2-factor-level-names-not-parameterized`. (d) Fourteen declared outputs become sixteen — see `deseq2-normalized-counts-and-plots-promoted-per-contrast`.
Bookkeeping, not a design gap: every one of the four changes is itself settled and recorded, and the draft is the current statement of the interface. What is unmet is that no Mold in this run's roster owns freeform-galaxy-interface.md after phase 2, so the edits have no writer. Recorded here so the test-plan phase reads labels off the draft rather than off the stale brief, and so a later maturation pass knows which artifact was authoritative. Closable by editing the brief's sections 2.3 and 3, or by declaring the draft authoritative for the interface and saying so in the brief.
IWC has no single workflow spanning FastQC to DESeq2. At corpus fe41a79 the journey is published as two workflows joined at the count-table boundary: transcriptomics/rnaseq-pe/rnaseq-pe ends at per-sample count tables, and transcriptomics/rnaseq-de/rnaseq-de-filtering-plotting begins there, taking pre-grouped `list` collections of count tables as its workflow inputs. This run's design is one workflow spanning both halves, which is why it needs an in-workflow condition split at all — a workflow that starts from count tables can demand pre-grouped collections at its interface and never reconstruct grouping internally.
Not a defect in either design, and not a reason to re-scope: the paper describes one analysis and the run was asked to build it. Recorded because it is the root of the single largest divergence from corpus practice (see `no-iwc-precedent-for-sample-sheet-workflow-input`) and because any later mature-galaxy-workflow-for-iwc pass will meet it as a packaging question — a reviewer may ask why this is not two workflows. Whoever revisits it should weigh that the split also costs reusability of the DE half, which is presumably why IWC made it.
A tool mapped over a `sample_sheet`-family collection produces a `sample_sheet`-shaped output without `column_definitions` (packaged note galaxy-sample-sheet-collections, "Mapping rules"). The interface's output table calls outputs 1-8 `list` / `list:paired`. Behaviourally that is accurate — such a collection maps, reduces and filters exactly like a list — but the declared collection type string is not `list`, which matters to anything that type-checks, including workflow test assertions on collection type.
No topology consequence; the fix is either a corrected type column in the interface brief or an explicit statement that the promoted outputs are sample_sheet-shaped lists without column metadata. Flagged primarily for the test-plan phase, which is where a wrong collection_type assertion would surface as a confusing failure. ADVANCED, NOT CLOSED, by implement-galaxy-workflow-test. The test file authors no `attributes: {collection_type: ...}` assertion on any of the eight mapped outputs, as the test plan's `promoted- collection-type-string-unverified` instructs. What it asserts instead is `element_count` plus a named `element_tests` entry per identifier - twelve for the flattened FastQC outputs, six for every output on the sample axis - which is shape-agnostic and pins the thing that actually matters, the identifier space the condition split joins on. So no assertion in this run depends on the declared type string and nothing here forces the question. It stays open as an interface-brief accuracy item, and the real type strings should be read off the first successful invocation before anyone decides whether a collection_type assertion is worth having.
Searched across all of `workflows/` at corpus fe41a79: `sample_sheet` as a collection type appears in zero workflows; `__SAMPLE_SHEET_TO_TABULAR__` appears in zero workflows; `column_definitions` appears only as the literal `null` that newer Galaxy serializes onto ordinary collection inputs, never with a value. `__FILTER_FROM_FILE__` does appear, in six workflows, but none in transcriptomics and none for a condition split. So the second half of the data-flow brief's section 4.2 route is a real used Galaxy idiom while the sample-sheet half is unprecedented in published IWC practice, and no corpus fixture declares a sample-sheet collection either.
Absence of precedent is not refutation, and nothing in the corpus contradicts the route — the phase-3 reasoning stands on its own. What changes is the risk profile: this is the one region of the workflow with no worked example to pattern-match against, and two open entries ride on it (`sample-sheet-to-tabular-identifier-column-unverified`, `sample-sheet-input-test-fixture-expressibility`). Guidance for the template, from the comparison notes section 3.1: build the split as a clearly delimited region with the phase-2 fallback (a `data` input `Sample metadata table`) reachable by deleting one node, so that if either open entry resolves against the sample sheet the cost is a deletion rather than a redesign. Worth knowing for the record that IWC made the opposite trade deliberately — it pushes grouping into the interface as two pre-grouped count collections, which is the same shape the interface brief's section 2.1 rejected for hard-coding the design into the public API.
No T-DNA integration sites are produced by this run. The scope decision was taken by the harness before this phase.
Uncited in the strict sense: no Tool Shed search for extractSoftClipped and no Addgene or ref. 52 lookup for pTO128 was run in this phase; both reasons are carried from the source summary. Treat as a debt, not a finding. If a later phase's discovery step contradicts either reason, this entry's `because` is refuted and the entry must be reopened rather than left standing.
The interface settles the genome as a history `fasta` dataset on portability grounds (a remote-URL fixture resolves on any server; a CVMFS built-in index does not), with RNA STAR building its index at run time. Whether usegalaxy.org carries a built-in index for GCA_002759435.2 was not checked, and this run's phase roster contains no reference-data Mold that owns the question.
Provisionally settled, not verified. A built-in index would be cheaper at run time but less portable for tests; revisiting it changes workflow input 2.
The paper states a ~29-fold SCF1 expression difference between AR0382 and AR0387. Direct measurement of the deposited runs disagrees on magnitude by roughly an order of magnitude: aligned fragments over the SCF1 exon at 200,000 read pairs per sample give 414 and 391 in the two AR0382 replicates against 2 and 1 in AR0387 (~270-fold), and exact 31-mer matching against the SCF1 CDS over 5.2 M reads gives 1942.7 vs 12.3 per million (~158-fold). Direction and significance are not in doubt; only the ratio is. Candidates not distinguished here: the 29-fold may be the RT-qPCR assay rather than the RNA-seq, a shrunk rather than raw log2FC, or a different normalization. Reference bias is ruled out as the explanation — AR0387 IS the B8441 reference strain, so its SCF1 reads cannot be undercounted; if anything AR0382's divergent allele is undercounted, which makes the true ratio larger, not smaller.
Consequence for the test plan, and the reason this is filed rather than glossed: no assertion may pin the paper's 29-fold figure. `test-data-refs.json` asserts direction and significance only (`log2FoldChange < -2`, `padj < 0.05`), which both the paper and the data support. Settling this needs the paper's source data (Data S1) or the authors, not more compute. REINFORCED by freeform-summary-to-galaxy-test-plan, and one candidate explanation is now the leading one. The disagreement was previously measured only on raw counts and k-mer matches; phase 8 ran an actual differential-expression model — pydeseq2, two separate two-level n=2 analyses with NO LFC shrinkage, mirroring the workflow's two DESeq2 nodes, on a strand-aware count matrix built from phase 7's six BAMs. AR0387 vs AR0382 gives SCF1 log2FC -7.95 at padj 4.1e-18, i.e. ~247-fold, consistent with the ~270-fold raw fragment ratio and not with the paper's ~29-fold. tnSWI1 vs AR0382 gives log2FC -6.93 at padj 2.1e-31. Because that run applied no shrinkage, it does not rule out the "shrunk rather than raw log2FC" candidate — log2(29) = 4.86 against an unshrunk 7.95 is exactly the direction and roughly the magnitude a shrinkage estimator would move a gene whose counts are near zero in one group, so that candidate is now the most plausible of the three and the RT-qPCR candidate is not excluded either. This matters for the workflow, not just the paper: the workflow pins `lfc_shrinkage_type: none`, so ITS log2FoldChange for SCF1 will be the unshrunk ~-8 and will never reproduce the paper's figure by construction. Do not treat that as a defect when the first real invocation lands. Caveat on all of the above: pydeseq2 is not the R DESeq2 the Galaxy wrapper runs, and the counts came from bwa-mem rather than RNA STAR plus featureCounts -s 2; see `deseq2-statistics-unverified-against-the-galaxy-wrapper`.
`genomeSAindexNbases` exists ONLY in the RNA STAR history branch — it configures the in-job `--runMode genomeGenerate` that this workflow's history-FASTA delivery choice introduces, so it is not a mapping parameter the paper's "default parameters" could have covered. The wrapper's own help carries STAR's formula verbatim: "For small genomes, the parameter --genomeSAindexNbases must be scaled down to min(14, log2(GenomeLength)/2 - 1)". The step binds `'10'`, from min(14, log2(12.5e6)/2 - 1) = min(14, 10.79). The 12.5 Mb is the GCA_002759435.2 assembly record's length as carried through this run's briefs; nothing in this run measured the dataset that will actually be wired.
Silent-degradation class, not a hard gate: the wrapper default of 14 is the documented seg-fault-at-mapping hazard for a genome this small (the wrapper's own test data uses 5), and a value too small merely costs search speed. Two ways this becomes wrong: a different reference is supplied at run time (a mammalian genome would want 14 back), or the B8441 FASTA in hand differs materially in length from the assembly record. Settle it by reading the length of the actual dataset, or by reading STAR's own recommendation out of the promoted `STAR mapping summary` / job stderr on the first real run — STAR prints the recommended value when the supplied one is too large. Tied to `reference-genome-delivery-shape-unverified`: if that resolves to a built-in index, this parameter disappears with the branch. ADVANCED, NOT CLOSED, by paper-to-test-data. The entry asked for a measured length of the reference actually wired in. The reference this run will wire is now pinned — NCBI `GCA_002759435.2_Cand_auris_B8441_V2_genomic.fna.gz`, md5 a008b270d3aaa04736a8bb7daf3f6dd5 — and it was downloaded and measured: 12,365,959 bp across 15 contigs. min(14, log2(12365959)/2 - 1) = min(14, 10.78), so the bound `'10'` is correct for this dataset. What keeps the entry open is exactly the exposure it already named: the parameter is correct for the reference the TEST supplies, and a user who supplies a different genome at run time still gets a silently wrong value. The first real STAR run should confirm from the promoted `STAR mapping summary`.
The 200,000-read-pair subsets were generated and hashed but not published. They exist only in the producing session's scratchpad and do not survive it. At ~61 MB compressed for 12 files they are too large to commit alongside the workflow, so the IWC idiom applies: host them (Zenodo or equivalent) and reference them by URL. `test-data-refs.json` carries `<FIXTURE_BASE>` as the placeholder in `planemo_test_job_block`.
Not a research gap — the recipe is deterministic and the artifact pins it. Regenerate by streaming each ENA FASTQ and taking `head -n 800000`, then verify against the `md5_uncompressed` value recorded per file (hash the UNCOMPRESSED `.fastq`; gzip output is not byte-stable across gzip versions). The reference genome and annotation need no hosting: both are pinned to stable NCBI FTP URLs with md5s and are staged with `decompress: true`. ADVANCED, NOT CLOSED, by implement-galaxy-workflow-test. All twelve 200,000-read-pair fixtures were regenerated in this phase with the recorded recipe (`curl -L <ENA url> | gzip -dc | head -n 800000`) and every one verified byte-for-byte against its `md5_uncompressed` in test-data-refs.json before compression, so the recipe and the hashes are now proven reproducible rather than merely recorded. They live at `<run>/test-data/*.fastq.gz` (59 MB) and `galaxy-workflow.gxwf-tests.yml` addresses them as local `path:` entries, each with the SHA-1 of the `.gz` artifact as produced here. A twelve-file 25,000-read-pair set for the smoke case was derived from the verified 200k files by taking their first 100,000 lines, at `<run>/test-data/smoke-25k/` (6.7 MB). The test therefore runs against real data in this run directory and nowhere else: the fixtures are still unhosted and do not survive the run, which is the whole of what remains unmet. Publishing them is now a mechanical step - upload the exact `.gz` bytes, replace each `path:` with the published `location:`, and leave the SHA-1 values unchanged, because they pin those bytes. Keep the uncompressed md5s in test-data-refs.json as the regeneration check; the SHA-1s are the fetch-integrity check and the two answer different questions.
The workflow can build only the two contrasts whose input data is public. A test that tries to reproduce Fig. S2 has no input.
Cut by data availability, not by a design decision. Adding a fourth condition later needs no interface change: the reads input is a sample sheet whose `condition` restrictions widen.
The paper reports two contrasts (tnSWI1 vs AR0382, AR0387 vs AR0382) and states the significance thresholds, but writes no design formula. One factor, three levels, n = 2 is inferred from the six deposited runs. Whether the two contrasts come from one DESeq2 run over a three-level factor or two runs over two-level factors depends on the chosen wrapper's contrast handling, which is not known at interface time.
The interface fixes only the output surface — two result tables and two filtered tables, labelled per contrast. Either realization satisfies it. Closed by the IWC exemplar comparison, section 3.2, against transcriptomics/rnaseq-de/rnaseq-de-filtering-plotting at corpus fe41a79. Two DESeq2 nodes, each with a two-level factor, the reference-level counts collection feeding both: H1 (AR0382, tnSWI1) and H2 (AR0382, AR0387). Evidence: the exemplar's DESeq2 step uses `select_data.how: datasets_per_level` with a `rep_factorName_0.rep_factorLevel` repeat of exactly two levels, each port consuming a whole collection through the wrapper's multiple=true input; its `deseq_out` is a single dataset, not a collection, proved by the single-dataset tools it feeds (deg_annotate, tp_cat, two Filter1 steps) and by the sibling -tests.yml asserting has_text_matching on the derived output as a dataset; and the workflow README bounds itself to "exactly 2 conditions with at least 2 replicates per condition". One DESeq2 job therefore yields one results table, so the interface's two distinctly labelled result tables require two jobs. A three-level factor is expressible via the repeat but would still emit one `deseq_out`, which cannot satisfy a two-output interface — so the output surface settles the arity regardless of the wrapper's three-level contrast behaviour. Upstream wiring is unaffected, exactly as the data-flow brief's section 4.4 predicted: nodes E, F1-F3 and G1-G3 are identical either way. One consequence for the interface: under two DESeq2 nodes, `DESeq2 normalized counts` (output 9) and `DESeq2 diagnostic plots` (output 14) are each produced twice while the interface declares one of each. Promote from one designated run and say which, or relabel per contrast; do not leave it implicit in the template.
The condition split needs one literal condition value per factor level to filter the sample metadata table (AR0382, AR0387, tnSWI1). The interface exposes only `Reference condition level` (input 4, default AR0382). The two contrast levels have no parameter, so as the interface stands they would be baked into two filter steps — re-hard-coding into the steps the three-level design that the sample-sheet input was chosen to keep out of the public interface.
Two ways to settle it, both changing the interface's parameter surface rather than the topology: expose two more text parameters (one per contrast level) alongside the existing reference-level parameter, or accept the bake and state plainly in the interface that the three level names are fixed in the steps. Whichever is chosen must be reflected back into freeform-galaxy-interface.md section 2.3, not applied silently in the template. Closed by the template on the first of those two options. The draft exposes two further `text` workflow inputs, `First contrast condition level` (default tnSWI1) and `Second contrast condition level` (default AR0387), alongside the existing `Reference condition level` (default AR0382). Each feeds a `compose_text_param` step that builds the row predicate for its level, so no level literal is baked into any step. The deciding argument is symmetry: the reference level was already a parameter for exactly this reason, and baking the other two would have left the interface half-generalized while re-hard-coding into the steps the design the sample-sheet input was chosen to keep out of the interface. The cost is two inputs the phase-2 brief does not declare, which is one facet of the interface drift recorded in `interface-brief-output-and-parameter-surface-drifted-from-draft`; that entry carries the owed edit to freeform-galaxy-interface.md section 2.3, since this Mold does not own that artifact. One caveat the draft states on both new inputs: the contrast output labels (`... tnSWI1 vs AR0382`, `... AR0387 vs AR0382`) are the public API and are not derived from these parameters, so changing a level value makes the labels stale.
`deseq2-contrast-realization-unsettled` closed on two DESeq2 nodes, which makes `DESeq2 normalized counts` (interface output 9) and `DESeq2 diagnostic plots` (interface output 14) each produced twice while the interface declares one of each. The IWC comparison flagged the consequence and instructed the template not to leave it implicit: promote from one designated run and say which, or relabel per contrast.
Settled by relabelling per contrast. The draft promotes four outputs where the interface declared two: `DESeq2 normalized counts: tnSWI1 vs AR0382`, `DESeq2 normalized counts: AR0387 vs AR0382`, `DESeq2 diagnostic plots: tnSWI1 vs AR0382` and `DESeq2 diagnostic plots: AR0387 vs AR0382`, taking the workflow to sixteen outputs. This is not a tie broken on taste. Each DESeq2 job sees only its own four samples, so its size factors, normalized values and PCA/dispersion plots are computed over that subset: the two normalized-count tables are different tables, not two copies of one. Promoting either and calling it "the" normalized counts would be wrong rather than merely arbitrary, and dropping one would discard a real artifact while leaving the surviving one silently contrast-scoped. Relabelling also makes the four outputs symmetric with the result and filtered-gene tables, which are already labelled per contrast. One consequence for the test plan: the paper's sanity check that SCF1 sits in the top 2.5% of AR0382 expression can be asserted against either normalized-counts table, because both carry the two AR0382 replicates. Assert it on one and say which. The owed edit to freeform-galaxy-interface.md section 3 rides on `interface-brief-output-and-parameter-surface-drifted-from-draft`; labels are the public API and this is a breaking change to be made once, before any test is written.
The corpus exemplar binds `header_lines: "1"` on both of its Filter1 steps, but it filters a table that has been through `deg_annotate` and then `tp_cat`, and the tp_cat step exists precisely to concatenate a separately generated single-line header onto the DESeq2 output. That strongly implies the raw `deseq_out` is headerless. This workflow omits the annotation pair — the paper names no annotation step and the interface's tool set is closed — so its Filter1 steps consume `deseq_out` directly and the corpus binding cannot be copied across. Whether the pinned iuc/deseq2 wrapper writes a header row on `deseq_out` is not established by anything this run read.
Consequential if missed rather than untidy, which is why it is a ledger entry and not only a `_plan_state` line. Binding "1" against a headerless table silently drops the first gene row of every filtered result; binding "0" against a headed table passes the header line into a numeric comparison, where Filter1 discards it with a warning rather than an error. Either way the workflow produces a plausible table. The same binding applies to the three row filters in the condition-split region, where the unknown is the header behaviour of `__SAMPLE_SHEET_TO_TABULAR__` rather than of DESeq2 — a different producer, the same class of silent loss, tracked there by `sample-sheet-to-tabular-identifier-column-unverified`. Closable by a summarize-galaxy-tool pass on iuc/deseq2 during the per-step loop, or by inspecting the first real run's output. SETTLED in the per-step loop, at the step that pins the producer (`Differential expression: first contrast vs reference`, iuc/deseq2/deseq2 @ 2.11.40.8+galaxy4, changeset 05f9e54d7e81): `deseq_out` carries NO header row, so all four significance filters bind `header_lines: '0'` — the same value the three condition-split row filters already use, for an unrelated reason. Three independent lines of evidence, all at the pinned version. (1) The wrapper's own `deseq2.R` writes the result table with `write.table(out_df, file = opt$outfile, sep = "\t", quote = FALSE, row.names = FALSE, col.names = FALSE)` — `col.names = FALSE`, at both of the two call sites that write a result table (the single-contrast path and the `many_contrasts` loop). The same script writes `counts_out` with `col.names = NA`, so the normalized-counts table DOES carry a header: one tool, opposite answers for its two tabular outputs, which is exactly why this could not be settled by analogy. (2) `deseq2.xml`'s test assertions for `deseq_out` match a data row first (`FBgn0003360\t1933.9504…\t-2.8399…`) with `has_n_lines n="3999"` and assert no header, while `vst_out` and `counts_out` assertions in the same test block DO assert a sample-name header line. (3) The corpus chain explains its own "1": rnaseq-de MANUFACTURES the header it later skips — `tp_text_file_with_recurring_lines` emits the single line `GeneID__tc__Base mean__tc__log2(FC)…`, `tp_sed_tool` turns `__tc__` into tabs, and `tp_cat` (labelled `Annotate DESeq2 table`) concatenates it on top of the deg_annotate output; only then do its two Filter1 steps bind `header_lines: "1"`. That "1" is about the manufactured header, not about `deseq_out`, which confirms rather than contradicts the reading above. One rider the four filters inherit: the c3 = log2FC / c7 = padj column contract holds only while `lfc_shrinkage_type` is `none` — `get_result_output_columns()` drops the `stat` column under any shrinkage, moving padj to c6. The DESeq2 steps pin `none` explicitly for that reason.
The interface declares `FastQC raw reads: text summary` and `: HTML report` as `list` collections. FastQC consumes a single dataset, so mapping it over a `sample_sheet:paired` input fans out over the inner paired axis as well — 12 jobs, and outputs nested one report per read direction per sample, not six flat elements. As declared, outputs 1 and 2 are not what the workflow computes.
The data-flow brief (section 7.1) recommends promoting the nested collection and correcting the interface: per-read-direction QC is what a reader wants from raw-read QC, and it preserves the element identifier space that every checkpoint assertion keys on. The alternative is an explicit flatten node, which satisfies the declared `list` but rewrites identifiers to a doubled vocabulary (`AR0382_A_forward`) for no analytical gain. Either way the test plan must know which, because it changes every assertion on outputs 1 and 2. Closed by the IWC exemplar comparison, section 3.3, in favour of the explicit flatten — the option the data-flow brief rated as having no analytical gain. The brief's preference was a reasonable call made with no corpus evidence available; the evidence now exists and points the other way. transcriptomics/rnaseq-pe/rnaseq-pe at corpus fe41a79 puts an explicit `__FLATTEN__` step with `join_identifier: _` between the list:paired reads input and the per-fastq QC tool, and the consuming QC subworkflow declares its input as `collection_type: list` — flat. This is published IWC convention rather than a workaround, and it satisfies the interface's declared `list` shape for outputs 1 and 2 as originally written, so no interface correction is needed. Consequences the test plan must key on: the identifier vocabulary on outputs 1 and 2 doubles to twelve (AR0382_A_forward, AR0382_A_reverse, and so on for each of the six samples), while outputs 3-8 keep the six-element sample identifier space untouched, because the flatten is a side branch off the workflow input and does not enter the map-over region. The data-flow brief's placeholder transformation 5.5 is therefore required, not conditional.
The paper names the genome assembly (GCA_002759435.2, C. auris B8441) but never names a GTF/GFF for it. featureCounts requires one and RNA STAR uses one for splice junctions. NCBI RefSeq GFF and FungiDB B8441 GFF differ in gene ID space and attribute keys (`gene_id` vs `ID`), which changes the featureCounts `-g` attribute, every downstream gene identifier, and whether SCF1 appears as `B9J08_001458` at all. FungiDB and CGOB are cited in the paper only for synteny inspection, not as the counting annotation. The interface fixes the datatype as `gtf`; a GFF3 source would require a different datatype or a conversion step.
Largest single gap for reproducing this analysis. Whoever picks a source must record which one, because count values and gene IDs are not comparable across the two. CARRIED FORWARD by advance-galaxy-draft-step at `Count reads per gene`, which is now concrete and STILL DOES NOT CLOSE THIS. Both consumers are wired to the declared `Gene annotation GTF` input (RNA STAR `sjdbGTFfile`, featureCounts `anno|reference_gene_sets` under `anno_select: history`, case 2), so the PORTS are settled. What this step adds to the entry is a second dependent binding: `gff_feature_attribute: gene_id` (featureCounts `-g`), taken from the wrapper default and from corpus transcriptomics/rnaseq-pe/rnaseq-pe at fe41a79. That is correct for a GTF and wrong for a FungiDB B8441 GFF3, which keys on `ID` — and the failure mode is a complete, plausible counts table of the WRONG identifiers, not an error. `gff_feature_type: exon` has the same shape of exposure. So this entry now gates three things, not one: the input's datatype, the `-g` attribute, and the gene ID space every downstream result is addressed in. Naming the source settles all three at once; nothing else will. CLOSED by paper-to-test-data. The annotation is NCBI's own GTF for the exact accession the paper names: `GCA_002759435.2_Cand_auris_B8441_V2_genomic.gtf.gz` under `https://ftp.ncbi.nlm.nih.gov/genomes/all/GCA/002/759/435/GCA_002759435.2_Cand_auris_B8441_V2/` (md5 6e5b9528d48c0a8fc2c8588e6eeea929). It was downloaded and inspected, not merely cited. It settles all three things this entry gated, in the direction the workflow already assumes: it is a true GTF (`#gtf-version 2.2`), so the input's `gtf` datatype needs no conversion step; its `gene_id` values ARE the paper's locus tags, so `gff_feature_attribute: gene_id` is correct as bound and SCF1 resolves as `B9J08_001458` with no identifier translation (PEKT02000003.1:864995-867292, + strand, single exon, 2298 bp); and it carries 6057 `exon` features across 5586 distinct genes, so `gff_feature_type: exon` is correct too. The FungiDB-GFF3 hazard this entry described is avoided by not using FungiDB — nothing about that hazard was wrong, it simply does not arise for this source.
Workflow input 7 is `Minimum absolute fold change`, default 2.0, in linear units — the form the paper states (|fold change| > 2). DESeq2 reports log2FoldChange. The significance filter must therefore compare abs(log2FoldChange) > log2(threshold), which needs either a conversion the filter expression may not support or a restatement of the parameter. Nothing in the interface brief notes the mismatch.
Consequential if missed rather than merely untidy: comparing abs(log2FoldChange) > 2.0 applies a 4-fold cut and silently fails to reproduce the paper's gene lists, while still producing a plausible-looking filtered table. Options are a log2 conversion node before the filter, a filter expression that computes the conversion inline, or restating input 7 as a log2 threshold (default 1.0) with its label changed — the last changes the interface. Closed by the IWC exemplar comparison, section 2, on the third of those three options. transcriptomics/rnaseq-de/rnaseq-de-filtering-plotting at corpus fe41a79 exposes `log2 fold change threshold` as a float workflow input, default 1.0, documented as "A log2 FC of 3 equals to an absolute fold change of 8 (2^3)", and filters with the raw column and no conversion anywhere in the workflow: `abs(c3)>` concatenated with the connected float. There is no linear fold-change parameter, no log2() call and no conversion node in the corpus exemplar. So: restate interface input 7 as a log2 threshold with default 1.0 — which is exactly the paper's |fold change| > 2 — and change its label; teach the conversion in the doc string as the corpus does. Three implementation details come with the answer. (a) Column indices are c3 for log2FC and c7 for adjusted p-value; deg_annotate appends columns 8-13 and leaves 1-7 untouched, so the indices hold whether or not an annotation step is added. (b) Filter1's predicate is a text parameter, so a numeric workflow input cannot reach it directly: the corpus idiom is one iuc/compose_text_param step per threshold concatenating a literal prefix (`c7<`, `abs(c3)>`) with the connected float, making node I two Galaxy steps per filter rather than one. (c) The two thresholds are two chained Filter1 steps, p-adj then log2FC, each with header_lines "1", not one compound predicate. Restating input 7 changes the interface brief's section 2.3 and its output-3 table entry; that edit must be made there, not silently in the template.
The paper states only the library kit (Illumina Stranded Total RNA Prep with Ribo-Zero Plus). Reverse-stranded (dUTP) is inferred from that kit name and is not stated anywhere in the supplement. The interface exposes the parameter with default `reverse` so the inference is visible and changeable rather than buried in a step default.
Empirically checkable without new information: the promoted output `featureCounts assignment summary` shows a large Unassigned_NoFeatures fraction when the setting is wrong. A test-plan or run phase can close this from evidence. CLOSED by paper-to-test-data, empirically rather than by argument. The entry itself named the check; it was performed a step earlier than expected, on alignments rather than on a featureCounts summary. 200,000 read pairs from each of AR0382_A, AR0387_A and tnSWI1_A were aligned to GCA_002759435.2 with bwa-mem and each R1 that unambiguously overlapped a single annotated gene was compared against that gene's strand. 98.4% / 98.4% / 98.2% map ANTISENSE. That is the dUTP reverse-stranded signature and it is not a close call. `stranded - reverse` → featureCounts `-s 2` is correct; the value is now measured, not inferred from the kit name. The `featureCounts assignment summary` check this entry proposed remains valid as a regression guard and is carried into the test plan as an assertion.
The interface carries condition and replicate as `column_definitions` on a `sample_sheet:paired` reads input. Galaxy does not propagate `column_definitions` or per-row `columns` through map-over, so by the time featureCounts has produced counts the condition metadata is gone and an explicit step must reattach it (`__SAMPLE_SHEET_TO_TABULAR__` plus a filter/split, or the rules DSL) before DESeq2 can receive one multi-data input per factor level.
Named fallback if the wiring proves unbuildable: replace the sample-sheet input with a `list:paired` reads collection plus a `data` input `Sample metadata table` (tabular: sample_id, condition, replicate) and split on that table. Taking the fallback changes workflow input 1 and must be reflected back into the interface brief, not applied silently in the template. Closed by the data-flow brief, section 4. The condition reaches DESeq2 by an identifier-keyed split taken off the workflow input, where the column metadata still lives: `__SAMPLE_SHEET_TO_TABULAR__` projects the sample sheet to a tabular (element identifier, condition, replicate); a row filter plus column projection yields one identifier list per factor level; `__FILTER_FROM_FILE__` filters the featureCounts collection to each level's identifiers; each per-level counts sub-collection reduces into one DESeq2 multi-data factor port. The map-over region and every promoted per-sample output are untouched, and the join key is the element identifier, which Galaxy preserves across map-over. The route is independent of how the DESeq2 contrasts are realized (`deseq2-contrast-realization-unsettled`) and survives the named fallback intact: under the fallback the tabular node simply disappears and the user-supplied metadata table lands in its place, with the filter and collection-split nodes unchanged. Two narrower successors carry what remains: `sample-sheet-to-tabular-identifier-column-unverified` (is the element identifier emitted as a column) and `deseq2-factor-level-names-not-parameterized` (where the per-level literal comes from). Rejected alternatives and why are recorded in the brief's section 4.6 — notably filtering by element-identifier regex, which is unsafe here because `AR0382_tnSWI1_A` contains the reference level's own identifier as a substring.
Whether a `sample_sheet`-family workflow input — element identifiers plus per-row typed `columns` and collection-level `column_definitions` — can be expressed in a Planemo/IWC `-tests.yml` job block. If it cannot, the workflow's primary input is untestable as designed and the fallback in `sample-sheet-condition-to-deseq2-factor-wiring` becomes mandatory.
Not verified in this phase; no fixture syntax for sample-sheet inputs appears in the references packaged with this Mold. Naturally closed by the test-plan phase. CLOSED by paper-to-test-data: YES, it is expressible, and the `list:paired` fallback is not required. Galaxy's job-block loader has an explicit branch for it — `lib/galaxy/tool_util/cwl/util.py`, `replacement_collection()`: `if collection_type.startswith("sample_sheet"): kwds["rows"] = value.get("rows")`, carried to the collections API by `lib/galaxy/tool_util/client/staging.py`. The nested `sample_sheet:paired` shape specifically is covered by `test/unit/tool_util/test_cwl_util.py::test_galactic_job_json_sample_sheet_paired_collection`, and an end-to-end worked example ships as `lib/galaxy_test/workflow/collection_semantics_cat_sample_sheet.gxwf-tests.yml`. Two things a test author must get right. (1) `rows` is a mapping of element identifier to a POSITIONAL LIST of column values ordered to match `column_definitions` — authority is `validate_row()` in `lib/galaxy/model/dataset_collections/types/sample_sheet_util.py`, which rejects on `len(row) != len(column_definitions)` then `zip(row, column_definitions)`. The dict form appearing in Galaxy's own non-paired unit tests never reaches that validator and will not work. (2) Neither `galactic_job_json` nor `staging.py` passes `column_definitions` when creating the collection, although the API payload supports it, so a test-staged sample sheet has collection-level `column_definitions: None`. This is NOT fatal: per-element `columns` are still populated from `rows`, and the two `column_definitions_compatible()` call sites are both in `DataCollectionToolParameter` option-building (UI dropdown filtering), which a test bypasses by supplying the HDCA by id. The one real consequence is that `__SAMPLE_SHEET_TO_TABULAR__` emits its header line only `#if $include_headers and $input.collection.column_definitions` — harmless here because `Project sample sheet to tabular` sets `include_headers: false`, but a later phase that flips it to true will see the header silently vanish under test while it appears in the UI. Concrete job block is in `test-data-refs.json` under `planemo_test_job_block`.
The settled condition wiring joins the sample metadata table to the featureCounts collection on element identifier, so node E's output must carry that identifier as a column. The packaged note galaxy-sample-sheet-collections documents `__SAMPLE_SHEET_TO_TABULAR__` only as iterating elements and tab-joining "for downstream tabular consumers"; it does not state the output column set or ordering, and no other packaged reference covers it. Galaxy source was not readable from inside this run.
RESOLVED, affirmatively, by the tool summary itself. `__SAMPLE_SHEET_TO_TABULAR__` v1.0.0 caches cleanly from the Tool Shed API by bare id (unlike `__FLATTEN__`, whose collection output defeats gxwf's summary decoder), and its packaged help states the contract directly: "The first column is always the element identifier (sample name). The remaining columns match the metadata fields defined in the sample sheet." With the optional `include_headers` enabled the first header cell is literally `element_identifier`. So the identifier IS emitted, as column 1, and the ordering this region assumed throughout -- (element identifier, condition, replicate), metadata in column_definitions order -- is confirmed. All six provisional column bindings downstream (three Filter1 predicates on c2, three Cut1 projections of c1) stand unchanged; the Apply Rules substitute node named in the data-flow brief section 4.2 is not needed and was not built. The step is pinned with `include_headers: false`, which also settles the three row filters' `header_lines` at 0 -- that binding is no longer blocked on this entry. Evidence is the wrapper's own documented contract, not a corpus exemplar; `no-iwc-precedent-for-sample-sheet-workflow-input` is unaffected and stays open.
Every RNA STAR step in the corpus at fe41a79 — there are exactly two, in transcriptomics/rnaseq-pe/rnaseq-pe and transcriptomics/rnaseq-sr/rnaseq-sr — uses `refGenomeSource.geneSource: indexed`, selecting a built-in `genomeDir` through a `restrictOnConnections: true` string parameter with `sjdbGTFfile` supplied from the history. Their test jobs pass a plain genome string (`Reference genome: sacCer3`). The interface settles input 2 as a history FASTA, which is the right call for C. auris B8441 since no public server indexes it, but that is a different `__current_case__` in the iuc/rgrnastar wrapper with a different set of required sub-parameters, and the corpus has no example of it.
RESOLVED, affirmatively, from the wrapper rather than from the corpus — which is what the original note asked for. iuc/rgrnastar/rna_star @ 2.7.11b+galaxy1 caches and summarizes cleanly (all ten of its outputs are plain `data`, so the collection-output decode failure that blocks `__FLATTEN__` and lparsons/cutadapt does not apply here), and its schema answers every part of the question. `refGenomeSource.geneSource` publishes exactly two options, `indexed` and `history`; the HISTORY branch is real, is `__current_case__: 1`, and carries `genomeFastaFiles` (gx_data, formats fasta/fasta.gz, optional false), `genomeSAindexNbases` (integer, min 2 max 16, default 14), its own TWO-case `GTFconditional`, and `diploidconditional` (Yes=0 / No=1, default No). So the two TODO ports resolve to `refGenomeSource|genomeFastaFiles` and `refGenomeSource|GTFconditional|sjdbGTFfile`; the exemplar's `refGenomeSource|GTFconditional|genomeDir` exists only under `indexed` and is correctly absent. Under history, GTFconditional's option order (without-gtf, with-gtf) is the REVERSE of its `<when>` order (with-gtf, without-gtf), so `with-gtf` is case 0 — a case index that cannot be read off the dropdown. That the case index follows `<when>` document order is not assumed: `gxwf convert --to format2` over transcriptomics/rnaseq-pe/rnaseq-pe.ga at fe41a79 emits `geneSource: indexed` with `__current_case__: 0` and `GTFselect: without-gtf-with-gtf` with `__current_case__: 1`, matching the indexed branch's `<when>` order (with-gtf, without-gtf-with-gtf, without-gtf) and not its option order. Cross-read against rg_rnaStar.xml and macros.xml at tools-iuc main, which carries @TOOL_VERSION@ 2.7.11b / @VERSION_SUFFIX@ 1 — this exact version. Two consequences worth carrying: the history branch builds its index in-job via `STAR --runMode genomeGenerate` into tempstargenomedir, confirming the data-flow brief's no-separate-index-node call; and `output_log` and `mapped_reads` carry no `<filter>` whatsoever in `<outputs>`, so the promoted `STAR mapping summary` cannot be suppressed by any parameter choice here. The step now validates for real against the cache (`12 ok, 0 fail, 2 skip`), not by skip. `reference-genome-delivery-shape-unverified` is untouched and STAYS OPEN — this entry establishes that the history route WORKS, never that it is preferable to a built-in index nobody checked for. `featurecounts-annotation-source-unnamed` also stays open: the GTF PORT is now wired, the GTF SOURCE is still unnamed by the paper.
The interface declares `Trimmed reads` as `list:paired`. Whether the trimmer emits one paired-inner collection per sample or two parallel single-ended collections (R1, R2) is wrapper-dependent and was not resolvable in this phase, which pins no Tool Shed tools. If it is the latter, the design needs a re-pair node between trimming and alignment, or the alignment step must take two parallel collections in dot-product.
Conditional shape repair, not method: recorded in the data-flow brief as placeholder transformation 5.6 so the template does not assume one reading. Naturally closed by the IWC exemplar comparison or by tool discovery on the trimmer. Closed by the IWC exemplar comparison, section 3.4: one paired-inner collection per sample, no re-pair node. Neither transcriptomics exemplar uses Cutadapt — both use fastp — so the evidence comes from epigenetics/cutandrun/cutandrun at corpus fe41a79, cited for the wrapper's IO shape only and for nothing else, since CUT&RUN is a different domain. toolshed.g2.bx.psu.edu/repos/lparsons/cutadapt/cutadapt/5.2+galaxy2 driven from a list:paired collection with `library.type: paired_collection` declares outputs `out_pairs` (type `input`, i.e. the input collection's own shape) and `report` — one paired collection plus one report per element. Placeholder transformation 5.6 is therefore not needed and node C takes a single collection input, as the data-flow brief's primary reading assumed. The finding is consistent across both wrappers that could fill the trimmer slot: transcriptomics/rnaseq-pe/rnaseq-pe's fastp emits `output_paired_coll` the same way. The residual is version-scoped rather than structural — a future Cutadapt wrapper could change its output set, so the per-step loop should confirm the output name against whatever version it pins.
61 entries about the Foundry's own assets. Triage them with the report-foundry-run-feedback skill rather than filing from here.
The procedure's first and only prerequisite step is "Clone or pull and merge the IWC corpus (https://github.com/galaxyproject/iwc) to `~/.foundry/iwc`". That path is hard-coded, and the Mold names no environment variable, no configuration key, and no fallback for a machine where it cannot be created. On this machine it cannot: `mkdir ~/.foundry` fails with `Operation not permitted`, with and without the Bash sandbox. The Mold is the corpus-first check of every Galaxy-targeting pipeline, so an unsatisfiable corpus location makes the whole phase unrunnable, and the only reason this run produced a comparison at all is that the harness supplied a clone at a different path out of band. Nothing in the bundle describes that route, and an unattended run has no way to discover it.
A `TODO_<port>` sentinel marks a port whose NAME is unresolved, but there is no way to mark a port that is simply ABSENT from `in:` -- and an absent port is invisible to `gxwf draft-next-step`, whose `work` list enumerates sentinels and `_plan_*` fields and so reports nothing at all. The obligation survives only as prose. Distinct from `draft-format-sentinel-hint-cannot-express-a-conditional-nested-port`, which is about a sentinel whose hint has the wrong SHAPE; here there is no sentinel to misread.
The procedure's resolve step branches on the draft tiers and sends the reader to `galaxy-workflow-draft-format` for them, but the Mold does not package that note: the bundle's references are three gxwf CLI pages, `galaxy-tool-job-failure-reference`, `open-requirements-ledger`, and the tool-summary and draft JSON Schemas. The Runtime Notes then forbid reading Foundry source files at runtime, so the one document the procedure names for the distinction it asks the iteration to make is unreachable from inside the bundle. The draft JSON Schema does not carry the tiers — they are prose.
The procedure names `galaxy-tool-cache list` as the way to resolve a stock tool's version, and the per-step validator it mandates only produces a real verdict when given `--cache-dir` with the step's tool cached. Neither the cache nor the `galaxy-tool-cache` binary appears anywhere in the bundle's contract: `_required_tools.json` declares `gxwf` alone, derived from the three `gxwf` commands the Mold cites, and no step of the procedure says to populate a cache. Without it `draft-validate --concrete` reports `skip_tool_not_found` for every step and the iteration's green is vacuous.
The validate step is written as a two-valued gate — "On green, return; on red, route per the failure-routing rules" — but `draft-validate --concrete` reports three tool-state outcomes, `ok`, `fail`, and `skip`. A step reported `skip` had its tool state checked against nothing at all, yet the run prints `Concrete: OK` and the procedure's green branch accepts it. The Mold never says which of the two branches a skip belongs to, that a skipped step's state is unvalidated, or that some skips are permanent and cannot be cleared by priming the cache. An iteration reading only this Mold would return green on a draft whose collection-plumbing steps have never been validated, and no later phase would know which steps those were.
A Mold that writes one large artifact and validates it at the end has a window in which the artifact exists on disk and nothing records whether it conforms to its schema. This run entered that window: phase 8's session was terminated after writing a 67 KB `galaxy-test-plan.yml` and before running `foundry validate-galaxy-workflow-test-plan`, and before its ledger pass. The run-lifecycle section addresses only run status — it says a hard interruption leaves the run `running`, "which is distinguishable from success without a recovery write" — and says nothing about the phase's declared OUTPUT. So `status: running` on a phase is ambiguous in a way that matters: the artifact may be absent, present and valid, present and invalid, or present and half-written, and the four look identical from the ledger. A resuming agent, or a next phase that simply reads the artifact because it is there, has nothing telling it the verify step never ran.
The artifact is described as a "Cleaned gxformat2 conversion (via convert --to format2 --compact) of the nearest IWC exemplar's relevant subgraph", and separately as "bounded to the relevant subgraph, not the whole workflow". Those two requirements cannot both hold literally. `convert` emits the whole workflow; bounding it to a subgraph is hand surgery that necessarily leaves dangling `source:` references to removed steps and outputs, so the result is well-formed YAML but not a loadable gxformat2 workflow. The Mold never says which property matters, and the downstream consumer is named only as something that "pattern-matches against" the file — which does not distinguish a human-read reference from a parsed one. This run resolved it by treating the artifact as a reading aid, eliding non-structural tool_state and marking every elision inline, but a template Mold that tried to load the file would fail, and nothing warned it.
This CORRECTS the root cause and the scope recorded in `gxwf-cannot-decode-collection-outputs-of-builtin-collection-operations`, which is filed against the same decode failure. Two things in that entry are wrong. (1) The collection output does NOT "carry no `structure`". The Tool Shed serializes a collection output FLAT — `collection_type`, `collection_type_source`, `collection_type_from_rules`, `structured_like` and `discover_datasets` all sit at the top level of the output object — while the decoder expects exactly those five fields nested inside a `structure` object. Every field the decoder wants is present in the payload; only the nesting differs. The correction that entry proposes — make `structure` optional, or default it — would therefore decode cutadapt's `out_pairs` with a NULL collection type when the API plainly said `"collection_type": "paired"`. That is worse than the current loud failure: downstream shape reasoning would silently lose the one fact the output exists to carry. (2) It is not a built-in phenomenon. The trigger is a `<collection>` output, wherever it occurs. `toolshed.g2.bx.psu.edu/repos/lparsons/cutadapt/cutadapt` — a mainstream IUC-maintained wrapper pinned by eight workflows in IWC at fe41a79 — fails identically at every version tried, because its paired-collection branch declares `<collection name="out_pairs" type="paired">`. So the blast radius is not "the collection-operation built-ins a draft uses for plumbing"; it is every tool with a collection output, which includes a large share of the wrappers real Galaxy workflows are built from. Any such step is permanently unvalidatable by `draft-validate --concrete`, and no cache priming can help.
The contract partitions ownership between data-flow, template and step implementation, and the Mold declares the interface brief as an input that "pins inputs, outputs, and labels". Neither says what to do when the data-flow analysis shows a pinned interface decision is not achievable under Galaxy semantics. That happened three times in this run: FastQC mapped over a sample_sheet:paired input fans out per read direction, so two outputs the interface declared as flat lists are nested; the outer axis of every mapped output is sample_sheet- shaped, not the declared `list`; and a linear fold-change parameter is declared against a DESeq2 column reported in log2. Whether the data-flow brief may correct the interface, must defer to it, or must only record the conflict is not stated anywhere in the bundle. The route taken here — ledger each one and state the correction in the brief — was invented.
All four packaged pattern references are MOC index pages. Each is a list of wiki-links to operation and recipe pages — sync-collections-by-identifier, collection-cleanup-after- mapover-failure, tabular-to-collection-by-row, tabular-filter-by-column-value and roughly thirty others — and none of those pages is in the bundle, while the Mold's runtime notes forbid reading Foundry source at runtime. The collection MOC states its own role plainly: "the operation and recipe pages are the actionable references". So the Mold packages the index and withholds the content it indexes. Every collection-idiom choice in this run's brief was made from a one-line MOC description, with no corpus-observed recipe available to check the shape, the built-in tool id, or the failure modes against.
`gxwf draft-extract` re-serializes the workflow from a parsed data model, so every YAML `#` comment in the draft is destroyed. The note describes the command as three subtractive operations — drop drafty steps, strip `_plan_*`, promote `class` — and its Output and Gotchas sections say nothing about comments. Nothing in this Mold's bundle does either. The loss is total and silent: it is not reported in `--report-json` (which counts only dropped steps, dropped outputs and rewritten inputs) and the extracted file validates clean, so no gate in the pipeline can see it. It matters because a per-step draft loop has nowhere else to put the reasoning: `_plan_*` fields MUST be deleted at concretion or `draft-next-step` re-selects the step forever, and the gxformat2 `doc:` field is user-facing prose, not a place for a corpus citation or a cross-step warning. A run that parks that material in comments above each step — as this one did, and as the concrete steps of a template naturally invite — loses all of it at the exact moment the artifact becomes the one downstream Molds consume. The two surfaces that DO survive are `doc:` and the gxformat2 `comments:` block (frames/markdown), and the note names neither as the place to put anything load-bearing.
Every annotation the draft format offers is scoped to ONE step (`TODO_*`, `_plan_*`, the tier vocabulary), so an invariant that binds two or more steps has nowhere to live except prose inside one of them -- and `advance-galaxy-draft-step` advances exactly one step per invocation, so it is also structurally unable to check one. The step's own `_plan_state` said it outright: "Bind both steps together so the factor name, level ordering, header flag and output_selector cannot drift apart between the two contrasts." That is an instruction with no mechanism behind it. The author is the only enforcement, and only if they happen to read a sibling step's plan prose while implementing a different step.
The note's "Example (sketch)" declares a workflow input as `format: fastqsanger.gz` — a bare scalar. The draft schema the same Mold packages types `format` as `null | ReadonlyArray<string>`, so the sketch is not valid against the contract it illustrates. An author who copies the sketch, as it invites, gets a structure error.
The note never says which step field a connection's `source:` resolves against. Its only example uses the map form, where the step key and its identity are the same string, so the question cannot arise there. In the list form a step may carry both `id` and `label`, and `gxwf draft-validate` resolves `source:` against the LABEL when one is present — an `id` that differs from the label is not addressable at all.
The `TODO_<port>` input sentinel carries the port's semantic hint as a FLAT identifier, so it cannot express a port that lives inside a conditional -- and it reads as though no qualification were needed. `__FILTER_FROM_FILE__` has one top-level `input` and a `filter_source` that exists only inside the `how` conditional, but the template wrote both as siblings, `TODO_input` and `TODO_filter_source`. The concrete `in:` key for the second is `how|filter_source`; a literal reading of the sentinel yields a bare `filter_source:`, which connects nothing. `gxwf draft-next-step` then restates the flat hint verbatim -- "assign the real wrapper input port name (semantic hint: 'filter_source')" -- so the loop's own work list repeats the wrong shape at the moment the author acts on it. The note does define qualified keys elsewhere (concrete steps in the same draft carry `select_data|rep_factorName_0|rep_factorLevel_0|countsFile`), so the shape is expressible; what is missing is any statement that the sentinel's hint is a NAME rather than a PATH, and that concretion may have to qualify it.
The Identity-pinned tier admits "a source summary that names a specific `tool_id` with evidence", and the Mold's source-tendency paragraph relaxes that to "a free-form source that does name a specific tool/version with evidence hardens to the matching tier". Two paragraphs earlier the same note forbids pinning "on plausibility". A paper naming "FastQC" names a piece of software, not a Galaxy `tool_id`; reading the tier rule literally licenses writing a Tool Shed path from memory, which is exactly the plausibility pin the note forbids. The note never distinguishes the two kinds of naming, and they come apart on every free-form source.
A sample_sheet collection staged from a Planemo/gxwf test `job:` block never carries collection-level `column_definitions`, although the API it posts to accepts them. `galactic_job_json`'s `replacement_collection()` passes only `rows` and `name` for a sample_sheet collection type, and `StagingInterface`'s `create_collection_func` has no `column_definitions` parameter to pass — while `CreateNewCollectionPayload` declares the field and `SampleSheetDatasetCollectionType.generate_elements` reads it. The result is that a workflow input declaring `column_definitions` is, under test, fed a collection that has none. Per-element `columns` survive, so most workflows still behave, and the two `column_definitions_compatible()` call sites are both in `DataCollectionToolParameter` option-building (UI dropdown filtering) which a test bypasses by supplying the HDCA by id — so the divergence is silent rather than caught. It becomes visible at `__SAMPLE_SHEET_TO_TABULAR__`, whose header line is emitted only `#if $include_headers and $input.collection.column_definitions`: a workflow using that tool with headers on produces a header in the UI and no header under test, and no gate reports the difference.
`gxwf draft-validate --concrete` cannot bring the built-in `__FLATTEN__` into its tool cache. The fetch is reported as `toolshed fetch failed ... for __FLATTEN__`, but the accompanying dump is a decode failure against the tool schema, not a transport error: `["outputs"][0]` is decoded as the collection-output branch and `["structure"]` `is missing`. The tool's collection output carries no `structure`, and the schema requires one. The step is then reported `skip_tool_not_found`, so its tool state is never validated — and no amount of cache priming can fix it, because the tool cannot be decoded into the cache in the first place. The same shape is likely to hit the other collection-operation built-ins (`__UNZIP_COLLECTION__`, `__FILTER_FROM_FILE__`, `__APPLY_RULES__`), which are exactly the steps a Galaxy draft uses for collection plumbing.
`gxwf validate` (and `draft-validate --concrete`) drops a `component_value: {__class__: ConnectedValue}` from a step's tool state before validating it when the state is written under the format2 key `state:`, but honours it under `tool_state:`. The same bytes under the two keys therefore get opposite verdicts. Any conditional parameter case whose generated `workflow_step_linked` branch marks `component_value` as required — every non-text case — then fails as "component_value: is missing", and the anyOf fallback reports the misleading "select_param_type: Expected \"text\", actual \"float\"" alongside it.
Three surfaces of the same CLI disagree about whether format2 `tool_state` may carry the `__`-prefixed bookkeeping keys Galaxy writes into conditionals and repeats. For `iuc/map_param_value/map_param_value` 0.2.0, `galaxy-tool-cache summarize` generates `input_schemas.workflow_step_linked` with `additionalProperties: false` on every conditional branch object (allowing only `type`, `input_param`, `mappings`) and on every `mappings` item (allowing only `from`, `to`). That schema rejects `__current_case__` and `__index__`. But `gxwf convert --to format2` of a real Galaxy workflow emits exactly those keys, and `gxwf draft-validate --concrete` accepts a state block containing them. The generated schema is therefore stricter than both the converter that produces format2 and the validator that checks it — and it is the one surface a Mold is told to author against. implement-galaxy-tool-step step 2 says to "shape the step's `state` against `input_schemas.workflow_step_linked`"; following that literally produces a state block that diverges from what Galaxy round-trips, and no gate reports the divergence, because the validator is the permissive one.
gxwf's two validation paths demand opposite tool-state keys, so no format2 workflow can satisfy both. `--strict-encoding` rejects `tool_state:` on every step with `uses "tool_state" instead of "state" (format2 should use "state")` and exits 2, while the tool-state validator silently drops a `{__class__: ConnectedValue}` placeholder under `state:` and honours it only under `tool_state:` (this ledger's `gxwf-drops-connectedvalue-under-format2-state-key`). Moving to the key `--strict-encoding` wants reintroduces that bug; staying on the key that works means `--strict` can never be part of a Foundry gate. `gxwf convert --to format2` emits `tool_state:`, so gxwf's own converter produces output its own `--strict-encoding` rejects.
`gxwf validate --json` does not put JSON, and only JSON, on stdout, so the documented machine interface cannot be consumed by parsing stdout. Two separate breaches. (1) Every uncached tool that fails to decode prints a multi-line `toolshed fetch failed (...) for <id>:` block to stdout ahead of the JSON document - the full Effect schema type, roughly 55 lines for seven such tools - so `JSON.parse(stdout)` throws and the report has to be located by scanning backwards for a line that is exactly `{`. (2) Adding any strict flag makes gxwf abandon JSON entirely: exit 2, plain-text diagnostics on stderr, and stdout completely empty, although `--json` was passed.
`gxwf validate` never checks that a step's `in:` keys name real parameters of the pinned tool, even when that tool is fully cached and its state validates. A step wiring a connection to a port the tool does not have is reported `tool_state: OK` and counted in the validated total. The same permissiveness runs the other way inside `tool_state:`: an unknown extra parameter is accepted, and a required parameter that is simply absent is accepted. What the validator actually checks is the type and value of the parameters that happen to be present. No strictness flag changes this - not `--strict-structure`, not `--strict-state`, not `--strict`. This matters most exactly where Galaxy's own syntax is easiest to get wrong: a conditional's nested port must be qualified (`how|filter_source`), and the unqualified spelling is silently accepted.
Both Molds assume a tool summary always exists. advance-galaxy-draft-step's sequence step 3 is an unconditional "Invoke summarize-galaxy-tool on the resolved wrapper", and implement-galaxy-tool-step's sequence step 2 is an unconditional "Read the galaxy-tool-summary manifest". Neither says what to do when the summary cannot be produced at all. That is not a hypothetical: summarize-galaxy-tool can only summarize what `galaxy-tool-cache add` could decode, and a wrapper with a collection output cannot be decoded (see `collection-output-decode-is-a-flat-vs-nested-shape-mismatch-not-a-missing-field`). The Mold's one nearby escape hatch does not cover it either — "If `input_schemas` is `null`, consult `warnings[]`" presupposes a manifest with a warnings array, and here there is no manifest. The result is an author improvising the most consequential part of a step, its tool state, with no stated evidence standard, on the first genuinely complex wrapper of the run.
The procedure says to "shape the step's `state` against `input_schemas.workflow_step_linked`" and speaks of `state` throughout, but the galaxy-workflow-draft schema it packages admits both `state` and `tool_state` on a step and the Mold never says which to write. The two are not interchangeable in practice: under gxwf 1.10.1 a connected non-text parameter validates under `tool_state:` and fails under `state:` (filed as `gxwf-drops-connectedvalue-under-format2-state-key`), so following the Mold's own wording is what produces the red verdict.
The Mold speaks of "nearest IWC exemplar(s)" in its summary, its procedure and its confidence table, but the gxformat2 artifact is specified strictly in the singular — one declared filename, "the nearest exemplar's relevant subgraph", "Once the nearest exemplar is chosen (High or Medium confidence), convert it". Nothing says what to emit when the honest answer is several exemplars covering disjoint parts of the subject. That is what happened here and it was not an edge case: IWC publishes the subject's journey as two workflows joined at the count-table boundary, so one exemplar covers the map-over head and a different one covers the differential-expression tail, and a third workflow in another domain was the only corpus source for one tool's output shape. Whether that is one file with several YAML documents, several files, or a single forced choice was guessed at.
Section 2e states normatively "Note: outer `collection_type: list:paired`, inner `type: paired` (not `collection_type:`)", and section 2f's nested-output example omits the `class: Collection` discriminator on the outer `element_tests` entry. Both forms are faithful transcriptions of the IWC corpus and both are rejected by `references/schemas/ tests-format.schema.json`, which the same Mold packages and names as the gate its output must pass. An agent that follows the note produces a file that fails the Mold's own step 4, with 48 unhelpful `oneOf` errors pointing at the whole job input rather than at the offending key. The note is the only packaged guidance on these two shapes, so there is nothing else to fall back to.
The open-requirements ledger is the only artifact that carries a decision from one iteration of the per-step loop to a later one, but the Mold never reads it as an INPUT to binding. Its `Inputs` section declares the ledger carries "the run's open, resolved, and surrendered entries", and then the procedure's single ledger touchpoint is step 5, AFTER implement: "Inspect the open-requirements-ledger for a new `open` blocking entry ... appended against this step." New, open, post-hoc. A decision settled five iterations earlier lives in a `resolved` entry -- whose `note` field is where the ledger note itself says the closure reasoning goes -- and no procedure step ever routes an author back to it before they bind state. The packaged `open-requirements-ledger` note does not close the gap either: its "how downstream reads it" paragraph covers only repair-galaxy-draft-topology reading OPEN blocking entries, and its one line about resolved entries ("Resolving is not deleting -- a resolved entry stays in the ledger as the audit trail") frames them as provenance, not as an input.
Some step bindings depend on a property of an UPSTREAM output's CONTENT -- does this tabular output carry a header row, is the element identifier emitted as a column -- and no artifact the Mold names can answer that class of question. `parsed_tool.outputs` carries name, label, type, format and discovery rules; it carries nothing about the rows inside the file, and it never could, because the property is a fact about the wrapper's script rather than about its declaration. The Mold has no instruction for the case, so the author either guesses or invents a method. This is not the same wall as `implement-step-mold-has-no-binding-path-when-no-tool-summary-exists`: there the summary is merely absent, and the fallback list that entry proposes -- wrapper XML at the pinned changeset, then a corpus round-trip, then "nothing else" -- would still not answer this, because the XML's `<param>`/`<data>` declarations are silent on it and the round-trip actively misleads. The cost is silent: a wrong `header_lines` drops the first data row of every filtered table, or passes a header into a numeric comparison, and either way the workflow emits a plausible result.
Several authoring paths depend on reading a wrapper's XML at the pinned changeset -- the correction already filed as `implement-step-mold-has-no-binding-path-when-no-tool-summary-exists` names it as fallback evidence "authoritative for parameter names, defaults and output filters", and it is the only source for output filters at all (see `tool-summary-drops-output-filter-expressions`). No declared tool can fetch it. The summary manifest sets `artifacts.raw_tool_source_path: null` for a toolshed-sourced tool; the Tool Shed's `repos/<owner>/<repo>/raw-file/<changeset>/<path>` endpoint answers 403; and `/api/tools/<trs-id>/versions/<v>/raw_tool_source` answers 404 with "No route". The only thing that worked this run was guessing the upstream repository layout from the repository record's `remote_repository_url` and fetching the file from GitHub at the default branch -- where the filename was `rg_rnaStar.xml`, not the `<repo>/<tool_id>.xml` the shed path implies, so the first two guesses 404'd. That fetch is also UNPINNED: it happened to match the pinned version here only because tools-iuc main still carried @TOOL_VERSION@ 2.7.11b / @VERSION_SUFFIX@ 1, which a later iteration on a lagging wrapper cannot count on.
The Nextflow path has a dedicated Mold for deciding the Galaxy-side shape of external reference data (nextflow-summary-to-galaxy-reference-data, listed among the producers of the open-requirements ledger). The paper path has no equivalent, and this run's phase roster contains no reference-data phase. The source paper nonetheless pins a specific non-model assembly (GCA_002759435.2, C. auris B8441) and requires an annotation file, so the decision between a built-in index, a data-table string, and a portable history dataset had to be made somewhere. The interface Mold made it, unprompted: nothing in its procedure or references assigns reference-data shape to the interface tier.
The Mold emits `test-data-refs.json` and specifies no shape for it. The whole procedure is one sentence — "derive concrete workflow test inputs and expected outputs — resolvable URLs, file shapes, and expected hashes — emitted as `test-data-refs`" — and the bundle packages no schema, template, or worked example (`refs: []`, "Load Upfront" and "Load On Demand" both "None declared"). The whole JSON structure was invented at runtime: how to key an input against a workflow input label, how to express a collection input's elements and per-row metadata, where expected outputs live and how to separate an assertion the data supports from one it does not. Two downstream Molds in this pipeline consume the artifact (freeform-summary-to-galaxy-test-plan at phase 8, implement-galaxy-workflow-test at phase 9) and neither can rely on any particular key existing. This is the same defect class already filed as `summarize-paper-declares-no-freeform-summary-shape`, at the other end of the same pipeline.
The Mold is silent on subsetting, which is the central problem of deriving test data from a paper. A paper's deposited data is essentially always orders of magnitude too large to be a workflow test fixture — this run's six SRA runs are 22–30 M read pairs each, ~6.5 GB — so the Mold's real task is not "resolve URLs" but "resolve URLs AND decide a subset AND establish whether that subset still supports the paper's claims". The procedure's phrasing ("resolvable URLs, file shapes, and expected hashes") reads as though the deposited data is used as-is. Nothing tells the runtime to subset, how to choose a depth, how to keep the recipe deterministic, or how to decide whether the result is a fixture that reproduces the paper's finding or one that only exercises the workflow's shape. That last decision is exactly what the Foundry's own fixture rule makes mandatory — "a fixture must also be able to produce the outcome the scenario bound to it claims" (AGENTS.md) — and the Mold never routes to it. The subsetting strategy, the depth, the evidence standard and the tiering of assertions by whether the data supports them were all invented here.
The note's closing guidance is that carry-forward of sample-sheet metadata past map-over "must be explicit (re-attaching metadata via `__SAMPLE_SHEET_TO_TABULAR__` or rules DSL `add_column_from_sample_sheet_index`)", and the note is packaged into this Mold precisely to drive that re-attachment. But it describes the tool in one clause — "iterates and tab-joins for downstream tabular consumers" — and never states its output columns. Whether the element identifier is emitted as a column is the single fact the re-attachment turns on, because the identifier is the only key that survives map-over and therefore the only possible join key back to a downstream collection. The note recommends the mechanism without supplying what is needed to wire it.
The procedure's query-normalization recipe for a tool-id-shaped need is to strip any `owner/` prefix, split on `_` / `-` into space-separated words, and also try the bare significant word — and it states that "a `miss` is only honest after the name variants have been tried". Every variant that recipe generates fails for `iuc/map_param_value/map_param_value`, a tool that is published, current, and pinned by the IWC corpus. The recipe does not cover the one transform that works: expanding an abbreviated word in the id to the word the human tool name actually uses. Galaxy tool ids abbreviate routinely (`param`, `val`, `seq`, `align`, `qc`, `col`), so this is a recurring shape, not a one-off. An iteration following the packaged recipe literally would have declared `miss` and fallen through to author-galaxy-tool-wrapper — authoring a new wrapper for a tool that already exists, which is the most expensive possible wrong answer this Mold can produce.
Procedure step 2's built-in/stock branch offers exactly two ways to get a stock tool's version — "read it from a populated cache via `galaxy-tool-cache list` or take a known pin from the step plan" — and then forbids the only remaining move: "never hand-guess a stock version". Both offered sources presuppose the version is already known. For a stock tool meeting the run for the first time, with `tool_version: TODO` in the draft and no step-plan pin, the cache is empty of it precisely because populating the cache requires an `add` that takes `--tool-version`. The branch is circular, and its stated prohibition closes the only exit. There is no third route to fall back on: the Tool Shed publishes no version listing for a bare stock id at all.
The Mold produces the shared freeform-summary handoff but specifies no shape for it. The procedure says only "a free-form Markdown summary capturing the workflow's steps, tools, parameters, and sample/reference-data leads", the cast bundle packages no template, example, or reference, and the runtime notes forbid reading Foundry source at runtime. Four downstream Molds consume this artifact (freeform-summary-to-galaxy-interface, -data-flow, -template, -test-plan), so the section structure they can rely on was guessed at, not settled by the instructions.
The Mold's entire job is to extract methods from a paper, but it declares no required tools and describes no procedure for actually obtaining the text. For this run the article's main text contained no Methods section at all: every computational detail lived only in a supplementary PDF. The publisher page returned HTTP 403, the article is not in the Europe PMC open-access set, and a single fetch of the PMC article page yielded a tool list with no parameters and no pipeline. The retrieval route that worked was improvised and is not described anywhere in the bundle.
The Mold packages four pattern references and all four are MOC index pages. Each names the recipe pages that carry the actual tool ids, port names and worked wiring, and none of those pages is in the bundle. The runtime notes forbid reading Foundry source, so a recipe named in a packaged MOC is unreachable at runtime — the reference resolves to a one-line description and a dead wiki-link.
The procedure's "Labels and fixtures are assumed, not bound" section instructs the Mold to bind assertions to interface-brief labels with `label_status: assumed` and `workflow.label_source: interface-brief`, and to record fixtures as `storage: unresolved` with `location: null`, on the stated premise that the concrete workflow and the resolved test-data refs "exist in the harness run-state by the time the plan is authored, but they are reconciled downstream rather than here". In this run both were supplied as phase-8 inputs and both were settled: `galaxy-workflow.gxwf.yml` (27 steps, 9 inputs, 16 outputs) carries the real labels, and `test-data-refs.json` carries resolved URLs, md5s and a measured verdict. Following the instruction would have meant writing `assumed` over labels read byte-for-byte from the workflow and `unresolved` over fixtures with pinned NCBI URLs and md5s — discarding verified information and handing implement-galaxy-workflow-test a reconciliation job already done. The Mold was deliberately disobeyed on both counts, and the deviation had to be argued inside the artifact rather than settled by the procedure.
Section 5 ("Design inputs with fixtures in mind") instructs the designer to match workflow input collection types to realistic fixture shapes and to choose labels readable as test `job:` keys, and its evidence covers `list`, `list:paired`, data, string, boolean and int inputs. It says nothing about the `sample_sheet` family, even though the sibling note galaxy-sample-sheet-collections is packaged in the same bundle and is what pushes the designer toward that shape. Nothing in the bundle states whether a `sample_sheet:paired` workflow input — element identifiers plus per-row typed `columns` plus collection-level `column_definitions` — can be expressed in a Planemo/IWC `-tests.yml` job block at all. This run's most consequential interface decision, the primary reads input, had to be made without knowing whether the same run's own test phase can express it.
The tests-format schema rejects two shapes that production IWC workflow tests use and that Galaxy accepts at run time. (1) The inner element of a `list:paired` job input written as `class: Collection` + `type: paired` — the `Collection` `$def` is `additionalProperties: false` with only `collection_type`, so `type` is an unknown property and the whole job input fails `oneOf`. (2) A nested-collection output assertion whose outer `element_tests` entry carries `elements:` without a `class: Collection` discriminator — the schema's `if/then` on `class` routes it to the dataset-element model, which has no `elements` property. Run over the pinned IWC corpus (fe41a79) with `gxwf validate-tests` from @galaxy-tool-util/cli 1.10.1, 31 of 122 committed `*-tests.yml` files fail, and the two clusters above account for the bulk of them. This makes the static gate unusable as a conformance check against the corpus the Foundry treats as normative, and it silently invalidates the corpus-derived recipes the Mold's own packaged notes teach.
`toNative` mistakes a raw format2 workflow for an already-normalized one whenever `inputs:` and `steps:` are written in list form, skips normalization, and then aborts with an uncaught `TypeError: step.in is not iterable` as soon as a step's `in:` uses the mapping form gxformat2 equally permits. `_isNormalizedFormat2` decides on three top-level facts that say nothing about per-step shape - `class === "GalaxyWorkflow"`, `Array.isArray(inputs)`, `Array.isArray(steps)` - so `normalizedFormat2`, whose `normalizeStepIn`/`normalizeStepOut` handle both spellings correctly, is never called and `_extractConnections` iterates a mapping. The defect is in `toNative`, not in the connection validator that surfaced it: `gxwf convert --to native` and `ensureNative` crash on the same input with no connection flag involved. For this run the consequence is that `gxwf validate --connections` - the only static gate covering connection types, collection algebra and map-over - could not be run at all.
`parsed_tool.outputs` carries name, label, hidden, type, format, format_source, metadata_source, discover_datasets, from_work_dir and precreate_directory -- but not the output's `<filter>` expression. A Galaxy output filter is what decides whether a declared output EXISTS for a given parameter branch, so the summary can enumerate ten outputs for a tool that will produce four, with nothing marking the difference. That is not cosmetic for this Mold: a step's `out:` list and every workflow output wired to it are only valid if the chosen tool state keeps those outputs alive, and the summary is the artifact procedure step 3 produces expressly so step 4 can bind the step against it. Concretely, this run's STAR step had to establish that `output_log` and `mapped_reads` survive `quantMode: '-'` and `outWigType: None`, because its plan makes a suppressed `output_log` a hard failure -- and the summary cannot answer that question at all. The answer (`reads_per_gene` and `transcriptome_mapped_reads` are filtered on quantMode; the two promoted outputs carry no filter) came only from reading the wrapper XML. Iteration 8 hit the same wall on lparsons/cutadapt, where the `report` and `out_pairs` filters are what keep two promoted outputs alive.
A conditional's `__current_case__` is an INDEX, and nothing in the packaged summary schema, the Mold, or any packaged note says what it indexes. The summary gives each conditional a `test_parameter.options` array and a `whens` array with a `discriminator` per element. The index that `__current_case__` must carry is the position in `whens` (i.e. `<when>` document order), NOT the position of the matching value in `options`. The two are not interchangeable and this run hit a live divergence: in iuc/rgrnastar/rna_star 2.7.11b+galaxy1, `refGenomeSource[history].GTFconditional` lists its options as `without-gtf, with-gtf` but its whens as `with-gtf, without-gtf`, so `with-gtf` is case 0 while the dropdown shows it second. An author reading the field an author would naturally read gets 1. The bundle offers no way to know which array is authoritative, so the rule had to be re-established from outside the bundle -- by round-tripping a corpus `.ga` that happens to use the same tool and reading the wrapper XML's `<when>` order to confirm the correspondence.
The note's Gotchas section warns about the one flag that weakens validation visibly and is silent on the two ways it fails invisibly. It says `--no-tool-state` weakens validation, but never says that omitting `--cache-dir` has the same effect and worse - every tool step becomes a skip and the command still exits 0. `--cache-dir` appears only as a bare option line, "Tool cache directory", with nothing about what happens without it. Separately, the note actively recommends a flag that cannot run: "Use `--connections` when tool cache metadata is available and data-shape compatibility matters, especially around collections and map-over", and its Examples block lists `gxwf validate workflow.gxwf.yml --json --connections --strict` - a command that at 1.10.1 crashes on any format2 workflow with a tool step, and would abandon JSON even if it did not.
The Mold owns the run's last automated gate and its procedure never mentions the tool cache, never mentions skips, and gives the artifact a three-valued status - `pass`, `fail`, `not-run` - with no value for the outcome this gate actually produces. Two consequences, both hit in this run. (1) The invocation the procedure implies, `gxwf validate <file> --json`, omits `--cache-dir` and returns `Tool state: 0 validated, 27 skipped` with exit code 0: a green that checked nothing, on the terminal gate, with no warning anywhere in the bundle. (2) When the cache is supplied, seven steps still skip permanently, and the Mold offers no way to say so - `pass` overstates it, `not-run` understates it - and no route to discharge a skip, although one exists and is cheap. The Mold's one instruction on this, "A `not-run` status is never reported as a pass", guards the case that cannot happen quietly and not the one that can.
The procedure's resolve-then-summarize sequence is written as if each iteration met its wrapper for the first time. It has no notion of a wrapper already resolved earlier in the same run: this draft uses one wrapper at five steps, and the identity-pinned branch still directs the iteration to confirm the pin via discover-shed-tool and then invoke summarize-galaxy-tool, both of which a sibling step's already-concrete `tool_id` + `tool_version` pair has settled. The Mold never describes what this iteration actually did, which was to take the sibling's pin and read the summary already in the cache.
Concretizing a step forces its `_plan_*` fields to be deleted — `draft-next-step` counts any surviving `_plan_*` as remaining work, so a finished step that keeps one is selected again on the next iteration and the harness loop cannot terminate. Those fields are also where the corpus provenance lives: the step implemented here carried a `_plan_context` citing the exemplar workflow and step that fixes its component text. The Mold says only that the implement phase "resolves the chosen step's remaining `TODO_*` / `_plan_*` slots", and never says whether that provenance should be carried into `doc:`, dropped, or recorded elsewhere. This run has been keeping it in a YAML comment above the step, a convention it invented, and nothing states whether `draft-extract` preserves comments when it re-serializes the concrete workflow (this iteration did not test that).
Section 5 is the note an agent reaches for when writing collection-output assertions, and its nested-collection rule — "outer `element_tests:` keyed by outer identifier; inner `elements:` (note plural, no `_tests` suffix on the inner)" — is incomplete in the one way that matters to the gate: the outer entry must also carry `class: Collection`, or the schema routes it to the dataset-element model and rejects `elements` as an additional property. The section also does not mention that an `asserts:` mapping cannot repeat a key, so any output needing two `has_text` probes has to use the list form (`- that: has_text`) — which this run needed on five outputs and which the section's own examples never show.
The caster emits a Validation instruction of the form "run `foundry <cmd> <artifact>` from `@galaxy-foundry/gxwf-foundry`; if the command is not on PATH, run `npx --package @galaxy-foundry/gxwf-foundry foundry <cmd> <artifact>`". The fallback names no version, so it resolves whatever `latest` is on the registry at runtime, while the bundle already carries the schema it was cast against, verbatim, under `references/schemas/`. Those two can disagree, and when they do the fallback returns a confident green verdict against a contract the skill is not bound by. In this run the fallback was the only available route — the checkout had no installed dependencies and no package manager on PATH — and the published 0.1.2 schema had to be diffed against the bundled copy by hand to know the verdict meant anything, which the runtime notes ("do not read Foundry source files at runtime") discourage in the first place.
The Mold's output contract requires an "open questions" section in the Markdown brief and a ledger of open entries, with no rule for which destination an unresolved choice belongs in. Every unresolved item in this run was genuinely both an obligation a later Mold must discharge and a thing a human reviewer should see, so all ten were written twice, in two different shapes, and cross-referenced by entry id by hand to stop them drifting. Confirming a real-run instance of something the note already lists under Open work ("Reconcile the design-tier briefs' free-text 'open questions' sections with the ledger"); filed as corroboration rather than as a new finding.
`alternates[]` reuses the full `ToolCandidate` shape, which requires `version` and `changeset_revision` (both `minLength: 1`) plus a numeric `score`. There is no way to record a candidate the discovery deliberately did NOT pin. The procedure asks for exactly that — "multiple plausible hits ... → `weak` with the leading candidate plus alternates", and the surrounding Molds hand this skill named alternatives in a step's `_plan_context` — but an alternate is only representable after it has been fully resolved to a changeset. So an author recording a runner-up faces two bad options: spend Tool Shed calls pinning a wrapper being rejected, or write placeholder strings. The second validates green. `"changeset_revision": "unresolved"` and `"score": 0` pass `validate-galaxy-tool-discovery` without a murmur, in a field the schema itself documents as "Selected Tool Shed Mercurial changeset revision for reproducible gxformat2 tool_shed_repository pinning" and one documented as "Higher is better". A machine-read pin contract should not be able to carry a fabricated pin.
The draft superset can mark a STEP as unresolved (`TODO`, `_plan_*`) but has no way to mark a REGION as provisional, or to record the named alternative a later phase should swap in. The `_plan_*` family is per-step and explicitly wrapper-tier, so a multi-step topology choice that is settled-but-unprecedented has nowhere durable to live.
`test_galactic_job_json_sample_sheet_collection_with_rows` asserts that `rows` round-trips as `{"el1": {"condition": "treatment"}, "el2": {"condition": "control"}}` — a mapping of column name to value. The real contract is a POSITIONAL LIST per row: `validate_row` in lib/galaxy/model/dataset_collections/types/sample_sheet_util.py rejects on `len(row) != len(column_definitions)` and then zips `row` against `column_definitions` in order. The unit test never catches this because it mocks `collection_create_func`, so nothing downstream of `galactic_job_json` is exercised. These unit tests are the most discoverable documentation of the job-block `rows` syntax, and the shape they document will not validate whenever the target collection has column definitions.
`gxwf --version` prints `1.0.0` regardless of the installed package version. The version actually installed here is `@galaxy-tool-util/cli@1.10.1`, confirmed by `npm ls -g`. Every Foundry Mold that requires gxwf pins a package version in `_required_tools.json` — this one pins `^1.8.1` — and none of those pins can be checked at runtime, because the only version the CLI reports is a constant that satisfies no pin and matches no release. The packaged availability check works around this by grepping `--help` for a subcommand name, which detects presence but says nothing about version.
The procedure names inputs, outputs, labels, collection shapes and checkpoints as the things to settle, but gives no rule for which source-stated parameters become exposed typed workflow inputs and which are baked into step defaults. The only nearby guidance is one line in the packaged testability note ("Keep typed parameters explicit when tests need to set them"), which answers the test-facing half and not the design half. For a paper source the distinction is sharp and recurring: a value the paper pins is settled and can be baked, while a value inferred from a kit name or an accession list is exactly what a reviewer must be able to see and change. That rule was invented here, not read.
Nothing the Mold names settles how a scalar parameter's VALUE is encoded in the step's state block, and the two authorities an author can reach disagree. Procedure step 2 sends the author to `parsed_tool` for ports and datatypes and to `input_schemas.workflow_step_linked` for the state shape. For `Filter1`'s `header_lines`, `parsed_tool` declares `parameter_type: gx_integer`, `type: integer`, `value: 0` — read literally that says write the YAML integer `0`. Every real Galaxy workflow writes the string `'0'`. This is a different axis from the two disagreements already filed here: `implement-step-mold-says-state-without-saying-which-key` is about WHICH block key, and `gxwf-linked-step-schema-rejects-the-keys-its-own-converter-emits` is about `__`-prefixed bookkeeping keys. This one is about the scalar leaf value, and it is unaddressed by either correction. The Mold's guidance is not merely silent — following the one authority it names produces the form the corpus never uses.
`AssertionIntent.evidence` is a two-value enum, `test-evidence` or `intent`, described as "whether this assertion was translated from upstream test evidence or synthesized from intent". A third case is routine on the paper and interview paths and occurred throughout this run: an assertion neither translated from an upstream fixture nor merely synthesized, but MEASURED against the run's own resolved test data before the plan was written. The same gap exists at plan level, where `source.derived_from` has the same two values. Forced to pick, every such assertion is recorded `evidence: intent`, so a reviewer filtering on the field sees a uniformly speculative plan and systematically undercounts its grounding. `confidence: high` is the only signal left, and it means something different.
When `galaxy-tool-cache add` caches a stock Galaxy tool by its bare id, it writes the id into `index.json` with a Tool Shed repository path glued on the front, producing `toolshed.g2.bx.psu.edu/repos/__SAMPLE_SHEET_TO_TABULAR__` — an identifier that names nothing: there is no such shed repository, and no workflow may legally carry that tool id. The cached summary body alongside it is correct (`"id": "__SAMPLE_SHEET_TO_TABULAR__"`), so the damage is confined to the index, but the index is what `galaxy-tool-cache list` prints. That matters because `list` is the surface the advance-galaxy-draft-step procedure directs an author to read a stock tool's version off, and what it shows there cannot be pasted into a draft.
With no `--max-results`, `gxwf tool-search` returns every hit three times — in the table rendering and in `--json` alike. Passing any explicit `--max-results` returns distinct hits, so the duplication is confined to the default page size. It is not cosmetic for a discovery procedure that triages by counting and comparing candidates: the default view of a search makes a field of twenty wrappers look like sixty, and the triage rule "multiple plausible hits ... → weak" reads a repeated single candidate as a cluster.
no checkpoint history — re-run with --checkpoint to get a per-phase and per-iteration record.
13 of 23 files map to a declared artifact. The rest are below. 3 path(s) were ignored entirely.
| path | size | modified | declared by |
|---|---|---|---|
| galaxy-tool-pin.json | 2.5 KB | 2026-09-16 17:59 | discover-shed-tool |
| path | size | modified | declared by |
|---|---|---|---|
| planemo-biocontainers.log | 369.4 KB | 2026-09-18 16:41 | — |
| planemo-smoke.attempt1.log | 317.3 KB | 2026-09-16 22:09 | — |
| planemo-smoke.html | 340.4 KB | 2026-09-17 14:26 | — |
| planemo-smoke.json | 14.8 KB | 2026-09-17 14:26 | — |
| planemo-smoke.log | 290.7 KB | 2026-09-17 14:26 | — |
| tool_test_output.html | 340.4 KB | 2026-09-18 16:41 | — |
| tool_test_output.json | 14.8 KB | 2026-09-18 16:41 | — |
| path | size | modified | declared by |
|---|---|---|---|
| CASE_STUDY.md | 15.1 KB | 2026-09-18 16:36 | — |
| README.md | 3.9 KB | 2026-09-18 16:39 | — |
| path | size | modified | declared by |
|---|---|---|---|
| test-data | 24 file(s) | 2026-09-18 16:38 | — |