BV-BRC Copilot: Agentic Workflow Automation
Advancing and systematically evaluating BV-BRC Copilot for generating, executing, and reproducing bioinformatics workflows from research questions and published methods.
Goal
Advance and systematically evaluate BV-BRC Copilot for generating, executing, and reproducing bioinformatics workflows from research questions and published methods.
Three-Day MVP
Build a benchmark of bioinformatics tasks spanning assembly, annotation, comparative genomics, phylogeny, AMR, RNA-seq, and data discovery. Use BV-BRC Copilot and MCP-enabled agents to generate and execute workflows while capturing parameters, data, provenance, logs, and replay instructions.
As a reproducibility challenge, select published studies and have Copilot reconstruct their analyses from methods sections, identifying data, tools, parameters, and workflow steps and reporting unresolved information as a reproducibility gap report.
Evaluation
Workflow completion and scientific correctness; reproducibility on rerun; provenance completeness; recovery of methods, tools, and parameters from publications; agreement with published results; hallucination/error rate; and human intervention required.
Team
Team assignments are being finalized ahead of the codeathon. Participants can review project teams, and request reassignment, in the shared participant spreadsheet circulated by the organizing team.