CAMISIM

Software for simulating metagenomes and microbial communities
https://github.com/CAMI-challenge/CAMISIM
Fritz*, Hofmann*, et al. CAMISIM: Simulating metagenomes and microbial communities. Microbiome, 7:17. doi:10.1186/s40168-019-0633-6 (2019)

AMBER

Software for genome and taxon binning assessment implementing metrics used in CAMI I and II
https://github.com/CAMI-challenge/AMBER
Meyer, F. et al. AMBER: Assessment of Metagenome BinnERs, GigaScience, 7, 6, doi:10.1093/gigascience/giy069 (2018)

OPAL

Software for assessing taxonomic profilers implementing metrics used in CAMI I and II
https://github.com/CAMI-challenge/OPAL
Meyer, F. et al. Assessing taxonomic metagenome profilers with OPAL, Genome Biology, 20:51, doi:10.1186/s13059-019-1646-y (2019)

MetaQUAST

Software to evaluate and compare metagenome assemblies based on alignments to close references
https://quast.sourceforge.net/metaquast.html
Mikheenko, A., Saveliev, V., and Gurevich, A. MetaQUAST: evaluation of metagenome assemblies. Bioinformatics, 32, 7 (2016), 1088-1090

Bioboxes

Docker containers with standardized interfaces for interchangeable bioinformatics software
http://bioboxes.org/
Belmann, P. et al. Bioboxes: standardised containers for interchangeable bioinformatics software. GigaScience, 4, 47 (2015)

The online benchmarking service uses MetaQUAST 5.2.0, AMBER 2.0.7, and OPAL 1.0.13.