mbf-fastq-processor documentation

Mon, 01 Jan 0001 00:00:00 +0000

Report #

Capture data for the final report (see the output section).

You can add multiple reports, at any stage of your transformation chain to get e.g. before/after filtering reports.

[[step]]
 action = 'Report'
 name = "report" # Key that the report will be listed under. Must be distinct
 count = true # count reads at this position
 base_statistics = false # include base distribution at each read position, q20, q30, total, gc bases
 length_distribution = false # capture read length distribution
 duplicate_count_per_read = false # count duplicates using Cukoo filter on each read1/read2/index1/index2
 duplicate_count_per_fragment = false # count duplicates using Cukoo filter, on concatenated read1/read2/index1/index2
 count_oligos = [] # if set, count these oligos in the segment. Full match only, no iupac
 count_oligos_segment = "all" # segment to count oligos in, can be 'all', 'read1', ...

Statistics available (for each ‘segment’. If demultiplexd, per barcode combination):

Mon, 01 Jan 0001 00:00:00 +0000

QuantifyTag #

Count the occurrences of each tag-sequence.

[[step]]
 action = "QuantifyTag"
 in_label = "mytag"
 infix = "tagcount" # output file is output{ix_separator}tagcount.qr.json (default '_' → output_tagcount.qr.json)

This transformation counts how many times each unique tag value appears and outputs the results to a JSON file.

Demultiplex interaction #

Barcodes are counted per demultiplexed stream.

Mon, 01 Jan 0001 00:00:00 +0000

Inspect #

Dump a few reads to a FASTQ file for inspection at this point in the graph.

[[step]]
 action = "Inspect"
 n = 1000 # how many molecules 
 infix = "inspect_at_point" # output filename infix
 segment = "read1" # Any of your input segments (use "all" for interleaved output)
 format = "FASTQ" # output format: FASTQ, FASTA or BAM
 suffix = "compressed" # (optional) custom suffix for filename
 compression = "gzip" # (optional) compression format: raw, gzip, zstd. Defaults to uncompressed
 compression_level = 1 # (optional) compression level for gzip/zstd/bam (gzip, zstd: 1-22)
 # defaults: gzip=6, zstd=5

Output filename pattern:

Mon, 01 Jan 0001 00:00:00 +0000

Progress #

Emit progress to stdout (default) or a .progress log file, if output_infix is set. (filename is {output_prefix}{ix_separator}{infix}.progress, default separator _).

[[step]]
 action = "Progress"
 n = 100_000
 output_infix = "progress" # optional

Every n reads, report on total progress, total reads per second. At the end, report final runtime and reads/second.