PositionBasedDownsampleSam (Picard)

Summary

Class to downsample a SAM/BAM/CRAM file based on the position of the read in a flowcell. As with DownsampleSam, all the reads with the same queryname are either kept or dropped as a unit.

Details

The downsampling is _not_ random (and there is no random seed). It is deterministically determined by the position of each read within its tile. Specifically, it draws an ellipse that covers a FRACTION of the total tile's area and of all the edges of the tile. It uses this area to determine whether to keep or drop the record. Since reads with the same name have the same position (mates, secondary and supplemental alignments), the decision will be the same for all of them. The main concern of this downsampling method is that due to "optical duplicates" downsampling randomly can create a result that has a different optical duplicate rate, and therefore a different estimated library size (when running MarkDuplicates). This method keeps (physically) close read together, so that (except for reads near the boundary of the circle) optical duplicates are kept or dropped as a group. By default the program expects the read names to have 5 or 7 fields separated by colons (:) and it takes the last two to indicate the x and y coordinates of the reads within the tile whence it was sequenced. See DEFAULT_READ_NAME_REGEX for more detail. The program traverses the INPUT twice: first to find out the size of each of the tiles, and next to perform the downsampling. Downsampling invalidates the duplicate flag because duplicate reads before downsampling may not all remain duplicated after downsampling. Thus, the default setting also removes the duplicate information. Example java -jar picard.jar PositionBasedDownsampleSam \ I=input.bam \ O=downsampled.bam \ FRACTION=0.1 Caveats Note 1: This method is technology and read-name dependent. If the read-names do not have coordinate information embedded in them, or if your BAM contains reads from multiple technologies (flowcell versions, sequencing machines) this will not work properly. It has been designed to work with Illumina technology and reads-names. Consider modifying {@link #READ_NAME_REGEX} in other cases. Note 2: The code has been designed to simulate, as accurately as possible, sequencing less, not for getting an exact downsampled fraction (Use {@link DownsampleSam} for that.) In particular, since the reads may be distributed non-evenly within the lanes/tiles, the resulting downsampling percentage will not be accurately determined by the input argument FRACTION. Note 3:Consider running {@link MarkDuplicates} after downsampling in order to "expose" the duplicates whose representative has been downsampled away. Note 4:The downsampling assumes a uniform distribution of reads in the flowcell. Input already downsampled with PositionBasedDownsampleSam violates this assumption. To guard against such input, PositionBasedDownsampleSam always places a PG record in the header of its output, and aborts whenever it finds such a PG record in its input.

Category Read Data Manipulation

Overview

Summary

Class to downsample a SAM/BAM file based on the position of the read in a flowcell. As with {@link DownsampleSam}, all the reads with the same queryname are either kept or dropped as a unit.

Details

The downsampling is not random (and there is no random seed). It is deterministically determined by the position of each read within its tile. Specifically, it draws an ellipse that covers a {@link #FRACTION} of the total tile's area and of all the edges of the tile. It uses this area to determine whether to keep or drop the record. Since reads with the same name have the same position (mates, secondary and supplemental alignments), the decision will be the same for all of them. The main concern of this downsampling method is that due to "optical duplicates" downsampling randomly can create a result that has a different optical duplicate rate, and therefore a different estimated library size (when running MarkDuplicates). This method keeps (physically) close read together, so that (except for reads near the boundary of the circle) optical duplicates are kept or dropped as a group. By default the program expects the read names to have 5 or 7 fields separated by colons (:), and it takes the last two to indicate the x and y coordinates of the reads within the tile whence it was sequenced. See {@link ReadNameParser#DEFAULT_READ_NAME_REGEX} for more detail. The program traverses the {@link #INPUT} twice: first to find out the size of each of the tiles, and next to perform the downsampling. Downsampling invalidates the duplicate flag because duplicate reads before downsampling may not all remain duplicated after downsampling. Thus, the default setting also removes the duplicate information.

Example

 java -jar picard.jar PositionBasedDownsampleSam \
       I=input.bam \
       O=downsampled.bam \
       FRACTION=0.1

Caveats

This method is technology and read-name dependent. If the read-names do not have coordinate information embedded in them, or if your BAM contains reads from multiple technologies (flowcell versions, sequencing machines). this will not work properly. It has been designed to work with Illumina technology and reads-names. Consider modifying {@link #READ_NAME_REGEX} in other cases.
The code has been designed to simulate, as accurately as possible, sequencing less, not for getting an exact downsampled fraction (Use {@link DownsampleSam} for that.) In particular, since the reads may be distributed non-evenly within the lanes/tiles, the resulting downsampling percentage will not be accurately determined by the input argument {@link #FRACTION}.
Consider running {@link MarkDuplicates} after downsampling in order to "expose" the duplicates whose representative has been downsampled away.
The downsampling assumes a uniform distribution of reads in the flowcell. Input already downsampled with PositionBasedDownsampleSam violates this assumption. To guard against such input, PositionBasedDownsampleSam always places a PG record in the header of its output, and aborts whenever it finds such a PG record in its input.

@author Yossi Farjoun

PositionBasedDownsampleSam (Picard) specific arguments

This table summarizes the command-line arguments that are specific to this tool. For more details on each argument, see the list further down below the table or click on an argument name to jump directly to that entry in the list.

Argument name(s)	Default value	Summary
Required Arguments
--FRACTION -F		The (approximate) fraction of reads to be kept, between 0 and 1.
--INPUT -I		The input SAM/BAM/CRAM file to downsample.
--OUTPUT -O		The output, downsampled, SAM/BAM/CRAM file.
Optional Tool Arguments
--ALLOW_MULTIPLE_DOWNSAMPLING_DESPITE_WARNINGS	false	Allow downsampling again despite this being a bad idea with possibly unexpected results.
--arguments_file		read one or more arguments files and add them to the command line
--help -h	false	display the help message
--READ_NAME_REGEX		Use these regular expressions to parse read names in the input SAM file. Read names are parsed to extract three variables: tile/region, x coordinate and y coordinate. The x and y coordinates are used to determine the downsample decision. Set this option to null to disable optical duplicate detection, e.g. for RNA-seq The regular expression should contain three capture groups for the three variables, in order. It must match the entire read name. Note that if the default regex is specified, a regex match is not actually done, but instead the read name is split on colons (:). For 5 element names, the 3rd, 4th and 5th elements are assumed to be tile, x and y values. For 7 element names (CASAVA 1.8), the 5th, 6th, and 7th elements are assumed to be tile, x and y values.
--REMOVE_DUPLICATE_INFORMATION	true	Determines whether the duplicate tag should be reset since the downsampling requires re-marking duplicates.
--STOP_AFTER		Stop after processing N reads, mainly for debugging.
--version	false	display the version number for this tool
Optional Common Arguments
--COMPRESSION_LEVEL	5	Compression level for all compressed files created (e.g. BAM and VCF).
--CREATE_INDEX	false	Whether to create an index when writing VCF or coordinate sorted BAM output.
--CREATE_MD5_FILE	false	Whether to create an MD5 digest for any BAM or FASTQ files created.
--MAX_RECORDS_IN_RAM	500000	When writing files that need to be sorted, this will specify the number of records stored in RAM before spilling to disk. Increasing this number reduces the number of file handles needed to sort the file, and increases the amount of RAM needed.
--QUIET	false	Whether to suppress job-summary info on System.err.
--REFERENCE_SEQUENCE -R		Reference sequence file.
--TMP_DIR		One or more directories with space available to be used by this program for temporary storage of working files
--USE_JDK_DEFLATER -use_jdk_deflater	false	Use the JDK Deflater instead of the Intel Deflater for writing compressed output
--USE_JDK_INFLATER -use_jdk_inflater	false	Use the JDK Inflater instead of the Intel Inflater for reading compressed input
--VALIDATION_STRINGENCY	STRICT	Validation stringency for all SAM files read by this program. Setting stringency to SILENT can improve performance when processing a BAM file in which variable-length data (read, qualities, tags) do not otherwise need to be decoded.
--VERBOSITY	INFO	Control verbosity of logging.
Advanced Arguments
--showHidden	false	display hidden arguments

Validation stringency for all SAM files read by this program. Setting stringency to SILENT can improve performance when processing a BAM file in which variable-length data (read, qualities, tags) do not otherwise need to be decoded.

The --VALIDATION_STRINGENCY argument is an enumerated type (ValidationStringency), which can have one of the following values:

STRICT
LENIENT
SILENT

ValidationStringency STRICT

--VERBOSITY

Control verbosity of logging.

The --VERBOSITY argument is an enumerated type (LogLevel), which can have one of the following values:

ERROR
WARNING
INFO
DEBUG

LogLevel INFO

--version

display the version number for this tool

boolean false

Return to top

GATK version 4.6.2.0 built at Sun, 13 Apr 2025 13:21:43 -0400.