Bioinformatics File Formats Explained: FASTA, FASTQ, SAM/BAM, VCF, GTF and BED

Chapter 5 of 5: Troubleshooting, practice and next steps

Which format do I need?

Other formats you will meet

Troubleshooting

These messages are copied from the tools (samtools/bcftools/htslib 1.25, bedtools 2.31).

A quick consistency check for any project

Before a long analysis, confirm that the reference, the alignments and the variants use the same sequence names. This loop prints the names from each file side by side:

cut -f 1 toy.fa.fai > names_fasta.txt
samtools view -H toy.bam | grep "^@SQ" | cut -f 2 | sed 's/^SN://' > names_bam.txt
bcftools view -h toy.vcf.gz | grep "^##contig" | sed -E 's/.*ID=([^,>]+).*/\1/' > names_vcf.txt
paste names_fasta.txt names_bam.txt names_vcf.txt
samtools quickcheck -v toy.bam && echo "BAM OK"

Three identical columns mean the files agree. On a human project you would see chr1 chr1 chr1… and any 1 vs chr1 row stands out at once.

Practice exercise

Use the files in ~/bhl-formats. Try first, then open the solution.

1. Decode FLAG 147 and FLAG 2064. For each, say: paired or not, which strand, R1 or R2, primary or not.
2. Read x3 in toy.sam has CIGAR 9M4I13M and POS 6. How long is its sequence, and what is its last reference position?
3. Write the SNP at ref2:17 as a one-line BED file, then use bedtools getfasta to print the reference base. Which base do you get?
4. Without opening the file, which column of a VCF would you cut to get the genotype column of the first sample, and why does bcftools query do this better?

Bioinformatics file formats: frequently asked questions

Is a .fa file the same as .fasta, .fna and .faa?

Yes, all are FASTA. By convention .fna holds nucleotides and .faa amino acids, but tools look at the content, not the extension.

Should I keep SAM files?

No. Convert to sorted, indexed BAM (or CRAM) right away and delete the SAM. Better still, pipe the aligner straight into samtools sort so the SAM is never written.

Can I gunzip a .vcf.gz or a BAM?

A bgzipped VCF can be read with zcat or gunzip, but keep it compressed so it stays indexable. A BAM is binary even after decompression, so always read it with samtools view.

Why does my BAM have no index, or an old one?

samtools index must be rerun every time the BAM changes. Many tools warn that the index is older than the data file; some silently use the old one.

GTF or GFF3 for RNA-seq?

Use whatever your counting tool expects (usually GTF for STAR and featureCounts), from the same source and release as your genome FASTA.

When should I use CRAM instead of BAM?

For long-term storage and sharing of large projects, when you can keep the exact reference FASTA with it. For day-to-day work, BAM is still the most widely supported.

How do I write this in my Methods?

Name every format with its tool and version, and the reference. For example: “Reads (FASTQ) were aligned to GRCh38 (Ensembl release 115 primary assembly). Alignments were sorted and indexed with samtools 1.25 and stored as BAM. Variants were called and filtered with bcftools 1.25, normalised with bcftools norm (left-aligned, multi-allelic sites split), and released as bgzip-compressed, tabix-indexed VCFv4.2. Gene annotation: Ensembl release 115 GTF. All intervals in BED files are 0-based, half-open.”

Glossary

• BGZF: block gzip; gzip made of independent blocks of up to 64 KiB, so files can be indexed. Made by bgzip.
• CIGAR: the string in SAM column 6 that describes how a read aligns (M, I, D, N, S, H, P, =, X).
• FLAG: SAM column 2; a sum of bits that answer yes/no questions about a read.
• Half-open interval: [start, end); start included, end excluded. Used by BED.
• Index: a side file (.fai, .bai, .csi, .crai, .tbi) that lets a tool jump to a region without reading the whole file.
• MAPQ: mapping quality; Phred-scaled confidence that the read is placed at the right position.
• Meta-information line: a VCF header line starting with ## that defines contigs, INFO, FORMAT and FILTER fields.
• Normalisation: writing variants in one standard form (left-aligned, minimal, one ALT per line) so they can be compared.
• Phase (GFF3): for a CDS piece, the number of bases (0–2) to skip to reach the first full codon.
• Primary alignment: the one representative line per read, with neither the secondary (0x100) nor the supplementary (0x800) bit set.
• Soft-masking: lowercase letters in a reference FASTA marking repeats.
• Read group: @RG; tags reads with sample, library and lane, needed by variant callers and duplicate marking.

References

1. SAM/BAM Format Specification Working Group. Sequence Alignment/Map Format Specification (SAMv1), 12 Aug 2025. samtools.github.io/hts-specs/SAMv1.pdf
2. SAM/BAM Format Specification Working Group. Sequence Alignment/Map Optional Fields Specification (SAMtags). samtools.github.io/hts-specs/SAMtags.pdf
3. The Variant Call Format Specification, VCFv4.5 and BCFv2.2, 25 Feb 2026. samtools.github.io/hts-specs/VCFv4.5.pdf
4. CRAM format specification (version 3.x). samtools.github.io/hts-specs/CRAMv3.pdf
5. Niu J, Denisko D, Hoffman MM. The Browser Extensible Data (BED) format, v1, 5 Jan 2022. samtools.github.io/hts-specs/BEDv1.pdf
6. Stein L. Generic Feature Format Version 3 (GFF3), v1.26, 18 Aug 2020. github.com/The-Sequence-Ontology/Specifications
7. Ensembl. GFF/GTF file format. ensembl.org/info/website/upload/gff.html
8. htslib. faidx(5): an index enabling random access to FASTA and FASTQ files. htslib.org/doc/faidx.html
9. Li H, Handsaker B, Wysoker A, et al. The Sequence Alignment/Map format and SAMtools. Bioinformatics. 2009;25(16):2078–2079. doi:10.1093/bioinformatics/btp352
10. Danecek P, Auton A, Abecasis G, et al. The variant call format and VCFtools. Bioinformatics. 2011;27(15):2156–2158. doi:10.1093/bioinformatics/btr330
11. Danecek P, Bonfield JK, Liddle J, et al. Twelve years of SAMtools and BCFtools. GigaScience. 2021;10(2):giab008. doi:10.1093/gigascience/giab008
12. Bonfield JK, Marshall J, Danecek P, et al. HTSlib: C library for reading/writing high-throughput sequencing data. GigaScience. 2021;10(2):giab007. doi:10.1093/gigascience/giab007
13. Cock PJA, Fields CJ, Goto N, Heuer ML, Rice PM. The Sanger FASTQ file format for sequences with quality scores, and the Solexa/Illumina FASTQ variants. Nucleic Acids Research. 2010;38(6):1767–1771. doi:10.1093/nar/gkp1137
14. Pearson WR, Lipman DJ. Improved tools for biological sequence comparison. PNAS. 1988;85(8):2444–2448. doi:10.1073/pnas.85.8.2444
15. Hsi-Yang Fritz M, Leinonen R, Cochrane G, Birney E. Efficient storage of high throughput DNA sequencing data using reference-based compression. Genome Research. 2011;21(5):734–740. doi:10.1101/gr.114819.110
16. Li H. Tabix: fast retrieval of sequence features from generic TAB-delimited files. Bioinformatics. 2011;27(5):718–719. doi:10.1093/bioinformatics/btq671
17. Quinlan AR, Hall IM. BEDTools: a flexible suite of utilities for comparing genomic features. Bioinformatics. 2010;26(6):841–842. doi:10.1093/bioinformatics/btq033
18. Kent WJ, Zweig AS, Barber G, Hinrichs AS, Karolchik D. BigWig and BigBed: enabling browsing of large distributed datasets. Bioinformatics. 2010;26(17):2204–2207. doi:10.1093/bioinformatics/btq351
19. Tan A, Abecasis GR, Kang HM. Unified representation of genetic variants. Bioinformatics. 2015;31(13):2202–2204. doi:10.1093/bioinformatics/btv112
20. Shen W, Le S, Li Y, Hu F. SeqKit: a cross-platform and ultrafast toolkit for FASTA/Q file manipulation. PLoS ONE. 2016;11(10):e0163962. doi:10.1371/journal.pone.0163962
21. Eilbeck K, Lewis SE, Mungall CJ, et al. The Sequence Ontology: a tool for the unification of genome annotations. Genome Biology. 2005;6(5):R44. doi:10.1186/gb-2005-6-5-r44
22. Pertea G, Pertea M. GFF Utilities: GffRead and GffCompare. F1000Research. 2020;9:304. doi:10.12688/f1000research.23297.2
23. Frankish A, Carbonell-Sala S, Diekhans M, et al. GENCODE: reference annotation for the human and mouse genomes in 2023. Nucleic Acids Research. 2023;51(D1):D942–D949. doi:10.1093/nar/gkac1071

What to learn next

• Check your FASTQ files before alignment: Quality Control of NGS Reads with FastQC and MultiQC.
• Browse all free tutorials on the Bioinformatics Tutorials page. Coming next in this path: aligning reads with BWA-MEM, then variant calling.
• Go further with guided projects in our courses: Introduction to NGS and Sequencing QC, Whole Exome Sequencing (WES) Data Analysis and RNA-Seq Data Analysis.