VAT
From GersteinInfo
(→Modify the header line to capture the population information) |
(→Generation of summary statistics) |
||
Line 348: | Line 348: | ||
=== Generation of summary statistics === | === Generation of summary statistics === | ||
- | + | $ mkdir ALL.2of4intersection.20100804 | |
- | vcfSummary ALL.2of4intersection.20100804.vcf.gz gencode.interval | + | $ vcfSummary ALL.2of4intersection.20100804.vcf.gz gencode.interval |
- | vcfSubsetByGene ALL.2of4intersection.20100804.vcf.gz gencode.interval ./ALL.2of4intersection.20100804 | + | $ vcfSubsetByGene ALL.2of4intersection.20100804.vcf.gz gencode.interval ./ALL.2of4intersection.20100804 |
- | vcf2images ALL.2of4intersection.20100804.vcf.gz gencode.interval ./ALL.2of4intersection.20100804 | + | $ vcf2images ALL.2of4intersection.20100804.vcf.gz gencode.interval ./ALL.2of4intersection.20100804 |
Revision as of 19:48, 8 March 2011
Contents |
Introduction
The Variant Annotation Tool (VAT) consists of a set of modules to annotate genetic variants including SNPs and indels. This software package also contains a program to aggregate SNP and indel variants at the gene level. Subsequently, an image is generated for each gene to visualize the functional impact of these variants. This information can then be viewed and shared using a web-interface. In addition to annotation of the coding variants, this tool also integrates allele frequencies and genotype data providing population-specific information from published high quality variation databases such as 1000 Genomes Project.
Data formats
VCF
The Variant Call Format (VCF) is a tab-delimited text file format to represent a number of different genetic variants including SNPs and Indels. This format was developed as part of the 1000 Genomes Project. A detailed summary of this file format can be found here. The annotation information is captured as part of the INFO field using the VA (Variant Annotation) tag. The string with the variant information has the following format:
AlleleNumber:GeneName:GeneId:Strand:Type:FractionOfTranscriptsAffected:{List of transcripts}TranscriptName:TranscriptId:TranscriptLength_RelativePositionOfVariant_RelativePositionOfAminoAcid_Substitution
The allele number refers to the numbering of the alleles. By definition, the reference allele has zero as the allele number, whereas the alternate alleles are numbered starting at one (some variants have more than one alternate alleles). The type refers to the type of variant. For SNPs, the types can take on the following values (generated by snpMapper): synonymous, nonsynonymous, prematureStop, removedStop, and spliceOverlap. For indels (generated by indelMapper), the types can take on the following values: spliceOverlap, startOverlap, endOverlap, insertionFS, insertionNFS, deletionFS, deletionNFS, where FS denotes 'frameshift' and NFS indicates 'non-frameshift'. The term spliceOverlap (for both SNPs and indels) refers to a genetic variant that overlaps with a splice site (either two nucleotides downstream of an exon or two nucleotides upstream of an exon).
Example 1: A SNP is introducing a premature stop codon. This variant affects one out of five transcripts for this gene.
chr1 23112837 . A T . PASS AA=A;AC=7;AN=118;DP=168;SF=2;VA=1:EPHB2:ENSG00000133216:+:prematureStop:1/5:EPHB2-001:ENST00000400191:3165_3055_1019_K->*
Example 2: A SNP leads to a non-synonymous substitution. This variant affects two out of four transcripts for this gene.
chr1 1110357 . G A . PASS AA=G;AC=3;AN=118;DP=203;SF=2;VA=1:TTLL10:ENSG00000162571:+:nonsynonymous:2/4:TTLL10-001:ENST00000379288:1212_1187_396_R->H:TTLL10-202:ENST00000400931:1212_1187_396_R->H
Example 3: A SNP causing a non-synonymous substitution in one transcript and a splice overlap in another transcript of the same gene.
chr9 35819390 rs2381409 C T . PASS AA=N;AC=157;AN=240;DP=49;SF=0,1;VA=1:TMEM8B:ENSG00000137103:+:nonsynonymous:1/7:TMEM8B-202:ENST00000360192:2109_166_56_P->S,1:TMEM8B:ENSG00000137103:+:spliceOverlap:1/7:TMEM8B-001:ENST00000450762:2106
Example 4: An indel with two alternate alleles. Each alternate allele leads to a non-frameshift deletion.
chr7 140118541 . TACAACAACA T,TACA . PASS HP=1;VA=1:AC006344.1:ENSG00000236914:+:deletionNFS:1/1:AC006344.1-201:ENST00000434223:66_23_8_LQQQ->L,2:AC006344.1:ENSG00000236914:+:deletionNFS:1/1:AC006344.1-201:ENST00000434223:66_23_8_LQQ->L
Notice that multiple annotation entries are comma-separated. Multiple annotation entries arise when a variant causes different types of effects on different transcripts (Example 3) or if there are multiple alternate alleles (Example 4).
Interval
The Interval format consists of eight tab-delimited columns and is used to represent genomic intervals such as genes. This format is closely associated with the intervalFind module, which is part of BIOS. This module efficiently finds intervals that overlap with a query interval. The underlying algorithm is based on containment sublists: Alekseyenko, A.V., Lee, C.J. "Nested Containment List (NCList): A new algorithm for accelerating interval query of genome alignment and interval databases" Bioinformatics 2007;23:1386-1393 [1].
1. Name of the interval 2. Chromosome 3. Strand 4. Interval start (with respect to the "+") 5. Interval end (with respect to the "+") 6. Number of sub-intervals 7. Sub-interval starts (with respect to the "+", comma-delimited) 8. Sub-interval end (with respect to the "+", comma-delimited)
Note: For the purpose of VAT, the name field in the Interval file must contain four pieces of information delimited by the '|' symbol (geneId|transcriptId|geneName|transcriptName). Using gencode2interval program ensures proper formatting:
Example file:
ENSG00000008513|ENST00000319914|ST3GAL1|ST3GAL1-201 chr8 - 134472009 134488267 6 134472009,134474117,134475656,134477020,134478136,134487961 134472180,134474237,134475702,134477200,134478333,134488267 ENSG00000008513|ENST00000395320|ST3GAL1|ST3GAL1-202 chr8 - 134472009 134488267 6 134472009,134474117,134475656,134477020,134478136,134487961 134472180,134474237,134475702,134477200,134478333,134488267 ENSG00000008513|ENST00000399640|ST3GAL1|ST3GAL1-203 chr8 - 134472009 134488267 6 134472009,134474117,134475656,134477020,134478136,134487961 134472180,134474237,134475702,134477200,134478333,134488267 ENSG00000008516|ENST00000325800|MMP25|MMP25-201 chr16 + 3097544 3105947 4 3097544,3100009,3100254,3105830 3097548,3100145,3100546,3105947 ENSG00000008516|ENST00000336577|MMP25|MMP25-202 chr16 + 3096918 3109096 10 3096918,3097415,3100009,3100254,3107033,3107310,3107531,3108181,3108412,3108827 3097017,3097548,3100145,3100547,3107210,3107395,3107614,3108334,3108670,3109096
In this example, each interval (line) represents a transcript, while the sub-intervals denote exons. The geneId is utilized to determine if multiple transcripts belong to the same gene model.
Note: the coordinates in the Interval format are zero-based and the end coordinate is not included.
List of programs
VAT Core Modules
snpMapper
snpMapper is a program to annotate a set of SNPs in VCF format. The program determines the effect of a SNP on the coding potential (synonymous, nonsynonymous, prematureStop, removedStop, spliceOverlap) of each transcript of a gene.
Usage:
snpMapper <annotation.interval> <annotation.fa>
- Inputs: Takes a VCF input from STDIN
- Outputs: Outputs annotated SNPs in VCF format. The annotation information is captured as part of the INFO field. For details refer to the VCF format specification.
- Required arguments
- annotation.interval - Annotation file representing the genomic coordinates of the gene models in Interval format. Each line in this file represents a transcript. This file is typically generated using the gencode2interval program.
- annotation.fa - File with the transcript sequences in FASTA format for each entry specified in annotation.interval. This file is typically generated by the interval2sequences program using the 'exonic' mode.
- Optional arguments
- None
snpMapperGeneric
snpMapperGeneric is a program to annotate a set of SNPs in VCF format. The program checks for containment of a SNP in the specified annotation set.
Usage:
snpMapper <annotation.interval> <nameFeature>
- Inputs: Takes a VCF input from STDIN
- Outputs: Outputs annotated SNPs in VCF format. The annotation information is captured as part of the INFO field. For details refer to the VCF format specification.
- Required arguments
- annotation.interval - Annotation file representing the genomic coordinates of the gene models in Interval format. Each line in this file represents a transcript. This file is typically generated using the gencode2interval program.
- nameFeature - Specifies the type of the annotation feature (for example promotor regions). The name of the feature is included as part of the annotation information (in the INFO field) in the resulting VCF file.
- Optional arguments
- None
indelMapper
indelMapper is a program to annotate a set of indels in VCF format. The program determines the effect of an indel on the coding potential (frameshift insertion, non-frameshift insertion, frameshift deletion, non-frameshift deletion, spliceOverlap, startOverlap, endOverlap) of each transcript of a gene.
Usage:
indelMapper <annotation.interval> <annotation.fa>
- Inputs: Takes a VCF input from STDIN
- Outputs: Outputs annotated indels in VCF format. The annotation information is captured as part of the INFO field. For details refer to the VCF format specification.
- Required arguments
- annotation.interval - Annotation file representing the genomic coordinates of the gene models in Interval format. Each line in this file represents a transcript. This file is typically generated using the gencode2interval program.
- annotation.fa - File with the transcript sequences in FASTA format for each entry specified in annotation.interval. This file is typically generated by the interval2sequences program using the 'exonic' mode.
- Optional arguments
- None
vcfSummary
vcfSummary is a program to aggregate annotated variants across genes and samples.
Usage:
vcfSummary <file.vcf.gz> <annotation.interval>
- Inputs: None
- Outputs: Generates two output files. The first file, named file.geneSummary.txt, contains the number of variants categorized by type for each gene. A second file, named file.sampleSummary.txt, summarizes number of variants categorized by type for each sample.
- Required arguments
- file.vcf.gz - VCF file with annotated variants (this can be a mixture of indels and SNPs). This file must be compressed using bgzip and indexed using the tabix program.
- annotation.interval - Annotation file representing the genomic coordinates of the gene models in Interval format. Each line in this file represents a transcript. This file is typically generated using the gencode2interval program.
- Optional arguments
- None
vcf2images
vcf2images is a program to generate an image for each gene to visualize effect of the annotated variants.
Usage:
vcf2images <file.vcf.gz> <annotation.interval> <outputDir>
- Inputs: None
- Outputs: Generates an image in PNG format for each gene that has at least one annotated variant.
- Required arguments
- file.vcf.gz - VCF file with annotated variants (this can be a mixture of indels and SNPs). This file must be compressed using bgzip and indexed using the tabix program.
- annotation.interval - Annotation file representing the genomic coordinates of the gene models in Interval format. Each line in this file represents a transcript. This file is typically generated using the gencode2interval program.
- outputDir - The output directory where the images are stored
- Optional arguments
- None
vcfSubsetByGene
vcfSubsetByGene is a program to subset a VCF file with annotated variants by gene.
Usage:
vcf2images <file.vcf.gz> <annotation.interval> <outputDir>
- Inputs: None
- Outputs: Generates a VCF file for each gene that has at least one annotated variant.
- Required arguments
- file.vcf.gz - VCF file with annotated variants (this can be a mixture of indels and SNPs). This file must be compressed using bgzip and indexed using the tabix program.
- annotation.interval - Annotation file representing the genomic coordinates of the gene models in Interval format. Each line in this file represents a transcript. This file is typically generated using the gencode2interval program.
- outputDir - The output directory where VCF files are stored
- Optional arguments
- None
Auxiliary programs
gencode2interval
gencode2interval converts a GENCODE annotation file (in GTF format) to the Interval format.
Usage:
gencode2interval
- Inputs: Takes a GENCODE annotation file in GTF format from STDIN
- Outputs: Outputs the GENCODE annotation file in Interval format to STDOUT
- Required arguments
- None
- Optional arguments
- None
Note: To obtain the coding sequences of the elements with gene_type protein_coding and transcript_type protein_coding the following command should be used:
awk '/\t(HAVANA|ENSEMBL)\tCDS\t/ {print}' gencode.v3c.annotation.GRCh37.gtf | grep 'gene_type "protein_coding"' | grep 'transcript_type "protein_coding"' > gencode.v3c.annotation.GRCh37.cds.gtpc.ttpc.gtf gencode2interval < encode.v3c.annotation.GRCh37.cds.gtpc.ttpc.gtf > encode.v3c.annotation.GRCh37.cds.gtpc.ttpc.interval
interval2sequences
Module to retrieve genomic/exonic sequences for an annotation set in Interval format.
Usage:
interval2sequences <file.2bit> <file.annotation> <exonic|genomic>
- Inputs: None
- Outputs: Reports the extracted sequences in FASTA format
- Required arguments
- file.2bit - genome reference sequence in 2bit format
- file.annotation - annotation set in Interval format (each line represents one transcript)
- < exonic | genomic > - exonic means that only the exonic regions are extracted, while genomic indicates that the intronic sequences are extracted as well
- Optional arguments
- None
External programs
bgzip/tabix
Tabix is generic tool that indexes position-sorted files in tab-delimited formats to facilitate fast retrieval. This tool was developed by Heng Li. For more information consult the tabix documentation page.
VCF tools
VCF tools consists of a suite of very useful modules to manipulate VCF files. For more information consult the documentation page.
Example workflow
This workflow shows how the 1000 Genomes Project, Phase I, SNP calls data set was processed.
Prerequisites
Download the GENCODE annotation set (version 3c, hg19):
$ wget ftp://ftp.sanger.ac.uk/pub/gencode/release_3c/gencode.v3c.annotation.GRCh37.gtf.gz
Download the human genome (hg19) in 2bit format. This is used by interval2sequences to extract the genomic sequences for the entries specified in the annotation set:
$ wget http://hgdownload.cse.ucsc.edu/goldenPath/hg19/bigZips/hg19.2bit
Download the SNPs in VCF format:
$ ftp://ftp-trace.ncbi.nih.gov/1000genomes/ftp/release/20100804/ALL.2of4intersection.20100804.genotypes.vcf.gz
Preprocessing of the annotation file
Decompress the annotation file:
$ gunzip gencode.v3c.annotation.GRCh37.gtf.gz
Extract the coding sequence (CDS) elements where the both the gene_type and transcript_type are protein_coding:
$ awk '/\t(HAVANA|ENSEMBL)\tCDS\t/ {print}' gencode.v3c.annotation.GRCh37.gtf | grep 'gene_type "protein_coding"' | grep 'transcript_type "protein_coding"' > gencode.v3c.annotation.GRCh37.cds.gtpc.ttpc.gtf
Convert the GENCODE GTF file into Interval format:
$ gencode2interval < gencode.v3c.annotation.GRCh37.cds.gtpc.ttpc.gtf > gencode.v3c.annotation.GRCh37.cds.gtpc.ttpc.interval
Retrieve the genomic sequences for the transcripts specified in the annotation file.
$ interval2sequences hg19.2bit gencode.v3c.annotation.GRCh37.cds.gtpc.ttpc.interval exonic > gencode.v3c.annotation.GRCh37.cds.gtpc.ttpc.fa
Annotation of variants
Annotate the variants using snpMapper
$ zcat ALL.2of4intersection.20100804.genotypes.vcf.gz | snpMapper gencode.v3c.annotation.GRCh37.cds.gtpc.ttpc.interval gencode.v3c.annotation.GRCh37.cds.gtpc.ttpc.fa > ALL.2of4intersection.20100804.genotypes.annotated.vcf
Modify the header line
Modify the VCF header line to assign individual samples to populations (groups). This is done by using the following syntax: group:sample (i.e. CEU:NA0705).
First get the old meta-data lines:
$ grep "#" ALL.2of4intersection.20100804.genotypes.annotated.vcf > ALL.2of4intersection.20100804.genotypes.annotated.oldHeader.vcf
Store the annotated variants in a separate file:
$ grep "#" -v ALL.2of4intersection.20100804.genotypes.annotated.vcf > ALL.2of4intersection.20100804.genotypes.annotated.variants.vcf
Create the new meta-data lines:
$ vcfModifyHeader ALL.2of4intersection.20100804.genotypes.annotated.oldHeader.vcf 20100804.ALL.panel > ALL.2of4intersection.20100804.genotypes.annotated.newHeader.vcf
Merge the new meta-data line with the annotated variants:
$ cat ALL.2of4intersection.20100804.genotypes.annotated.newHeader.vcf ALL.2of4intersection.20100804.genotypes.annotated.variants.vcf > ALL.2of4intersection.20100804.vcf
Compress the newly created file:
$ bgzip ALL.2of4intersection.20100804.vcf
Index the newly created file:
$ tabix -p vcf ALL.2of4intersection.20100804.vcf.gz
Generation of summary statistics
$ mkdir ALL.2of4intersection.20100804 $ vcfSummary ALL.2of4intersection.20100804.vcf.gz gencode.interval $ vcfSubsetByGene ALL.2of4intersection.20100804.vcf.gz gencode.interval ./ALL.2of4intersection.20100804 $ vcf2images ALL.2of4intersection.20100804.vcf.gz gencode.interval ./ALL.2of4intersection.20100804