https://bioinformaticsworkbook.org/

Address of the bookmark: https://bioinformaticsworkbook.org/

You can also add your favourite NGS educational material, or workshop tutorial by commenting on this bookmarks for user benefit. 

Understanding the basics of genome sequencing:

Tutorial by Luke Jostins.

http://www.genetic-inference.co.uk/blog/2009/04/basics-sequencing-dna-part-1/

http://www.genetic-inference.co.uk/blog/2009/08/basics-sequencing-dna-part-2/

A window into third-generation sequencing

http://hmg.oxfordjournals.org/content/19/R2/R227.full.pdf

==============================================

NGS data analysis pipelines

• Detecting and annotating genetic variations using the HugeSeq pipeline  DOI: 10.1038/nbt.2134
• NARWHAL, a primary analysis pipeline for NGS data http://bioinformatics.oxfordjournals.org/cgi/content/abstract/28/2/284?etoc
• RseqFlow: Workflows for RNA-Seq data analysis  DOI: 10.1093/bioinformatics/btr441
• ngs_backbone: a pipeline for read cleaning, mapping and SNP calling using Next Generation Sequence  10.1186/1471-2164-12-285
• A framework for variation discovery and genotyping using next-generation DNA sequencing data  PubMed: 21478889
• SNiPlay: a web-based tool for detection, management and analysis of SNPs. Application to grapevine diversity projects  DOI: 10.1186/1471-2105-12-134 Abstract: http://www.biomedcentral.com/1471-2105/12/134/abstract
• WEP: a high-performance analysis pipeline for whole-exome data http://www.biomedcentral.com/1471-2105/14/S7/S11
• DDBJ read annotation pipeline: a cloud computing-based pipeline for high-throughput analysis of next-generation sequencing data. http://www.ncbi.nlm.nih.gov/pubmed/23657089
• GATK: a Toolkit for Genome Analysis http://www.broadinstitute.org/gatk/
• Metagenomics:http://www.nbic.nl/education/nbic-phd-school/course-schedule/ngsmetagenomics/
• RNASeq:http://www.nbic.nl/education/nbic-phd-school/course-schedule/ngsrnaseq/
• Bioinformatics and Seq courses: http://www.isb-sib.ch/training/training-activities-schedule/archive-2013.html
• Variant Detection (Model organism) Advanced tutorial https://docs.google.com/document/pub?id=1CuKkKylVDb03tnN7RSWl5EUzleetn0ctjmvaidPKLxM
• Variant Detection Introductory tutorial https://docs.google.com/document/pub?id=1ZRzrjjOCvtAu3m-IKL-rbJ1f4On60dDL_IEwG7oejdI
• Microbial de novo Assembly for Illumina Data Introductory tutorial https://docs.google.com/document/pub?id=1N3AB9ptISUu4zULqe1kXpVF0BDyGb5f5yzxWSJd_WNM
• RNAseq Differential Gene Expression Introductory tutorial https://docs.google.com/document/pub?id=1KbTiBHtvHLfPRZ39AY3uriazrINA8TJzgjjwn1zPP7Y

" Please add your favourite NGS link below in comment section for the benefit of bioinformatics community ". 

Address of the bookmark: http://chagall.med.cornell.edu/NGScourse/

Address of the bookmark: http://bioinformatics.oxfordjournals.org/content/23/14/1713.full

A list of programs for genotype and SNP calling :

SOAP2 http://soap.genomics.org.cn/index.html

Single-sample High-quality variant database (for example, dbSNP) Package for NGS data analysis, which includes a single individual genotype caller (SOAPsnp)

realSFS http://128.32.118.212/thorfinn/realSFS/

Single-sample Aligned reads Software for SNP and genotype calling using single individuals and allele frequencies. Site frequency spectrum (SFS) estimation

Samtools http://samtools.sourceforge.net/

Multi-sample Aligned reads Package for manipulation of NGS alignments, which includes a computation of genotype likelihoods (samtools) and SNP and genotype calling (bcftools)

GATK http://www.broadinstitute.org/gsa/wiki/index.php/The_Genome_Analysis_Toolkit Multi-sample Aligned reads Package for aligned NGS data analysis, which includes a SNP and genotype caller (Unifed Genotyper), SNP filtering (Variant Filtration) and SNP quality recalibration (Variant Recalibrator)

Beagle http://faculty.washington.edu/browning/beagle/beagle.html

Multi-sample LD Candidate SNPs, genotype likelihoods Software for imputation, phasing and association that includes a mode for genotype calling

IMPUTE2 http://mathgen.stats.ox.ac.uk/impute/impute_v2.html

Multi-sample LD Candidate SNPs, genotype likelihoods Software for imputation and phasing, including a mode for genotype calling. Requires fine-scale linkage map

QCall ftp://ftp.sanger.ac.uk/pub/rd/QCALL

Multi-sample LD ‘Feasible’ genealogies at a dense set of loci, genotype likelihoods Software for SNP and genotype calling, including a method for generating candidate SNPs without LD information (NLDA) and a method for incorporating LD information (LDA). The ‘feasible’ genealogies can be generated using Margarita (http://www.sanger.ac.uk/resources/software/margarita)

MaCH http://genome.sph.umich.edu/wiki/Thunder

Multi-sample LD Genotype likelihoods Software for SNP and genotype calling, including a method (GPT_Freq) for generating candidate SNPs without LD information and a method (thunder_glf_freq) for incorporating LD information

Variant calling and/or genotyping

• DiscoSNP++ and discoSnpRAD: Reference-free small variant discovery (SNPs and indels)
• MindTheGap: Detection and assembly of large insertion variants
• TakeABreak: reference-free inversion discovery tool
• SVJedi: Structural Variant genotyper with long read data
• SVJedi-graph: Structural Variant genotyper with long read data using a variation graph

Sequence assembly

• MinYS: reference-guided genome assembly in metagenomics data
• MTG-link: local assembly tool for linked-read data
• Minia: De novo short read assembler
• de-novo pipeline: de-novo assembly pipeline (error correction / contigs / scaffolding) for genomes and meta-genomes
• Mapsembler2: Targeted assembly (not maintained)

Managing k-mers & indexation

• findere: simple strategy for speeding up queries and for reducing false positive calls from any Approximate Membership Query data structure.
  • fimpera extends findere adding the abundance information.
• kmtricks: modular tool suite for counting kmers, and constructing Bloom filters or kmer matrices, for large collections of sequencing data.
• kmindex is a tool for indexing and querying sequencing samples. It is built on top of kmtricks.
• back to sequences: Find sequences (reads, unitigs, genes) related to a set of kmers in large datasets, in a matter of seconds.
• Backpack Quotient Filter: k-mer indexing data structure with abundance
• short read connector: Detect similar reads from potentially large read set
• DSK: Count K-mer in sequences

Pangenome graph manipulation

• Pancat: Pangenome Comparison and Analysis Toolkit
• GFAGraphs: a Python library to handle pangenome graph files in GFA format.

Comparative metagenomics with k-mers

• Simka and SimkaMin: Comparative metagenomics for large-scale datasets
• Comparead & Commet: comparison of metagenomic datasets

Species and bacterial strains identification

• ORI: software using long nanopore reads to identify bacteria present in a sample at the strain level
• StrainFLAIR: STRAIN-level proFiLing using vArIation gRaph

General-purpose sequencing data manipulation

• GASSST: long read mapper
• Leon: short read compressor (now included in GATB-core)
• Bloocoo: short read corrector
• BCALM: Construct compacted de Bruijn graphs (unitigs)

 Protein Structure

• A_Purva: Contact Map Overlap solver
• MD-Jeep: Distance Geometry solver
• CSA: Comparative Structural Alignment

Workflow

• SLICEE: parallel execution of bioinformatics workflows

Comparative Genomics

• CASSIS: detection of rearrangement breakpoints
• PLAST: intensive bank-to-bank sequence comparison
• DRJBreakpointFinder: detection and precise localization of excision sites in proviral segments

Basic Galaxy Tutorial

• RNA-Seq tutorial based on Trapnell et al. (2012) Nature Protocols

In this tutorial we cover the concepts of RNA-Seq differential gene expression (DGE) analysis using a very small synthetic dataset from a well studied organism.

Advanced Galaxy Tutorial

• RNA-Seq (Advanced) Tutorial

In this tutorial we compare the performance of three statistically-based differential expression tools:

* CuffDiff

* EdgeR

* DESeq2

Advanced Command Line Tutorial

• Graphical Output with CummeRbund introduces some basic commands using the cummeRbund package of the R programming language

You will need to install R, RStudio and cummeRbund on your PC (explained in the Tutorial). You will learn how to produce graphical output from RNA-Seq analysis previously done using a Cuffdiff analysis.

Variant Detection

Basic Galaxy Tutorial

• Variant Detection tutorial

In this tutorial we cover the concepts of detecting small variants (SNVs and indels) in human genomic DNA using a small set of reads from chromosome 22.

Advanced Galaxy Tutorial

• Variant Detection (Advanced) Tutorial

In this tutorial we compare the performance of three statistically-based variant detection tools:

* SAMtools: Mpileup

* GATK: Unified Genotyper

* FreeBayes

Each of these tools takes as its input a BAM file of aligned reads and generates a list of likely variants in VCF format

Pipelines are for those who are comfortable with using the UNIX command line; and often allow more control over branching and iteration logic.

• WGS/exome GATK-based variant calling pipeline

This is a basic variant-calling and annotation pipeline developed at the Victorian Life Sciences Computation Initiative (VLSCI), University of Melbourne. It is based around BWA, GATK and ENSEMBL and was originally designed for human (or similar) data. The master branch is configured for WGS data; there is an exome branch configured for variant calling in exome data.

To run the pipeline you will need Rubra: https://github.com/bjpop/rubra. Rubra uses the python Ruffus library: http://www.ruffus.org.uk/.

Protocols

• Familial Variant Calling

In this protocol we discuss and outline the process of calling familial related mutations.

• Somatic Variant Calling

In this protocol we discuss and outline the process of identifying somatic variants or mutations.

Assembly

Basic Galaxy Tutorial

• Genome assembly tutorial

In this tutorial we carry out de novo assembly of a microbial genome. We have also written a De novo Genome Assembly for Illumina Data Protocol for a more generic description of the method.

Protocol

• De novo Genome Assembly for Illumina Data

In this protocol we discuss and outline the process of de novo assembly for small to medium sized genomes. Use our Genome assembly tutorial to learn a specific case of using Galaxy to carry out de novo assembly of a microbial genome.

Small RNAs

Basic Galaxy Tutorial

• Quality control for small RNA

This tutorial covers initial steps of the workflow for analysis of short RNA expression such as a quality control of the raw reads, processing of the raw reads for the subsequent analysis and initial quality assessment of the library.

ChIP Seq

Protocol

• ChIP-Seq

In this protocol we discuss ChIP-Seq: a method to analyze the interaction between proteins and DNA.

Amplicons

Protocol

• Amplicon Alignment

In this protocol we discuss and outline the process of aligning custom amplicons using primers for high precision.

Learn Galaxy

Introduction to Galaxy, for those who are very new to Galaxy.

Using Histories and Workflows, for those with some Galaxy knowledge.

The Galaxy project website has many tutorials and screencasts about using Galaxy and the tools, and developing new tools.

Address of the bookmark: https://genome.edu.au/wiki/Learn

More at http://www.ploscompbiol.org/article/info%3Adoi%2F10.1371%2Fjournal.pcbi.1000369

Address of the bookmark: http://www.ploscompbiol.org/article/info%3Adoi%2F10.1371%2Fjournal.pcbi.1000369

Perl one-liners are small and awesome Perl programs that fit in a single line of code and they do one thing really well. These things include changing line spacing, numbering lines, doing calculations, converting and substituting text, deleting and printing certain lines, parsing logs, editing files in-place, doing statistics, carrying out system administration tasks, updating a bunch of files at once, and many more. Perl one-liners will make you the shell warrior. Anything that took you minutes to solve, will now take you seconds!

perl -pe '$\="\n"'   
#double space a file

perl -pe '$_ .= "\n" unless /^$/'
#double space a file except blank lines

perl -pe '$_.="\n"x7'
#7 space in a line.

perl -ne 'print unless /^$/'
#remove all blank lines

perl -lne 'print if length($_) < 20'
#print all lines with length less than 20.

perl -00 -pe ''
#If there are multiple spaces, delete all leaving one(make the file a single spaced file).

perl -00 -pe '$_.="\n"x4'
#Expand single blank lines into 4 consecutive blank lines

perl -pe '$_ = "$. $_"'
#Number all lines in a file

perl -pe '$_ = ++$a." $_" if /./'
#Number only non-empty lines in a file

perl -ne 'print ++$a." $_" if /./'
#Number and print only non-empty lines in a file

perl -pe '$_ = ++$a." $_" if /regex/'
#Number only lines that match a pattern

perl -ne 'print ++$a." $_" if /regex/'
#Number and print only lines that match a pattern

perl -ne 'printf "%-5d %s", $., $_ if /regex/'
#Left align lines with 5 white spaces if matches a pattern (perl -ne 'printf "%-5d %s", $., $_' : for all the lines)

perl -le 'print scalar(grep{/./}<>)'
#prints the total number of non-empty lines in a file

perl -lne '$a++ if /regex/; END {print $a+0}'
#print the total number of lines that matches the pattern

perl -alne 'print scalar @F'
#print the total number fields(words) in each line.

perl -alne '$t += @F; END { print $t}'
#Find total number of words in the file

perl -alne 'map { /regex/ && $t++ } @F; END { print $t }'
#find total number of fields that match the pattern

perl -lne '/regex/ && $t++; END { print $t }'
#Find total number of lines that match a pattern

perl -le '$n = 20; $m = 35; ($m,$n) = ($n,$m%$n) while $n; print $m'
#will calculate the GCD of two numbers.

perl -le '$a = $n = 20; $b = $m = 35; ($m,$n) = ($n,$m%$n) while $n; print $a*$b/$m'
#will calculate lcd of 20 and 35.

perl -le '$n=10; $min=5; $max=15; $, = " "; print map { int(rand($max-$min))+$min } 1..$n'
#Generates 10 random numbers between 5 and 15.

perl -le 'print map { ("a".."z",”0”..”9”)[rand 36] } 1..8'
#Generates a 8 character password from a to z and number 0 – 9.

perl -le 'print map { ("a",”t”,”g”,”c”)[rand 4] } 1..20'
#Generates a 20 nucleotide long random residue.

perl -le 'print "a"x50'
#generate a string of ‘x’ 50 character long

perl -le 'print join ", ", map { ord } split //, "hello world"'
#Will print the ascii value of the string hello world.

perl -le '@ascii = (99, 111, 100, 105, 110, 103); print pack("C*", @ascii)'
#converts ascii values into character strings.

perl -le '@odd = grep {$_ % 2 == 1} 1..100; print "@odd"'
#Generates an array of odd numbers.

perl -le '@even = grep {$_ % 2 == 0} 1..100; print "@even"'
#Generate an array of even numbers

perl -lpe 'y/A-Za-z/N-ZA-Mn-za-m/' file
#Convert the entire file into 13 characters offset(ROT13)

perl -nle 'print uc'
#Convert all text to uppercase:

perl -nle 'print lc'
#Convert text to lowercase:

perl -nle 'print ucfirst lc'
#Convert only first letter of first word to uppercas

perl -ple 'y/A-Za-z/a-zA-Z/'
#Convert upper case to lower case and vice versa

perl -ple 's/(\w+)/\u$1/g'
#Camel Casing

perl -pe 's|\n|\r\n|'
#Convert unix new lines into DOS new lines:

perl -pe 's|\r\n|\n|'
#Convert DOS newlines into unix new line

perl -pe 's|\n|\r|'
#Convert unix newlines into MAC newlines:

perl -pe '/regexp/ && s/foo/bar/'
#Substitute a foo with a bar in a line with a regexp.

Reference/Sources:

http://genomics-array.blogspot.in/2010/11/some-unixperl-oneliners-for.html

http://genomespot.blogspot.com/2013/08/a-selection-of-useful-bash-one-liners.html

http://biowize.wordpress.com/2012/06/15/command-line-magic-for-your-gene-annotations/

http://genomics-array.blogspot.com/2010/11/some-unixperl-oneliners-for.html

http://bioexpressblog.wordpress.com/2013/04/05/split-multi-fasta-sequence-file/

Reference

Cuccuru G1, Orsini M, Pinna A, Sbardellati A, Soranzo N, Travaglione A, Uva P, Zanetti G, Fotia G. (2014) Orione, a web-based framework for NGS analysis in microbiology. Bioinformatics [Epub ahead of print]. [article]

Address of the bookmark: http://orione.crs4.it/

Address of the bookmark: http://pevsnerlab.kennedykrieger.org/php/?q=software