BOL: Related items

odgi: optimized dynamic genome/graph implementation

Abhimanyu Singh — Tue, 01 Feb 2022 23:42:21 -0600

odgi provides an efficient and succinct dynamic DNA sequence graph model, as well as a host of algorithms that allow the use of such graphs in bioinformatic analyses.

Careful encoding of graph entities allows odgi to efficiently compute and transform pangenomes with minimal overheads. odgi implements a dynamic data structure that leveraged multi-core CPUs and can be updated on the fly.

The edges and path steps are recorded as deltas between the current node id and the target node id, where the node id corresponds to the rank in the global array of nodes. Graphs built from biological data sets tend to have local partial order and, when sorted, the deltas be small. This allows them to be compressed with a variable length integer representation, resulting in a small in-memory footprint at the cost of packing and unpacking.

The RAM and computational savings are substantial. In partially ordered regions of the graph, most deltas will require only a single byte.

Address of the bookmark: https://github.com/pangenome/odgi

Synteny Portal: a web-based application portal for synteny block analysis

Jit — Wed, 24 May 2017 10:39:23 -0500

Synteny Portal, a versatile web-based application portal for constructing, visualizing and browsing synteny blocks. With Synteny Portal, users can easily (i) construct synteny blocks among multiple species by using prebuilt alignments in the UCSC genome browser database, (ii) visualize and download syntenic relationships as high-quality images, (iii) browse synteny blocks with genetic information and (iv) download the details of synteny blocks to be used as input for downstream synteny-based analyses, all in an intuitive and easy-to-use web-based interface. We believe that Synteny Portal will serve as a highly valuable tool that will enable biologists to easily perform comparative genomics studies by compensating limitations of existing tools. Synteny Portal is freely available at http://bioinfo.konkuk.ac.kr/synteny_portal.

http://bioinfo.konkuk.ac.kr/synteny_portal/

Address of the bookmark: http://bioinfo.konkuk.ac.kr/synteny_portal/

collinearity: scripts to parse and analyse MCScanX collinearity output

Jit — Wed, 29 Nov 2017 16:47:54 -0600

scripts to parse and analyse MCScanX collinearity output

Address of the bookmark: https://github.com/reubwn/collinearity

String graph based genome assembly software and tools !

Rahul Nayak — Tue, 19 Dec 2017 17:17:38 -0600

In graph theory, a string graph is an intersection graph of curves in the plane; each curve is called a "string". String graphs were first proposed by E. W. Myers in a 2005 publication. In recent Genome Research paper describing an innovative approach for assembling large genomes from NGS data caught our attention for several reasons. i) it give different "string graph" prospective of long lasting genome assembly problem ii) the paper is coauthored by Jared Simpson, the developer of ABySS assembler and Richard Durbin. iii) Simpson-Durbin algorithm is that it does not rely on de Bruijn graphs, and instead employs a different graph construction approach called ‘string graph’.

Following are the genome assembly tools based on string graph:

1.SGA (String Graph Assembler) https://github.com/jts/sga

Assembles large genomes from high coverage short read data. SGA is designed as a modular set of programs, which are used to form an assembly pipeline. SGA implements a set of assembly algorithms based on the FM-index. As the FM-index is a compressed data structure, the algorithms are very memory efficient. The SGA assembly has three distinct phases. The first phase corrects base calling errors in the reads. The second phase assembles contigs from the corrected reads. The third phase uses paired end and/or mate pair data to build scaffolds from the contigs. The output of this software is a PDF report that allows the properties of the genome and data quality to be visually explored. By providing more information to the user at the start of an assembly project, this software will help increase awareness of the factors that make a given assembly easy or difficult, assist in the selection of software and parameters and help to troubleshoot an assembly if it runs into problems.

2. SAGE: String-overlap Assembly of GEnomes https://github.com/lucian-ilie/SAGE2

SAGE, for de novo genome assembly. As opposed to most assemblers, which are de Bruijn graph based, SAGE uses the string-overlap graph. SAGE builds upon great existing work on string-overlap graph and maximum likelihood assembly, bringing an important number of new ideas, such as the efficient computation of the transitive reduction of the string overlap graph, the use of (generalized) edge multiplicity statistics for more accurate estimation of read copy counts, and the improved use of mate pairs and min-cost flow for supporting edge merging. The assemblies produced by SAGE for several short and medium-size genomes compared favourably with those of existing leading assemblers.

3. FSG: Fast String Graph

The new integrated assembler has been assessed on a standard benchmark, showing that fast string graph (FSG) is significantly faster than SGA while maintaining a moderate use of main memory, and showing practical advantages in running FSG on multiple threads. Moreover, we have studied the effect of coverage rates on the running times.

4. BASE https://github.com/dhlbh/BASE

It enhances the classic seed-extension approach by indexing the reads efficiently to generate adaptive seeds that have high probability to appear uniquely in the genome. Such seeds form the basis for BASE to build extension trees and then to use reverse validation to remove the branches based on read coverage and paired-end information, resulting in high-quality consensus sequences of reads sharing the seeds. Such consensus sequences are then extended to contigs. BASE is a practically efficient tool for constructing contig, with significant improvement in quality for long NGS reads. It is relatively easy to extend BASE to include scaffolding.

5. Fermi https://github.com/lh3/fermi/

Fermi is a de novo assembler with a particular focus on assembling Illumina short sequence reads from a mammal-sized genome. In addition to the role of a typical assembler, fermi also aims to preserve heterozygotes which are often collapsed by other assemblers. Its ultimate goal is to find a minimal set of unitigs to represent all the information in raw reads.

If you want to learn about String Graph assembler, please read the following papers -

i) The Fragment Assembly String Graph - E. W. Myers

This paper describes the String Graph concept.

ii) Efficient construction of an assembly string graph using the FM-index - Jared T. Simpson and Richard Durbin

This earlier paper from Simpson and Durbin

iii) Efficient de novo assembly of large genomes using compressed data structures - Jared T. Simpson and Richard Durbin

Genome U-Plot: a whole genome visualization

Rahul Nayak — Fri, 13 Jul 2018 19:50:41 -0500

Genome U-Plot for producing clear and intuitive graphs that allows researchers to generate novel insights and hypotheses by visualizing SVs such as deletions, amplifications, and chromoanagenesis events. The main features of the Genome U-Plot are its layered layout, its high spatial resolution and its improved aesthetic qualities.

https://github.com/gaitat/GenomeUPlot

Address of the bookmark: https://github.com/gaitat/GenomeUPlot

GRSR: a tool for deriving genome rearrangement scenarios from multiple unichromosomal genome sequences

Jit — Fri, 28 Sep 2018 09:35:10 -0500

GRSR is a Tool for Deriving Genome Rearrangement Scenarios for Multiple Uni-chromosomal Genomes. This tool will do the following steps:

Step 1. Run mugsy to get multiple sequence alignment results.
Step 2 & 3. Extraction of the Coordinates of Core Blocks, Construction of Synteny Blocks and Generating Signed Permutations.
Step 4. Generate pairwise genome rearrangement scenarios and find repeats at the breakpoints of each rearrangement events.

https://github.com/DanwangJessica/GRSR

Address of the bookmark: https://github.com/DanwangJessica/GRSR

MitoZ: a toolkit for animal mitochondrial genome assembly, annotation and visualization

Jit — Fri, 24 Jan 2020 04:09:15 -0600

MitoZ is a Python3-based toolkit which aims to automatically filter pair-end raw data (fastq files), assemble genome, search for mitogenome sequences from the genome assembly result, annotate mitogenome (genbank file as result), and mitogenome visualization. MitoZ is available from https://github.com/linzhi2013/MitoZ.

https://academic.oup.com/nar/article/47/11/e63/5377471

Address of the bookmark: https://github.com/linzhi2013/MitoZ

Ancient whole genome duplication (WGD) detection tools !

Rahul Nayak — Sun, 07 Mar 2021 00:32:44 -0600

There are two methods for ancient WGD detection, one is collinearity analysis, and the other is based on the Ks distribution map. Among them, Ks is defined as the average number of synonymous substitutions at each synonymous site, and there is also a Ka corresponding to it, which refers to the average number of non-synonymous substitutions at each non-synonymous site.

At present, some people have posted articles about the analysis process of WGD. I searched for the keyword "wgd pipeline" and found the following:

GenoDup: https:// github.com/MaoYafei/GenoDup-Pipeline
https://peerj.com/articles/6303/
WGDdetector: https:// github.com/yongzhiyang2 012/WGDdetector
https://bmcbioinformatics.biomedcentral.com/articles/10.1186/s12859-019-2670-3
wgd: https:// github.com/arzwa/wgd
https://bmcbioinformatics.biomedcentral.com/articles/10.1186/s12859-016-1142-2#Sec1
https://bmcbiol.biomedcentral.com/articles/10.1186/s12915-017-0399-x
GeNoGAP https://bmcbioinformatics.biomedcentral.com/articles/10.1186/s12859-016-1142-2
https://bmcbiol.biomedcentral.com/articles/10.1186/s12915-017-0399-x
https://github.com/dfguan/purge_dups
https://www.biorxiv.org/content/10.1101/2020.01.24.917997v1

This article introduces the usage of wgd.

Wgd cannot be installed directly with bioconda at present, so it is a little troublesome to install, because it depends on a lot of software. wgd depends on the following software

BLAST
MCL
MUSCLE/MAFFT/PRANK
PAML
PhyML/FastTree
i-ADHoRe

But the good news is that most of the software it depends on can be installed with bioconda

conda create -n wgd python=3.5 blast mcl muscle mafft prank paml fasttree cmake libpng mpi=1.0=mpich
conda activate wgd

Here mpi=1.0=mpich is selected, because i-adhore depends on mpich. If openmpi is installed, an error will appear while loading shared libraries: libmpi_cxx.so.40: cannot open shared object file: No such file or directory

After that, the installation is much simpler

git clone https://github.com/arzwa/wgd.git
cd wgd
pip install .
pip install git+https://github.com/arzwa/wgd.git
For i-ADHoRe, you need to register at http:// bioinformatics.psb.ugent.be /webtools/i-adhore/licensing/Agree to the license to download i-ADHoRe-3.0

Since my miniconda3 installed ~/opt/, the installation path is so~/opt/miniconda3/envs/wgd/

tar -zxvf i-adhore-3.0.01.tar.gz
cd i-adhore-3.0.01
mkdir -p build && cd build
cmake .. -DCMAKE_INSTALL_PREFIX=~/opt/miniconda3/envs/wgd/
make -j 4
make insatall

Take the sugarcane genome Saccharum spontaneum L as an example. The genome is 8-ploid with 32 chromosomes (2n = 4x8 = 32)

Download the tutorial for CDS and GFF annotation files

mkdir -p wgd_tutorial && cd wgd_tutorial
wget http://www.life.illinois.edu/ming/downloads/Spontaneum_genome/Sspon.v20190103.cds.fasta.gz
wget http://www.life.illinois.edu/ming/downloads/Spontaneum_genome/Sspon.v20190103.gff3.gz
gunzip *.gz

First conda activate wgdstart our analysis environment, and then start the analysis

Step 1 : Use to wgd mclidentify homologous genes in the genome

wgd mcl -n 20 --cds --mcl -s Sspon.v20190103.cds.fasta -o Sspon_cds.out

Step 2 : Use to wgd ksdbuild Ks distribution

wgd ksd --n_threads 80 Sspon_cds.out/Sspon.v20190103.cds.fasta.blast.tsv.mcl Sspon.v20190103.cds.fasta

Step 3 : If the quality of the genome is good, then wgd syncollinearity analysis can be used . It can help us find the collinearity block in the genome and the corresponding anchor point

wgd syn --feature gene --gene_attribute ID \
-ks wgd_ksd/Sspon.v20190103.cds.fasta.ks.tsv \
Sspon.v20190103.gff3 Sspon_cds.out/Sspon.v20190103.cds.fasta.blast.tsv.mcl

For more reading - There are 9 sub-modules in WGD

kde: KDE fitting to the Ks distribution
ksd: Ks distribution construction
mcl: BLASP comparison of All-vs-ALl + MCL classification analysis.
mix: Hybrid modeling of Ks distribution.
pre: preprocess the CDS file
syn: Call I-ADHoRe 3.0 to use GFF files for collinearity analysis
viz: draw histogram and density plot
wf1: Ks standard analysis procedure of the whole genome paranome (paranome), call mcl, ksd and syn
wf2: Ks standard analysis procedure of one-vs-one homologous gene (ortholog), call wcl and kSD

JBrowse 2: a modular genome browser with views of synteny and structural variation

Abhi — Tue, 25 Apr 2023 20:58:52 -0500

igvjs - a create-react-app with igv package from npm installed. the igv.js is instrumented to output "DONE" to the console when finished, and to have an increased fetchSizeLimit (which is otherwise git in CRAM longread tests)
jb2-web - stock instance of jbrowse-web v1.7.5
jb1 - stock instance of jbrowse 1 v1.16.11
jb2 embedded - a create-react-app with @jbrowse/react-linear-genome-view

Address of the bookmark: https://github.com/GMOD/jb2profile

Tools to detect synteny blocks regions among multiple genomes

Jitendra Narayan — Mon, 16 Sep 2013 17:12:02 -0500

The synteny block (which etymologically means “on the same ribbon”) is a collection of contiguous genes located on the same chromosome. These block regions have mostly been preserved by genome rearrangements, and so synteny blocks from two related species (e.g., humans and mice) will be roughly similar but flipped around on the respective genomes. Ovcharenko et. al. define it as ‘any conserved sequence blocks, regardless of whether it encompasses multiple genes, an area containing single genes, or areas devoid of known genes to be considers as synteny block as long as there is conservation at the sequence level. Today, however, biologists usually refer to synteny as the conservation of blocks of order within two sets of chromosomes that are being compared with each other. This concept can also be referred to as shared synteny. The NHBLI/NCBI Glossary define synteny as “Two genes which occur on the same chromosome are syntenic; however, syntenic genes may or may not be "linked."

Now a day, geneticists have developed a language of their own. They are pouring lots of money and energy to read the entire genomic text and understand the gods own code ATGC. It is somewhat fascinating, not only for geneticist but also for non-biologist to know that there are several conserved blocks in genome which remain conserved over hundreds of millions of years. There have been several researches on conserved blocks and non-conserved regions to understand the mechanism and importance of all these regions (http://www.ncbi.nlm.nih.gov/pmc/articles/PMC2675965/). The finding indicates conservation and rearrangements of certain evolutionary important genes play an important role in evolution/adaptive changes (http://www.nature.com/nature/journal/v491/n7424/abs/nature11622.html https://academic.oup.com/gbe/article/8/8/2442/2198198/Novel-Insights-into-Chromosome-Evolution-in-Birds , http://science.sciencemag.org/content/346/6215/1311).

But the puzzle remains open, how to correctly define the synteny (presence of two or more genes on the same chromosome) and conserved synteny (presence of two or more genes on chromosome of each of the two species) on several genomes.

Figure: Image generated with Evolution Highway (EH) tool http://eh-demo.ncsa.illinois.edu/

Keeping the new approach to define conserved synteny in mind there have been various algorithms developed to identify the conserved homologous synteny blocks (HSB) amongst species. Some of them which were commonly used for synteny detections are:

SyntenyTracker ( http://www-app.igb.uiuc.edu/labs/lewin/donthu/Synteny_assign/html/),

SyntenyTracker was shown to be an efficient and accurate automated tool for defining HSBs using datasets that may contain minor errors resulting from limitations in map construction methodologies.

CoGe (http://genomevolution.org/CoGe/SynFind.pl )

Satsuma (http://evomics.org/learning/genomics/satsuma/)

Cinteny (http://cinteny.cchmc.org/) ,

Cinteny server can be used for finding regions syntenic across multiple genomes and measuring the extent of genome rearrangement using reversal distance as a measure.

OrthoCluster (http://krono.act.uji.es/noticias/orthocluster-a-new-tool-for-mining-syntenic-blocks)

A new tool for mining syntenic blocks in comparative genomics

SynMap (http://genomevolution.org/wiki/index.php/SynMap),

SyMAP (http://www.symapdb.org/)

SyMAP (Synteny Mapping and Analysis Program) v4.0 is an automated system for identifying and displaying genome synteny alignments. The genomes may be represented by sequenced chromosomes (pseudomolecules), by draft sequence contigs, or by FPC physical maps (with BAC-end or marker sequence).

http://genomevolution.org/CoGe/SynMap.pl

RegionMiner (http://www.genomatix.de/online_help/help_regionminer/orthologous.html)

SyntenyMiner is being developed as an application to visualize and interrogate comparisons among multiple complete genome sequences. http://syntenyminer.sourceforge.net/

AutoGRAPH ( http://autograph.genouest.org/),

AutoGRAPH is an integrated web server for multi-species comparative genomic analysis. It is designed for constructing and visualizing synteny maps between two or three species, determination and display of macrosynteny and microsynteny relationships among species, and for highlighting evolutionary breakpoints.

SynChro(http://www.lgm.upmc.fr/CHROnicle/SynChro.html)

SynChro is a tool designed to define conserved synteny blocks. It reconstructs synteny blocks between pairwise comparison of multiple genomes. The reconstructed synteny blocks may overlap each other, be included in one another or duplicated due to micro-rearrangements.

SyntenyView ( http://www.cbs.dtu.dk/dtucourse/cookbooks/nikob/exercises/gf1_output_5.html),

Ensembl 'SyntenyView' shows conservation of large-scale gene order between species pairs. A brief summary of the calculation method appears at the bottom of this help page. The left of a 'SyntenyView' page displays a diagram of chromosomes with blocks of conserved synteny. The right of a page shows homology matches between individual genes within syntenic blocks.

SynBrowse ( http://www.synbrowse.org/),

SynBrowse (Synteny Browser) is a generic sequence comparison tool for visualizing genome alignments both within and between species. It is intended to help scientists study and analyze synteny, homologous genes and other conserved elements between sequences. This software is useful in studying genome duplication and evolution. It can also aid in identifying uncharacterized genes, putative regulatory elements and novel structural features of study species by comparing to a well annotated reference sequence, thus enabling genome curators to refine and edit annotations of species that have incomplete genome annotations.

Sibelia (http://arxiv.org/abs/1307.7941).

A comparative genomic tool: It assists biologists in analysing the genomic variations that correlate with pathogens, or the genomic changes that help microorganisms adapt in different environments. Sibelia will also be helpful for the evolutionary and genome rearrangement studies for multiple strains of microorganisms.

GSV (http://cas-bioinfo.cas.unt.edu/gsv/homepage.php)

Genome Synteny Viewer allows users to upload files which contain synteny regions between two or more genomes and interactively visualize the synteny between them. GSV also allows users to upload annotation files to visualize annotated regions in addition to synteny regions.

MicroSyn (http://www.lgm.upmc.fr/CHROnicle/SynChro.html)

MicroSyn software as a means of detecting microsynteny in adjacent genomic regions surrounding genes in gene families. MicroSyn searches for conserved, flanking colinear homologous gene pairs between two genomic fragments to determine the relationship between two members in a gene family.

SynOrth (http://synorth.genereg.net/)

Synorth [s n ôrth], named in combination of "synteny" and "ortholog", is designed for the study of evolutionary changes of genomic regulatory blocks (GRBs) in vertebrate genomes, and especially the changes following the whole-genome duplication in teleost fish, by tracing the ortholog genes gain and loss in ancient synteny blocks.

SyDiG (http://www.ncbi.nlm.nih.gov/pubmed/21441096)

Uncovering Synteny in Distant Genomes.

MapSynteny (http://www.automatizacionysistemas.com/download.html)

MapSynteny is a macro in MS Excel® able to create images to show the relationship between genetic maps and large sequences (scaffolds, chromosomes, BACs, etc.). Based on tab – delimited BLAST results and some formulas, a suitable image of syntenic relationships or physical mapping can be obtained. http://www.automatizacionysistemas.com/Poster_MapSynteny.pdf

One of the best synteny tutorial for beginer @ http://www.nature.com/scitable/topicpage/synteny-inferring-ancestral-genomes-44022

Reference:

http://www.nature.com/scitable/topicpage/synteny-inferring-ancestral-genomes-44022

http://www.nature.com/nature/journal/v491/n7424/full/nature11622.html

http://en.wikipedia.org/wiki/Synteny

http://www.ncbi.nlm.nih.gov/pmc/articles/PMC2675965/