BOL: Related items

JCVI:Python utility libraries on genome assembly, annotation and comparative genomics

Jit — Tue, 17 Mar 2020 06:19:06 -0500

Collection of Python libraries to parse bioinformatics files, or perform computation related to assembly, annotation, and comparative genomics.

https://github.com/tanghaibao/jcvi

More at https://github.com/tanghaibao/jcvi/wiki

Address of the bookmark: https://github.com/tanghaibao/jcvi

odgi: optimized dynamic genome/graph implementation

Abhimanyu Singh — Tue, 01 Feb 2022 23:42:21 -0600

odgi provides an efficient and succinct dynamic DNA sequence graph model, as well as a host of algorithms that allow the use of such graphs in bioinformatic analyses.

Careful encoding of graph entities allows odgi to efficiently compute and transform pangenomes with minimal overheads. odgi implements a dynamic data structure that leveraged multi-core CPUs and can be updated on the fly.

The edges and path steps are recorded as deltas between the current node id and the target node id, where the node id corresponds to the rank in the global array of nodes. Graphs built from biological data sets tend to have local partial order and, when sorted, the deltas be small. This allows them to be compressed with a variable length integer representation, resulting in a small in-memory footprint at the cost of packing and unpacking.

The RAM and computational savings are substantial. In partially ordered regions of the graph, most deltas will require only a single byte.

Address of the bookmark: https://github.com/pangenome/odgi

Genomicus: genome browser that enables users to navigate in genomes in several dimensions

Jit — Mon, 28 Feb 2022 23:27:37 -0600

Genomicus is a genome browser that enables users to navigate in genomes in several dimensions: linearly along chromosome axes, transversaly across different species, and chronologicaly along evolutionary time.

Once a query gene has been entered, it is displayed in its genomic context in parallel to the genomic context of all its orthologous and paralogous copies in all the other sequenced metazoan genomes. Moreover, Genomicus stores and displays the predicted ancestral genome structure in all the ancestral species within the phylogenetic range of interest.

All the data on extant species displayed in this browser are from Ensembl.

Summary statistics of Genomicus version 105.01: (view species tree in pdf or newick)


Number of extant species	200
Number of extant genes	4303993
Number of ancestral species	196
Number of ancestral genes	4624213
Number of ancestral synteny blocks	83342

Address of the bookmark: https://www.genomicus.bio.ens.psl.eu/genomicus-105.01/cgi-bin/search.pl

Comparative Genomics Workshops !

Rahul Nayak — Tue, 25 Jan 2022 20:39:58 -0600

This meeting's objective was to obtain a big picture look at the current state of the field of comparative genomics with a focus on commonalities across genomic investigations into humans, model organisms (both traditional and non-traditional), agricultural species, wildlife species and microbes.

https://www.genome.gov/event-calendar/perspectives-in-comparative-genomics-and-evolution

Address of the bookmark: https://www.genome.gov/event-calendar/perspectives-in-comparative-genomics-and-evolution

NVIDIA and Arc Institute Unveil Evo 2: A Breakthrough AI for DNA Design

BioStar — Fri, 21 Feb 2025 10:39:47 -0600

NVIDIA and the Arc Institute have introduced Evo 2, a groundbreaking AI model designed to understand, predict, and generate DNA sequences. This marks a major advancement in computational biology, offering scientists an unprecedented tool to decode the genetic blueprint of life and even design entirely new biological systems.

The Power of Evo 2: AI Meets DNA

Evo 2 is the largest AI model for biology ever created, trained on an astonishing 9.3 trillion DNA "letters" (nucleotides) carefully selected from genomes spanning the entire tree of life. This massive dataset ensures that Evo 2 can recognize patterns and relationships in genetic sequences at an unparalleled scale.

For the first time, scientists can design DNA with AI, moving beyond simple sequence analysis to active DNA generation. Evo 2 enables researchers to predict, modify, and even create entire genetic sequences, opening new possibilities in medicine, agriculture, and synthetic biology.

Decoding the Dark Genome

One of the biggest challenges in genetics is understanding the non-coding regions of DNA—vast stretches of the genome that do not code for proteins but play crucial roles in regulating gene expression. These regions control when and how genes are activated, influencing everything from development to disease.

Evo 2 is designed to decode these non-coding elements, helping researchers uncover their functions and use this knowledge to develop gene-based therapies, synthetic life forms, and precision agriculture solutions.

From Reading DNA to Writing It

To put Evo 2’s impact into perspective:

Previous AI models could "read" DNA like a book, analyzing genetic sequences and identifying patterns.
Evo 2 can "write" entirely new DNA, designing functional genes, chromosomes, and even full genomes from scratch.

This means scientists can now engineer biological systems with AI, designing new proteins, metabolic pathways, and genetic circuits to address real-world challenges.

A Step Toward Generative Biology

The Arc Institute describes Evo 2 as a major step toward "generative biology"—a revolutionary approach where AI is used to create novel biological structures rather than just analyzing existing ones. This could lead to breakthroughs such as:

New medicines: AI-generated enzymes and proteins tailored for targeted therapies.
Disease-resistant crops: Genetically optimized plants for higher yield and climate resilience.
Synthetic organisms: Custom-designed microbes for bioremediation, biofuel production, and industrial applications.

An Open-Source Revolution

Unlike many proprietary AI models, Evo 2 is open source, making its capabilities accessible to researchers worldwide. This democratization of AI-driven biology means that scientists from different disciplines can collaborate, experiment, and innovate, accelerating discoveries in genetic engineering and synthetic biology.

With Evo 2, the boundaries of what’s possible in DNA design, genetic engineering, and biological innovation are being redrawn. The future of life sciences is no longer just about understanding life’s code—it’s about writing it.

The Hiller Lab at the LOEWE-TBG in Frankfurt is looking for an ambitious PostDoc– Comparative Genomics

Sat, 31 Oct 2020 23:01:41 -0500

The mission of our group is to discover genomic determinants of phenotypic differences between species, which is important to understand how nature's fascinating phenotypic diversity evolved and how it is encoded in the genome. Work in the lab ranges from genome assembly and alignment to annotating genes, developing and applying comparative genomic methods to discover key differences in genes (such as loss, gain, selection) and regulatory elements, and using statistical approaches to associate genomic to phenotypic differences [1-8].

The postdoc will join our efforts to extend our methods repertoire to accurately detect additional types of genomic changes, to adopt them to other taxonomic groups, and to apply them on a large-scale to existing and numerous newly-sequenced genomes generated by us and our TBG collaborators.

Candidate requirements
· PhD degree in bioinformatics / computational biology, genomics or a related area

· a strong publication record

· excellent programming skills in a Linux environment as well as experience with shell scripting and Unix tools

· previous experience in large-scale comparative genomic data analysis is an advantage.

More at https://www.lifescience.net/jobs/71529/postdoc-mfd-comparative-genomics/

Exploring Bacterial Comparative Genomics: A Bioinformatics Approach

LEGE — Sat, 14 Dec 2024 12:31:14 -0600

In the world of microbiology, bacteria have long fascinated scientists for their diversity, adaptability, and crucial roles in ecosystems and human health. Comparative genomics—a field that involves analyzing and comparing the genomes of different organisms—has revolutionized our understanding of bacterial evolution, adaptation, and pathogenicity. By leveraging bioinformatics tools and techniques, researchers can uncover genomic insights that were once hidden. This blog delves into the principles, methodologies, and applications of bacterial comparative genomics from a bioinformatics perspective.

What is Bacterial Comparative Genomics?

Comparative genomics involves the systematic comparison of genomes across different bacterial species or strains. This approach allows scientists to:

Identify conserved and unique genes.
Explore genetic determinants of pathogenicity.
Understand bacterial evolution and phylogenetics.
Investigate horizontal gene transfer and its role in antibiotic resistance.

Bioinformatics is central to these analyses, enabling the processing and interpretation of large-scale genomic data.

Key Steps in Bacterial Comparative Genomics

Genome Sequencing and Assembly: The process begins with obtaining high-quality bacterial genome sequences. Advances in next-generation sequencing (NGS) technologies have made it faster and more affordable to sequence bacterial genomes. Tools such as SPAdes and Velvet are commonly used for genome assembly.
Genome Annotation: Annotating a genome involves identifying genes, regulatory elements, and other genomic features. Automated tools like Prokka and RAST provide functional annotations, allowing researchers to predict the roles of genes and proteins.
Genome Alignment: Aligning genomes is crucial for identifying conserved regions, single-nucleotide polymorphisms (SNPs), and structural variations. Tools like Mauve and progressiveMauve are commonly employed for whole-genome alignments.
Comparative Analyses:
- Core and Pan-genome Analysis: The core genome consists of genes shared across all strains of a species, while the pan-genome includes all genes found in any strain. Software like Roary and BPGA can perform core and pan-genome analyses.
- Phylogenetic Analysis: Comparative genomics often involves reconstructing evolutionary relationships. Tools such as MEGA and IQ-TREE facilitate phylogenetic tree construction based on genomic data.
- Functional Enrichment Analysis: To understand the biological significance of unique or shared genes, functional enrichment analysis using databases like GO (Gene Ontology) and KEGG is essential.

Recommended Bioinformatics Tools for Comparative Genomics

Here are some additional bioinformatics tools that can aid bacterial comparative genomics:

OrthoFinder: For accurate ortholog identification across multiple genomes.
PanOCT: Specifically designed for pan-genome clustering and annotation.
FASTANI: A tool for calculating Average Nucleotide Identity (ANI) for microbial genome comparisons.
CIRCOS: For visually comparing genomic data through circular genome plots.
Galaxy Platform: A user-friendly web-based platform offering numerous genomic analysis tools.
BLAST: Essential for sequence alignment and similarity searches.
PhyloSift: Focused on phylogenetic analysis of microbial genomes using marker genes.

These tools, in combination with the methods discussed, provide a robust framework for conducting comprehensive comparative genomic studies.

Applications of Bacterial Comparative Genomics

Understanding Pathogenicity: Comparative genomics helps identify virulence factors that distinguish pathogenic strains from non-pathogenic relatives. For instance, comparing genomes of Escherichia coli strains has revealed key genetic determinants of pathogenicity in enterohemorrhagic strains.
Antibiotic Resistance Research: The spread of antibiotic resistance genes through horizontal gene transfer is a major global concern. Comparative analyses can trace the origins and dissemination of resistance genes, aiding in the development of countermeasures.
Microbial Ecology and Evolution: By studying genomic variations, researchers can understand how bacteria adapt to different environments. This is particularly relevant for extremophiles and symbiotic bacteria.
Vaccine Development: Identifying conserved antigens across pathogenic strains is critical for vaccine design. Comparative genomics has been instrumental in developing vaccines against pathogens like Neisseria meningitidis.
Biotechnology Applications: Comparative studies can uncover unique metabolic pathways in bacteria, paving the way for applications in bioremediation, synthetic biology, and industrial microbiology.

Challenges in Bacterial Comparative Genomics

While the field has made significant strides, several challenges remain:

Data Overload: The rapid growth of sequencing data requires robust computational infrastructure and efficient algorithms.
Genome Plasticity: High rates of horizontal gene transfer and genome rearrangements in bacteria complicate comparative analyses.
Annotation Accuracy: Automated annotation tools are not infallible, and manual curation is often needed for high-confidence results.
Interpreting Non-Coding Regions: Understanding the functional significance of non-coding genomic regions remains a challenge.

Future Directions

The integration of bacterial comparative genomics with other ‘omics’ approaches—such as transcriptomics, proteomics, and metabolomics—promises a more comprehensive understanding of bacterial biology. Additionally, advancements in machine learning and artificial intelligence are likely to further enhance bioinformatics analyses, enabling the prediction of complex phenotypes from genomic data.

Conclusion

Bacterial comparative genomics, driven by bioinformatics, continues to unravel the complexities of bacterial life. From combating antibiotic resistance to uncovering the secrets of microbial evolution, this interdisciplinary field holds immense potential for addressing pressing challenges in microbiology and beyond. As technology advances, so too will our ability to harness the power of comparative genomics for scientific and societal benefit.

MEDEA: Comparative Genomic Visualization with Adobe Flash

Jit — Tue, 26 Apr 2016 12:15:16 -0500

As the number of sequence and annotated genomes grows larger, the need to understand, compare, and contrast the data becomes increasingly important. Using the power of the human visual system to detect trends and spot outliers is necessary in such large and complex data sets.

More at http://www.broadinstitute.org/annotation/medea/

Address of the bookmark: http://www.broadinstitute.org/annotation/medea/

JCVI utility libraries

Jit — Sat, 08 May 2021 22:04:02 -0500

Collection of Python libraries to parse bioinformatics files, or perform computation related to assembly, annotation, and comparative genomics.

Address of the bookmark: https://github.com/tanghaibao/jcvi

Public Databases for Bioinformatics !

Jit — Tue, 23 Mar 2021 05:32:15 -0500

https://www.nature.com/articles/s41467-020-17155-y

Server Infrastructure:

File Server:

dhara: Synology 3614 Storage Appliance
4 Core Xeon
108TB disk storage
10Gb ethernet to SCG3
Access atx: dhara:5000
Has btsync server (try it - its much better than dropbox)

Compute Servers:

nandi: Kundaje and Phi Server
24 intel cores
256GB RAM
500GB of SSD storage 
36TB RAID6 local storage
4 Intel Phi's (space for 4 more GPU's)


durga: Montgomery and sensitive data
24 intel cores
256GB RAM
500GB of SSD RAID0 storage 
60TB RAID6 local storage

mitra: Bassik and Web/DB Server
24 core
256GB RAM 
500GB of SSD RAID0 storage 
36TB RAID6 local storage

vayu: Kundaje GPU server
4 core
64GB RAM 
200GB of SSD storage 
8TB RAID10 local storage
4 Nvidia GTX 970 4GB GPUs

amold: Bickel and SGE server
32 AMD core
128GB RAM 
200GB of SSD storage 
12TB RAID5 local storage

wotan: Bickel and SGE server
64 AMD core
256GB RAM 
200GB of SSD storage 
12TB RAID5 local storage

Filesystem:

/users/$USER
default home directory
full backups nightly 
nfs mount to dhara
should store code, papers, and other highly processed data here

/mnt/data/
globally accessible data
should store common data here
e.g. genomes and indexes, annotations, ENCODE data  
if you dont want this to count towards your quote you must chown

/mnt/lab_data/$LAB/
lab accessible data
should store lab project data here 
e.g. ATAC-seq prediction data, enhancer prediction, motif calls

/srv/scratch/$USER
fast local storage
not backed up, but on raid and data will never be deleted
most analysis should be performed here

/srv/persistent/$USER
fast local storage
synced nightly, but not backed up
       ie if the hard drives fail or you delete something and notice 
       within 24 hours we can recover. Otherwise not. (vs home which is 
       properly backed up )  
intermediate analysis products that would be hard to recover should be stored here 
       e.g. stochastic analysis results that need to be kept so that paper 
       results can be reproduced

/srv/www/$LABNAME/
web accessible from mitra.stanford.edu
*NOT BACKED UP*

Some parallel programming patterns:

# gzip a bunch of files
parallel gzip -- *.FILESTOGZIP

# fork example in python:
(for more detailed examples look at 
 https://github.com/nboley/grit/ grit/lib/multiprocessing_utils.py)

import os
import time
import random

import multiprocessing

class ProcessSafeOPStream( object ):
    def __init__( self, writeable_obj ):
        self.writeable_obj = writeable_obj
        self.lock = multiprocessing.Lock()
        self.name = self.writeable_obj.name
        return
    
    def write( self, data ):
        self.lock.acquire()
        self.writeable_obj.write( data )
        self.writeable_obj.flush()
        self.lock.release()
        return
    
    def close( self ):
        self.writeable_obj.close()

def worker(queue, ofp):
    # Try without this
    random.seed()
    while True:
        i = queue.get()
        if i == 'FINISHED': return
        # simulate an expensive function
        x = random.random()
        time.sleep(x/10)
        print i, x
        ofp.write("%i\t%s\n" % (i, x))

NSIMS = 10000
NPROC = 25

# populate queue
todo = multiprocessing.Queue()
for i in xrange(NSIMS): todo.put(i)
for i in xrange(NPROC): todo.put('FINISHED')

ofp = ProcessSafeOPStream( open("output.txt", "w") )

pids = []
for i in xrange(NPROC):
    pid = os.fork()
    if pid == 0:
       worker(todo, ofp)
       os._exit(0)
    else:
       pids.append(pid)  

for pid in pids:
    os.waitpid(pid, 0)

ofp.close()

print "FINISHED"

For use case 1 we obtained the following ENCODE and ROADMAP datasets https://www.encodeproject.org/files/ENCFF446WOD/@@download/ENCFF446WOD.bed.gz, https://www.encodeproject.org/files/ENCFF546PJU/@@download/ENCFF546PJU.bam, https://www.encodeproject.org/files/ENCFF059BEU/@@download/ENCFF059BEU.bam. Blacklisted regions were obtained from http://mitra.stanford.edu/kundaje/akundaje/release/blacklists/hg38-human/hg38.blacklist.bed.gz. The human genome version hg38 was obtained from http://hgdownload.cse.ucsc.edu/goldenPath/hg38/bigZips/hg38.fa.gz.

For use case 2 we used the set of narrowPeak files summarized in https://github.com/wkopp/janggu_usecases/tree/master/extra/urls.txt (archived version v1.0.1). The human genome version hg19 was obtained from http://hgdownload.cse.ucsc.edu/goldenPath/hg19/bigZips/hg19.fa.gz

For use case 3 we used the ENCODE datasets https://www.encodeproject.org/files/ENCFF591XCX/@@download/ENCFF591XCX.bam, https://www.encodeproject.org/files/ENCFF736LHE/@@download/ENCFF736LHE.bigWig, https://www.encodeproject.org/files/ENCFF177HHM/@@download/ENCFF177HHM.bam as we as the GENCODE annotation v29 from ftp://ftp.ebi.ac.uk/pub/databases/gencode/Gencode_human/release_29/gencode.v29.annotation.gtf.gz.

Address of the bookmark: http://mitra.stanford.edu/