BOL: Related items

MetaGraph: Ultra Scalable Framework for DNA Search, Alignment, Assembly

Abhi — Sat, 08 Jun 2024 16:15:25 -0500

The MetaGraph framework is designed to work with a wide range of input data sets, indexing from a few samples up to the contents of entire archives with hundreds of thousands of records. The indexing workflow always follows the same principle, transforming single input samples into error-removed, refined sample graphs, which are then merged into a joint metagraph index. Each input sample is annotated in the joint index as a subgraph. This graph index enriched with metadata can then be used for downstream applications such as sequence search or differential assembly.

Searcg link https://metagraph.ethz.ch/search

Pre-print https://www.biorxiv.org/content/10.1101/2020.10.01.322164v4

Address of the bookmark: https://metagraph.ethz.ch/

DNA in Pieces, Survival in Progress: The Rotifer’s Incredible Secret

Jitendra Narayan — Fri, 18 Sep 2026 03:07:47 -0500

A creature smaller than a grain of sand has discovered a surprising method of surviving damage that can break its DNA apart. It lives in the mosses and lichens, in the thin films of water that show up after rain and disappear when the area dries out. Although it has no hard armor, no sharp claws, and no venom, it is still able to survive even when its chromosomes are shattered by radiation and can pass on some of that damaged DNA to its offspring so that the repair process can go on for several generations. The animal in question is a bdelloid rotifer named Adineta vaga, and its unusual method of survival is providing scientists with new ideas regarding how DNA damage can be repaired.

Bdelloid rotifers are small animals well known for their ability to survive in harsh conditions. They can endure extreme dehydration by entering a state of dormancy and then revive when water becomes available again. Scientists have always been interested in the degree to which these animals resist radiation, since radiation can severely damage DNA by causing both of its strands to break. Interestingly, Adineta vaga's ability to survive radiation may be connected to the way in which it tolerates drying out, as dehydration can also damage DNA. The rotifer probably evolved strong DNA protection and repair mechanisms primarily as a response to its severe environment. Its resistance to radiation is merely a byproduct of adapting to the cycles of drying and rehydration.

In order to find out more about this ability, the researchers subjected A. vaga to proton radiation and then examined what effect it had on its chromosomes. The radiation caused serious damage, resulting in hundreds of breaks in the DNA. For the vast majority of organisms, such extensive chromosome damage would be disastrous; it's similar to going through an instruction manual in a shredder and ending up with a large number of separate fragments. The scientists believed that the rotifers would repair their chromosomes before producing offspring. However, what they discovered was unexpected: some of the broken pieces of chromosome did survive and were passed on to the next generations, where the repair process continued.

The discovery took on even greater interest when the scientists examined the offspring of the irradiated rotifers. Rather than observing the same DNA damage in all the generations, they found different sized sections missing and evidence that some of the genetic regions which had been lost were being slowly restored. The researchers proposed that the rotifers use a repair mechanism known as Break-Induced Homologous Extension Repair, or BIHER. In simple terms, a broken piece of DNA can locate a matching sequence on the other chromosome and use it as a guide to rebuild itself. If the damage is beyond what can be repaired in one generation, the repair process can continue in the next.

What makes the rotifer's method of repairing DNA so unusual is that, normally, we consider DNA repair to take place immediately after damage has occurred, with the cell rapidly repairing the damaged DNA. In A. vaga, however, the repair process can be a lot longer. Certain pieces of chromosome survive the initial damage, are inherited, and then serve as the basis for further repairs in later generations. It's as if one generation begins mending a broken book, the next one continues the work, and another eventually completes the job until all the missing parts have been restored.

One reason why this might work is that bdelloid rotifers have holocentric chromosomes. Rather than having a single point, as most chromosomes do, which controls movement during cell division, the ability to move is distributed along the entire length of their chromosomes. As a result, fragments of chromosomes are sometimes able to retain the parts that are necessary for them to move and divide, and this may account for how broken pieces of chromosome can last long enough to be passed on and then repaired.

This finding could have implications that go beyond those concerning rotifers. Scientists are interested in understanding how genomes cope with severe chromosome damage, for example in the case of chromothripsis, where chromosomes break and are then reassembled in new configurations. Investigating A. vaga could help us to learn how cells manage large genetic issues. Moreover, its high resistance to radiation might also prove useful in research into extreme environments, such as the radiation hazards encountered on long space missions. However, these are ideas for future investigation; at present the rotifer's DNA repair mechanism cannot be used to protect people from radiation.

What is perhaps the most interesting aspect of this discovery is that the rotifer's remarkable ability could have developed for a very simple reason. Since an animal living in moss constantly undergoes periods of drying out and then getting wet again, the same mechanisms that protect its DNA when it is dehydrated might also enable it to survive radiation which would damage most other animals. Therefore, the rotifer probably did not evolve solely in order to resist radiation; instead, it became extremely good at coping with the difficult conditions it encounters, and radiation resistance was just a byproduct.

The case of Adineta vaga demonstrates that survival does not always mean avoiding damage; on some occasions it involves being able to recover from damage which appears to be impossible to repair. This small animal can survive having its chromosomes broken into many pieces, retain some of those pieces, pass them on to its offspring, and continue to repair them over successive generations. Although we generally regard DNA as something fragile that must always remain intact, the rotifer indicates an alternative possibility: perhaps a genome does not have to be perfect in order to survive, since sometimes the pieces last long enough for life to reassemble them, bit by bit, generation after generation.

Ream more at https://www.science.org/doi/10.1126/sciadv.aee0891

DNA RNA MEME

Neel — Thu, 28 Jan 2021 11:23:14 -0600

Explain the DNA and RNA with picture ...

Genetics for Fun and Profit: Andrew Hessel at TEDxVilnius

Sun, 01 Sep 2013 19:15:06 -0500

Andrew Hessel co-chairs the Bioinformatics and Biotechnology track at the Singularity University, an institution founded by futurist Ray Kurzweil and X Prize Foundation CEO Peter Diamandis, with sponsorship from world-leading organizations that include Google, Autodesk, and NASA. He is also the founder of the Pink Army Cooperative, a venture aiming to make open source personalized cancer therapies. His work has been featured in The New York Times, Futurist Magazine, H+, and Wired News. In the spirit of ideas worth spreading, TEDx is a program of local, self-organized events that bring people together to share a TED-like experience. At a TEDx event, TEDTalks video and live speakers combine to spark deep discussion and connection in a small group. These local, self-organized events are branded TEDx, where x = independently organized TED event. The TED Conference provides general guidance for the TEDx program, but individual TEDx events are self-organized.* (*Subject to certain rules and regulations)

Introduction to programming. Write short programs that generate graphics and animation.

Ram Yash Pal — Thu, 14 Aug 2014 23:29:04 -0500

Introduction to programming. Write short programs that generate graphics and animation.

http://funprogramming.org/

Biologist versus computational biologist !

Abhimanyu Singh — Mon, 29 Oct 2018 04:23:24 -0500

This is how it work :)

Public Databases for Bioinformatics !

Jit — Tue, 23 Mar 2021 05:32:15 -0500

https://www.nature.com/articles/s41467-020-17155-y

Server Infrastructure:

File Server:

dhara: Synology 3614 Storage Appliance
4 Core Xeon
108TB disk storage
10Gb ethernet to SCG3
Access atx: dhara:5000
Has btsync server (try it - its much better than dropbox)

Compute Servers:

nandi: Kundaje and Phi Server
24 intel cores
256GB RAM
500GB of SSD storage 
36TB RAID6 local storage
4 Intel Phi's (space for 4 more GPU's)


durga: Montgomery and sensitive data
24 intel cores
256GB RAM
500GB of SSD RAID0 storage 
60TB RAID6 local storage

mitra: Bassik and Web/DB Server
24 core
256GB RAM 
500GB of SSD RAID0 storage 
36TB RAID6 local storage

vayu: Kundaje GPU server
4 core
64GB RAM 
200GB of SSD storage 
8TB RAID10 local storage
4 Nvidia GTX 970 4GB GPUs

amold: Bickel and SGE server
32 AMD core
128GB RAM 
200GB of SSD storage 
12TB RAID5 local storage

wotan: Bickel and SGE server
64 AMD core
256GB RAM 
200GB of SSD storage 
12TB RAID5 local storage

Filesystem:

/users/$USER
default home directory
full backups nightly 
nfs mount to dhara
should store code, papers, and other highly processed data here

/mnt/data/
globally accessible data
should store common data here
e.g. genomes and indexes, annotations, ENCODE data  
if you dont want this to count towards your quote you must chown

/mnt/lab_data/$LAB/
lab accessible data
should store lab project data here 
e.g. ATAC-seq prediction data, enhancer prediction, motif calls

/srv/scratch/$USER
fast local storage
not backed up, but on raid and data will never be deleted
most analysis should be performed here

/srv/persistent/$USER
fast local storage
synced nightly, but not backed up
       ie if the hard drives fail or you delete something and notice 
       within 24 hours we can recover. Otherwise not. (vs home which is 
       properly backed up )  
intermediate analysis products that would be hard to recover should be stored here 
       e.g. stochastic analysis results that need to be kept so that paper 
       results can be reproduced

/srv/www/$LABNAME/
web accessible from mitra.stanford.edu
*NOT BACKED UP*

Some parallel programming patterns:

# gzip a bunch of files
parallel gzip -- *.FILESTOGZIP

# fork example in python:
(for more detailed examples look at 
 https://github.com/nboley/grit/ grit/lib/multiprocessing_utils.py)

import os
import time
import random

import multiprocessing

class ProcessSafeOPStream( object ):
    def __init__( self, writeable_obj ):
        self.writeable_obj = writeable_obj
        self.lock = multiprocessing.Lock()
        self.name = self.writeable_obj.name
        return
    
    def write( self, data ):
        self.lock.acquire()
        self.writeable_obj.write( data )
        self.writeable_obj.flush()
        self.lock.release()
        return
    
    def close( self ):
        self.writeable_obj.close()

def worker(queue, ofp):
    # Try without this
    random.seed()
    while True:
        i = queue.get()
        if i == 'FINISHED': return
        # simulate an expensive function
        x = random.random()
        time.sleep(x/10)
        print i, x
        ofp.write("%i\t%s\n" % (i, x))

NSIMS = 10000
NPROC = 25

# populate queue
todo = multiprocessing.Queue()
for i in xrange(NSIMS): todo.put(i)
for i in xrange(NPROC): todo.put('FINISHED')

ofp = ProcessSafeOPStream( open("output.txt", "w") )

pids = []
for i in xrange(NPROC):
    pid = os.fork()
    if pid == 0:
       worker(todo, ofp)
       os._exit(0)
    else:
       pids.append(pid)  

for pid in pids:
    os.waitpid(pid, 0)

ofp.close()

print "FINISHED"

For use case 1 we obtained the following ENCODE and ROADMAP datasets https://www.encodeproject.org/files/ENCFF446WOD/@@download/ENCFF446WOD.bed.gz, https://www.encodeproject.org/files/ENCFF546PJU/@@download/ENCFF546PJU.bam, https://www.encodeproject.org/files/ENCFF059BEU/@@download/ENCFF059BEU.bam. Blacklisted regions were obtained from http://mitra.stanford.edu/kundaje/akundaje/release/blacklists/hg38-human/hg38.blacklist.bed.gz. The human genome version hg38 was obtained from http://hgdownload.cse.ucsc.edu/goldenPath/hg38/bigZips/hg38.fa.gz.

For use case 2 we used the set of narrowPeak files summarized in https://github.com/wkopp/janggu_usecases/tree/master/extra/urls.txt (archived version v1.0.1). The human genome version hg19 was obtained from http://hgdownload.cse.ucsc.edu/goldenPath/hg19/bigZips/hg19.fa.gz

For use case 3 we used the ENCODE datasets https://www.encodeproject.org/files/ENCFF591XCX/@@download/ENCFF591XCX.bam, https://www.encodeproject.org/files/ENCFF736LHE/@@download/ENCFF736LHE.bigWig, https://www.encodeproject.org/files/ENCFF177HHM/@@download/ENCFF177HHM.bam as we as the GENCODE annotation v29 from ftp://ftp.ebi.ac.uk/pub/databases/gencode/Gencode_human/release_29/gencode.v29.annotation.gtf.gz.

Address of the bookmark: http://mitra.stanford.edu/

Ribbon: Visualizing complex genome alignments and structural variation:

Jit — Wed, 29 Nov 2017 07:40:22 -0600

Ribbon can be used for long reads, short reads, paired-end reads, and assembly/genome alignments. Instructions for each data format are available by clicking on "instructions" in each tab on the right.

Local installation:

You can install Ribbon locally from Github by following the instructions here: https://github.com/MariaNattestad/Ribbon

Address of the bookmark: http://genomeribbon.com/

jobTree based python wrapper to run the genome simulation tool suite Evolver

Jit — Fri, 08 Dec 2017 16:26:32 -0600

evolverSimControl (eSC) can be used to simulate multi-chromosome genome evolution on an arbitrary phylogeny (Newick format). In addition to simply running evolver, eSC also automatically creates statistical summaries of the simulation as it runs including text and image files. Also included are convenience scripts to: check on a running simulation and see detailed status and logging information; extract fasta sequence files from the leaf nodes of a completed simulation; extract pairwise multiple alignment files (.maf) from leaf and branch nodes from a completed simulation and with the help of mafJoin, join them together into a single maf covering the entire simulation.

Address of the bookmark: https://github.com/dentearl/evolverSimControl

Mash: fast genome and metagenome distance estimation using MinHash

Jit — Tue, 12 Dec 2017 17:30:12 -0600

Mash is normally distributed as a dependency-free binary for Linux or OSX (see https://github.com/marbl/Mash/releases). This source distribution is intended for other operating systems or for development. Mash requires c++11 to build, which is available in and GCC >= 4.8 and OSX >= 10.7.

See http://mash.readthedocs.org for more information.

Address of the bookmark: https://github.com/marbl/Mash/releases