BOL: Related items

ALF--a simulation framework for genome evolution.

Jit — Tue, 22 Oct 2019 22:05:58 -0500

Artificial Life Framework (ALF) simulates a root genome into a number of related genomes. Result files include the resulting gene sequences, true tree and true MSAs. A description of ALF can be found in the following article:

Daniel A Dalquen, Maria Anisimova, Gaston H Gonnet, Christophe Dessimoz: ALF - A Simulation Framework for Genome Evolution. Mol Biol Evol, 29(4):1115-1123, April 2012.
http://mbe.oxfordjournals.org/content/29/4/1115

Address of the bookmark: http://alfsim.org/#index

Genetic basis of tail-loss evolution

LEGE — Tue, 04 Mar 2025 12:12:36 -0600

The paper "On the genetic basis of tail-loss evolution in humans and apes (https://www.nature.com/articles/s41586-024-07095-8)", published in Nature, investigates the genetic mechanisms that led to the loss of tails in humans and apes. The study suggests that a specific genetic mutation, involving the insertion of an Alu element (a type of transposable DNA sequence), played a critical role in the evolutionary transition from tailed primates to tailless hominoids.

Key Findings of the Study:

Alu Insertion and Tail Loss:
The researchers discovered an Alu-mediated genetic change in a common ancestor of modern apes and humans. This change disrupted the normal function of a gene involved in tail development, leading to the suppression of tail formation.
Gene Disruption Mechanism:
The Alu insertion was found within a regulatory region of the TBXT gene (also known as T or Brachyury), which is crucial for tail development in vertebrates. This insertion likely altered the gene's expression patterns, leading to tail reduction over evolutionary time.
Functional Evidence from Model Organisms:
To test their hypothesis, the researchers introduced similar genetic modifications in mice. The modified mice exhibited shortened or absent tails, supporting the idea that the identified mutation played a role in tail loss in hominoids.
Evolutionary Implications:
The findings suggest that small, random genomic changes—such as transposable element insertions—can have profound effects on body morphology. This study provides evidence that mobile DNA elements (like Alu) can drive major evolutionary transitions.
Relevance to Human Evolution:
Understanding the genetic basis of tail loss helps in reconstructing the evolutionary history of hominins (the lineage that includes humans and our extinct relatives). It also sheds light on how genetic variations contribute to anatomical diversity among primates.

Significance of the Study:

This research highlights the role of transposable elements in shaping evolutionary traits and provides a concrete genetic explanation for a defining characteristic of humans and great apes. It also demonstrates how mutations in regulatory regions of developmental genes can lead to significant anatomical changes.

Public Databases for Bioinformatics !

Jit — Tue, 23 Mar 2021 05:32:15 -0500

https://www.nature.com/articles/s41467-020-17155-y

Server Infrastructure:

File Server:

dhara: Synology 3614 Storage Appliance
4 Core Xeon
108TB disk storage
10Gb ethernet to SCG3
Access atx: dhara:5000
Has btsync server (try it - its much better than dropbox)

Compute Servers:

nandi: Kundaje and Phi Server
24 intel cores
256GB RAM
500GB of SSD storage 
36TB RAID6 local storage
4 Intel Phi's (space for 4 more GPU's)


durga: Montgomery and sensitive data
24 intel cores
256GB RAM
500GB of SSD RAID0 storage 
60TB RAID6 local storage

mitra: Bassik and Web/DB Server
24 core
256GB RAM 
500GB of SSD RAID0 storage 
36TB RAID6 local storage

vayu: Kundaje GPU server
4 core
64GB RAM 
200GB of SSD storage 
8TB RAID10 local storage
4 Nvidia GTX 970 4GB GPUs

amold: Bickel and SGE server
32 AMD core
128GB RAM 
200GB of SSD storage 
12TB RAID5 local storage

wotan: Bickel and SGE server
64 AMD core
256GB RAM 
200GB of SSD storage 
12TB RAID5 local storage

Filesystem:

/users/$USER
default home directory
full backups nightly 
nfs mount to dhara
should store code, papers, and other highly processed data here

/mnt/data/
globally accessible data
should store common data here
e.g. genomes and indexes, annotations, ENCODE data  
if you dont want this to count towards your quote you must chown

/mnt/lab_data/$LAB/
lab accessible data
should store lab project data here 
e.g. ATAC-seq prediction data, enhancer prediction, motif calls

/srv/scratch/$USER
fast local storage
not backed up, but on raid and data will never be deleted
most analysis should be performed here

/srv/persistent/$USER
fast local storage
synced nightly, but not backed up
       ie if the hard drives fail or you delete something and notice 
       within 24 hours we can recover. Otherwise not. (vs home which is 
       properly backed up )  
intermediate analysis products that would be hard to recover should be stored here 
       e.g. stochastic analysis results that need to be kept so that paper 
       results can be reproduced

/srv/www/$LABNAME/
web accessible from mitra.stanford.edu
*NOT BACKED UP*

Some parallel programming patterns:

# gzip a bunch of files
parallel gzip -- *.FILESTOGZIP

# fork example in python:
(for more detailed examples look at 
 https://github.com/nboley/grit/ grit/lib/multiprocessing_utils.py)

import os
import time
import random

import multiprocessing

class ProcessSafeOPStream( object ):
    def __init__( self, writeable_obj ):
        self.writeable_obj = writeable_obj
        self.lock = multiprocessing.Lock()
        self.name = self.writeable_obj.name
        return
    
    def write( self, data ):
        self.lock.acquire()
        self.writeable_obj.write( data )
        self.writeable_obj.flush()
        self.lock.release()
        return
    
    def close( self ):
        self.writeable_obj.close()

def worker(queue, ofp):
    # Try without this
    random.seed()
    while True:
        i = queue.get()
        if i == 'FINISHED': return
        # simulate an expensive function
        x = random.random()
        time.sleep(x/10)
        print i, x
        ofp.write("%i\t%s\n" % (i, x))

NSIMS = 10000
NPROC = 25

# populate queue
todo = multiprocessing.Queue()
for i in xrange(NSIMS): todo.put(i)
for i in xrange(NPROC): todo.put('FINISHED')

ofp = ProcessSafeOPStream( open("output.txt", "w") )

pids = []
for i in xrange(NPROC):
    pid = os.fork()
    if pid == 0:
       worker(todo, ofp)
       os._exit(0)
    else:
       pids.append(pid)  

for pid in pids:
    os.waitpid(pid, 0)

ofp.close()

print "FINISHED"

For use case 1 we obtained the following ENCODE and ROADMAP datasets https://www.encodeproject.org/files/ENCFF446WOD/@@download/ENCFF446WOD.bed.gz, https://www.encodeproject.org/files/ENCFF546PJU/@@download/ENCFF546PJU.bam, https://www.encodeproject.org/files/ENCFF059BEU/@@download/ENCFF059BEU.bam. Blacklisted regions were obtained from http://mitra.stanford.edu/kundaje/akundaje/release/blacklists/hg38-human/hg38.blacklist.bed.gz. The human genome version hg38 was obtained from http://hgdownload.cse.ucsc.edu/goldenPath/hg38/bigZips/hg38.fa.gz.

For use case 2 we used the set of narrowPeak files summarized in https://github.com/wkopp/janggu_usecases/tree/master/extra/urls.txt (archived version v1.0.1). The human genome version hg19 was obtained from http://hgdownload.cse.ucsc.edu/goldenPath/hg19/bigZips/hg19.fa.gz

For use case 3 we used the ENCODE datasets https://www.encodeproject.org/files/ENCFF591XCX/@@download/ENCFF591XCX.bam, https://www.encodeproject.org/files/ENCFF736LHE/@@download/ENCFF736LHE.bigWig, https://www.encodeproject.org/files/ENCFF177HHM/@@download/ENCFF177HHM.bam as we as the GENCODE annotation v29 from ftp://ftp.ebi.ac.uk/pub/databases/gencode/Gencode_human/release_29/gencode.v29.annotation.gtf.gz.

Address of the bookmark: http://mitra.stanford.edu/

DFAST: a flexible prokaryotic genome annotation pipeline for faster genome publication

Jit — Tue, 14 Nov 2017 10:26:16 -0600

We developed a prokaryotic genome annotation pipeline, DFAST, that also supports genome submission to public sequence databases. DFAST was originally started as an on-line annotation server, and to date, over 7,000 jobs have been processed since its first launch in 2016. Here, we present a newly implemented background annotation engine for DFAST, which is also available as a standalone command-line program. The new engine can annotate a typical-sized bacterial genome within 10 minutes, with rich information such as pseudogenes, translation exceptions, and orthologous gene assignment between given reference genomes. In addition, the modular framework of DFAST allows users to customize the annotation workflow easily and will also facilitate extensions for new functions and incorporation of new tools in the future.

Availability and Implementation

The software is implemented in Python 3 and runs in both Python 2.7 and 3.4– on Macintosh and Linux systems. It is freely available at https://github.com/nigyta/dfast_core/ under the GPLv3 license with external binaries bundled in the software distribution. An on-line version is also available at https://dfast.nig.ac.jp/.

Address of the bookmark: https://dfast.nig.ac.jp/

mScaffolder: A comparative genome scaffolding tool

Jit — Fri, 15 Jun 2018 04:48:01 -0500

A comparative genome scaffolding tool based on MUMmer

mScaffolder scaffolds a genome using an existing high quality genome as the reference. It aligns the two genomes using nucmer utility from MUMmer and then orders and orients the contigs of the candidate genome guided by their alignments to the reference genome. Please send your questions and comments to mchakrab@uci.edu.

Citation https://www.nature.com/articles/s41588-017-0010-y

Address of the bookmark: https://github.com/mahulchak/mscaffolder

5700 year-old human genome !

Jit — Thu, 19 Dec 2019 11:22:18 -0600

A Landmark in genomics, scientists have done something that hasn't been done ever.

Scientists have reconstructed the genome of an ancient human who lived nearly 5,700 years ago in Southern Denmark from the birch pitch- an ancient tar-like substance.

By sequencing the sample, researchers not only discovered the ancient human DNA but also microbial DNA reflecting the oral microbiome of the person who chewed the pitch, along with plant and animal DNA that could be the recent meal she might have consumed.

The DNA sample is comparable in quality to well-preserved teeth and skull bones. The DNA suggests that the chewer was a female, most likely with dark skin, dark brown hair and blue eyes.

https://www.nature.com/articles/s41467-019-13549-9

Artistic reconstruction. (Tom Björklund)

More at https://gizmodo.com/scientists-reconstruct-lola-after-finding-her-dna-in-1840481633

Biological databases !

BioStar — Wed, 12 Feb 2020 01:16:29 -0600

Now a days there are a lots of genomics databases available around the world. This bookmark is created to provide all links in one place ...

ftp://ftp.ncbi.nih.gov/genomes/

https://hgdownload.soe.ucsc.edu/downloads.html

Address of the bookmark: ftp://ftp.ncbi.nih.gov/genomes/

VCF Compare !

Rahul Nayak — Wed, 19 Jan 2022 10:30:14 -0600

compare two BWA mapping methods with the online hg18-mapped data

We first operate a rapid inspection of the different BAM files using samtools flagstat. Illumina provided chr21 read mapping obtained with their GA IIx deep sequencing platform <ftp://webdata:webdata@ussd-ftp.illumina.com/Data/SequencingRuns/NA18507_GAIIx_100_chr21.bam>, aligned to the b36/hg18 reference genome)

Address of the bookmark: https://wiki.bits.vib.be/index.php/NGS_Exercise.6#compare_aln_.26_mem_results_with_vcf-compare

The complete sequence of a human genome

Neel — Thu, 31 Mar 2022 23:58:18 -0500

The completed regions include all centromeric satellite arrays, recent segmental duplications, and the short arms of all five acrocentric chromosomes, unlocking these complex regions of the genome to variational and functional studies.

Address of the bookmark: https://www.science.org/doi/10.1126/science.abj6987

Reference Sequence Resource!

LEGE — Wed, 15 Sep 2021 21:15:22 -0500

The ENCODE project uses Reference Genomes from NCBI or UCSC to provide a consistent framework for mapping high-throughput sequencing data. In general, ENCODE data are mapped consistently to 2 human (GRCH38, hg19) and 2 mouse (mm9/mm10) genomes for historical comparability. Drosophia melanogaster experiments are mapped to either dm3 or dm6 and Caenorhabdilis elegans experiments are mapped to ce10 or ce11. T

Address of the bookmark: https://www.encodeproject.org/data-standards/reference-sequences/