BOL: Related items

Many-Core Engine (MCE) for Perl example

Jit — Tue, 31 Jan 2017 05:37:50 -0600

MCE spawns a pool of workers and therefore does not fork a new process per each element of data. Instead, MCE follows a bank queuing model. Imagine the line being the data and bank-tellers the parallel workers. MCE enhances that model by adding the ability to chunk the next n elements from the input stream to the next available worker.

CORE MODULES

Three modules make up the core engine for MCE.

MCE::Core: Provides the Core API for Many-Core Engine. The various MCE options are described here.
MCE::Signal: Temporary directory creation, cleanup, and signal handling.
MCE::Util: Utility functions for Many-Core Engine.

MCE EXTRAS

There are 4 add-on modules for use with MCE.

MCE::Candy: Provides a collection of sugar methods and output iterators for preserving output order.
MCE::Mutex: Provides a simple semaphore implementation supporting threads and processes.
MCE::Queue: Provides a hybrid queuing implementation for MCE supporting normal queues and priority queues from a single module. MCE::Queue exchanges data via the core engine to enable queuing to work for both children (spawned from fork) and threads.
MCE::Relay: Enables workers to receive and pass on information orderly with zero involvement by the manager process while running.

MCE MODELS

The models take Many-Core Engine to a new level for ease of use. Two options (chunk_size and max_workers) are configured automatically as well as spawning and shutdown.

MCE::Loop: Provides a parallel loop utilizing MCE for building creative loops.
MCE::Flow: A parallel flow model for building creative applications. This makes use of user_tasks in MCE. The author has full control when utilizing this model. MCE::Flow is similar to MCE::Loop, but allows for multiple code blocks to run in parallel with a slight change to syntax.
MCE::Grep: Provides a parallel grep implementation similar to the native grep function.
MCE::Map: Provides a parallel map model similar to the native map function.
MCE::Step: Provides a parallel step implementation utilizing MCE::Queue between user tasks. MCE::Step is a spin off from MCE::Flow with a touch of MCE::Stream. This model, introduced in 1.506, allows one to pass data from one sub-task into the next transparently.
MCE::Stream: Provides an efficient parallel implementation for chaining multiple maps and greps together through user_tasks and MCE::Queue. Like with MCE::Flow, MCE::Stream can run multiple code blocks in parallel with a slight change to syntax from MCE::Map and MCE::Grep.

MISCELLANEOUS

Miscellaneous additions included with the distribution.

MCE::Examples: Describes various demonstrations for MCE including a Monte Carlo simulation.
MCE::Subs: Exports functions mapped directly to MCE methods; e.g. mce_wid. The module allows 3 options; :manager, :worker, and :getter.

REQUIREMENTS

Perl 5.8.0 or later. PDL::IO::Storable is required in scripts running PDL.

SOURCE AND FURTHER READING

The source, cookbook, and examples are hosted at GitHub.

3d-dna: 3D de novo assembly (3D DNA) pipeline

Jit — Thu, 28 Dec 2017 10:09:37 -0600

This code is designed to enable anyone to reproduce the Hs2-HiC and the AaegL4 genomes reported in: Dudchenko et al., De novo assembly of the Aedes aegypti genome using Hi-C yields chromosome-length scaffolds. Science, 2017.

Unless otherwise noted, all terminology below is consistent with this paper, and all references to figures and tables in this readme refer to this paper. Specifically, some of the terminology used below is outlined in Figure S2. The assembly procedure is described in detail in the Supporting Online Materials, specifically in the section labelled “Pipeline description”.

In addition, the pipeline uses tools and methods from Juicer (Durand & Shamim et al., Cell Systems, 2016) and Juicebox (Durand & Robinson et al., Cell Systems, 2016), as well as additional dependencies noted below.

Feel free to post your questions and comments at: http://www.aidenlab.org/forum.html

http://aidenlab.org/documentation.html

Address of the bookmark: https://github.com/theaidenlab/3d-dna

Open Positions in Pasini’s lab

Sat, 04 Feb 2017 08:17:18 -0600

Computational Biologists
Open to PhD-student and Post-doc candidates
We are looking for wet and computational biologists to work on an ERC funded project in our
laboratory located at the Department of Experimental Oncology of the European Institute of
Oncology in Milan (Italy). The project will focus on different aspects of the function of Polycomb
Group proteins and other chromatin modifying activities in relation to their role in regulating cellular
identity in the development of adult tissues.
The candidates will be in charge of computational analysis and data management related to the
project. She/he will directly interact with the wet scientists working in our laboratory while working
embedded in the community of computational biologists present at our institution. The work will
involve the analysis of sequencing data produced with cutting edge technologies to study gene
expression and chromatin environment including data produced on rare cell populations and single
cells. The applicants must have a good knowledge of programming in python/perl/java along with
strong statistical background and performing analysis in R platform. A biological background is
also recommended however it’s not mandatory for application.
Each applicant should submit a full CV (with a detailed description of her/his background,
expertise, achievements and publication records) together with a letter of intent and at least two
contacts for recommendations (for a post-doc position). Competitive salary will be offered based
on the experience of the candidate. Non Italian as well as Italian applicants that have been working
outside Italy (>3yrs.) will have the opportunity to benefit of a full tax deduction for the first three
years of contract.
Applications should be submitted as single PDF to diego.pasini@ieo.it

Lab https://www.ieo.it/en/RESEARCH/People/Researchers/Pasini-Diego/

ALPACA: A hybrid strategy for assembly of genomic DNA shotgun sequencing reads.

Seema Singh — Mon, 30 Apr 2018 04:38:40 -0500

ALPACA requires Celera Assembler 8.3 or later. It is recommended to build Celera Assembler from source. (Why? The pre-built binaries CA_8.3rc1 and CA8.3rc2 will work for any large data set.

Detail paper at https://bmcgenomics.biomedcentral.com/articles/10.1186/s12864-017-3927-8

Address of the bookmark: https://github.com/VicugnaPacos/ALPACA

fineSTRUCTURE v2 & GLOBETROTTER

Shruti Paniwala — Mon, 13 Feb 2017 08:40:23 -0600

Software available at this site

FineSTRUCTURE version 2, a pipeline for running ChromoPainter and FineSTRUCTURE for population inference. A GUI is available for interpretation. Download from the Downloads page.
FineSTRUCTURE R scripts, a facility for exploring the results when the GUI is unavailable.
GLOBETROTTER, the admixture dating method based on ChromoPainter. Download from the Downloads page.
AdmixturePainting, A set of R tools to inmterpret the results of ADMIXTURE and STRUCTURE-like mixture models.
RADpainter, finestructure and ChromoPainter for RAD tag data used for non-model organisms.
Scripts to perform many types of conversion. Included in the main software download from the Downloads page.

What this page is This page provides information about and downloads for methodology for Chromosome Painting. It is not a facility to analyse your genome. Sorry if you were misled by the punchy name!
About Chromosome Painting Painting is an efficient way of identifying important haplotype information from dense genotype data. It describes ancestry in an efficient way suitable for a range of further analyses, including population identification and admixture dating.

Address of the bookmark: http://paintmychromosomes.com/

PERGA: A Paired-End Read Guided De Novo Assembler for Extending Contigs Using SVM and Look Ahead Approach

Rahul Nayak — Tue, 05 Jun 2018 09:57:11 -0500

PERGA - Paired End Reads Guided Assembler PERGA is a novel sequence reads guided de novo assembly approach which adopts greedy-like prediction strategy for assembling reads to contigs and scaffolds. Instead of using single-end reads to construct contig, PERGA uses paired-end reads and different read overlap sizes from O ≥ Omax to Omin to resolve the gaps and branches. Moreover, by constructing a decision model using machine learning approach based on branch features, PERGA can determine the correct extension in 99.7% of cases. PERGA will try to extend the contigs by all feasible nucleotides and determine if these multiple extensions due to sequencing errors or repeats by using looking ahead technology, and it also try to separate the different repeats of nearby genomic regions to make the assembly result more longer and accurate. The simulated E.coli paired-end reads data are generated using GemSim (KE McElroy, F Luciani, T Thomas. Gemsim: General, Error-Model Based Simulator of Next-Generation Sequencing Data. BMC Genomics 2012, 13:74), with coverage 50x, 60x, 100x, read lengths 100-bp, and can be downloaded from https://github.com/zhuxiao/data_PERGA.

Address of the bookmark: https://github.com/hitbio/PERGA

BioDownloader

Surabhi Chaudhary — Sat, 25 Feb 2017 17:52:33 -0600

BioDownloader is a program for downloading and/or updating files from ftp/http servers. The program has unique features that are specifically designed to deal with bioinformatics data files and servers:

optimized to work with vast amount of data and very large file sets (~ 10,000 - 100,000).
allows the selective retrieval of only the required files (file masks, ls-lR parsing, recursive search, updates)
has a built-in repository containing the settings for the most common bioinformatics download needs
built-in wizard for batch post-processing of downloaded files (archive extraction, file conversion, etc.)
capable of performing multiple download or update tasks simultaneously

BioDownloader has a built-in repository containing the settings for common bioinformatics file-synchronization needs, including the Protein Data Bank (PDB) and National Center for Biotechnology Information (NCBI) databases. It can post-process downloaded files, including archive extraction and file conversions.

http://dunbrack.fccc.edu/BioDownloader/

Address of the bookmark: http://dunbrack.fccc.edu/BioDownloader/

ASplice: a scalable and memory-efficient algorithm for de novo transcriptome assembly

Rahul Nayak — Tue, 03 Jul 2018 04:09:46 -0500

With increased availability of de novo assembly algorithms, it is feasible to study entire transcriptomes of non-model organisms. While algorithms are available that are specifically designed for performing transcriptome assembly from high-throughput sequencing data, they are very memory-intensive, limiting their applications to small data sets with few libraries. Texas A&M University researchers develop a transcriptome assembly algorithm that recovers alternatively spliced isoforms and expression levels while utilizing as many RNA-Seq libraries as possible that contain hundreds of gigabases of data. New techniques are developed so that computations can be performed on a computing cluster with moderate amount of physical memory. Availability – A software program that implements the algorithm is available at: http://faculty.cse.tamu.edu/shsze/asplice. Sze SH, Pimsler ML, Tomberlin JK, Jones CD, Tarone AM. (2017) A scalable and memory-efficient algorithm for de novo transcriptome assembly of non-model organisms. BMC Genomics 18(Suppl 4):387.

Address of the bookmark: http://faculty.cse.tamu.edu/shsze/asplice/

DAGchainer: Computing Chains of Syntenic Genes in Complete Genomes

Abhimanyu Singh — Fri, 17 Feb 2017 16:13:35 -0600

The DAGchainer software computes chains of syntenic genes found within complete genome sequences. As input, DAGchainer accepts a list of gene pairs with sequence homology along with their genome coordinates. Using a scoring function which accounts for the distance between neighboring genes on each DNA molecule and the BLAST E-value score between homologs, maximally scoring chains of ordered gene pairs are computed and reported. This algorithm can be used to mine large evolutionary conserved regions of genomes between two organisms. Alternatively, by examining colinear sets of homologous genes found within a single genome, segmental genome duplications can be revealed.

This software distribution includes both the DAGchainer utility and a Java-based graphical interface that allows the inputs and outputs to be navigated and interrogated dynamically.

Address of the bookmark: http://dagchainer.sourceforge.net/

YASRA: Reference based assembler

Abhimanyu Singh — Wed, 01 Mar 2017 08:32:45 -0600

YASRA (Yet Another Short Read Assembler) performs comparative assembly of short reads using a reference genome, which can differ substantially from the genome being sequenced. Mapping reads to reference genomes makes use of LASTZ (Harris et al), a pairwise sequence aligner compatible with BLASTZ. Special scoring sets were derived to improve the performance, both in runtime and quality for 454 and Illumina sequence reads.

YASRA uses LASTZ (http://bx.psu.edu/miller_lab for released version and http://www.bx.psu.edu/~rsharris/lastz/newer for newer version) for aligning the sequences to the reference genome. Please install LASTZ (the newest version on http://www.bx.psu.edu/~rsharris/lastz/newer) and add the LASTZ binary in your executable/binary search path before installing YASRA.

Address of the bookmark: https://github.com/aakrosh/YASRA