BOL: Related items

Import R Data

Abhimanyu Singh — Wed, 12 Jul 2017 08:30:46 -0500

It is often necessary to import sample textbook data into R before you start working on your homework.

Excel File

Quite frequently, the sample data is in Excel format, and needs to be imported into R prior to use. For this, we can use the function read.xls from the gdata package. It reads from an Excel spreadsheet and returns a data frame. The following shows how to load an Excel spreadsheet named "mydata.xls". This method requires Perl runtime to be present in the system.

> library(gdata)                   # load gdata package
> help(read.xls)                   # documentation
> mydata = read.xls("mydata.xls")  # read from first sheet

Alternatively, we can use the function loadWorkbook from the XLConnect package to read the entire workbook, and then load the worksheets with readWorksheet. The XLConnect package requires Java to be pre-installed.

> library(XLConnect) # load XLConnect package
> wk = loadWorkbook("mydata.xls")
> df = readWorksheet(wk, sheet="Sheet1")

Minitab File

If the data file is in Minitab Portable Worksheet format, it can be opened with the function read.mtp from the foreign package. It returns a list of components in the Minitab worksheet.

> library(foreign)                 # load the foreign package
> help(read.mtp)                   # documentation
> mydata = read.mtp("mydata.mtp")  # read from .mtp file

SPSS File

For the data files in SPSS format, it can be opened with the function read.spss also from the foreign package. There is a "to.data.frame" option for choosing whether a data frame is to be returned. By default, it returns a list of components instead.

> library(foreign) # load the foreign package
> help(read.spss) # documentation
> mydata = read.spss("myfile", to.data.frame=TRUE)

Table File

A data table can resides in a text file. The cells inside the table are separated by blank characters. Here is an example of a table with 4 rows and 3 columns.

100   a1   b1
200   a2   b2
300   a3   b3
400   a4   b4

Now copy and paste the table above in a file named "mydata.txt" with a text editor. Then load the data into the workspace with the function read.table.

> mydata = read.table("mydata.txt")  # read text file
> mydata                             # print data frame
   V1 V2 V3
1 100 a1 b1
2 200 a2 b2
3 300 a3 b3
4 400 a4 b4

For further detail of the function read.table, please consult the R documentation.

> help(read.table)

CSV File

The sample data can also be in comma separated values (CSV) format. Each cell inside such data file is separated by a special character, which usually is a comma, although other characters can be used as well.

The first row of the data file should contain the column names instead of the actual data. Here is a sample of the expected format.

Col1,Col2,Col3
100,a1,b1
200,a2,b2
300,a3,b3

After we copy and paste the data above in a file named "mydata.csv" with a text editor, we can read the data with the function read.csv.

> mydata = read.csv("mydata.csv")  # read csv file
> mydata
  Col1 Col2 Col3
1  100   a1   b1
2  200   a2   b2
3  300   a3   b3

In various European locales, as the comma character serves as the decimal point, the function read.csv2 should be used instead. For further detail of the read.csv and read.csv2 functions, please consult the R documentation.

> help(read.csv)

Working Directory

Finally, the code samples above assume the data files are located in the R working directory, which can be found with the function getwd.

> getwd() # get current working directory

You can select a different working directory with the function setwd(), and thus avoid entering the full path of the data files.

> setwd("") # set working directory

Note that the forward slash should be used as the path separator even on Windows platform.

> setwd("C:/MyDoc")

NCBI to assist in Virus Hunting Data Science Hackathon

BioStar — Thu, 15 Nov 2018 12:55:01 -0600

NCBI Hackathon are pleased to announce the second installment of the SoCal Bioinformatics Hackathon. From January 9-11, 2019, the NCBI will help run a bioinformatics hackathon in Southern California hosted by the Computational Sciences Research Center at San Diego State University!

NCBI Hackathon specifically looking for folks who have experience in computational virus hunting or adjacent fields to identify known, taxonomically-definable and novel viruses from a few hundred thousand metagenomic datasets that we’ll put on cloud infrastructure. This event is for researchers, including students and postdocs, who are already engaged in the use of bioinformatics data or in the development of pipelines for virological analyses from high-throughput experiments. If this describes you, please apply! The event is open to anyone selected for the hackathon and willing to travel to SDSU (see below).

https://ncbiinsights.ncbi.nlm.nih.gov/2018/11/09/ncbi-sdsu-virus-hunting-data-science-hackathon-january-2019/

De novo Genome Assembly for Illumina Data

Rahul Nayak — Mon, 20 Jan 2020 05:13:29 -0600

Written and maintained by Simon Gladman - Melbourne Bioinformatics (formerly VLSCI)

Protocol Overview / Introduction

In this protocol we discuss and outline the process of de novo assembly for small to medium sized genomes.

https://www.melbournebioinformatics.org.au/tutorials/tutorials/assembly/assembly-protocol/

Address of the bookmark: https://www.melbournebioinformatics.org.au/tutorials/tutorials/assembly/assembly-protocol/

BioJupies: Automatically Generates RNA-seq Data Analysis Notebooks

Rahul Nayak — Sun, 20 Dec 2020 11:43:45 -0600

With BioJupies you can produce in seconds a customized, reusable, and interactive report from your own raw or processed RNA-seq data through a simple user interface

BioJupies now supports user accounts! Sign in from the top right corner of the page for access to unlimited private notebooks, RNA-seq datasets and alignment jobs.

Address of the bookmark: https://amp.pharm.mssm.edu/biojupies/

Fundamentals of Data Visualization by Claus O. Wilke

Abhi — Sat, 08 Jun 2024 16:07:19 -0500

The book is meant as a guide to making visualizations that accurately reflect the data, tell a story, and look professional. It has grown out of my experience of working with students and postdocs in my laboratory on thousands of data visualizations. Over the years, I have noticed that the same issues arise over and over. I have attempted to collect my accumulated knowledge from these interactions in the form of this book.

The entire book is written in R Markdown, using RStudio as my text editor and the bookdown package to turn a collection of markdown documents into a coherent whole. The book’s source code is hosted on GitHub, at https://github.com/clauswilke/dataviz.

Address of the bookmark: https://clauswilke.com/dataviz/

Quick tour of Genetic Algorithms !

Jit — Thu, 17 Jan 2019 03:42:48 -0600

The R package GA provides a collection of general purpose functions for optimization using genetic algorithms. The package includes a flexible set of tools for implementing genetic algorithms search in both the continuous and discrete case, whether constrained or not. Users can easily define their own objective function depending on the problem at hand.

https://cran.r-project.org/web/packages/GA/vignettes/GA.html

Address of the bookmark: https://cran.r-project.org/web/packages/GA/vignettes/GA.html

List of generic simulation software/tools/resource with brief description and homepage !!!

Jit — Mon, 10 Feb 2014 05:57:29 -0600

List of generic simulation software/tools/resource with brief description and homepage

ALF
A Simulation Framework for Genome Evolution
http://www.cbrg.ethz.ch/alf

Bayesian Serial SimCoal
Bayesian Serial SimCoal, (BayeSSC) is a modification of SIMCOAL 1.0, a program written by Laurent Excoffier, John Novembre, and Stefan Schneider.
http://www.stanford.edu/group/hadlylab/ssc/index.html

BEERS
BEERS was designed to benchmark RNA-Seq alignment algorithms and also algorithms that aim to reconstruct different isoforms and alternate splicing from RNA-Seq data
http://cbil.upenn.edu/beers/

BOTTLENECK
Bottleneck is a program for detecting recent effective population size reductions from allele data frequencies
http://www.ensam.inra.fr/urlb/bottleneck/bottleneck.html

BottleSim
BottleSim is a computer simulation program for simulating the process of population bottlenecks
http://chkuo.name/software/bottlesim.html

CASS
Protein Sequence Simulation
http://www.wyomingbioinformatics.org/liberlesgroup/cass/

CDPOP
CDPOP is a landscape genetics tool for simulating the emergence of spatial genetic structure in populations resulting from specified landscape processes governing organism movement behavior.
http://cel.dbs.umt.edu/cdpop

CoalFace
CoalFace is a simulation of the coalescent process with the visual display of gene genealogies.
http://web.up.ac.za/default.asp?ipkcategoryid=3283

CoaSim
CoaSim is a tool for simulating the coalescent process with recombination and geneconversion under various demographic models.
http://users-birc.au.dk/mailund/coasim/index.html

cosi
The cosi package is written in C and is available as a tar file.
http://www.broadinstitute.org/~sfs/cosi/

CS-PSeq-Gen
A program to simulate the evolution of protein sequences under the constraints of the information of a particular reconstructed phylogeny
http://bioserv.rpbs.univ-paris-diderot.fr/software/cs-pseq-gen.html

DAWG
An application designed to simulate the evolution of recombinant DNA sequences in continuous time
http://scit.us/projects/dawg

Easypop
EASYPOP is an individual based model intended to simulate datasets under a very broad range of conditions
http://www.unil.ch/dee/page36926_fr.html

EggLib
EggLib is a C++/Python library and program package for evolutionary genetics and genomics.
http://egglib.sourceforge.net/

EvolSimulator
A simulation test bed for hypotheses of genome evolution
http://acb.qfab.org/acb/evolsim/

EvolveAGene
A realistic coding sequence simulation program that separates mutation from selection and allows the user to set selection conditions
http://bellinghamresearchinstitute.com/software/index.html

fastsimcoal
A continuous-¬‐time coalescent simulator of genomic diversity under arbitrarily complex evolutionary scenarios
http://cmpg.unibe.ch/software/fastsimcoal/

FastSLINK
Simulation of Marker and Phenotype Data in Pedigrees
http://watson.hgen.pitt.edu/

FFPopSim
C++/Python library for population genetics.
http://webdav.tuebingen.mpg.de/ffpopsim/

FLUX SIMULATOR
The Flux Simulator aims at providing a deterministic in silico reproduction of the experimental pipelines for RNA-Seq, employing a minimal set of parameters.
http://flux.sammeth.net/simulator.html

ForSim
ForSim: A Forward Evolutionary Computer Simulation
http://www.anthro.psu.edu/weiss_lab/research.shtml

ForwSim
The program given below is based on the algorithm described in Padhukasahasram et al. 2008 to simulate genetic drift in a standard Wright-Fisher process.
http://badri-populationgeneticsimulators.blogspot.com/

FPG
Forward Population Genetic simulation
http://genfaculty.rutgers.edu/hey/software#fpg

FREGENE
FREGENE is a C++ program that simulates sequence-like data over large genomic regions in large diploid populations.
http://www.ebi.ac.uk/projects/bargen/download/fregen/documentation_html.html

GAMETES
Genetic Architecture Model Emulator for Testing and Evaluating Software: Simulates complex SNP models with pure, strict epistatic interactions with n-loci.
http://sourceforge.net/projects/gametes/?source=navbar

GASP
Genometric Analysis Simulation Program. A software tool for testing and investigating methods in statistical genetics by generating samples of family data based on user specified models.
http://research.nhgri.nih.gov/gasp/

GemSIM
Next generation sequencing read simulator
http://sourceforge.net/projects/gemsim/

GeneArtisan
Simulation of Markers in Case-Control Study Designs
http://www.rannala.org/?page_id=241

GENOME
A rapid coalescent-based whole genome simulator
http://www.sph.umich.edu/csg/liang/genome/

GenomePop2
GenomePop2 is a specialization of the program GenomePop just to manage SNPs under more flexible and useful settings. If you need models with more than 2 alleles please use the GenomePop program version.
http://webs.uvigo.es/acraaj/genomepop2.htm

GenomeSimla
GenomeSIMLA is currently under development- however, we have a beta release that we are asking to be tested
http://chgr.mc.vanderbilt.edu/genomesimla/

GENS2
Simulates interactions among two genetic and one environmental factor and also allows for epistatic interactions.
https://sourceforge.net/projects/gensim/

GWAsimulator
A rapid whole genome simulation program
http://biostat.mc.vanderbilt.edu/wiki/main/gwasimulator

HAP-SAMPLE
An association simulator for candidate regions or genome scans
http://www.hapsample.org/

HAPGEN
A simulator for the simulation of case control datasets at SNP markers
https://mathgen.stats.ox.ac.uk/genetics_software/hapgen/hapgen2.html

HapSim
A simulation tool for generating haplotype data with pre-specified allele frequencies and LD coefficients
http://cran.r-project.org/web/packages/hapsim/index.html

HAPSIMU
A program that simulates heterogeneous populations with various known and controllable structures under the continuous migration model or the discrete model
http://l.web.umkc.edu/liujian/

IBDsim
IBDSim is a computer package for the simulation of genotypic data under general isolation by distance models.
http://raphael.leblois.free.fr/

indel-Seq-Gen
A biological sequence simulation program that simulates highly divergent DNA sequences and protein superfamilies
http://bioinfolab.unl.edu/~cstrope/isg/

Indelible
A powerful and flexible simulator of biological evolution
http://abacus.gene.ucl.ac.uk/software/indelible/

invertFREGENE
InvertFREGENE is a forward-in-time simulator of inversions in population genetic data
http://www.ebi.ac.uk/projects/bargen/

kernalPop
A spatially explicit population genetic simulation engine
http://cran.r-project.org/src/contrib/archive/kernelpop/

MaCS
Markovian Coalescent Simulator
http://www-hsc.usc.edu/~garykche/

Mason
A package for the simulation of nucleotide data.
http://www.seqan.de/projects/mason/

mbs
modifying Hudson's ms software to generate samples of DNA sequences with a biallelic site under selection
http://www.sendou.soken.ac.jp/esb/innan/innanlab/software.html

Mendel's Accountant
Mendel's Accountant (MENDEL) is an advanced numerical simulation program for modeling genetic change over time and was developed collaboratively by Sanford, Baumgardner, Brewer, Gibson and ReMine
http://mendelsaccount.sourceforge.net/

MetaSim
A tool to generate collections of synthetic reads that reflect the diverse taxonomical composition of typical metagenome data sets
http://ab.inf.uni-tuebingen.de/software/metasim/

mlcoalsim
Multilocus Coalescent Simulations
http://code.google.com/p/mlcoalsim-v1/

ms
The purpose of this program is to allow one to investigate the statistical properties of such samples, to evaluate estimators or statistical tests, and generally to aid in the interpretation of polymorphism data sets.
http://home.uchicago.edu/~rhudson1/source/mksamples.html

msHOT
The purpose of this program is to allow one to investigate the statistical properties of such samples, to evaluate estimators or statistical tests, and generally to aid in the interpretation of polymorphism data sets.
http://home.uchicago.edu/~rhudson1/

msms
A coalescent Simlation tool with selection.
http://www.mabs.at/ewing/msms/index.shtml

MySSP
A program for the simulation of DNA sequence evolution across a phylogenetic tree
http://www.rosenberglab.net/software.php

Nemo
A forward-time, individual-based, genetically explicit, and stochastic simulation program designed to study the evolution of genetic markers, life history traits, and phenotypic traits in a flexible (meta-)population framework.
http://nemo2.sourceforge.net/

NetRecodon
Coalescent simulation of coding DNA sequences with recombination (inter and intracodon), migration and demography
http://code.google.com/p/netrecodon/

PEDAGOG
Software for simulating eco-evolutionary population dynamics
https://bcrc.bio.umass.edu/pedigreesoftware/node/5

phenosim
A tool to add phenotypes to simulated genotypes
http://evoplant.uni-hohenheim.de/doku.php?id=software:software

PhyloSim
An R package for the Monte Carlo simulation of sequence evolution
http://bit.ly/rlsim-git

pIRS
Profile-based Illumina pair-end reads simulator
https://code.google.com/p/pirs/

ProteinEvolver
Simulation of protein evolution along phylogenies under structure-based substitution models
http://code.google.com/p/proteinevolver/

QMSim
QTL and Marker Simulator
http://www.aps.uoguelph.ca/~msargol/qmsim/

quantiNEMO
An individual-based program for the analysis of quantitative traits with explicit genetic architecture potentially under selection in a structured population
http://www2.unil.ch/popgen/softwares/quantinemo/

RECOAL
Simulates new haplotype data from a reference population of haplotypes.
ftp://popgen.usc.edu/

Recodon
Coalescent simulation of coding DNA sequences with recombination, migration and demography
http://code.google.com/p/recodon/

rlsim
A package for simulating RNA-seq library preparation with parameter estimation
http://bit.ly/rlsim-git

Rmetasim
Rmetasim is a front-end for the metasim engine that is implemented as a package that runs in the statistical computing environment R
http://linum.cofc.edu/software.html#metasim

RNA Seq Simulator
RSS takes SAM alignment files from RNA-Seq data and simulates over dispersed, multiple replica, differential, non-stranded RNA-Seq datasets.
http://useq.sourceforge.net/cmdlnmenus.html#rnaseqsimulator

Rose
Random model of sequence evolution
http://bibiserv.techfak.uni-bielefeld.de/rose/

SelSim
SelSim is a program for Monte Carlo simulation of DNA polymorphism data for a recom- bining region within which a single bi-allelic site has experienced natural selection
http://www.well.ox.ac.uk/~spencer/selsim/

Seq-Gen
An application for the Monte Carlo simulation of molecular sequence evolution along phylogenetic trees.
http://tree.bio.ed.ac.uk/software/seqgen/

SEQPower
Statistical power analysis for sequence-based association studies
http://bioinformatics.org/spower/

SeqSIMLA
SeqSIMLA can simulate sequence data with user-specified disease and quantitative trait models. Family or unrelated case-control data can be simulated.
http://seqsimla.sourceforge.net/

Serial NetEvolve
A flexible utility for generating serially-sampled sequences along a tree or recombinant network
http://biorg.cis.fiu.edu/sne/

SFS_CODE
SFS_CODE can perform forward population genetic simulations under a general Wright-Fisher model with arbitrary migration, demographic, selective, and mutational effects.
http://sfscode.sourceforge.net/sfs_code/index/index.html

SIBSIM
Quantitative phenotype simulation in extended pedigrees
http://sourceforge.net/projects/sibsim/

SIMCOAL2
A coalescent program for the simulation of complex recombination patterns over large genomic regions under various demographic models
http://cmpg.unibe.ch/software/simcoal2/

SimCopy
An R package simulating the evolution of copy number profiles along a tree.
http://bit.ly/simcopy

SIMLA
SIMLA is a SIMuLAtion program that generates data sets of families for use in Linkage and Association studies.
http://www.chg.duke.edu/research/simla.html

SimPed
A Simulation Program to Generate Haplotype and Genotype Data for Pedigree Structures
http://www.hgsc.bcm.tmc.edu/content/simped

Simprot
A program to simulate protein evolution by substitution, insertion and deletion
http://www.uhnresearch.ca/labs/tillier/software.htm#3

SimRare
Rare variant simulation and analysis tool
http://code.google.com/p/simrare/

simuGWAS
A forward-time simulator that simulates realistic samples for genome-wide association studies.
http://simupop.sourceforge.net/cookbook/simucomplexdisease

simuPOP
simuPOP is a general-purpose individual-based forward-time population genetics simulation environment.
http://simupop.sourceforge.net/

SISSI
A software tool to generate data of related sequences along a given phylogeny, taking into account user defined system of neighbourhoods and instantaneous rate matrices.
http://www.cibiv.at/software/sissi/

SNPsim
Coalescent simulation of hotspot recombination
http://code.google.com/p/phylosoftware/

SPIP
SPIP simulates the transmission of genes from parents to offspring in a population having demographic structure defined by the user
http://swfsc.noaa.gov/textblock.aspx?division=fed&id=3434

Splatche
Spatial and Temporal Coalescences in Heterogeneous Environment
http://www.splatche.com/

srv
Simulator of Rare Varaints (srv) is a simulator for the simulation of the introduction and evolution of (rare) genetic variants.
http://simupop.sourceforge.net/cookbook/simurarevariants

SUP
SLINK/FastSLINK utility program
http://mlemire.freeshell.org/software.html

TreesimJ
A flexible, forward-time population genetic simulator
http://code.google.com/p/treesimj/

Vortex
VORTEX is an individual-based simulation model for population viability analysis (PVA).
http://www.vortex9.org/vortex.html

References:

Image www.evolution-of-life.com

www.cancer.gov

simNGS and simLibrary – Software for Simulating Next-Gen Sequencing Data

Jit — Tue, 28 Nov 2017 06:49:11 -0600

simNGS is software for simulating observations from Illumina sequencing machines using the statistical models behind the AYB base-calling software. By default, observations only incorporate noise due to sequencing and do not incorporate effects from more esoteric sources of noise that may be present in real data ("dust", bubbles, merged clusters, sequence-heterogeneous clusters, etc). Many of these additional sources may optionally applied.

simNGS takes fasta format sequences and a file describing the covariance of noise between bases and cycles observed in an actual run of the machine, randomly generates noisy intensities representing the signals for the sequence at each cycle and calculates likelihoods for all possible base calls.

Address of the bookmark: https://www.ebi.ac.uk/goldman-srv/simNGS/

Recombination detection tool

Jit — Tue, 02 Feb 2016 10:11:14 -0600

A program to detect recombination hotspots using population genetic data.

More at https://github.com/auton1/LDhot

Address of the bookmark: https://github.com/auton1/LDhot

RASTtk : algorithm for building custom annotation pipelines and annotating batches of genomes

Abhi — Wed, 27 Apr 2016 11:07:59 -0500

The RAST (Rapid Annotation using Subsystem Technology) annotation engine was built in 2008 to annotate bacterial and archaeal genomes. It works by offering a standard software pipeline for identifying genomic features (i.e., protein-encoding genes and RNA) and annotating their functions. Recently, in order to make RAST a more useful research tool and to keep pace with advancements in bioinformatics, it has become desirable to build a version of RAST that is both customizable and extensible. In this paper, we describe the RAST tool kit (RASTtk), a modular version of RAST that enables researchers to build custom annotation pipelines. RASTtk offers a choice of software for identifying and annotating genomic features as well as the ability to add custom features to an annotation job. RASTtk also accommodates the batch submission of genomes and the ability to customize annotation protocols for batch submissions. This is the first major software restructuring of RAST since its inception.

More at http://www.nature.com/articles/srep08365

Address of the bookmark: http://rast.nmpdr.org/