BOL: Related items

RNA-Seq Data Pathway and Gene-set Analysis Workflows

Jit — Fri, 25 Oct 2013 08:00:48 -0500

It describe the GAGE (Luo et al., 2009) /Pahview (Luo and Brouwer, 2013) workflows on RNA-Seq data pathway analysis and gene-set analysis. The gage package (2.12.0) now includes a new tutorial, “RNA-Seq Data Pathway and Gene-set Analysis Workflows“.

First cover a full workflow from preparation, reads counting, data preprocessing, gene set test, to pathway visualization in about 40 lines of codes. The same workflow can be used for GO analysis or other types of gene set analysis too. We also describe joint workflows, i.e. to do gene-level analysis using one of the major RNA-Seq analysis tools, DEseq/DEseq2, edgeR, limma and Cufflinks, and feed the results into GAGE/Pahview for pathway analysis or visualization. All these workflows are implemented in R/Bioconductor.

The work ows cover the most common situations and issues for RNA-Seq data pathway analysis. Issues like data quality assessment are relevant for data analysis in general yet out the scope of this tutorial. Although we focus on RNA-Seq data here, but pathway analysis work ow remains similar for microarray, particularly step 3-4 would be the same. Please check gage and pathview vigenttes for details.

Note: You need to update to current release versions of R(3.0.2)/ Bioconductor(2.13) to use all the features.

Reference:

Please check it out:
http://bioconductor.org/packages/release/bioc/html/gage.html
http://bioconductor.org/packages/release/bioc/vignettes/gage/inst/doc/RNA-seqWorkflow.pdf

Managing and Analyzing Next-Generation Sequence Data

Rahul Agarwal — Sat, 10 May 2014 06:28:06 -0500

Centralized Bioinformatics Core Facilities provide shared resources for the computational and IT requirements of the investigators in their department or institution. As such, they must be able to effectively react to new types of experimental technology. Recently faced with an unprecedented flood of data generated by the next generation of DNA sequencers, these groups found it necessary to respond quickly and efficiently to the informatics and infrastructure demands. Centralized Facilities newly facing this challenge need to anticipate time and design considerations of necessary components, including infrastructure upgrades, staffing, and tools for data analyses and management ...

More at http://www.ploscompbiol.org/article/info%3Adoi%2F10.1371%2Fjournal.pcbi.1000369

Address of the bookmark: http://www.ploscompbiol.org/article/info%3Adoi%2F10.1371%2Fjournal.pcbi.1000369

One page R survival guide !!

Rahul Nayak — Thu, 28 May 2015 21:10:12 -0500

There any many of the documents have been developed and tested by scientist around the world. I found this one really useful. The data used is available for download as data.zip.

Reference@http://www.datasciencecentral.com/profiles/blogs/one-page-r-a-survival-guide-to-data-science-with-r

Templates for the Data Scientist
1. A Template for Preparing Data: *OnePageR - *R
2. A Template for Building Models: *OnePageR - *R
Getting Started as a Data Scientist
1. Getting Started with R and Rattle: *Lecture - *Laboratory
2. Introducing and Interacting with R: *Lecture - *Laboratory
3. BasicR - OnePage(R) - Writing R scripts
Dealing With Data
1. Read Data into R: *OnePageR - *R
2. Explore and Summarise Data: *OnePageR - *R
3. Transform Data: *OnePageR - *R
4. Dealing with Dates and Time: (PDF, R) Dates and Time
5. Visualising Data with GGPlot2: *OnePageR - *R
6. Visualising Data with Maps *OnePageR - *R
7. Spatial (R) Spatial Analysis
8. Handling Big Data *OnePageR - *R
Descriptive Analytics
1. Cluster Analysis: *Lecture - *OnePageR - *R
2. Association Analysis: *Lecture
Predictive Analytics
1. Decision Trees: *Lecture - *OnePageR - *R - *Rattle
2. Ensembles of Decision Trees: *Lecture - *OnePageR - *R
3. SVM (R)
4. KernLab (R)
5. NeuralNetworks (R)
6. NNet (R)
Model Delivery
1. Evaluating Models: *OnePageR - *R
2. Evaluation (R)
3. Scoring (R)
4. PMML (R) Exporting Models for Deployment
Advanced Topics
1. Text Mining: *OnePageR - *R
Advanced R Topics
1. Plots (PDF, R) Miscellaneous Plots
2. Functions (PDF, R) Writing Functions in R
3. Parallel (PDF, R) Parallel Execution
4. Packaging (R) Pulling it Together into a Package
5. Doing R with Style: *OnePageR - *R
6. Literate Data Mining with KnitR: *Lecture - *OnePageR - *

Visiting Scientist - Computational Genomics (two positions)

Mon, 07 Jul 2014 22:53:41 -0500

Scientific/Managerial & International Recruitment

ICRISAT seeks applications from Indian nationals Visiting Scientist-Computational Genomics (2 positions), to be part of a team of Centre of Excellence in Genomics (CEG), (www.icrisat.org/ceg) to work on legume genomics projects. The positions will be based at ICRISAT’s Headquarters in Patancheru, Hyderabad, India.

ICRISAT is a non-profit, non-political organization that conducts agricultural research for development in Asia and sub-Saharan Africa with a wide array of partners throughout the world. Covering 6.5 million square kilometers of land in 55 countries, the semi-arid tropics is home to over 2 billion people, with 650 million of these are the poorest of the poor. ICRISAT and its partners help empower those living in the semi-arid tropics, especially smallholder farmers, to overcome poverty, hunger, malnutrition and a degraded environment through more efficient and profitable agriculture. ICRISAT is headquartered in Greater Hyderabad, Andhra Pradesh, India and belongs to the Consortium of Centers supported by the Consultative Group on International Agricultural Research (CGIAR).

The Job: Responsibilities for these positions include:

Analyzing and handling large-scale next generation sequencing DNA and RNA data
Data mining and development of pipelines and troubleshooting
Genome diversity analysis such as SNPs, Indels, Structural Variations, population structure
Genome wide association study (GWAS) related analysis- LD analysis, hapmap and trait mapping
Expression analysis based on RNA-Seq data, annotation, gene ontology and metabolic pathway analysis
Epigenome analysis, small RNA identification
Gene family analysis, sequence level protein analysis, orthology/paralogy and molecular modelling
Compiling and analysis of results, writing reports and research papers

The Person: Ph.D. or MSc/MTech/PGDCA with two years research experience in Biotechnology, Computational biology, Agricultural/ Plant Biotechnology, Genetics, Molecular Biology or related discipline. Good knowledge of programming/scripting in at least two of following languages: Perl, C, C++, R, Shell Scripting and Python is plus.

How to apply: Please apply latest by 20 July 2014. The application should include the name of the position applied for, a letter of motivation, a full Curriculum Vita (CV), and the names and contact information of three references that are knowledgeable of the candidate’s professional qualifications and work experience. Technical details and more information about these positions can be obtained from R.K.VARSHNEY@CGIAR.ORG. All applications will be acknowledged, however only short listed candidates will be contacted.

Apply here https://recruit.zoho.com/ats/Portal.na?digest=T642sgLYWZOStExJ77cPrcM*sIMGZETWw4yPxngbmHA-

Import R Data

Abhimanyu Singh — Wed, 12 Jul 2017 08:30:46 -0500

It is often necessary to import sample textbook data into R before you start working on your homework.

Excel File

Quite frequently, the sample data is in Excel format, and needs to be imported into R prior to use. For this, we can use the function read.xls from the gdata package. It reads from an Excel spreadsheet and returns a data frame. The following shows how to load an Excel spreadsheet named "mydata.xls". This method requires Perl runtime to be present in the system.

> library(gdata)                   # load gdata package
> help(read.xls)                   # documentation
> mydata = read.xls("mydata.xls")  # read from first sheet

Alternatively, we can use the function loadWorkbook from the XLConnect package to read the entire workbook, and then load the worksheets with readWorksheet. The XLConnect package requires Java to be pre-installed.

> library(XLConnect) # load XLConnect package
> wk = loadWorkbook("mydata.xls")
> df = readWorksheet(wk, sheet="Sheet1")

Minitab File

If the data file is in Minitab Portable Worksheet format, it can be opened with the function read.mtp from the foreign package. It returns a list of components in the Minitab worksheet.

> library(foreign)                 # load the foreign package
> help(read.mtp)                   # documentation
> mydata = read.mtp("mydata.mtp")  # read from .mtp file

SPSS File

For the data files in SPSS format, it can be opened with the function read.spss also from the foreign package. There is a "to.data.frame" option for choosing whether a data frame is to be returned. By default, it returns a list of components instead.

> library(foreign) # load the foreign package
> help(read.spss) # documentation
> mydata = read.spss("myfile", to.data.frame=TRUE)

Table File

A data table can resides in a text file. The cells inside the table are separated by blank characters. Here is an example of a table with 4 rows and 3 columns.

100   a1   b1
200   a2   b2
300   a3   b3
400   a4   b4

Now copy and paste the table above in a file named "mydata.txt" with a text editor. Then load the data into the workspace with the function read.table.

> mydata = read.table("mydata.txt")  # read text file
> mydata                             # print data frame
   V1 V2 V3
1 100 a1 b1
2 200 a2 b2
3 300 a3 b3
4 400 a4 b4

For further detail of the function read.table, please consult the R documentation.

> help(read.table)

CSV File

The sample data can also be in comma separated values (CSV) format. Each cell inside such data file is separated by a special character, which usually is a comma, although other characters can be used as well.

The first row of the data file should contain the column names instead of the actual data. Here is a sample of the expected format.

Col1,Col2,Col3
100,a1,b1
200,a2,b2
300,a3,b3

After we copy and paste the data above in a file named "mydata.csv" with a text editor, we can read the data with the function read.csv.

> mydata = read.csv("mydata.csv")  # read csv file
> mydata
  Col1 Col2 Col3
1  100   a1   b1
2  200   a2   b2
3  300   a3   b3

In various European locales, as the comma character serves as the decimal point, the function read.csv2 should be used instead. For further detail of the read.csv and read.csv2 functions, please consult the R documentation.

> help(read.csv)

Working Directory

Finally, the code samples above assume the data files are located in the R working directory, which can be found with the function getwd.

> getwd() # get current working directory

You can select a different working directory with the function setwd(), and thus avoid entering the full path of the data files.

> setwd("") # set working directory

Note that the forward slash should be used as the path separator even on Windows platform.

> setwd("C:/MyDoc")

MinION_GC: An R script to do some QC on MinION data

Radha Agarkar — Sun, 03 Dec 2017 15:19:18 -0600

Other tools focus on getting data out of the fastq or fast5 files, which is slow and computationally intensive. The benefit of this approach is that it works on a single, small, .txt summary file. So it's a lot quicker than most other things out there: it takes about a minute to analyse a 4GB flowcell on my laptop.

https://github.com/roblanf/minion_qc

Address of the bookmark: https://github.com/roblanf/minion_qc

Perl one-liner for bioinformatician !!!

Abhimanyu Singh — Fri, 30 May 2014 05:49:07 -0500

With the emergence of NGS technologies, and sequencing data most of the bioinformaticians mung and wrangle around massive amounts of genomics text. There are several "standardized" file formats (FASTQ, SAM, VCF, etc.) and some tools for manipulating them (fastx toolkit, samtools, vcftools, etc.), there are still times where knowing a little bit of Perl onliner is extremely helpful.

Perl one-liners are small and awesome Perl programs that fit in a single line of code and they do one thing really well. These things include changing line spacing, numbering lines, doing calculations, converting and substituting text, deleting and printing certain lines, parsing logs, editing files in-place, doing statistics, carrying out system administration tasks, updating a bunch of files at once, and many more. Perl one-liners will make you the shell warrior. Anything that took you minutes to solve, will now take you seconds!

perl -pe '$\="\n"'
#double space a file

perl -pe '$_ .= "\n" unless /^$/'
#double space a file except blank lines

perl -pe '$_.="\n"x7'
#7 space in a line.

perl -ne 'print unless /^$/'
#remove all blank lines

perl -lne 'print if length($_) < 20'
#print all lines with length less than 20.

perl -00 -pe ''
#If there are multiple spaces, delete all leaving one(make the file a single spaced file).

perl -00 -pe '$_.="\n"x4'
#Expand single blank lines into 4 consecutive blank lines

perl -pe '$_ = "$. $_"'
#Number all lines in a file

perl -pe '$_ = ++$a." $_" if /./'
#Number only non-empty lines in a file

perl -ne 'print ++$a." $_" if /./'
#Number and print only non-empty lines in a file

perl -pe '$_ = ++$a." $_" if /regex/'
#Number only lines that match a pattern

perl -ne 'print ++$a." $_" if /regex/'
#Number and print only lines that match a pattern

perl -ne 'printf "%-5d %s", $., $_ if /regex/'
#Left align lines with 5 white spaces if matches a pattern (perl -ne 'printf "%-5d %s", $., $_' : for all the lines)

perl -le 'print scalar(grep{/./}<>)'
#prints the total number of non-empty lines in a file

perl -lne '$a++ if /regex/; END {print $a+0}'
#print the total number of lines that matches the pattern

perl -alne 'print scalar @F'
#print the total number fields(words) in each line.

perl -alne '$t += @F; END { print $t}'
#Find total number of words in the file

perl -alne 'map { /regex/ && $t++ } @F; END { print $t }'
#find total number of fields that match the pattern

perl -lne '/regex/ && $t++; END { print $t }'
#Find total number of lines that match a pattern

perl -le '$n = 20; $m = 35; ($m,$n) = ($n,$m%$n) while $n; print $m'
#will calculate the GCD of two numbers.

perl -le '$a = $n = 20; $b = $m = 35; ($m,$n) = ($n,$m%$n) while $n; print $a*$b/$m'
#will calculate lcd of 20 and 35.

perl -le '$n=10; $min=5; $max=15; $, = " "; print map { int(rand($max-$min))+$min } 1..$n'
#Generates 10 random numbers between 5 and 15.

perl -le 'print map { ("a".."z",”0”..”9”)[rand 36] } 1..8'
#Generates a 8 character password from a to z and number 0 – 9.

perl -le 'print map { ("a",”t”,”g”,”c”)[rand 4] } 1..20'
#Generates a 20 nucleotide long random residue.

perl -le 'print "a"x50'
#generate a string of ‘x’ 50 character long

perl -le 'print join ", ", map { ord } split //, "hello world"'
#Will print the ascii value of the string hello world.

perl -le '@ascii = (99, 111, 100, 105, 110, 103); print pack("C*", @ascii)'
#converts ascii values into character strings.

perl -le '@odd = grep {$_ % 2 == 1} 1..100; print "@odd"'
#Generates an array of odd numbers.

perl -le '@even = grep {$_ % 2 == 0} 1..100; print "@even"'
#Generate an array of even numbers

perl -lpe 'y/A-Za-z/N-ZA-Mn-za-m/' file
#Convert the entire file into 13 characters offset(ROT13)

perl -nle 'print uc'
#Convert all text to uppercase:

perl -nle 'print lc'
#Convert text to lowercase:

perl -nle 'print ucfirst lc'
#Convert only first letter of first word to uppercas

perl -ple 'y/A-Za-z/a-zA-Z/'
#Convert upper case to lower case and vice versa

perl -ple 's/(\w+)/\u$1/g'
#Camel Casing

perl -pe 's|\n|\r\n|'
#Convert unix new lines into DOS new lines:

perl -pe 's|\r\n|\n|'
#Convert DOS newlines into unix new line

perl -pe 's|\n|\r|'
#Convert unix newlines into MAC newlines:

perl -pe '/regexp/ && s/foo/bar/'
#Substitute a foo with a bar in a line with a regexp.

Reference/Sources:

http://genomics-array.blogspot.in/2010/11/some-unixperl-oneliners-for.html

http://genomespot.blogspot.com/2013/08/a-selection-of-useful-bash-one-liners.html

http://biowize.wordpress.com/2012/06/15/command-line-magic-for-your-gene-annotations/

http://genomics-array.blogspot.com/2010/11/some-unixperl-oneliners-for.html

http://bioexpressblog.wordpress.com/2013/04/05/split-multi-fasta-sequence-file/

R 3.5.0 has been Released!

Jit — Thu, 26 Apr 2018 11:31:58 -0500

The latest version of R is a major release! It comes with a ton of new features, including performance and speed improvements
All R packages will now be byte-compiled, hence boosting packages installed from GitHub
You may need to re-install all previously installed R packages; old scripts however will continue to work normally

More at https://cran.r-project.org/doc/manuals/r-release/NEWS.html

Stephen Friend: The hunt for "unexpected genetic heroes"

Sat, 31 May 2014 14:31:47 -0500

What can we learn from people with the genetics to get sick — who don't? With most inherited diseases, only some family members will develop the disease, while others who carry the same genetic risks dodge it. Stephen Friend suggests we start studying those family members who stay healthy. Hear about the Resilience Project, a massive effort to collect genetic materials that may help decode inherited disorders. TEDTalks is a daily video podcast of the best talks and performances from the TED Conference, where the world's leading thinkers and doers give the talk of their lives in 18 minutes (or less). Look for talks on Technology, Entertainment and Design -- plus science, business, global issues, the arts and much more. Find closed captions and translated subtitles in many languages at http://www.ted.com/translate Follow TED news on Twitter: http://www.twitter.com/tednews Like TED on Facebook: https://www.facebook.com/TED Subscribe to our channel: http://www.youtube.com/user/TEDtalksDirector

Julia Programming Language, a Python and R rival

Radha Agarkar — Sat, 25 Aug 2018 04:46:39 -0500

Big data has grown to become one of the most lucrative fields. In fact, data scientists are some of the most sought people. They are usually hired to analyze, control and parse large chunks of data. Implementing these actions using traditional techniques is not a walk in the park. This is why most data scientists prefer using programming languages such as R and Python. However, there is one more programming language that can do the job. That is Julia programming language.

What Is Julia Language?

Julia is a programming language that came into the limelight in 2012. It is a general-purpose programming language that was designed for solving scientific computations. Julia was meant to be an alternative to Python, R and other programming languages that were mainly used for manipulating data. This is because it has numerous features that can minimize the complexities of numerical computations.

Julia optimizes on the best features of Python and R while at the same time overlooks their weaknesses. This explains why it is viewed as an alternative to these programming languages. For instance, it utilizes the readability and simplicity of Python then performs faster.

Julia is the most preferred programming language for data scientists and mathematicians. This is because its core features are similar to the ones that are used on most data software. Also, the language is ideal for these two subjects because its syntax is similar to the standard mathematical formulas.

Key Features Of Julia Language
Uses JIT Compilation
Parallelism
Dynamic Typing
Simple Syntax
Allows Metaprogramming
Accessible to Libraries
-1-Array Indexing

Julia Vs Python And R Programming Languages
1. Speed
Julia is faster than both Python and R. This is a very critical aspect that is given special attention in the big data programming. The high speed of Julia is because of JIT compilers. You will need to install external libraries on Python to achieve similar speed.

2. Syntax
Julia has a math-friendly syntax. The syntax of this programming language is similar to the mathematical formulas hence can be used to perform mathematical and scientific computations. This syntax makes it easier to learn than Python.

3. Parallelism
Although both Python and R use parallelism, Julia uses a top-level parallelism. Julia allows the processor to perform to the optimum level than what Python and R can achieve.

4. Versatility
Julia programming language is more versatile than Python and R. It allows a programmer to move from different codes and functions with ease.

The only area that Python and R are superior to Julia is in terms of community. Given that Julia is a new programming language, it has a small community as compared to others which have been around for years.

In overall Julia programming language is a better alternative that you can use to handle Big data projects. Despite having a small community, it is one of those programming languages that you can easily learn.