Address of the bookmark: https://astrobiomike.github.io/

Reference@http://www.datasciencecentral.com/profiles/blogs/one-page-r-a-survival-guide-to-data-science-with-r

• Templates for the Data Scientist
  1. A Template for Preparing Data: *OnePageR - *R
  2. A Template for Building Models: *OnePageR - *R
• Getting Started as a Data Scientist
  1. Getting Started with R and Rattle: *Lecture - *Laboratory
  2. Introducing and Interacting with R: *Lecture - *Laboratory
  3. BasicR - OnePage(R) - Writing R scripts
• Dealing With Data
  1. Read Data into R: *OnePageR - *R
  2. Explore and Summarise Data: *OnePageR - *R
  3. Transform Data: *OnePageR - *R
  4. Dealing with Dates and Time: (PDF, R) Dates and Time
  5. Visualising Data with GGPlot2: *OnePageR - *R
  6. Visualising Data with Maps *OnePageR - *R
  7. Spatial (R) Spatial Analysis
  8. Handling Big Data *OnePageR - *R
• Descriptive Analytics
  1. Cluster Analysis: *Lecture - *OnePageR - *R
  2. Association Analysis: *Lecture
• Predictive Analytics
  1. Decision Trees: *Lecture - *OnePageR - *R - *Rattle
  2. Ensembles of Decision Trees: *Lecture - *OnePageR - *R
  3. SVM (R)
  4. KernLab (R)
  5. NeuralNetworks (R)
  6. NNet (R)
• Model Delivery
  1. Evaluating Models: *OnePageR - *R
  2. Evaluation (R)
  3. Scoring (R)
  4. PMML (R) Exporting Models for Deployment
• Advanced Topics
  1. Text Mining: *OnePageR - *R
• Advanced R Topics
  1. Plots (PDF, R) Miscellaneous Plots
  2. Functions (PDF, R) Writing Functions in R
  3. Parallel (PDF, R) Parallel Execution
  4. Packaging (R) Pulling it Together into a Package
  5. Doing R with Style: *OnePageR - *R
  6. Literate Data Mining with KnitR: *Lecture - *OnePageR - *

Address of the bookmark: http://cbb.sjtu.edu.cn/course/database/beginning.pdf

More at https://bioinformaticsonline.com/snippets/view/42919/perl-ipcopen2-module

2. exec””

 syntax: exec "command";

3. Backticks “ or qx//

syntax: `command`;

syntax: qx/command/;

4. IPC::Open3

syntax: $output = open3(\*CHLD_IN, \*CHLD_OUT, \*CHLD_ERR, 'command arg1 arg2', 'optarg',...);

It is very similar to IPC::Open2 with the capability to capture all three file handles of the process, i.e., STDIN, STDOUT, and STDERR. It can also be used with or without the shell. You can read about it more in the documentation: IPC::Open3.

$output = open3(my $o ut, my $in, 'command arg1 arg2');

OR without using the shell

$output = open3(my $out, my $in, 'command', 'arg1', 'arg2');

5.a2p

syntax: a2p [options] [awkscript]

There is a Perl utility known as a2p which translates awk command to Perl. It takes awk script as input and generates a comparable Perl script as the output. Suppose, you want to execute the following awk statement

awk '{$1 = ""; $2 = ""; print}' f1.txt

This statement gives error sometimes even after escaping the variables (\$1, \$2) but by using a2p it can be easily converted to Perl script:

For further information, you can read it’s documentation: a2p

More help at https://bioinformaticsonline.com/snippets/view/42920/perl-script-to-run-awk-inside-perl

6. system()

syntax: system( "command" );

It is also a Perl function (perlfunc) that executes a command and waits for it to get finished first and then resume the Perl script. The return value is the exit status of the command. It can be called in two ways:

system( "command arg1 arg2" );

OR

system( "command", "arg1", "arg2" );

HELP

Here are some useful Perl cheat sheets that can be used as a quick reference guide.-- https://www.pcwdld.com/perl-cheat-sheet

• -e Makes the line of code be executed instead of a script
• -n Forces your line to be called in a loop. Allows you to take lines from the diamond operator (or stdin)
• -p Forces your line to be called in a loop. Prints $_ at the end

• This counts the number of quotation marks in each line and prints it

  perl -ne '$cnt = tr/"//;print "$cnt\n"' inputFileName.txt

• Adds string to each line, followed by tab

  perl -pe 's/(.*)/string\t$1/' inFile > outFile

• Append a new line to each line

  perl -pe 's//\n/' all.sent.classOnly > all.sent.classOnly.sep

• Replace all occurrences of pattern1 (e.g. [0-9]) with pattern2

  perl -p -i.bak -w -e 's/pattern1/pattern2/g' inputFile

• Go through file and only print words that do not have any uppercase letters.

  perl -ne 'print unless m/[A-Z]/' allWords.txt > allWordsOnlyLowercase.txt

• Go through file, split line at each space and print words one per line.

  perl -ne 'print join("\n", split(/ /,$_));print("\n")' someText.txt > wordsPerLine.txt

• or in other words, delete every character that is not a letter, white space or line end (replace with nothing)

  perl -pne 's/[^a-zA-Z\s]*//g' text_withSpecial.txt > text_lettersOnly.txt

• perl -pne 'tr/[A-Z]/[a-z]/' textWithUpperCase.txt > textwithoutuppercase.txt;

• Print only the second column of the data when using tabular as a separator

  perl -ne '@F = split("\t", $_); print "$F[1]";' columnFileWithTabs.txt > justSecondColumn.txt

• One-Liner: Sort lines by their length

  perl -e 'print sort {length $a <=> length $b} <>' textFile

• One-Liner: Print second column, unless it contains a number

  perl">perl -lane 'print $F[1] unless $F[1] =~ m/[0-9]/' wordCounts.txt

Originally design to map the location of objects in a 10 km map IrishGrid includes:

• native support of the Irish Grid System (see http://www.osi.ie/)
• optimize for speed (there's as less as possible data to conversion)
• customized color functions

https://code.google.com/p/irishgrid/downloads/detail?name=irishgrid.pl

Address of the bookmark: https://code.google.com/p/irishgrid/

It is possible to create a vector by typing data directly into R using the combine function ‘c’

x

same as

x

creates the vector x with the numbers between 1 and 5.

You can see what is in an object at any time by typing its name;

x

will produce the output ‘[1] 1 2 3 4 5′

Note that names need to be quoted

daysofweek ← c(‘Monday’, ‘Tuesday’, ‘Wednesday’, ‘Thursday’, ‘Friday’);

Usually however you want to input from a file. We have touched on the ‘read.table’ function already.

mydata

Now mydata is a data frame with multiple vectors

each vector can be identified by the default syntax

#if any of these are typed it will print to screen

mydata$V1 mydata$V2 mydata$V3

By default the function assumes certain things from the file

• The file is a plain text file (there are function to read excel files: not covered here)
• columns are separated by any number of tabs or spaces
• there is the same number of data points in each column
• there is no header row (labels for the columns)
• there is no column with names for the rows** [I’ll explain].

If any of these are false, we need to tell that to the function

If it has a header column

mydata header=T also works

Note that there is a comma between different parts of the functions arguments

If there is one less column in the header row, then R assumes that the 1^st column of data after the header are the row names

Now the vectors (columns) are identified by their name

#if any of these are typed it will print to screen

mydata$A mydata$B mydata$C

# Summary about the whole data frame

summary(mydata)

# Summary information of column A

summary(mydata$A)

We can shortcut having to type the data frame each time by attaching it

attach(mydata)

# summary of column B as ‘mydata’ is attached

summary(B)

Two other important options for read.table

If is is separated only by tabs and has a header

mydata

Really useful if you have spaces in the contents of some columns, so R does not mess up reading the columns . However if the columns or of an uneven length it will tell you.

If you know that the file has uneven columns

mydata

This causes R to fill empty spaces in a columns with ‘NA’ .

The last two examples will still work with our file and give the same result as with only headers=T

Graphs

to get an idea of what R is capable of type

demo(graphics)

steps through the examples, and the code is printed to the screen

We will work with simpler examples that have immediate use to biologists.

Remember to get more information about the options to a function type ‘?function’

Histogram of A

hist(mydata$A)

If there was more data we could increase the number of vertical columns with the option, breaks=50 (or another relevant number).

boxplot(mydata)

We can get rid of the need to type the data frame each time by using the attach function

# if not already done so

attach(mydata)

boxplot(mydata$A, mydata$B, name=c(“Value A”, “Value B”) , ylab=“Count of Something”)

same as

boxplot(A, B, name=c(“Value A”, “Value B”) , ylab=“Count of Something”)

Scatter plot

# if not already done so

attach(mydata)

plot(A,B) # or plot(mydata$A, mydata$B)

SAVING an image

Windows users (Rgui) RIGHT click on image and select which you want.

These instructions work for everyone.

You need to create a new device of the type of file you need, then send the data to that device

to save as a png file (easy to load into the likes of powerpoint, also great for web applications.

png(‘filename’)

boxplot(A, B, name=c(“Value A”, “Value B”) , ylab=“Count of Something”)

or to save as a pdf

pdf(‘filename’)

boxplot(A, B, name=c(“Value A”, “Value B”) , ylab=“Count of Something”)

Note

• Nothing will appear on screen, the output is going to the file
• Also it may not be saved immediately but will once the device (or R) is turned quit.

To quit R type

q() # If you save your session, next time you start R, you will have your data preloaded.

Or if you want to remain in R

dev.off() #turns of the png (or pdf etc) device, thus forces the data to save

The OpenCPU JavaScript client library provides the most seamless integration of R and JavaScript available today.

OpenCPU uses standard R packaging to develop, ship and deploy web applications. Several open source example apps are available from Github.

Installing your own OpenCPU server is super easy and only takes a few minutes.

More at https://www.opencpu.org/

You see, for the past couple of years, I’ve been a bit frustrated because OS X does not come with a whole lot of Perl modules pre-installed, and for all I googled, I couldn’t find an “idiot’s” guide for moderately-savvy-but-not-expert users like myself to install modules and dependencies on demand.

The only instructions I could find point to Fink, which basically installs modules in a path that isn’t included in the Perl @INC variable, meaning you have to manually specify the full path to the modules in every script — which is not a lot of fun if you’re developing on OS X and deploying on Red Hat, for instance.

Moreover, Fink doesn’t seem to make every module available, and it’s not very easy to determine which Fink package you need to install if you need a particular module.

So, with a script that called on several apparently unavailable modules, and a deadline looming, I finally decided to suck it up and figure out how to use CPAN to install them:

1) Make sure you have the Apple Developer Tools (XCode) installed.

These are on one of your install discs, or available as a huge but free download from the Apple Developer Connection [free registration required] or the Mac App Store. I thought I had them, but apparently when we upgraded that computer to Tiger, they went missing.

If you don’t have this stuff installed, your installation will fail with errors about unavailable commands.

1.5) Install Command Line Tools (Recent XCode versions only)

(Thank you to Tom Marchioro for informing me about this step.)

Older versions of XCode installed the command line tools (which are required to properly install CPAN modules) by default, but apparently newer ones do not. To check whether you have the command line tools already installed, run the following from the Terminal:

$ which make

This command checks the system for the “make” tool. If it spits out something like /usr/bin/make you’re golden and can skip ahead to Step 2. If you just get a new prompt and no output, you’ll need to install the tools:

1. Launch XCode and bring up the Preferences panel.
2. Click on the Downloads tab
3. Click to install the Command Line Tools

If you like, you can run which make again to confirm that everything’s installed correctly.

2) Configure CPAN.

$ sudo perl -MCPAN -e shell

perl> o conf init

This will prompt you for some settings. You can accept the defaults for almost everything (just hit “return”). The two things you must fill in are the path to make (which should be /usr/bin/make or the value returned when you run which make from the command line) and your choice of CPAN mirrors (which you actually choose don’t really matter, but it won’t let you finish until you select at least one). If you use a proxy or a very restrictive firewall, you may have to configure those settings as well.

If you skip Step 2, you may get errors about make being unavailable.

3) Upgrade CPAN

$ sudo perl -MCPAN -e 'install Bundle::CPAN'

Don’t forget the sudo, or it’ll fail with permissions errors, probably when doing something relatively unimportant like installing man files.

This will spend a long time downloading, testing, and compiling various files and dependencies. Bear with it. It will prompt you a few times about dependencies. You probably want to enter “yes”. I agreed to everything it asked me, and everything turned out fine. YMMV of course. If everything installs properly, it’ll give you an “OK” at the end.

4) Install your modules. For each module….

$ sudo perl -MCPAN -e 'install Bundle::Name'

or

$ sudo perl -MCPAN -e 'install Module::Name'

This will install the module and its dependencies. Nice, eh? Again, don’t forget the sudo.

The first time you run this after upgrading CPAN, it may prompt you to configure again (see Step 2). If you accept its offer to try to configure itself automatically, it may just run through everything without a problem.

There are a couple of potential pitfalls with specific modules (such as theLWP::UserAgent / HEAD issue), but most have workarounds, and I haven’t run into anything that wasn’t easily recoverable.

And that’s it!

Did you find this useful? Is there anything I missed?