BOL: Related items

BioDownloader

Surabhi Chaudhary — Sat, 25 Feb 2017 17:52:33 -0600

BioDownloader is a program for downloading and/or updating files from ftp/http servers. The program has unique features that are specifically designed to deal with bioinformatics data files and servers:

optimized to work with vast amount of data and very large file sets (~ 10,000 - 100,000).
allows the selective retrieval of only the required files (file masks, ls-lR parsing, recursive search, updates)
has a built-in repository containing the settings for the most common bioinformatics download needs
built-in wizard for batch post-processing of downloaded files (archive extraction, file conversion, etc.)
capable of performing multiple download or update tasks simultaneously

BioDownloader has a built-in repository containing the settings for common bioinformatics file-synchronization needs, including the Protein Data Bank (PDB) and National Center for Biotechnology Information (NCBI) databases. It can post-process downloaded files, including archive extraction and file conversions.

http://dunbrack.fccc.edu/BioDownloader/

Address of the bookmark: http://dunbrack.fccc.edu/BioDownloader/

NCBI Magic-BLAST

Jit — Tue, 14 Aug 2018 18:11:11 -0500

Magic-BLAST is a tool for mapping large next-generation RNA or DNA sequencing runs against a whole genome or transcriptome. Each alignment optimizes a composite score, taking into account simultaneously the two reads of a pair, and in case of RNA-seq, locating the candidate introns and adding up the score of all exons. This is very different from other versions of BLAST, where each exon is scored as a separate hit and read-pairing is ignored.

Magic-BLAST incorporates within the NCBI BLAST code framework ideas developed in the NCBI Magic pipeline, in particular hit extensions by local walk and jump (http://www.ncbi.nlm.nih.gov/pubmed/26109056), and recursive clipping of mismatches near the edges of the reads, which avoids accumulating artefactual mismatches near splice sites and is needed to distinguish short indels from substitutions near the edges.

Address of the bookmark: https://ncbi.github.io/magicblast/

Magic-BLAST

Shruti Paniwala — Fri, 20 Mar 2020 15:18:36 -0500

Address of the bookmark: https://ncbi.github.io/magicblast/

NCBIBLAST+ 2.14.1 now available

Neel — Wed, 30 Aug 2023 02:36:13 -0500

#NCBIBLAST+ 2.14.1 now available with improved documentation, faster and more reliable database downloads, and some bug fixes.

Check out the changes they made.

They added the cleanup-blastdb-volumes.py script to remove unused BLAST database volumes. Read the documentation here.

They also switched the protocol from ftp to https to access BLAST databases for increased performance and reliability when downloading data from the NCBI with the update_blastdb.pl script.

And fixed a few bugs related to downloading data from the NCBI, and mt_mode crashing blastn and blastx.

Check out the release notes.

Download BLAST+ 2.14.1

Questions or comments? Please write the BLAST help desk.

More info and download: https://blast.ncbi.nlm.nih.gov/doc/blast-news/2023-BLAST-News.html

How to install Perl modules manually, using CPAN command, and other quick ways

Jit — Fri, 12 Jul 2013 07:20:24 -0500

As a bioinformatics programmer, and crunchy data analyser you need to install several perl modules and dependencies. Installing Perl modules manually by resolving all the dependencies is tedious and annoying process. Some of the packages like GD is the real pain.

However, Installing Perl modules using CPAN is a better solution, as it resolves all the dependencies automatically. In this article, let us review how to install Perl modules on Linux ( which is prefereced amonst bioinformatician) using both manual and CPAN method.

When a Perl module is not installed, application will display the following error message. In this example, XML::Parser Perl module is missing.

Can't locate XML/parser.pm in @INC (@INC contains:
/usr/lib/perl5/5.10.0/i386-linux-thread-multi
/usr/lib/perl5/5.10.0
/usr/local/lib/perl5/site_perl/5.10.0/i386-linux-thread-multi
/usr/local/lib/perl5/site_perl/5.10.0
/usr/lib/perl5/vendor_perl/5.10.0/i386-linux-thread-multi
/usr/lib/perl5/vendor_perl/5.10.0 /usr/lib/perl5/vendor_perl
/usr/lib/perl5/site_perl/5.10.0 .)

Manual Method of Perl Module Installation

Install Perl Modules Manually

This manual method is very useful when your computer or server is not connected to the Internet.

Download Perl module:
Go to CPAN Search website and search for the module that you wish to download. In this example, let us search, download and install XML::Parser Perl module. I have downloaded the XML-Parser-2.36.tar.gz to /home/download

# cd /home/download
# gzip -d XML-Parser-2.36.tar.gz
# tar xvf XML-Parser-2.36.tar
# cd XML-Parser-2.36

Build the perl module:
Build by running Makefile.PL, remember the case sensitivity, make and make test.

# perl Makefile.PL
Checking if your kit is complete...
Looks good
Writing Makefile for XML::Parser::Expat
Writing Makefile for XML::Parser
# make
# make test

Install the perl module:
Now your package is ready to install.

# make install

As a newbie it looks pretty simple, and one go. But, luckily this is a very simple one module with no dependencies. Typically, Perl modules will be dependent on several other modules. Just imagine chasing all these dependencies one-by-one, thinking ... oh ye I got it. That will be very painful and annoying task. I recommend the CPAN method of installation as shown below.

Install Perl Modules using CPAN automatically

Logically, you should must have the CPAN perl module installed in your server or computer before you can install any other Perl modules using CPAN. I know you are laughing, "to install a perl module you need another perl module" ;)

Lets verify whether CPAN is already installed:

To install Perl modules using CPAN, make sure the cpan command is working. Following are the error message when CPAN module is not installed.

# cpan
-bash: cpan: command not found

# perl -MCPAN -e shell
Can't locate CPAN.pm in @INC (@INC contains:
/usr/lib/perl5/5.10.0/i386-linux-thread-multi
/usr/lib/perl5/5.10.0
/usr/local/lib/perl5/site_perl/5.10.0/i386-linux-thread-multi
/usr/local/lib/perl5/site_perl/5.10.0
/usr/lib/perl5/vendor_perl/5.10.0/i386-linux-thread-multi
/usr/lib/perl5/vendor_perl/5.10.0
/usr/lib/perl5/vendor_perl /usr/lib/perl5/site_perl/5.10.0 .).
BEGIN failed--compilation aborted.

Install the CPAN module using yum:
If CPAN in not installed in your system, you can use "yum" for the rescue. Dont worry biological data cruncher, this is true we are now dependent all these tiny magicians :).

# yum install perl-CPAN

Output of yum install perl-CPAN command:

Loaded plugins: refresh-packagekit
updates-newkey                       | 2.3 kB     00:00
primary.sqlite.bz2                   | 2.4 MB     00:00
Setting up Install Process
Parsing package install arguments

Resolving Dependencies
Transaction Summary
=============================================================================
Install      5 Package(s)
Update       0 Package(s)
Remove       0 Package(s)

Total download size: 1.0 M
Is this ok [y/N]: y
Downloading Packages:
(1/5): perl-ExtUtils-ParseXS-2.18-31.fc9.i386.rpm     | 30 kB     00:00
(2/5): perl-Test-Harness-2.64-31.fc9.i386.rpm         | 70 kB     00:00
(3/5): perl-CPAN-1.9205-31.fc9.i386.rpm               | 217 kB     00:00
(4/5): perl-ExtUtils-MakeMaker-6.36-31.fc9.i386.rpm   | 284 kB     00:00
(5/5): perl-devel-5.10.0-31.fc9.i386.rpm              | 408 kB     00:00

Installing     : perl-ExtUtils-ParseXS                             [1/5]
Installing     : perl-devel                                        [2/5]
Installing     : perl-Test-Harness                                 [3/5]
Installing     : perl-ExtUtils-MakeMaker                           [4/5]
Installing     : perl-CPAN                                         [5/5]

Installed: perl-CPAN.i386 0:1.9205-31.fc9
Dependency Installed:
perl-ExtUtils-MakeMaker.i386 0:6.36-31.fc9
perl-ExtUtils-ParseXS.i386 1:2.18-31.fc9
perl-Test-Harness.i386 0:2.64-31.fc9
perl-devel.i386 4:5.10.0-31.fc9
Complete!

Configure cpan the first time:
Once the CPAN is installed, you need to configure it by executing cpan, you should set some configuration parameters as shown below. I have shown only the important configuration parameters below. Accept all the default values by pressing enter.

Note: Make sure to execute “o conf commit” in the cpan prompt after the configuration to save the settings.

# cpan

Sorry, we have to rerun the configuration dialog for CPAN.pm due
to some missing parameters...

CPAN build and cache directory? [/root/.cpan]
Download target directory? [/root/.cpan/sources]
Directory where the build process takes place? [/root/.cpan/build]

Always commit changes to config variables to disk? [no]
Cache size for build directory (in MB)? [100]
Let the index expire after how many days? [1]

Perform cache scanning (atstart or never)? [atstart]
Cache metadata (yes/no)? [yes]
Policy on building prerequisites (follow, ask or ignore)? [ask]

Parameters for the 'perl Makefile.PL' command? []
Parameters for the 'perl Build.PL' command? []

Your ftp_proxy? []
Your http_proxy? []
Your no_proxy? []
Is it OK to try to connect to the Internet? [yes]

First, pick a nearby continent and country by typing in the number(s)
(1) Africa
(2) Asia
(3) Central America
(4) Europe
(5) North America
(6) Oceania
(7) South America
Select your continent (or several nearby continents) [] 5

(1) Bahamas
(2) Canada
(3) Mexico
(4) United States
Select your country (or several nearby countries) [] 4

(2) ftp://carroll.cac.psu.edu/pub/CPAN/
(3) ftp://cpan-du.viaverio.com/pub/CPAN/
(4) ftp://cpan-sj.viaverio.com/pub/CPAN/
(5) ftp://cpan.calvin.edu/pub/CPAN
(6) ftp://cpan.cs.utah.edu/pub/CPAN/
e.g. '1 4 5' or '7 1-4 8' [] 2-16

cpan[1]> o conf commit
commit: wrote '/usr/lib/perl5/5.10.0/CPAN/Config.pm'

cpan[2]> quit
No history written (no histfile specified).
Lockfile removed.

Install Perl Modules using CPAN

Hey smile please, now you are ready with CPAN and can download modules in one line command.

You can use one of the following method to install a Perl module using cpan:

# perl -MCPAN -e 'install Bundle::BioPerl'

(or)

# cpan
cpan shell -- CPAN exploration and modules installation (v1.9205)
ReadLine support available (maybe install Bundle::CPAN or Bundle::CPANxxl?)

cpan[1]> install "Bundle::BioPerl"

In the example above, CPAN will check for Bundle::BioPerl dependencies and automatically resolves and installs Bundle::BioPerl with all the dependent Perl modules.

Quick Ways

Oh, look at your face.. smily hmm :). This is what your are looking for, a quick and best way to install Perl modules, Bioperl. Following are the the steps to download BioPerl in your server/computer.

# sudo apt-cache search perl BioPerl

Output will be like as follows:

bioperl - Perl tools for computational molecular biology
bioperl-run - BioPerl wrappers: scripts
libbio-perl-perl - BioPerl core perl modules
libbio-perl-run-perl - BioPerl wrappers: modules
libbio-samtools-perl - Perl interface to SamTools library for DNA sequencing
libbiojava-java - Java API to biological data and applications (default version)
libbiojava3-java - Java API to biological data and applications (default version)
python-biopython-sql - Biopython support for the BioSQL database schema
libbtlib-perl - library for basic sequence manipulation

# sudo apt-get install bioperl

If it is installed then flash the following message:

Reading package lists... Done
Building dependency tree
Reading state information... Done
bioperl is already the newest version.
0 upgraded, 0 newly installed, 0 to remove and 10 not upgraded.

In it is found not installed in your server or system them install all with dependencies.

You can use the same approach to install all the modules, and packages if required.

Thanks for reading. Best of luck for your research.

Perl Maven tutorial by Gabor Szabo

Jitendra Narayan — Fri, 09 Aug 2013 05:43:10 -0500

I personally reccoment this tutorial for Perl users.

More detail http://perlmaven.com/

Cava Deployment Tools

Jitendra Narayan — Wed, 14 Aug 2013 15:00:48 -0500

Cava Packager allows packaging of applications written in Perl. Packaged applications can be distributed without the need for an installation of Perl on the target machine. Cava Packager is currently distributed under a free to use license.

Supported Perl Versions

Threaded Perl with a shared libperl 5.8, 5.10, 5.12, 5.14, 5.16

Supported Operating Systems

    MS Windows XP and later 32 and 64 bit.

    Linux 32 and 64 bit - minimum glibc 2.5, GTK+ 2.10, libstdc++6.

    Linux 32 bit RHEL 3 and 4.

    Mac OSX 10.4 and 10.5 - ppc and i386.

    Mac OSX 10.6, 10.7, 10.8 - i386 and x86_64 ( 64 bit for Perl 5.16 only ).

Address of the bookmark: http://www.cavapackager.com/

Detect the sequence pattern and its location in fasta file with match and mismatches information.

Jit — Thu, 26 Sep 2013 15:02:53 -0500

This script is one of my old script to detect some centromeric pattern in chromosomes. User can also control the number of mismatches allowed through command line ..

To run:

perl centro.pl

Perl One liner basics !!

Abhimanyu Singh — Sun, 24 May 2015 09:28:33 -0500

Perl has a ton of command line switches (see perldoc perlrun), but I'm just going to cover the ones you'll commonly need to debug code. The most important switch is -e, for execute (or maybe "engage" :) ). The -e switch takes a quoted string of Perl code and executes it. For example:

$ perl -e 'print "Hello, World!\n"'
Hello, World!

It's important that you use single-quotes to quote the code for -e. This usually means you can't use single-quotes within the one liner code. If you're using Windows cmd.exe or PowerShell, you must use double-quotes instead.

I'm always forgetting what Perl's predefined special variables do, and often test them at the command line with a one liner to see what they contain. For instance do you remember what $^O is?

$ perl -e 'print "$^O\n"'
linux

It's the operating system name. With that cleared up, let's see what else we can do. If you're using a relatively new Perl (5.10.0 or higher) you can use the -E switch instead of -e. This turns on some of Perl's newer features, like say, which prints a string and appends a newline to it. This saves typing and makes the code cleaner:

$ perl -E 'say "$^O"'
linux

Pretty handy! say is a nifty feature that you'll use again and again.

Frequent words problem solution by Perl

Jit — Tue, 09 Jun 2015 23:38:44 -0500

Solved with perl http://rosalind.info/problems/1a/

#Find the most frequent k-mers in a string.
#Given: A DNA string Text and an integer k.
#Return: All most frequent k-mers in Text (in any order).

use strict;
use warnings;

my $string="ACGTTGCATGTCGCATGATGCATGAGAGCT";
my $kmer=4;
my %myHash;
my $max=0;

for (my $aa=0; $aa<=(length($string)-4); $aa++) {
   my $myStr=substr $string, $aa,$kmer;
   #print "$myStr\n";
   my $km=kmerMatch ($string, $myStr, $kmer);
   if ($km > $max) { $max = $km;}
   #print "$km\t$myStr\n";
   $myHash{$myStr}=$km;

}

#Print all key which have matching values
foreach my $name (keys %myHash){
    print "$name " if $myHash{$name} == $max;
}

sub kmerMatch { #Check the exact matching kmers with sliding window
my ($string, $myStr, $kmer)=@_;
my $count=0;
for (my $aa=0; $aa<=(length($string)-4); $aa++) {
   my $myWin=substr $string, $aa,$kmer;
   if ($myWin eq $myStr) {
       #print "$myWin eq $myStr\n";
       $count++;
   }
}
return $count;
}