BOL: All

R script to visualize fastANI core-genome comparison

Shruti Paniwala — Fri, 30 Apr 2021 02:29:19 -0500

#######
# Purpose: Visualize fastANI core-genome comparison
# Usage: Rscript     
# Output: .pdf
# Uses genoPlotR package: http://genoplotr.r-forge.r-project.org

#Parse command line arguments
query_fasta=commandArgs(TRUE)[1]
subject_fasta=commandArgs(TRUE)[2]
fastANI_visual_file=commandArgs(TRUE)[3]

library(genoPlotR)

#Read fastANI output
comparison <- try(read_comparison_from_blast(fastANI_visual_file))

#Read sequences into genoPlotR objects
Query <- try(read_dna_seg_from_file(query_fasta))
Ref <- try(read_dna_seg_from_file(subject_fasta))

plotTitle = paste(query_fasta, subject_fasta, sep=" v/s ")

pdf( paste(fastANI_visual_file,".pdf",sep="") )

plot_gene_map(dna_segs=list(Query, Ref), comparisons=list(comparison), main=plotTitle, scale=FALSE, scale_cex=1, n_scale_ticks=4)

dev.off()

Download genome using NCBI esearch command line !

Neel — Fri, 30 Apr 2021 00:31:09 -0500

#Download using command line "NC_005956.1" /  "CP003784.1"

esearch -db nucleotide -query "NC_005956.1" | efetch -format fasta > seq1.fa
esearch -db nucleotide -query "CP003784.1" | efetch -format fasta > seq2.fa

#fastANI for alignment free comparision
#conda install -c bioconda fastani

fastANI -q seq1.fa -r seq2.fa --visualize -o fastani.out

#Plot using sequences
Rscript plotSeq.R seq1.fa seq2.fa fastani.out.visual

Install NCBI entrez-direct and esearch using conda !

Neel — Fri, 30 Apr 2021 00:25:35 -0500

$ conda install -c bioconda entrez-direct
Collecting package metadata (current_repodata.json): done
Solving environment: done


==> WARNING: A newer version of conda exists. <==
  current version: 4.9.0
  latest version: 4.10.1

Please update conda by running

    $ conda update -n base -c defaults conda



## Package Plan ##

  environment location: /home/urbe/anaconda3/envs/MetaENV

  added / updated specs:
    - entrez-direct


The following packages will be downloaded:

    package                    |            build
    ---------------------------|-----------------
    entrez-direct-13.9         | pl5262he881be0_2         5.2 MB  bioconda
    ------------------------------------------------------------
                                           Total:         5.2 MB

The following packages will be UPDATED:

  entrez-direct                        13.9-pl526h375a9b1_0 --> 13.9-pl5262he881be0_2


Proceed ([y]/n)? y


Downloading and Extracting Packages
entrez-direct-13.9   | 5.2 MB    | ############################################################################# | 100%
Preparing transaction: done
Verifying transaction: done
Executing transaction: done

Python script to download covid genome !

Surabhi Chaudhary — Fri, 26 Mar 2021 07:01:29 -0500

#!/usr/bin/env python3

# these are the publicly available "complete" sequences
# https://www.gisaid.org/ has more (1200?), but they require you to sign up

import requests
import yaml

seqs = yaml.load(requests.get("https://www.ncbi.nlm.nih.gov/core/assets/genbank/files/ncov-sequences.yaml").text)
seqs = seqs['genbank-sequences']
print("got %d sequences" % len(seqs))

from Bio import Entrez
allseq = {}
for x in seqs:
  if 'gene-region' in x and x['gene-region'] == "complete":
    nm = x['accession']
    print("downloading", nm)
    dna = Entrez.efetch(db='nucleotide',id=nm, rettype = 'fasta', retmode= 'text').read().split("\n")[1:]
    allseq[nm] = ''.join(dna)

import json
with open("data/allseq.json", "w") as f:
  json.dump(allseq, f)

Install BLAST in Ubuntu/Linux and Window !

Poonam Mahapatra — Mon, 15 Mar 2021 12:35:50 -0500

#On ubuntu
sudo apt-get install ncbi-blast+

#Ubuntu Conda installation 
conda install -c bioconda blast

#Windows installation
First:
Download ftp://ftp.ncbi.nlm.nih.gov/blast/executables/blast+/2.7.1/ncbi-blast-2.7.1+-win64.exe with your web browser. 

#If that FTP links does not work, try HTTP instead http://ftp.ncbi.nlm.nih.gov/blast/executables/blast+/2.7.1/ncbi-blast-2.7.1+-win64.exe

#Run this 
installer using the defaults, it should put BLAST under C:\Program Files\NCBI\blast-2.7.1+

Install ragout with conda !

Jit — Sun, 14 Mar 2021 10:09:32 -0500

(JitMetaENV) ➜  setu git:(master) ✗ conda install -c bioconda ragout
Collecting package metadata (current_repodata.json): done
Solving environment: failed with initial frozen solve. Retrying with flexible solve.
Solving environment: failed with repodata from current_repodata.json, will retry with next repodata source.
Collecting package metadata (repodata.json): done
Solving environment: done


==> WARNING: A newer version of conda exists. <==
  current version: 4.9.0
  latest version: 4.9.2

Please update conda by running

    $ conda update -n base -c defaults conda



## Package Plan ##

  environment location: /home/urbe/anaconda3/envs/JitMetaENV

  added / updated specs:
    - ragout


The following packages will be downloaded:

    package                    |            build
    ---------------------------|-----------------
    decorator-4.3.0            |             py_0          10 KB  conda-forge
    networkx-2.2               |             py_1         1.1 MB  conda-forge
    ragout-2.3                 |   py36hf1ae8f4_2         2.8 MB  bioconda
    sibelia-3.0.7              |       he1b5a44_2        24.8 MB  bioconda
    ------------------------------------------------------------
                                           Total:        28.6 MB

The following NEW packages will be INSTALLED:

  decorator          conda-forge/noarch::decorator-4.3.0-py_0
  networkx           conda-forge/noarch::networkx-2.2-py_1
  ragout             bioconda/linux-64::ragout-2.3-py36hf1ae8f4_2
  sibelia            bioconda/linux-64::sibelia-3.0.7-he1b5a44_2

The following packages will be UPDATED:

  ca-certificates    conda-forge::ca-certificates-2020.12.~ --> pkgs/main::ca-certificates-2021.1.19-h06a4308_1
  openssl                                 1.1.1i-h7f98852_0 --> 1.1.1j-h7f98852_0


Proceed ([y]/n)? y


Downloading and Extracting Packages
sibelia-3.0.7        | 24.8 MB   | ################################################################################# | 100%
networkx-2.2         | 1.1 MB    | ################################################################################# | 100%
ragout-2.3           | 2.8 MB    | ################################################################################# | 100%
decorator-4.3.0      | 10 KB     | ################################################################################# | 100%
Preparing transaction: done
Verifying transaction: done
Executing transaction: done

Install htop in elementary/Ubuntu OS !

Neel — Wed, 10 Mar 2021 08:18:27 -0600

neelam@neelam-Lenovo-ideapad-320-15ISK:~$ sudo apt-get install htop
Reading package lists... Done
Building dependency tree       
Reading state information... Done
The following packages were automatically installed and are no longer required:
  gir1.2-vte-2.91 libido3-0.1-0 libllvm6.0 libwayland-egl1-mesa
Use 'sudo apt autoremove' to remove them.
The following NEW packages will be installed:
  htop
0 upgraded, 1 newly installed, 0 to remove and 0 not upgraded.
Need to get 80.0 kB of archives.
After this operation, 221 kB of additional disk space will be used.
Get:1 http://in.archive.ubuntu.com/ubuntu bionic/main amd64 htop amd64 2.1.0-3 [80.0 kB]
Fetched 80.0 kB in 0s (239 kB/s)
Selecting previously unselected package htop.
(Reading database ... 176684 files and directories currently installed.)
Preparing to unpack .../htop_2.1.0-3_amd64.deb ...
Unpacking htop (2.1.0-3) ...
Setting up htop (2.1.0-3) ...
Processing triggers for desktop-file-utils (0.23-1ubuntu3.18.04.2+elementary2~ubuntu5.0.1) ...
Processing triggers for bamfdaemon (0.5.3+18.04.20180207.2-0ubuntu1) ...
Rebuilding /usr/share/applications/bamf-2.index...
Processing triggers for man-db (2.8.3-2ubuntu0.1) ...
Processing triggers for gnome-menus (3.13.3-11ubuntu1.1) ...
Processing triggers for mime-support (3.60ubuntu1) ...

Perl script to check modules installed !

Jit — Sat, 06 Mar 2021 21:41:41 -0600

#use strict;
use warnings;
use ExtUtils::Installed;

#
# First, check if all the required modules have been installed in the system 
#
BEGIN {
    my @import_modules = (
    'Cwd',
    'File::chdir',
    'File::Copy',
    'POSIX',
    'Tie::File',
    'Try::Tiny',
    'Data::Dumper',
    'File::Basename',
    'Bio::SeqIO',
    'FindBin',
    'File::Remove',
    'Capture::Tiny',
    'File::Temp',
    'File::Spec::Functions',
    'Statistics::Multtest',
    'File::Path',
    'Statistics::Distributions',
    'Getopt::Long',
    'Statistics::R',
    'Math::Round',
    'File::Find',
    'Bio::DB::Taxonomy',
    'Pod::Usage',
        );

    my ($inst) = ExtUtils::Installed->new();
    my (@installed_modules) = $inst->modules();

    for ( @import_modules ) {

        eval{ $inst->validate($_) };
        if($@) {
            print qq{\n Module $_   NOT OK!\n };
            #exit 1;
        } # end 'if'
        else { print "\n Module $_ OK!\n";}
    } # end 'for'
} # end 'BEGIN' block

Write bash in perl script !

Jit — Wed, 03 Mar 2021 22:48:53 -0600

#use strict;
use warnings;
use ExtUtils::Installed;
use LWP::Simple;
use Archive::Extract;
#
# First, check if all the required modules have been installed in the system and download the mandatory database
#
BEGIN {
    my @import_modules = (
    'Cwd',
    'File::chdir',
    'File::Copy',
    'POSIX',
    'Tie::File',
    'Try::Tiny',
    'Data::Dumper',
    'File::Basename',
    'Bio::SeqIO',
    'FindBin',
    'File::Remove',
    'Capture::Tiny',
    'File::Temp',
    'File::Spec::Functions',
    'Statistics::Multtest',
    'File::Path',
    'Statistics::Distributions',
    'Getopt::Long',
    'Statistics::R',
    'Math::Round',
    'File::Find',
    'Bio::DB::Taxonomy',
    'Pod::Usage',
        );

    my ($inst) = ExtUtils::Installed->new();
    my (@installed_modules) = $inst->modules();

    for ( @import_modules ) {

        eval{ $inst->validate($_) };
        if($@) {
            print qq{\n Module $_   NOT OK!\n };
            #exit 1;
        } # end 'if'
        else { print "\n Module $_ OK!\n";}
    } # end 'for'
} # end 'BEGIN' block

#Bash script else here

print "\nDownloading third party database for Alienomics\n";



print << `EOC`; # execute command and get results
set -o xtrace
echo hi there;
#wget ftp://ftp.ncbi.nlm.nih.gov/pub/taxonomy/taxdump.tar.gz;
echo "----";
echo "Jit here";
sleep 9s


#wget ftp://ftp.ncbi.nlm.nih.gov/pub/taxonomy/taxdump.tar.gz;
echo "JJ"
EOC

Perl script to check perl modules and download NCBI, BUSCO, Taonomy, Silva databases !

Jit — Mon, 01 Mar 2021 23:13:19 -0600

use strict;
use warnings;
use ExtUtils::Installed;
use LWP::Simple;
use Archive::Extract;
#
# First, check if all the required modules have been installed in the system and download the mandatory database
#
BEGIN {
    my @import_modules = (
    'Cwd',
    'File::chdir',
    'File::Copy',
    'POSIX',
    'Tie::File',
    'Try::Tiny',
    'Data::Dumper',
    'File::Basename',
    'Bio::SeqIO',
    'FindBin',
    'File::Remove',
    'Capture::Tiny',
    'File::Temp',
    'File::Spec::Functions',
    'Statistics::Multtest',
    'File::Path',
    'Statistics::Distributions',
    'Getopt::Long',
    'Statistics::R',
    'Math::Round',
    'File::Find',
    'Bio::DB::Taxonomy',
    'Pod::Usage',
        );

    my ($inst) = ExtUtils::Installed->new();
    my (@installed_modules) = $inst->modules();

    for ( @import_modules ) {

        eval{ $inst->validate($_) };
        if($@) {
            print qq{\n Module $_   NOT OK!\n };
            #exit 1;
        } # end 'if'
        else { print "\n Module $_ OK!\n";}
    } # end 'for'
} # end 'BEGIN' block

#Bash script else here

print "\nDownloading third party database for Alienomics\n";

#Install NCBI taxdump
my $url = 'ftp://ftp.ncbi.nlm.nih.gov/pub/taxonomy/taxdump.tar.gz';
my $file = 'taxdump.tar.gz';

my $code = getstore($url, $file);
#print $code;

my $ae = Archive::Extract->new( archive => $file);

my $ok = $ae->extract( to => 'third_party_DB/taxdump' );


#Install silvaDB
#https://www.arb-silva.de/no_cache/download/archive/release_138_1/Exports/
#LSU: Large subunit (23S/28S ribosomal RNAs)
#SSU: Small subunit (16S/18S ribosomal RNAs)

my $silva_url_LSU = 'https://www.arb-silva.de/fileadmin/silva_databases/release_138_1/Exports/SILVA_138.1_LSUParc_tax_silva.fasta.gz';
my $silva_LSU_fileName = 'SILVA_138.1_LSUParc_tax_silva.fasta.gz';

my $silva_LSU_code = getstore($silva_url_LSU, $silva_LSU_fileName);
#print $silva_LSU_code;

my $silva_LSU_ae = Archive::Extract->new( archive => $silva_LSU_fileName);

my $silva_ok = $silva_LSU_ae->extract( to => 'third_party_DB/silvaDB/SILVA_138.1_LSUParc_tax_silva.fasta' );


#Install buscoDB
#https://busco-archive.ezlab.org/

my $busco_url_LSU = 'https://busco-archive.ezlab.org/datasets/metazoa_odb9.tar.gz';
my $busco_LSU_fileName = 'metazoa_odb9.tar.gz';

my $busco_LSU_code = getstore($busco_url_LSU, $busco_LSU_fileName);
#print $busco_LSU_code;

my $busco_LSU_ae = Archive::Extract->new( archive => $busco_LSU_fileName);

my $busco_ok = $busco_LSU_ae->extract( to => 'third_party_DB/' );


#Install diamondDB


__END__

# BlastDB download
my $result = `mkdir taxdump; cd taxdump; wget ftp://ftp.ncbi.nlm.nih.gov/pub/taxonomy/taxdump.tar.gz; tar xzfp taxdump.tar.gz`;
parseResult ($result);

#do your magic here
sub parseResult {
  system ($_[0]);
}

__END__
my $content = get("ftp://ftp.ncbi.nlm.nih.gov/pub/taxonomy/taxdump.tar.gz");
die "Couldn't get it!" unless defined $content;

if (is_success(get("$content"))) {
    print "Download started";
}



if [ ! -d "$PWD/blastDB" ]; then
    if [ ! -d "$PWD/blastDB" ]; then
        mkdir $PWD/blastDB
    fi
    cd $PWD/blastDB
    wget "ftp://ftp.ncbi.nlm.nih.gov/blast/db/nt.*.tar.gz"
    for a in nt.*.tar.gz; do tar xzf $a; done
    #Out of the directory
    cd ..
else
   echo -e "exists \t blastDB, If not installed sucessfully, delete the folder and re-run it"
fi

echo -e "\nAll Done\n"

echo -e "\nInstall R package install.packages('Ckmeans.1d.dp') \n"

/;