BOL: Related items

Creating conda environment for python2.7

Jit — Mon, 07 May 2018 08:56:52 -0500

TIP: By default, environments are installed into the envs directory in your conda directory. Run conda create --help for information on specifying a different path.

Use the Terminal or an Anaconda Prompt for the following steps.

To create an environment:
```
conda create --name myenv
```
NOTE: Replace myenv with the environment name.
When conda asks you to proceed, type y:
```
proceed ([y]/n)?
```

This creates the myenv environment in /envs/. This environment uses the same version of Python that you are currently using, because you did not specify a version.

To create an environment with a specific version of Python:

conda create -n myenv python=3.4

To create an environment with a specific package:

conda create -n myenv scipy

OR:

conda create -n myenv python
conda install -n myenv scipy

To create an environment with a specific version of a package:

conda create -n myenv scipy=0.15.0

OR:

conda create -n myenv python
conda install -n myenv scipy=0.15.0

To create an environment with a specific version of Python and multiple packages:

conda create -n myenv python=3.4 scipy=0.15.0 astroid babel

TIP: Install all the programs that you want in this environment at the same time. Installing 1 program at a time can lead to dependency conflicts.

To automatically install pip or another program every time a new environment is created, add the default programs to the create_default_packages section of your .condarc configuration file. The default packages are installed every time you create a new environment. If you do not want the default packages installed in a particular environment, use the --no-default-packages flag:

conda create --no-default-packages -n myenv python

TIP: You can add much more to the conda create command. For details, run conda create --help.

➜ redundans git:(master) ✗ conda create --name py27 python=2.7
Solving environment: done

==> WARNING: A newer version of conda exists. <==
current version: 4.5.0
latest version: 4.5.2

Please update conda by running

$ conda update -n base conda

## Package Plan ##

environment location: /home/urbe/anaconda3/envs/py27

added / updated specs:
- python=2.7

The following packages will be downloaded:

package | build
---------------------------|-----------------
wheel-0.31.0 | py27_0 61 KB
python-2.7.15 | h1571d57_0 12.1 MB
certifi-2018.4.16 | py27_0 142 KB
sqlite-3.23.1 | he433501_0 1.5 MB
setuptools-39.1.0 | py27_0 582 KB
openssl-1.0.2o | h20670df_0 3.4 MB
pip-10.0.1 | py27_0 1.7 MB
ca-certificates-2018.03.07 | 0 124 KB
------------------------------------------------------------
Total: 19.6 MB

The following NEW packages will be INSTALLED:

ca-certificates: 2018.03.07-0
certifi: 2018.4.16-py27_0
libedit: 3.1-heed3624_0
libffi: 3.2.1-hd88cf55_4
libgcc-ng: 7.2.0-hdf63c60_3
libstdcxx-ng: 7.2.0-hdf63c60_3
ncurses: 6.0-h9df7e31_2
openssl: 1.0.2o-h20670df_0
pip: 10.0.1-py27_0
python: 2.7.15-h1571d57_0
readline: 7.0-ha6073c6_4
setuptools: 39.1.0-py27_0
sqlite: 3.23.1-he433501_0
tk: 8.6.7-hc745277_3
wheel: 0.31.0-py27_0
zlib: 1.2.11-ha838bed_2

Proceed ([y]/n)? y

Downloading and Extracting Packages
wheel 0.31.0: #################################################################################################################################################################################################### | 100%
python 2.7.15: ################################################################################################################################################################################################### | 100%
certifi 2018.4.16: ############################################################################################################################################################################################### | 100%
sqlite 3.23.1: ################################################################################################################################################################################################### | 100%
setuptools 39.1.0: ############################################################################################################################################################################################### | 100%
openssl 1.0.2o: ################################################################################################################################################################################################## | 100%
pip 10.0.1: ###################################################################################################################################################################################################### | 100%
ca-certificates 2018.03.07: ###################################################################################################################################################################################### | 100%
Preparing transaction: done
Verifying transaction: done
Executing transaction: done
#
# To activate this environment, use:
# > source activate py27
#
# To deactivate an active environment, use:
# > source deactivate
#

➜ redundans git:(master) ✗ source activate py27

Subprocess pkg

Rahul Agarwal — Sun, 17 Aug 2014 07:59:32 -0500

Subprocess is one of simplest way of running linux command from within python code

Example:

if you want to run fastqc for QC of fastq file:

from subprocess import Popen,PIPE,call

p=Popen(["fastqc","-f","fastq","-o", "/home/name/result/","/dev/stdin"],stdin=fopen("read.fastq","r") ,stdout=PIPE,stderr=PIPE)

print p.stderr

p.stdout.close()

http://pymotw.com/2/subprocess/

Installing ELGG on Ubuntu !

Neel — Wed, 25 May 2022 02:26:05 -0500

Elgg is an open-source and highly customizable framework used for building an online social environment. It provides a simple and powerful user interface that helps to manage and build your content through a web browser. Elgg offers a rich set of features including messaging, microblogging, file-sharing, RSS support, access control, groups, and many more.

In this tutorial, we will show you how to install and configure Elgg social networking platform on Ubuntu 20.04.

Prerequisites

• A fresh Ubuntu 20.04 VPS on the Atlantic.net Cloud Platform
• A valid domain name pointed to your server IP
• A root password configured on your server

Step 1 – Create Atlantic.Net Cloud Server

First, log in to your Atlantic.Net Cloud Server. Create a new server, choosing Ubuntu 20.04 as the operating system with at least 2GB RAM. Connect to your Cloud Server via SSH and log in using the credentials highlighted at the top of the page.

Once you are logged in to your Ubuntu 20.04 server, run the following command to update your base system with the latest available packages.

apt-get update -y

Step 2 – Install Apache, MariaDB and PHP

Elgg runs on Apache web server, is written in PHP, and uses MySQL/MariaDB as a database backend, so you will need to install the Apache, MariaDB, PHP and other required PHP extensions to your server. You can install all of them with the following command:

apt-get install apache2 mariadb-server php libapache2-mod-php php-common php-sqlite3 php-curl 
php-intl php-mbstring php-xmlrpc php-mysql php-gd php-xml php-cli php-zip unzip wget -y

After installing all the packages, edit the php.ini file and change some recommended settings.

nano /etc/php/7.4/apache2/php.ini

Change the following values:

max_execution_time = 300
memory_limit = 512M
upload_max_filesize = 100M
date.timezone = Asia/Kolkata

Save and close the file, then restart the Apache service to apply the configuration changes.

systemctl restart apache2

Step 3 – Create a Database for Elgg

Next, you will need to create a database and user for Elgg. First, log in to MySQL shell with the following command:

mysql

Once logged in, create a database and user with the following command:

CREATE DATABASE elgg;
CREATE USER 'elgg'@'localhost' IDENTIFIED BY 'secure-password';

Next, grant all the privileges to the elgg database with the following command:

GRANT ALL ON elgg.* TO 'elgg'@'localhost' IDENTIFIED BY 'secure-password' WITH GRANT 
OPTION;

Next, flush the privileges and exit from the MariaDB shell with the following command:

FLUSH PRIVILEGES;
EXIT;

At this point, the MariaDB database is created for Elgg.

Step 4 – Install Elgg

First, download the latest version of Elgg from its official website using the following command:

wget https://elgg.org/download/elgg-3.3.13.zip

Once the download is completed, unzip the downloaded file with the following command:

unzip elgg-3.3.13.zip

Next, move the extracted directory to the Apache root directory:

mv elgg-3.3.13 /var/www/html/elgg

Next, create a data directory and set proper ownership and permissions to the Elgg directory:

mkdir /var/www/html/data
chown -R www-data:www-data /var/www/html/elgg
chown -R www-data:www-data /var/www/html/data
chmod -R 755 /var/www/html/elgg

Once you are finished, you can proceed to the next step.

Step 5 – Configure Apache for Elgg

Next, you will need to configure Apache to serve Elgg. You can configure it by creating a new Apache virtual host configuration file:

nano /etc/apache2/sites-available/elgg.conf

Add the following lines:


ServerAdmin admin@example.com
DocumentRoot /var/www/html/elgg/
ServerName elgg.example.com
Options FollowSymLinks
AllowOverride All
ErrorLog /var/log/apache2/elgg-error_log
CustomLog /var/log/apache2/elgg-access_log common

Save and close the file, then enable the virtual host and Apache rewrite module with the following command:

a2ensite elgg.conf
a2enmod rewrite

Finally, restart the Apache service to apply the changes:

systemctl restart apache2

Step 6 – Access Elgg Web Interface

Now, open your web browser and access the Elgg web interface using the URL http://elgg.example.com. You should see the Elgg welcome screen:

Kubeflow: an open, community driven project to make it easy to deploy and manage an ML stack on Kubernetes

BioStar — Fri, 16 Nov 2018 15:05:14 -0600

The Kubeflow project is dedicated to making deployments of machine learning (ML) workflows on Kubernetes simple, portable and scalable. Our goal is not to recreate other services, but to provide a straightforward way to deploy best-of-breed open-source systems for ML to diverse infrastructures. Anywhere you are running Kubernetes, you should be able to run Kubeflow.

Address of the bookmark: https://www.kubeflow.org/

ChIPBase: open database for studying the transcription factor binding sites and motifs

Abhi — Wed, 29 Dec 2021 05:36:03 -0600

ChIPBase v2.0 is an open database for studying the transcription factor binding sites and motifs, and decoding the transcriptional regulatory networks of lncRNAs, miRNAs, other ncRNAs and protein-coding genes from ChIP-seq data. Our database currently contains ~10,200 curated peak datasets derived from ChIP-seq methods in 10 species.

Address of the bookmark: https://rna.sysu.edu.cn/chipbase/

NASA Open Science Data Repository

Abhi — Wed, 18 Dec 2024 11:54:47 -0600

The NASA Open Science Data Repository (OSDR) enables access to space-related data from experiments and missions that investigate biological and health responses of terrestrial life to spaceflight. The goal of OSDR is to enable multi-modal and multi-hierarchical fundamental space life science data be reused toward basic science, applied science, and operational outcomes for space exploration and knowledge discovery. These data include ‘omics, phenotypic, physiological, behavioral, hardware, environmental telemetry; raw, processed; tabular, text, code, bioimaging, and video.

https://www.nasa.gov/reference/osdr-data-processing/

Address of the bookmark: https://www.nasa.gov/osdr/

dcGOR

Martin Jones — Sat, 08 Nov 2014 14:54:28 -0600

An R package for analysing ontologies and protein domain annotations has been published in PLoS Computational Biology (http://dx.doi.org/10.1371/journal.pcbi.1003929). The package is distributed as part of CRAN (http://cran.r-project.org/package=dcGOR), and also at GitHub for version control.

The dedicated website is available in http://supfam.org/dcGOR, from which several demos are also provided:

1. Analysing SCOP domains: http://supfam.org/dcGOR/demo-Fang.html

2. Analysing Pfam domains: http://supfam.org/dcGOR/demo-Basu.html

3. Analysing InterPro domains: http://supfam.org/dcGOR/demo-Customisation.html

CANU: Assembling Large Genomes with Single-Molecule Sequencing and Locality Sensitive Hashing.

Jit — Tue, 26 Apr 2016 11:38:10 -0500

Canu is a fork of the Celera Assembler designed for high-noise single-molecule sequencing (such as the PacBio RSII or Oxford Nanopore MinION). The software is currently alpha level, feel free to use and report issues encountered.

Canu is a hierachical assembly pipeline which runs in four steps:

Detect overlaps in high-noise sequences using MHAP
Generate corrected sequence consensus
Trim corrected sequences
Assemble trimmed corrected sequences

Read the documentation

New release https://github.com/marbl/canu/releases

Address of the bookmark: https://github.com/marbl/canu

ORFfinder with smart BLAST

Jit — Tue, 17 May 2016 01:43:15 -0500

ORF Finder

ORFfinder is a graphical analysis tool for finding open reading frames (ORFs). We’ve been working on a few updates, and we’d like to find out what you think about them. Read on to find out what you can do with the new ORFfinder.

Smart BLAST (https://ncbiinsights.ncbi.nlm.nih.gov/2015/07/29/smartblast/)

Select one or a group of ORFs and BLAST several databases at once, and use the newly developed SmartBLAST to verify protein names. Looking for the traditional results from BLAST? They’re there too.

BBMap/BBTools package: Multipurpose tool designed for converting reads or other nucleotide data between different formats.

Jit — Mon, 13 Jun 2016 05:47:21 -0500

Reformatis a member of the BBMap/BBTools package. It is a multipurpose tool designed for converting reads or other nucleotide data between different formats. It supports, and can inter-convert:

fastq
fasta
fasta+qual
sam
scarf (an old Illumina format)
bam (if samtools is installed)
gzip
zip
ascii-33 (sanger)
ascii-64 (old Illumina)
paired files
interleaved files

It is multithreaded and can process data at over 500 megabytes per second, and can accept streams from standard in and write to standard out, allowing it to be easily dropped into the middle of a pipeline for format conversion. Reformat autodetects formats based on file extensions and content, making it very easy to use; and the autodetection can be overridden, allowing flexibility for people who don't like to follow naming conventions, or out-of-spec fastq files with qualities values like -17 or 120.

The program has been gradually expanded, and can now perform various other functions. None of these will break pairing, if the input is paired.

Quality trimming (either or both ends)
Quality filtering
Fixed-length trimming
Generation of histograms (base composition, quality, etc)
Subsampling (to a fraction of input reads, or an exact number of reads or bases)
Changing fasta line-wrapping length
Reverse-complementing (all reads or only read 2)
Adding /1 and /2 suffix to read names
GC-content filtering
Length-filtering
Testing for corrupted interleaved files

Reformat is compatible with any platform that supports Java 1.7 or higher. It also has a bash shellscript for simpler invocation. Typical usage examples:

Reformat fastq into fasta:
reformat.sh in=x.fq out=y.fa

Interleave paired reads:
reformat.sh in1=x1.fq in2=x2.fq out=y.fq

Note - you can actually use a shortcut if paired read files have the same name with a 1 and a 2. This is equivalent to the above command:
reformat.sh in=x#.fq out=y.fq

De-interleave reads:
reformat.sh in=x.fq out1=y1.fq out2=y2.fq

Verify that interleaving appears correct, assuming Illumina namimg conventions:
reformat.sh in=x.fq vint

Convert ASCII-33 to ASCII-64:
reformat.sh in=x.fq out=y.fq qin=33 qout=64

Quality-trim paired reads to Q10 on the left and right ends and discard reads shorter than 50bp after trimming:
reformat.sh in1=x1.fq in2=x2.fq out1=y1.fq out2=y2.fq outsingle=singletons.fq qtrim=rl trimq=10 minlength=50

Subsample 10% of the first 20000 pairs in an interleaved file:
reformat.sh in=x.fq out=y.fq reads=20000 samplerate=0.1 int=t
(in this case "int=t" overrides interleaving autodetection, to ensure reads are treated as pairs)

Pipe in a gzipped sam file and pipe out fasta:
reformat.sh in=stdin.sam.gz out=stdout.fa

Reverse-complement reads:
reformat.sh in=x.fq out=y.fq rcomp

For reformatting a file with very long sequences, Reformat will need more memory; just add the additional flag "-Xmx2g". For example, to change the line-wrapping length on the human genome (which has individual sequences over 200Mbp long) to 70 characters:
reformat.sh -Xmx2g in=HG19.fa.gz out=HG19_wrapped.fa.gz fastawrap=70

For additional functions, please run the shellscript with no arguments, or just read it with a text editor. If you have any questions, please post them in this thread.

For people using a non-bash terminal, you may need to type "bash reformat.sh" instead of just "reformat.sh".
For users of Windows or other platforms that do not support bash shellscripts, replace "reformat.sh" with "java -ea -Xmx200m /path/to/bbmap/current/ jgi.ReformatReads"
for example,
java -ea -Xmx200m C:\bbmap\current\ jgi.ReformatReads in=x.fq out=y.fa

Reformat can be downloaded with BBTools here:
https://sourceforge.net/projects/bbmap/