<![CDATA[BOL: Related items]]> https://bioinformaticsonline.com/related/38248?offset=20 https://bioinformaticsonline.com/pages/view/9204/keep-your-important-ssh-session-running-when-you-disconnect-from-server Sat, 15 Mar 2014 21:39:17 -0500 https://bioinformaticsonline.com/pages/view/9204/keep-your-important-ssh-session-running-when-you-disconnect-from-server <![CDATA[Keep Your Important SSH Session Running when You Disconnect from Server !!!]]> As a Bioinformatician/ Computational biologist we swim in the ocean of genomic/proteomics data, and play with them with an ease. In our day to day simulation, analysis, comparative study we do need to run exhaustive programs, which might take more than a week. In such cases we do need to disconnect from sever in a way that our program/session should not get terminated. To do so there are lots of software, tools such as tmux ( http://tmux.sourceforge.net/, nohup (http://ss64.com/bash/nohup.html) , byobu (https://help.ubuntu.com/10.04/serverguide/byobu.html) and other commands (disown -a && exit), but following are the ones I use the most.

Screen is like a window manager for your console. It will allow you to keep multiple terminal sessions running and easily switch between them. It also protects you from disconnection, because the screen session doesn’t end when you get disconnected.

You’ll need to make sure that screen is installed on the server you are connecting to. If that server is Ubuntu or Debian, just use this command:

sudo apt-get install screen

Now you can start a new screen session by just typing screen at the command line. You’ll be shown some information about screen. Hit enter, and you’ll be at a normal prompt.

To disconnect (but leave the session running)

Hit Ctrl + A and then Ctrl + D in immediate succession. You will see the message [detached]

To reconnect to an already running session

screen -r

To reconnect to an existing session, or create a new one if none exists

screen -D -r

To create a new window inside of a running screen session

Hit Ctrl + A and then C in immediate succession. You will see a new prompt.

To switch from one screen window to another

Hit Ctrl + A and then Ctrl + A in immediate succession.

To list open screen windows

Hit Ctrl + A and then W in immediate succession

]]> Jitendra Narayan https://bioinformaticsonline.com/blog/view/37627/setting-python-version-as-default-on-linux Tue, 04 Sep 2018 10:15:47 -0500 https://bioinformaticsonline.com/blog/view/37627/setting-python-version-as-default-on-linux <![CDATA[Setting python version as default on Linux]]> If you have a later version than 2.6 you'll need to set 2.6 as the default Python. Later versions would be 2.7 and 3.1; see what you have by typing

python -V

at the terminal. For purposes of this example we'll assume you have 3.1 installed. You'll next need to execute the following commands:

 

sudo apt-get install python2.6 idle-python2.6
sudo update-alternatives --install /usr/bin/python python /usr/bin/python3.1 1
sudo update-alternatives --install /usr/bin/python python /usr/bin/python2.6 10
sudo update-alternatives --config python

This last command will allow you to choose which version of python to use by default. If you have done everything above correctly, python2.6 should already be set as the default. If it is not, choose it to be the default. From now on, running python should start version 2.6.

Undoing These Changes

In some cases (e.g., installing or updating certain packages), you'll get an error message if you've run the commands above. To update these packages, you'll have to temporarily undo these changes. Here's how to do that:

sudo update-alternatives --remove-all python
sudo ln -s python3.1 /usr/bin/python

Once you're done updating these packages, execute the commands at the top to set python2.6 as the default again.

]]> Rahul Nayak https://bioinformaticsonline.com/pages/view/11582/monitor-running-jobs-on-linux-server Fri, 06 Jun 2014 16:18:43 -0500 https://bioinformaticsonline.com/pages/view/11582/monitor-running-jobs-on-linux-server <![CDATA[Monitor running jobs on Linux server]]> You as a bioinformatican run lots of program on your servers. Sometime the shared server is also used by your colleague. If server is busy you sometime need to check the running programs and want to monitor the running programs as well. The "top" command will come in handy when you need to find out if things are still running, how long they’ve been running, or how much memory is being used.

‘top’ is very simple to run: type

%% top

You’ll get a screen that looks like this, and is updated regularly:

image
Simple, right? Heh.

First! Note that you can use ‘q’ or ‘CTRL-C’ to exit from ‘top’.

Now let’s read and understand at each line independently.

The first line:

top - 23:00:48 up 39 days,  2 user,  load average: 0.00, 0.00, 0.00

The first line tells you the current time, how long the machine has been up, how many users are logged in, and the short/medium/long-term compute load on the machine. If you run something for a long time, you’ll see these numbers go up. Right now, the machine is basically just sitting there, so these are all close to 0.

The second line:

Tasks:  239 total,   1 running,  238 sleeping,   0 stopped,   0 zombie

This line tells you how many processes are running. If you are using laptops machines it’s not so interesting because you really are the only one using this machine.

Cpu(s):  0.0%us,  0.0%sy,  0.0%ni,100.0%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st

This line contains the CPU load. The first two numbers are how busy the system is doing computation (“us” stands for “user”) and how busy the system is doing system-y things like accessing disks or network (“sy” stands for “system”). We’ll talk more about this later.

Mem:   49457320k total,    3492174k used,  14535596k free,    1435148k buffers

This should be easy to understand – how much memory you’re using!

Swap:   539356k total,   28332k used,   836562k free,    29862014k cached

Swap is just on-disk memory that can be used to “swap” out programs from main memory. Again, we’ll talk about this later.:

PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND
  1 root      39   19  0  0  0 S  0.0  0.0   246:57.22 kipmi0
  2 root      RT   0     0    0    0 S  0.0  0.0   0:00.00 migration/0

And... finally! What’s actually running! The two most important numbers are the %CPU and %MEM towards the right, as well as the COMMAND. This tells you how compute- and memory-intensive your program is. Right now, nothing’s running so the numbers aren’t very interesting, but just wait until we run something...

]]> Jitendra Narayan https://bioinformaticsonline.com/bookmarks/view/19633/vital-it Thu, 18 Dec 2014 10:46:59 -0600 https://bioinformaticsonline.com/bookmarks/view/19633/vital-it <![CDATA[Vital-IT]]> Vital-IT is a bioinformatics competence center that supports and collaborates with life scientists in Switzerland and beyond. The multi-disciplinary team provides expertise, training and maintains a high-performance computing (HPC) and storage infrastructure, so as to help develop, maintain and extend life science and medical research (activities).

Address of the bookmark: http://www.vital-it.ch/

]]> Abhi https://bioinformaticsonline.com/pages/view/27321/slurm-basics Fri, 13 May 2016 04:42:24 -0500 https://bioinformaticsonline.com/pages/view/27321/slurm-basics <![CDATA[SLURM basics !]]> SLURM is a queue management system and stands for Simple Linux Utility for Resource Management. SLURM was developed at the Lawrence Livermore National Lab and currently runs some of the largest compute clusters in the world.

SLURM is similar in many ways to most other queue systems. You write a batch script then submit it to the queue manager. The queue manager then schedules your job to run on the queue (or partition in SLURM parlance) that you designate. Below we will provide an outline of how to submit jobs to SLURM, how SLURM decides when to schedule your job and how to monitor progress.

SLURM has a number of valuable features compared to other job management systems:

  • Kill and Requeue SLURM’s ability to kill and requeue is superior to that of other systems. It waits for jobs to be cleared before scheduling the high priority job. It also does kill and requeue on memory rather than just on core count.
  • Memory Memory requests are sacrosanct in SLURM. Thus the amount of memory you request at run time is guaranteed to be there. No one can infringe on that memory space and you cannot exceed the amount of memory that you request.
  • Accounting Tools SLURM has a back end database which stores historical information about the cluster. This information can be queried by the users who are curious about how much resources they have used.

Summary of SLURM commands

The table below shows a summary of SLURM commands. These commands are described in more detail below along with links to the SLURM doc site.

 SLURMSLURM Example
Submit a batch serial job sbatch sbatch runscript.sh
Run a script interatively srun srun --pty -p interact -t 10 --mem 1000 /bin/bash /bin/hostname
Kill a job scancel scancel 999999
View status of queues squeue squeue -u akitzmiller
Check current job by id sacct sacct -j 999999
]]> Radha Agarkar https://bioinformaticsonline.com/bookmarks/view/38378/gwaspro-a-high-performance-genome-wide-association-analysis-server Fri, 07 Dec 2018 08:04:57 -0600 https://bioinformaticsonline.com/bookmarks/view/38378/gwaspro-a-high-performance-genome-wide-association-analysis-server <![CDATA[GWASpro: A High-Performance Genome-Wide Association Analysis Server]]> GWASpro supports building complex design matrices, by which complex experimental designs that may include replications, treatments, locations and times, can be accounted for in the linear mixed model (LMM). GWASpro is optimized to handle GWAS data that may consist of up to 10 million markers and 10,000 samples from replicable lines or hybrids. GWASpro provides an interface that significantly reduces the learning curve for new GWAS investigators.

 

Address of the bookmark: https://bioinfo.noble.org/GWASPRO/

]]> Rahul Nayak https://bioinformaticsonline.com/blog/view/43896/list-of-comparative-genomics-resources Tue, 28 Jun 2022 04:08:06 -0500 https://bioinformaticsonline.com/blog/view/43896/list-of-comparative-genomics-resources <![CDATA[List of comparative genomics resources !]]>

Compare structural and functional annotations of proteins between sequenced genomes.

View AREs in the human transcriptome and study the comparative genomics of AREs in model organisms.

Find information about orthologous genes in prokaryotes.

Search for publicly available QTL data on livestocks and animal species.

Find information about bovine genomics data.

A multi-organism web-based resource system designed to easily retrieve, correlate and interpret data across species.

A database resource of developmentally associated conserved non-coding elements.

Delineate conserved non-coding blocks from upstream regions of putative orthologous gene pairs from man, mouse, rat, fugu, Mus musculus, Danio rerio, and zebrafish.

Find coexpressed gene lists and networks in human and mouse.

Construct phylogenetic tree of microorganisms based on oligopeptide content of their complete proteomes.

A novel database server that classifies GenBank's dbEST (database of expressed gene sequences) libraries and removes contaminants.

Find information about the ancestral relationship between genes.

Provides an overview of the genomic organization of microRNAs and extent of conservation during evolution in different metazoan species.

Conduct comparative biometric analysis of chromosomes of different organisms.

Search for Indices of gene and transcripts in human and mouse.

Search and compare 12 new and old Drosophila genomes.

Access to whole genome alignments of human, mouse, rat and fish sequences.

Find eukaryotic paralog/paralogon information.

Analyze the evolutionary process of overlapping genes when comparing different species.

The first publicly available system reported to handle inter-species gene mention normalization.

A web-based software/database system aimed at an improved and accelerated annotation of prokaryotic genomes.

Visualize gene indices of human, mouse, Arabidopsis, Zebrafish, Drosophila and Sheep.

Find information about mutated mouse genes.

A data management and analysis system for metagenomes

Search for influenza sequence, vaccine, and drug resistance information.

LAMHDI, the initiative to Link Animal Models to Human DIsease, is designed to accelerate the research process by providing biomedical researchers with a simple, comprehensive Web-based resource to find the best animal models for their research.

The missing link between multi-species full genome comparisons and functional analysis.

Conduct comparative analysis of completely sequenced microbial genomes.

A biologist-centric software for evolutionary analysis of DNA and protein sequences.

Conduct single nucleotide polymorphisms diversity measurements among homologous sequences from the Mammalia class.

Find information about 1000s of microbial genomes.

A database dedicated to the study of genome conservation.

Explore orthologous relations across 352 complete genomes.

Browse complete genomes in several clades.

A database of orthologous genomic markers for placental mammal phylogenetics.

Conduct genome wide functional and structural analysis.

Conduct genome-wide cis-regulatory module (CRM) predictions for both the human and the mouse genomes.

Compare phenotypes of a given gene or gene set in different model organisms.

Phylemon is a web server that integrates a selected suite of more than 20 different tools from the most popular stand-alone programs of phylogenetic and evolutionary analysis.

Use this database to see where in the evolution some phylogenetic lineages were started, and over which species they were contained.

Search for genomic information on nematode satellite species Pristionchus pacificus.

Find information about related protein sequences.

Database hosting genomics and post-genomics data from multiple protozoans.

A database of pseudogene families based on the protein families from the Pfam database.

Find genomic information about mammals.

Find information about predicted regulons in prokaryotic transcription regulation.

Perform systematic comparison of proteome data among species.

A comparative map viewer dedicated to the biology of the Solanaceae family.

Analyze data from functional analysis on fragmented microbial genetic material.

Visualize and explore comparative genomic hybridization data sets.

Search for genome-wide annotations of regulatory sites in yeast and prokaryotes genomes.

Search for information on adaptive evolution in gene families of higher plants and chordate.

Generate graphical maps of circular genomes that show sequence features, base composition plots, analysis results and sequence similarity plots.

Conduct a comprehensive analysis of genes and genomes.

An interactive online poster presentation on the Macaque genome, including high-quality images, video clips, and Web resources

Search for annotated genetic information of expressed sequence tags (ESTs) in different eukaryotic organisms.

Find mapping and expression information for a unigene cluster (ESTs and full-length mRNA sequences organized into clusters that each represent a unique known or putative gene)

A public online resource for identifying or designing 'universal' overgo-hybridization probes from conserved sequences that can be used to efficiently screen one or more genomic libraries from a designated group of species.

Comprehensive suite of programs and databases for comparative analysis of genomic sequences.

Find metabolic networks and phylogenomic information on a taxonomically diverse range of eukaryotes.

]]> Shruti Paniwala https://bioinformaticsonline.com/bookmarks/view/33219/dbcan-a-web-server-and-database-for-automated-carbohydrate-active-enzyme-annotation Mon, 29 May 2017 05:39:29 -0500 https://bioinformaticsonline.com/bookmarks/view/33219/dbcan-a-web-server-and-database-for-automated-carbohydrate-active-enzyme-annotation <![CDATA[dbCAN: a web server and DataBase for automated Carbohydrate-active enzyme ANnotation]]> dbCAN is a web server and DataBase for automated Carbohydrate-active enzyme ANnotation, funded by the BioEnergy Science Center of the DOE. Similar resources on the web include CAZy database and CAT. All data in dbCAN are generated based on the family classification from CAZy database while it has the following unique features compared with CAZy database and CAT:

  • dbCAN provides the capability of automated and comprehensive CAZyme annotation of a given genome submitted by the user;
  • dbCAN provides an explicitly defined signature domain for each and every CAZyme family along with its location in all the relevant full-length CAZyme proteins in all sequenced genomes;
  • dbCAN provides the most complete set of metagenomic CAZyme genes published so far and represents the first step towards discovering novel CAZyme catalysts in metagenomes;
  • dbCAN provides a subfamily classification of the existing CAZyme families based on sequence similarities;
  • dbCAN make all pre-computed data freely available to the public, including sequence alignments, hidden markov models (HMMs) and phylogenies of the signature domain regions in each and every CAZyme family and subfamily.

dbCAN is updated regularly when CAZy database created new families based on latest literature.

Address of the bookmark: http://csbl.bmb.uga.edu/dbCAN/index.php

]]> Jit https://bioinformaticsonline.com/bookmarks/view/40235/bioinformatics-web-development-course Wed, 06 Nov 2019 20:42:48 -0600 https://bioinformaticsonline.com/bookmarks/view/40235/bioinformatics-web-development-course <![CDATA[Bioinformatics web development course]]> This web development course, targeted at Biology and Bioinformatics students, aims at teaching from scratch all the skills needed to setup a fully working Linux web server and to develop and deploy web applications for Bioinformatics.

No previous programming knowledge is assumed. By following this tutorial you will learn the fundamental concepts of programming by using scripting languages: variables, types, arrays, cycles, conditional statements, functions, objects, regular expressions, files reading and manipulation et-cetera.

Address of the bookmark: http://www.cellbiol.com/bioinformatics_web_development/introduction/

]]> Jit https://bioinformaticsonline.com/bookmarks/view/41565/csar-web-a-web-server-of-contig-scaffolding-using-algebraic-rearrangements Fri, 10 Apr 2020 04:39:36 -0500 https://bioinformaticsonline.com/bookmarks/view/41565/csar-web-a-web-server-of-contig-scaffolding-using-algebraic-rearrangements <![CDATA[CSAR-web: a web server of contig scaffolding using algebraic rearrangements]]> CSAR-web is a web-based tool that allows the users to efficiently and accurately scaffold (i.e. order and orient) the contigs of a target draft genome based on a complete or incomplete reference genome from a related organism. 

CSAR-web can serve as a convenient and useful scaffolding tool allowing the users to efficiently and accurately scaffold their draft genomes according to a complete or incomplete reference genome. 

Address of the bookmark: http://genome.cs.nthu.edu.tw/CSAR-web

]]> BioStar