Why Remote Session Management is a Must-Have

In bioinformatics, tasks like genome assembly, RNA-seq analyses, and phylogenetic computations often take hours or days. A dropped SSH connection can result in:

• Lost Progress: Restarting a job from scratch wastes valuable time.
• Workflow Interruptions: Disruptions can derail your focus and productivity.
• Corrupted Data: Interrupted processes may lead to incomplete or corrupted outputs.

By integrating screen, tmux, or mosh into your workflow, you can avoid these setbacks and ensure a seamless experience.

Screen: The Classic Workhorse

Screen is a terminal multiplexer that comes pre-installed on most Linux systems. It allows you to manage multiple terminal sessions and reconnect to them even after being disconnected.

Getting Started with Screen:

1. Start a Session:
   screen
2. Detach from a Session:
   Press Ctrl+A, then D.
3. Reattach to a Session:
   screen -r

Pro Tip: Enhance your screen experience with a customized .screenrc configuration file. Download one here: Get .screenrc.

Tmux: A Modern Alternative

Tmux takes everything great about screen and adds modern features, including better key bindings and intuitive session management. It\u2019s perfect for bioinformaticians who want more control over their workflow.

Getting Started with Tmux:

1. Start a Session:
   tmux
2. Detach from a Session:
   Press Ctrl+B, then D.
3. Reattach to a Session:
   tmux attach

Customize Your Tmux Experience:
Use a .tmux.conf file to personalize your setup. Grab one here: Download .tmux.conf.

Mosh: The Mobile Shell for Unreliable Connections

SSH works well for stable networks, but it struggles in areas with spotty connectivity. Enter Mosh, the Mobile Shell. Designed for intermittent networks, Mosh keeps your session alive even when the connection drops temporarily.

Why Mosh is a Game-Changer:

• No lag over high-latency networks.
• Automatically reconnects when the network is restored.
• Ideal for working on the go, from cafes to trains.

Getting Started with Mosh:

1. Install Mosh:
   sudo apt install mosh # For Debian/Ubuntu
2. Connect to a Server:
   mosh username@server

Learn more at mosh.org.

Why This Matters for Bioinformatics

Every bioinformatician knows the value of time and data integrity. Tools like screen, tmux, and mosh provide a lifeline when running long analyses, enabling you to:

• Safeguard your work against disconnections.
• Easily manage multiple workflows in parallel.
• Stay productive, even in challenging environments.

Quickstart Cheat Sheet

• Screen:

  screen # Start a session Ctrl+A, D # Detach screen -r # Reattach

• Tmux:

  tmux # Start a session Ctrl+B, D # Detach tmux attach # Reattach

• Mosh:

  mosh username@server

Final Thoughts

As a bioinformatician, your time is too valuable to spend restarting analyses due to technical hiccups. With screen, tmux, and mosh in your toolkit, you can work smarter, protect your progress, and stay productive no matter where you are. Start using these tools today and transform the way you work with remote systems.

Let me know how these tools work for you, and don\u2019t forget to follow for more bioinformatics tips!

Here are the commands used to connect by Secure SHell:

On the server side

sudo apt-get install ssh

sudo apt-get install openssh-server

sudo /etc/init.d/ssh start

sudo nano /etc/ssh/sshd_config

Uncomment port 22
Uncomment HostKey /etc/ssh/ssh_host_rsa_key
Uncomment AuthorizedKeysFile .ssh/authorized_keys .ssh/authorized_keys2
Set pubkey authentication to "yes"

sudo systemctl restart sshd.service # or sudo /etc/init.d/ssh reload

On the client side:
in ~/.ssh

ssh-keygen -t rsa # set passphrase or not
ssh-copy-id -i ~/.ssh/id_rsa user@100.100.10.100

--> write "yes" then password in

ssh user@100.100.10.100

--> write password --> you should be in

--> in /home/user/.ssh/config type:
Host WhateverName
HostName 100.100.10.100
User username
ForwardX11 yes
ForwardAgent yes
IdentityFile ~/.ssh/id_rsa

--> you should now be able to connect with :

ssh WhateverName

CSAR-web can serve as a convenient and useful scaffolding tool allowing the users to efficiently and accurately scaffold their draft genomes according to a complete or incomplete reference genome. 

Address of the bookmark: http://genome.cs.nthu.edu.tw/CSAR-web

I thought that load average 1.0 on my two core machine means that the CPU usage is at 50%. That's not quite right. And also, why does it say 1.0?

I decided to look everything up and document it here.

Address of the bookmark: https://peteris.rocks/blog/htop/

https://academic.oup.com/ve/article/7/1/veab042/6248116

Address of the bookmark: https://academic.oup.com/ve/article/7/1/veab042/6248116

1. NCBI (National Center for Biotechnology Information):

   • NCBI provides various databases related to genetic information, including 16S rRNA sequences.
   • You can access the 16S ribosomal RNA sequences from NCBI's Nucleotide database (https://www.ncbi.nlm.nih.gov/nucleotide/).
   • Perform a search using keywords like "16S rRNA" or specific bacterial names to find relevant sequences.
   • You can download sequences individually or in batches using the provided tools.

2. GreenGenes:

   • GreenGenes is a widely used 16S rRNA gene sequence database.
   • You can access it at http://greengenes.secondgenome.com/.
   • GreenGenes provides precompiled databases for various purposes, including classification, alignment, and phylogenetic analysis.

3. SILVA:

   • SILVA (https://www.arb-silva.de/) is another comprehensive database for ribosomal RNA (rRNA) sequences.
   • It covers not only 16S rRNA but also other ribosomal RNA sequences.
   • SILVA provides precompiled databases for various purposes, including taxonomic classification and alignment.

4. Ribosomal Database Project (RDP):

   • RDP (http://rdp.cme.msu.edu/) is a curated database that offers 16S rRNA sequences.
   • It provides tools for sequence analysis and classification.
   • You can download sequences and taxonomy information from their website.

5. QIIME (Quantitative Insights Into Microbial Ecology):

   • QIIME (https://qiime2.org/) is a widely used bioinformatics platform for microbiome analysis.
   • It provides tools for analyzing microbial communities, including processing 16S rRNA sequences.
   • QIIME often includes its own preprocessed 16S rRNA databases that can be used for analysis within the platform.

Before downloading any database, make sure to read the terms of use and citation requirements, as some databases may have specific usage policies. Additionally, consider the compatibility of the database with your analysis pipeline and software tools.

NCBI 16s RNA database location ftp://ftp.ncbi.nih.gov/blast/db/16SMicrobial.tar.gz

Tasks:  239 total,   1 running,  238 sleeping,   0 stopped,   0 zombie

This line tells you how many processes are running. If you are using laptops machines it’s not so interesting because you really are the only one using this machine.

Cpu(s):  0.0%us,  0.0%sy,  0.0%ni,100.0%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st

This line contains the CPU load. The first two numbers are how busy the system is doing computation (“us” stands for “user”) and how busy the system is doing system-y things like accessing disks or network (“sy” stands for “system”). We’ll talk more about this later.

Mem:   49457320k total,    3492174k used,  14535596k free,    1435148k buffers

This should be easy to understand – how much memory you’re using!

Swap:   539356k total,   28332k used,   836562k free,    29862014k cached

Swap is just on-disk memory that can be used to “swap” out programs from main memory. Again, we’ll talk about this later.:

PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND
  1 root      39   19  0  0  0 S  0.0  0.0   246:57.22 kipmi0
  2 root      RT   0     0    0    0 S  0.0  0.0   0:00.00 migration/0

And... finally! What’s actually running! The two most important numbers are the %CPU and %MEM towards the right, as well as the COMMAND. This tells you how compute- and memory-intensive your program is. Right now, nothing’s running so the numbers aren’t very interesting, but just wait until we run something...

Address of the bookmark: http://www.vital-it.ch/

SLURM is similar in many ways to most other queue systems. You write a batch script then submit it to the queue manager. The queue manager then schedules your job to run on the queue (or partition in SLURM parlance) that you designate. Below we will provide an outline of how to submit jobs to SLURM, how SLURM decides when to schedule your job and how to monitor progress.

SLURM has a number of valuable features compared to other job management systems:

• Kill and Requeue SLURM’s ability to kill and requeue is superior to that of other systems. It waits for jobs to be cleared before scheduling the high priority job. It also does kill and requeue on memory rather than just on core count.
• Memory Memory requests are sacrosanct in SLURM. Thus the amount of memory you request at run time is guaranteed to be there. No one can infringe on that memory space and you cannot exceed the amount of memory that you request.
• Accounting Tools SLURM has a back end database which stores historical information about the cluster. This information can be queried by the users who are curious about how much resources they have used.

Summary of SLURM commands

The table below shows a summary of SLURM commands. These commands are described in more detail below along with links to the SLURM doc site.

Address of the bookmark: https://bioinfo.noble.org/GWASPRO/