Why Remote Session Management is a Must-Have

In bioinformatics, tasks like genome assembly, RNA-seq analyses, and phylogenetic computations often take hours or days. A dropped SSH connection can result in:

• Lost Progress: Restarting a job from scratch wastes valuable time.
• Workflow Interruptions: Disruptions can derail your focus and productivity.
• Corrupted Data: Interrupted processes may lead to incomplete or corrupted outputs.

By integrating screen, tmux, or mosh into your workflow, you can avoid these setbacks and ensure a seamless experience.

Screen: The Classic Workhorse

Screen is a terminal multiplexer that comes pre-installed on most Linux systems. It allows you to manage multiple terminal sessions and reconnect to them even after being disconnected.

Getting Started with Screen:

1. Start a Session:
   screen
2. Detach from a Session:
   Press Ctrl+A, then D.
3. Reattach to a Session:
   screen -r

Pro Tip: Enhance your screen experience with a customized .screenrc configuration file. Download one here: Get .screenrc.

Tmux: A Modern Alternative

Tmux takes everything great about screen and adds modern features, including better key bindings and intuitive session management. It\u2019s perfect for bioinformaticians who want more control over their workflow.

Getting Started with Tmux:

1. Start a Session:
   tmux
2. Detach from a Session:
   Press Ctrl+B, then D.
3. Reattach to a Session:
   tmux attach

Customize Your Tmux Experience:
Use a .tmux.conf file to personalize your setup. Grab one here: Download .tmux.conf.

Mosh: The Mobile Shell for Unreliable Connections

SSH works well for stable networks, but it struggles in areas with spotty connectivity. Enter Mosh, the Mobile Shell. Designed for intermittent networks, Mosh keeps your session alive even when the connection drops temporarily.

Why Mosh is a Game-Changer:

• No lag over high-latency networks.
• Automatically reconnects when the network is restored.
• Ideal for working on the go, from cafes to trains.

Getting Started with Mosh:

1. Install Mosh:
   sudo apt install mosh # For Debian/Ubuntu
2. Connect to a Server:
   mosh username@server

Learn more at mosh.org.

Why This Matters for Bioinformatics

Every bioinformatician knows the value of time and data integrity. Tools like screen, tmux, and mosh provide a lifeline when running long analyses, enabling you to:

• Safeguard your work against disconnections.
• Easily manage multiple workflows in parallel.
• Stay productive, even in challenging environments.

Quickstart Cheat Sheet

• Screen:

  screen # Start a session Ctrl+A, D # Detach screen -r # Reattach

• Tmux:

  tmux # Start a session Ctrl+B, D # Detach tmux attach # Reattach

• Mosh:

  mosh username@server

Final Thoughts

As a bioinformatician, your time is too valuable to spend restarting analyses due to technical hiccups. With screen, tmux, and mosh in your toolkit, you can work smarter, protect your progress, and stay productive no matter where you are. Start using these tools today and transform the way you work with remote systems.

Let me know how these tools work for you, and don\u2019t forget to follow for more bioinformatics tips!

Here are the commands used to connect by Secure SHell:

On the server side

sudo apt-get install ssh

sudo apt-get install openssh-server

sudo /etc/init.d/ssh start

sudo nano /etc/ssh/sshd_config

Uncomment port 22
Uncomment HostKey /etc/ssh/ssh_host_rsa_key
Uncomment AuthorizedKeysFile .ssh/authorized_keys .ssh/authorized_keys2
Set pubkey authentication to "yes"

sudo systemctl restart sshd.service # or sudo /etc/init.d/ssh reload

On the client side:
in ~/.ssh

ssh-keygen -t rsa # set passphrase or not
ssh-copy-id -i ~/.ssh/id_rsa user@100.100.10.100

--> write "yes" then password in

ssh user@100.100.10.100

--> write password --> you should be in

--> in /home/user/.ssh/config type:
Host WhateverName
HostName 100.100.10.100
User username
ForwardX11 yes
ForwardAgent yes
IdentityFile ~/.ssh/id_rsa

--> you should now be able to connect with :

ssh WhateverName

More at https://software.broadinstitute.org/gatk/download/bundle

ftp://ftp.ncbi.nlm.nih.gov/snp/organisms/human_9606/VCF/GATK/.

ftp://ftp.broadinstitute.org/bundle/hg38/hg38bundle/

Address of the bookmark: https://console.cloud.google.com/storage/browser/genomics-public-data/resources/broad/hg38/v0?pli=1

• viewing files and directories;
• editing, copying and renaming files;
• and many other actions.

https://www.farmanager.com/

Basic instruction at

https://conemu.github.io/en/FarManager.html

Plugins at 

https://plugring.farmanager.com/

Address of the bookmark: https://www.farmanager.com/

Address of the bookmark: https://academic.oup.com/bioinformatics/article/25/22/3005/179064

1) expression-based heat maps from transcriptomic, proteomic and metabolomic experiments; 2) pairwise distance maps;

3) correlation maps;

4) image overlay heat maps;

5) latitude and longitude heat maps and

6) geopolitical (choropleth) heat maps.

Heatmapper offers a number of simple and intuitive customization options for easy adjustments to each heat map’s appearance and plotting parameters. Heatmapper also allows users to interactively explore their numeric data values by hovering their cursor over each heat map, or by using a searchable/sortable data table view.

Ref https://www.ncbi.nlm.nih.gov/pubmed/27190236

Address of the bookmark: http://www2.heatmapper.ca/

Dash Bio is a suite of bioinformatics components that make it simpler to analyze and visualize bioinformatics data and interact with them in a Dash application.

The source can be found on GitHub at plotly/dash-bio.

These docs are using Dash Bio version 0.1.4.

Address of the bookmark: https://dash.plot.ly/dash-bio

Pst. hey, you, join our stargazers :)

Address of the bookmark: https://github.com/firecrawl/firecrawl

Perl's second wave of adoption came from the growth of the world wide web. Dynamic web pages—the precursor to modern web applications—were easy to create with Perl and CGI. Thanks to Perl's ubiquity as a language for system administrators and its power to manipulate text, it was the default choice for web programming. Its presence everywhere made it popular and, in some ways, the duct tape of the Internet.

Web Application Development

The old days of CGI programs and the simple development style that represented seem clunky. Web pages have become web applications. Development has moved from generating static HTML to both client and server side programming, with rich client interfaces and powerful backends.

Perl is still well suited for developing modern web apps. The language grows more powerful and easier to use every year, the available libraries are wonderful and keep getting better, and the inventions and discoveries available in modern Perl are unsurpassed.

In particular, a modern Perl developer can do amazing things with modern Perl tools. If you still think of Perl web development as a cgi-bin directory full of messy scripts that spew warnings to STDERR, you're a decade out of date. Better yet, you can replace that mess piecemeal, thanks to the new tools and techniques of modern Perl. See, for example, the ever-growing list of technologies Built in Perl.

Modern Perl Web Frameworks

While the old wave of web development may have made the CGI.pm module central, modern Perl web programming follows a stricter separation of business logic, URL and request routing, and output. The days of slinging a string here, an array there, a Perl hash yonder, declaring every variable at the top of the program, and maybe making a subroutine are gone. The Perl world has seen the value of abstraction and ways to mechanize away boilerplate. Perl has dozens of frameworks and toolkits designed to make web development and deployment simpler.

Any of a dozen of these frameworks will help you do great things, but three in particular stand out. You can build web sites and web applications of tremendous value with all three. These are neither the only good possibilities (think of POE or Jifty or Continuity or...) nor the only mechanisms for web programming with Perl (see Mechanize or LWP or Mojo::UserAgent for more). Yet if you want three good options to choose between, start here.

Catalyst

The Catalyst framework is a flexible and powerful system for building small to large web apps. It uses the Moose object system to provide great APIs for extension and further development. It's the most mature of the modern top Perl web frameworks, yet it retains its flexibility and vibrancy. In particular, its plugin and extension ecosystem allows it to evolve to provide new and essential features.

Catalyst has embraced the Plack/PSGI standard for Perl web deployment and recent versions are exploring high-scalability, event-based request handling models.

Dancer

The Dancer framework is deliberately minimal in syntax and scope, but it also has a vibrant plugin ecosystem. Dancer particularly excels for smaller sites and applications, though good programmers can build larger things with it.

The first version of Dancer was easy to use. Dancer 2 continues that ease while improving the internals and robustness of applications.

Mojolicious

The Mojolicious (Mojo) framework has a real-time design based on high performance event handling. Its focus is solving new and interesting problems in simple and effective ways, and the project has produced a lot of new code that does old things in better ways.

In particular, Mojolicious goes to great lengths to support new web standards, such as CSS 3, web sockets, and HTTP 2.

Where Catalyst embraces the CPAN fully, Mojolicious by design provides most of what an average app might need in a single download. It's still fully compatible with the CPAN, but the intention is to provide good working defaults in a package that's easy to start with. Mojo's fans are quick to praise it as fun to develop.

A modern Perl web developer should be familiar with at least one of these frameworks.

Modern Perl Storage Mechanisms

Perl's venerable DBI module has been the focal point of database access since its invention. Its design allows it to provide the same interface to huge relational databases and flat files alike through its DBD extension mechanism. Yet the DBI by itself isn't the be-all, end-all of data storage and access in Perl.

DBIx::Class

DBIx::Class sits on top of DBI to provide an API to your database based on the concept of queries and results. This is often sufficient to remove all but the most complicated of SQL from your code, leaving you to manipulate your business models instead of the small details of how a relational database works. The power and maintainability you receive is well the small cost of the learning curve.

Even better, DBIC can manage (and even generate) your database schema for you.

Recent versions of DBIC have demonstrated that a well-written ORM can perform much better than even clever hand-written code. Because it builds on the Perl DBI, it scales everywhere from SQLite to PostgreSQL, MySQL, Oracle, and more.

Rose::DB

The lesser-known but no less powerful Rose::DB::Object builds on Rose::DB to provide an object-relational mapper for Perl. While its high level features most directly compare to those of DBIx::Class, it's often measurably faster.

NoSQL on the CPAN

Of course the CPAN has modules for almost any NoSQL database or job queue or persistence mechanism you could name, and several you have never heard of. Everything you need is a quick CPAN or cpanm away!

Modern Perl Deployment Strategies

In the early days of the web, deploying a Perl web application meant putting one or more .cgi or .pl files in a special directory and hoping that your system administrator had everything configured correctly. The execution model was often slow and cumbersome, and accessing shared resources such as databases was often tricky.

Modern Perl has better choices. While deployment strategies are the source of many arguments, the return on your investment from learning the modern way is impressive.

Plack/PSGI

The PSGI specification (as exemplified by Plack) describes a strategy for building Perl web apps independent of server and with the possibility to share custom processing behaviors.

In other words, it's a standard for writing Perl apps to take advantage of the huge ecosystem of Perl development available on the CPAN without tying yourself to a server like Apache, Apache 2, nginx, or anything else.

Any good modern Perl web framework (including those listed here) supports PSGI. Several deployment mechanisms exist to meet various business needs which also support PSGI. In particular, you can deploy the same application with a local testing server on your own machine as you can to your production server or servers without changing your application at all.

mod_perl

The older but still viable mod_perl Apache httpd module embeds Perl into the web server. This was the first widespread persistence mechanism for Perl web applications themselves and it's still popular to this day, though PSGI compliance is often the choice for new development. (PSGI handlers to use mod_perl as the backend are available.)

Modern Perl developers should familiarize themselves with PSGI and the wealth of available Plack middleware.

Perl Web Development

Of course no discussion of Perl web development would be complete without mentioning the strength of the CPAN. Almost any project will benefit from the wealth of freely available libraries built to solve real problems. These distributions run the gamut from full-blown web frameworks and content management systems to APIs for web services, development tools, testing systems, and interfaces to document formats and external resources.

For example, if you need to write a web service which accepts JSON data and produces Excel spreadsheets, you can glue together a few CPAN distributions and get the job done early. If you need to consume XML from a remote service and emit a PDF, you're in luck.

Perl's prowess as a general purpose programming language as well as its flexibility and power in managing text and gluing systems together make it a wonderful fit for web development. The community's adoption of modern Perl standards such as PSGI and Plack only enhance your power.

Web application development in Perl is still viable, and modern Perl tools and techniques and libraries make it more powerful and pleasant than ever.

Address of the bookmark: http://wego.genomics.org.cn/