Linux tips & hacks#

This page is meant for documenting Linux and Shell (BASH, ZSH, etc) hacks and tips which are very useful but too complicated to remember for me.

Using VS Code with Remote-SSH#

Visual Studio Code, the almighty code editor, comes with an extension to allow you to connect and develop code in a remote server. To use this follow the instructions below (based on the official instructions here).

Setting up Remote-SSH#

Before you get started, obviously you need an OpenSSH compatible SSH client installed in your machines.
In VS code, install the Remote-SSH extension.
Depending on the remote server, you may also needed to set the below settings as described here.
- Set the following properties in settings.json to automatically prompt you to enter the password. An easy way to open the settings is to do Command+,.
  "remote.SSH.showLoginTerminal": true, "remote.SSH.useLocalServer": false
- If you are on macOS and Linux and want to reduce how often you have to enter a password or token, you can enable the ControlMaster feature on your local machine so that OpenSSH runs multiple SSH sessions over a single connection. To do this,
  Add the below lines to ~/.ssh/config,
  
  Host * ControlMaster auto ControlPath ~/.ssh/sockets/%r@%h-%p ControlPersist 600
  
  Run mkdir -p ~/.ssh/sockets to create the sockets folder
Do Command+Shift+p and select Remote-SSH: Connect to Host.... If you have your SSH host names in the ~/.ssh/config then they will automatically appear in the dropdown.
Select the server you wish to connect, enter the password, and you should log in without issues (hopefully!). For further troubleshooting, check here.

To connect through command-line#

Even the slightest increase in efficiency is something we look for 😉. So why not connect directly through Remote-SSH from your terminal.

code --remote ssh-remote+<remoteHost> <remotePath>

Make sure to have the full path for <remotePath>.

For managing, editing, viewing or searching files#

Get filename of log files not containing a particular string#

This is useful to identify (logs of) jobs that failed. Inn the below example, the log files are named log-0.out, log-1.out, …, log-333.out. The string name to check is successfully. Then we will also print the line containing the sampleID.

for i in {0..333};
    do
        VAR=`cat log-"$i".out | tail -1`;
        case $VAR in
            *"successfully"*) continue; ;;
            *) printf "\n""\033[1;36m""$i""\033[0m""\n"; grep sampleID log-"$i".out ; tail log-"$i".out; ;;
        esac;
    done

For making symbolic links for a bunch of files#

For making symbolic links for a bunch of files in a new directory (target directory), based on some specific files from a different directory (source directory). Here, the grep -v option is used to ignore files names we do not want to create symbolic links for.

# Create and array containing the files names we want
array=($(ls * | grep -v <source-directory-path>/*<file-name-string>* | xargs -n 1 basename))

#Then go to the target directory and do
for file in ${array[@]};
    do
        ln -s <source-directory-path>/"$file" "$file";
    done

Removing a bunch of files except one type#

Removing a bunch of files except for the ones that fit the <file-name-string>. The -v option if for verbose.
rm -v !(*<file-name-string>*)

File processing - PDF, PNG, JPEG, etc.#

Converting PDF to Images#

The package poppler (manual) installs several useful commands for processing PDF’s. You can install poppler through brew,

brew install poppler

And this will install several commands like pdfattach, pdftoppm, etc. You can use pdftoppm to convert images to JPEG or PNG. Below is a simple example where we convert a PDF to JPEG. The cropbox option is to trim the image into the frame.

pdftoppm -jpeg -cropbox Name.pdf NewName

You can also use the below command if you wish iteratively change the name of a bunch of files. Below is an example of converting all PDF files inside a folder to JPEG.

# Create and array containing the files names we want
array=($(ls *pdf))
#Then go to the target directory and do
for file in ${array[@]};
    do
        pdftoppm -jpeg -cropbox $file ${file%".pdf"};
    done

On this page